I think you need to talk to a local realator and see how a finished basement would impact the selling value. Before you make any other upgrades (like the bathroom) make sure you'll actually see a return on the money.

I have spoken with several realtors - one urges me to forget the basement given the the other amenities in the house. Several others say that they would list the house as a 3 bedroom/2.5 bath rather than a 4 bedroom/3.5 bath. The price would be listed much less. When I bought the house, finishing the basement would have cost an additional $15,000. At this point, I assume that the cost would be more like $40,000 as a part of the basement floor would need to be broken to run multiple pipes to the main outlet ($10,000).

While finishing the basement will certainly add to the value of the house the question is if it will be enough to offset the cost. I don't think people value basement "bedrooms" as they do ones on the main level(s). When I was doing such work over a decade ago in another part of the country we were at roughly $31-$33 per square foot for fairly basic finishes plus flooring.

The obvious unknown is who might be interested in your house. Some people are going to find a finished basement appealing and for others the higher price of the home is going to have them looking elsewhere. Unfortunately there isn't a crystal ball.

Get a couple of estimates and see what you're looking at in terms of cost.

Seems you're getting conflicting advice from a couple of realtors. At least in our area, basement space isn't as valuable as really great living space and some people would want to finish it in their own way. It sounds like if your home's main levels are far nicer than comparable houses, that might outweigh the number of beds and baths (it can in my area). Here it isn't only about finished square footage.

I am completely confused as we are discussing a non-walkout basement. The realtors are very confusing in their advice. I am not sure where to go to get a good estimate for the renovation.

It's worth even less without a walkout.