Your basic question, "....do you feel that much better after building to be worth the cost, knowing what that extra money could either be invested, or working less long term just to have a better house?" will get as many answers/opinions as there are people who build and reno. It's all situational and no one has your exact situation. I'm not sure personal opinions will help you clarify your thinking but maybe you'll get some useful perspective.

We chose to renovate our first very small home and add a true second story to a 1-1/2 for a number of reasons. We had lived there for years, liked the neighbors, convenient location to town, parks, beach, transportation (commuter train into Chicago), good school district and it would be a good investment as houses there are desirable, increase in value and always sell. A big factor was financial as we would be doing all the work ourselves so the added value of the house would not be outweighed by cost - our equity increased immediately well above the cost and continued to rise for the years we lived there. We did basically the same thing to our next house 4 blocks away in a higher end neighborhood that could handle a more upscale addition/reno. Same financial scenario and value added. Neither of those houses was perfect. There were plenty of compromises about space, floor plan, storage, energy efficiency and on and on. But we truly loved both houses and lived happily in each for many years.

Then we moved to a new state. If we had been able to find an existing house that ticked off the most important boxes - location, neighbors(hood), schools (empty nesters but good school help homes hold value), acreage and perhaps most elusive of all, an existing house constructed in a way as to make reno/addition worthwhile, we would have done it again. But that house/location proved to be a unicorn so we found land and built.

Do I love my house? Absolutely. I also would have loved an old home that we could make our own if we had found one. Living in new and shiny is wonderful but we updated our old homes with plenty of new and shiny too, so that's not exclusive to a new build. And honestly there were elements of charm in our 1924 tudor and 1936 colonial that can only be somewhat replicated in a new build and I still miss. Like all plaster walls. Living in an energy efficient house is new for us as is living on private acreage. There is no way for me to place living in a new build on a scale to measure against a scenario of reno on an old house that didn't happen here.

I think a lot of what you need to ask yourself is how much you like living where you are now. If you're just putting in your time because it happens to be where you bought, then fixing up the house isn't going to make you happier living there. Quality of life in terms of where you live is measured by a whole lot more than the walls that surround you. A fancy kitchen, Pinterest mudroom or Toto toilet won't make up for a location that doesn't meet your needs. OTOH if you like living where you are and investing in updates is not a waste in that location, then updates would probably serve you well and not add the financial (and other) stress of a new build. Good luck - tough decision, for sure.