If the radiant heat floor system is rarely used, then why spend a lot of money on the equipment to operate it? Floor radiant heating is very nice, but it does not lend itself to be turned on and off quickly. For that reason it is difficult to set back the temperature and save some energy costs. Perhaps you need to think about an alternate way of heating the bottom floor of the house.

A high recovery hot water heater has an extra large gas burner. Normally a 50 gallon hot water heater would have a 40,000 or 50,000 BTU gas input. The 65,000 BTU input allow it to recover faster when a lot of hot water is used. The extra large burner makes the how water heater about twice the price.

I think a 50 gallon 50,000 BTU hot water heater is fine if it only supply the domestic hot water. Consider replacing your shower heads. All shower heads sold today are low flow. You will be surprised how much hot water you will save per shower.

This is very frustrating:

The cost to operate involves only 3 things: The load (how much heat output is needed), the efficiency (how much heat is wasted - usually up the flue), and the cost of the fuel. Period.

Propane is one of the most expensive fuels to use - so if you are using it go with condensing (92%+) heaters. That leaves the load as the only other place to make improvements - and I'm guessing you could make a lot of improvements there. It's not about judgement, it's about math.

Within reason, the high btu input on a condensing heater doesn't translate into much difference in operating costs. Neither does the tank vs tankless.

The least expensive way to heat hot water is a hybrid electric tank heater - that's especially true compared to propane. It's not the answer for your floor though or if you have uninsulated circulation lines that run all the time.

If you absolutely want the ability to heat the floor on occasion then the answer is a 200k btu hour condensing tankless heater or a condensing tank heater. The condensing tank or tankless will cost about the same to operate.

Lastly, are your circulation lines insulated? Can you run your circulation pump intermittently (timer, motion detector, push button) to limit heat loss.

Regarding your question about keeping up over a long period of time (an hour) - Tankless heaters can deliver 184k btu/hr continuously until your propane tank runs out. That amounts to about 8.5 gpm of water heated from 60 to 105 (that's more than 3 showers at 2.5 gpm - a normal shower head).

The tank type can deliver about 70k btu/hr continuously until your propane tank runs out (about 3 gpm - or about 1 normal shower head of water heated from 60-105)

To answer your question about keeping up with demand we need to do some math. How many gallons per minute is one of your shower heads? How many showers at the same time? How cold is the inlet water? That's the only way the question can be answered.