Is a heat pump or propane the more economic way to heat with your rates?

I would find it hard to believe that propane is cheaper to use than a heat pump. Installation cost, that's another matter.

If you have no ductwork already, the ductless minisplit would probably be the way to go. They do make ceiling mounted units.

A heat pump minisplit would be more efficient than electric baseboard.

Thanks for the comments. Our house is tricky as it started out as a two room cabin in the 50's, then 2 bedrooms in the 70's and the newest part in the 90's. Section by section we are remodeling it. No one ever lived here in the winter, hence no furnace. Everyone up here uses propane so we will probably stick with that as our fireplace and stove use it.

Weed has a good point. Depending on your climate, a heat pump would make a lot of sense. You just have to run some numbers. As a bonus, you get cooling and dehumidification. What you should install depends on climate, your preferences and needs, as well as fuel costs if money is a issue (and it almost always is). I believe that in some areas, perhaps most, propane is the most costly heating fuel at this point.

Mini splits come in high wall, ceiling cassettes, short duct (pancake), floor (really low wall) and the most common high wall versions. Although mini splits are a good solution sometimes, more conventional ducted systems would be better. There are some extensive posts here by a guy heating and cooling a small home in central PA with mini splits. Perhaps you should search them up.

A big question is, do you have a basement? What is your climate? I suggest that you look into local energy efficiency programs to guide your reno efforts. They are often utility administered, but not always. Often the states run them. Often they will pay the whole cost of an energy rater to come in and evaluate your home and make calculations that will tell you where to invest. For example, their numbers might support the fact that if you are making a very well insulated and air-sealed home, you don't really need to spend extra money on efficient heat sources. Sticking with the electric baseboard might be your best move. If a program does not pay you for the evaluation, getting a Resnet or BPI energy rater in there on your own dime would still be a good move. Most houses are tricky!

A word of caution on "ductless" systems; they MUST be installed by personnel with very specific training. My neighbor required four follow-up visits to obtain optimum performance.

We live in northern Door County and the winters can be pretty brutal. Part of the house that we have remodeled is on a slab. The bedrooms and bath are over a crawl space and the newest addition is over a basement. It's a strange little house.

I have not looked at them in a while, but I think that even sealed combustion gas fireplaces are not that efficient compared to good gas furnaces. I think that you could make cold climate heat pumps work there. Have you considered a couple of modern wall-hung direct vent gas heaters?

Cold weather heat pump/minisplit is still likely to be less costly long-term (but do check the numbers based on your local propane and electricity prices). Keep in mind, most heat pumps (incl mini-splits) will have backup resistance heat - at the same efficiency as your electric baseboard - but will only turn that on for the coldest weather; that may be a few days a year, or a couple dozen. But in a relatively cold climate like yours, you may be heating (some of the time) for eight months or more a year, and the vast majority of the time at far greater efficiency than your baseboards (or propane). My heat is on some nights even in July/August.

Armored, which mini splits have backup heat?