Wood pellets; geothermal (but retroactively installing it can be very costly); photovoltaic panels? A more efficient oil burner?

Propane is a reasonable alternative to natural gas from the utility company and is commonly used in the country. An electric heat pump is very efficient and will save energy costs. Almost any new heat source/furnace is going to save money over the old system...even if it just 15 years old.