Is there a temperature below which a conventional heating system is cheaper to run than a heat pump?

Yes, for some heat pump/boiler combinations and electricity/heating fuel price ratios there is an outdoor temperature below which the boiler burning oil or propane is less expensive to operate. Our modeling has shown that using 2018 energy prices, the transition point is somewhere between 5F and -15F. Switching back and forth exactly when the temperature crosses the transition point could save up to 5% on heating costs. However, forgetting to switch back to the heat pump once the temperature climbs back above the transition point could end up costing more. Therefore, we generally recommend that you select a temperature set point on the heat pump that makes the room feel comfortable (usually several degrees higher than what you have used with your central heating system thermostat) and leave it at that setting throughout the winter.