I’ve been a little obsessed by the question of how well heat pumps perform in real world installations. It turns out to be a complicated question and hard to quantify definitively. However, it seems very clear that we should generally expect seasonal coefficients of performance materially below rated performance. In practice, we should expect common seasonal COPs to run at 2.0 to 2.5 with wide variability both above and below that range.
The automobile analogy
Everyone who has ever tracked their gas mileage has experienced mileage below the EPA rated miles-per-gallon for a vehicle. Not that the rating is invalid — it sets a comparison standard. But real world conditions differ from testing conditions, and real world mpg is usually below rated mpg.
If one is driving very conservatively on a flat country road with no intersections and little wind, one can sometimes beat EPA rated mileage. But usually, driving on a freeway, one is going a little above the speed limit and one will not achieve EPA rated highway mileage. Similarly, congested city streets may be much more congested than the EPA tests assume. For hybrid vehicles the disparity between actual and rated mpg can become dramatic when weather chills their battery.
According to Consumer Reports, the EPA mileage ratings are closer to reality than they used to be. But the Consumer Reports “real world” test conditions still don’t match highway conditions. Consumer Reports tests at highway speeds of 65mph. For better or worse, most drivers drive somewhat above 65 mph unless there is heavy traffic or a visible law enforcement presence.
Analogous real vs. test discrepancies appear for heat pumps. There is a rating system for heat pumps and rebate programs require heat pumps to meet certain rating standards, but the installed performance of heat pumps will often be below the rating.
Heat Pump Efficiency Basics
MassCEC offers a basic explanation about how heat pumps work here. It’s also helpful to know some basic college thermodynamics. For understanding real vs. tested performance, the key points about heat pumps are the following:
- The purpose of a heat pump is to pump heat from a cold place to a hot place. That’s a counter-intuitive idea. An air conditioner moves heat from the cold inside to the hot outside. A heat pump can move heat from the cold outside to the warm inside. All modern heat pumps can actually run either way and provide heat or cooling.
- It takes mechanical work to pump heat. Usually, electricity is used to drive motors that do that work. In addition to the intrinsic pumping work, there are friction and other losses in the conversion of electrical energy into mechanical work.
- The efficiency of a heat pump is the ratio of the heat moved to the electrical energy it consumes to do the mechanical work of pumping. This is known as the “coefficient of performance” — the COP.
- Efficiency depends not only on the pump itself, but also on the heat distribution system inside the house. An outdoor heat pump unit supplies warm fluid to heat the home, but the heat in that fluid needs to be transferred efficiently around the home. That is usually done by an “air handler” that blows air over pipes that are warmed by the fluid from the pump. The air handler could be a wall mounted unit that circulates air within a room or it could receive and expel air through a duct system. Either way, if that air handler cannot circulate enough air past those warmed pipes, it will not pick up all the available heat in the fluid. In a ducted system, ducts that are too narrow may make it harder to circulate air and diminish the efficiency of the pump. Heat pumps are always rated in combination with an indoor air handler that is assumed to be properly installed in a room or supported by an adequate duct system.
- The greater the difference between the temperatures on the hot side and the cold side, the more energy it will take to pump heat per unit of heat. Obviously enough, when the heat pump is heating a home, the colder it is outside, then the more heat a heat pump will need to supply. But also, the colder it is outside, then the greater the hot-side/cold-side difference and therefore the less efficient the pump will be. So, electrical energy consumed by a heat pump in very cold weather goes up for two reasons — greater need and lower efficiency. Some heat pumps do much better than others in maintaining their efficiency at colder temperatures, and can handle Massachusetts’ climate, but for any heat pump system, there is an outside temperature below which it becomes inefficient.
- Efficiency also depends on the physical layout of the house. For example, suppose a wall-mounted air handler unit is installed in one room. If the room is large or the unit is expected to heat other rooms as well, then heat may not spread across the whole area. The air near the wall unit might get to 80 if the pump is run enough to heat the farther reaches to 70. But that means the air going into the wall unit is 80 and the pump is performing as if it were heating to 80 degrees. And that means the pump is less efficient because it is pumping across a larger temperature difference. So, energy is being wasted in two ways — a part of the house is excessively hot, but also the pump is running less efficiently. The same kinds of problems can occur in ducted systems if ducts are not well placed.
- Finally, efficiency is best when the pump operates steadily. There is mechanical energy involved in cycling on and off and also the heat in the fluid in the lines connecting the indoor and outdoor components can be wasted when the fluid is not moving. One wants a pump that is large enough to supply the home’s needs in very cold weather, but also a pump that can dial down its output when the weather is not quite as cold so that it keeps up but doesn’t have to cycle on and off. An over-sized pump, incapable of adequately lowering its output, is a common cause of inefficiency.
Performance rating metrics for heat pumps
Across a season, a heat pump system will perform at different efficiencies on different days. So, one would like to know an average coefficient of performance (COP) over a representative season.
The most commonly published seasonal performance rating is the heating seasonal performance factor (“HSPF”) — a variant of the COP that uses heat BTUs in the numerator and watt-hours in the denominator and represents an expected average performance across a season. If it were perfectly predictive of reality, the HSPF would equal the total BTUs of heat pumped into the house across the season divided by the watt-hours used by the heat pump across the season. To compute a predicted seasonal average COP from the the HSPF one divides by 3.412 to convert the BTUs in the numerator to the same unit as the denominator (watt hours) — 3.412 is the number of BTUs of energy in a watt-hour of energy.
To qualify for Mass Save incentives heat pump systems had to have an HSPF of 9.5 or better in 2022. The Mass Save qualified product list is an inventory of specific combinations of outdoor compressors and indoor air handlers that have been rated to meet this standard.
While the HSPF offers a single comparative number for particular heat pump equipment, professionals choosing equipment will look at multiple available numbers that offer a sense of how the equipment will perform at different temperatures. In particular, Northeast Energy Efficiency Partnerships has developed a database including coefficients of performance at different temperatures. The data is presented together with a performance model that includes local temperature data.
For 2023, there is a new metric, HSPF2, which is intended to better reflect real conditions by requiring that the air handlers push against more air resistance than in the previous HSPF test. The same pumps will score about 15% lower on the HSPF2 than on the HSPF. This is expected to bring rated performance closer to real performance. Under the new HSPF2 rating methodology, Mass Save is requiring pumps to hit 8.1 for ducted and 8.5 for ductless which translate to SCOPs of 2.37 and 2.49.
Performance rating methods for heat pumps
The HSPF’s reported for heat pumps are the results of tests performed in certain ideal conditions. Those conditions are precisely and elaborately specified in a standards document developed by the Air-Conditioning, Heating and Refrigeration Institute (AHRI): AHRI Standard 210/240, Performance Rating of Unitary Air-conditioning & Air-source Heat Pump Equipment.
Summarizing a very complex process for computing the HSPF of a combined system including a heat pump and an air handler: The system is tested running at a steady state under defined air handling conditions at each of several different “outdoor” and “indoor” temperatures and then the coefficients of performance at each temperature are averaged with weighting corresponding to the average temperature mix in the climatic region — so the HSPF is dependent on the climatic region (of which there are six in the standard).
Discrepancies between predicted and actual performance will emerge from several factors:
- The particular climate of the home will rarely match the broad regional averages used for the HSPF. Western Massachusetts is in Region V which runs up to northern Maine. Eastern Massachusetts is in Region IV which runs south to Washington DC. There is a big difference in climate between Boston and Washington DC.
- Each installation requires a difficult set of judgment calls about sizing based on imperfect estimates of the heating load for the home. The pump may not be able to run steadily as contemplated in the HSPF testing process.
- Similarly, a variable speed system will by design run at speeds different than those in the steady state testing.
- Additionally, the real world air handler may face different circulation challenges than the air handler in the test — it may have to press harder to circulate air through a duct system and so may circulate less air, resulting in less heat transfer.
- The heat pump may be serving a home in which the layout does not support the distribution of heat well away from the air handler.
- Temperature profiles vary from winter to winter and different homeowners set their thermostats differently, resulting not only in variable heating needs, but in varying heat-pump efficiency as discussed above.
- Homeowners may choose lower fan speeds for quiet or air stability and this may reduce heat transfer.
- Electric back up systems that use expensive resistance heat may kick-in.
The HSPF provides a rough method for ranking and comparing heat pumps, but it only very crudely predicts real world performance.
Real world heat pump performance data
Public agencies evaluating their heat pump incentive programs are rapidly expanding the historically thin collection of real-world studies performance of heat pumps. We will have better perspective in each of the coming years. So far, the studies seem to support a general expectation of performance between 2.0 to 2.5 with wide variability above and below that level.
|Study||Seasonal Average COP Found|
|Department of Energy (2015): SCOP (seasonal coefficient of performance) for ductless units in 2013-2014. This was a carefully done study, but included only 10 homes. Wide variability.||2.0|
|Department of Energy (2018): Only two heat pumps studied; notes that low fan speed may have depressed observed averages.||2.5|
|Massachusetts Energy Efficiency Advisory Council Study (2014-6). Over 100 homes homes with ductless installations. Wide COP range (<1 to > 5). (Values extracted from Figures 30 and 31 by Mike Duclos.)||1.8 for cold winter;|
2.4 for a very mild winter
|Vermont Public Service (2017). “ccHPs operated at 88% of the average nameplate HSPF. In situ HSPF varied from 57% to 119% of nameplate HSPF.” Page 29. Many factors influenced in situ results.||88% of nameplate|
|Canadian Standards Association (2020). Tested multiple heat pumps using a new test methodology designed to simulate real world variability and challenge the control algorithms of the heat pumps. Measured performance for all tested pumps dropped substantially from HSPF and the rank ordering of heat pumps by performance changed.||Well below nameplate|
|MassCEC/NYSERDA (Cadmus, 2022). Measured seasonal COP of 2.3 across 43 homes, similar results for whole-home and partial conversions. Slide 33.||2.3|
|NYSERDA (DNV, 2022). 8 ASHPs intensively monitored within much larger sample. Mostly ductless. Found performance near only slightly below rated. Main concern raised by study was lower-than-projected use of heat pumps resulting in dramatically reduced savings.||3.3|
|NYSERDA (Urban, multiple consultants, 2022). Seasonal COP of 2.4 across 19 sites. Method is pre/post fuel use monitoring. Mostly multi-head ductless sytems. (Found no cost-savings at natural gas sites. Across all sites, estimated costs per ton of installed capacity of $4,483. Estimated a per year marginal 592 pounds of carbon dioxide per ton installed. The emissions and installation cost estimates result in a cost of carbon emission saved of $981/MTCO2, assuming a 17 year life.)||2.4|
|NYSERDA (Hudson Valley, multiple consultants, 2022). 20 homes. Mostly ductless. Range from 1.7 to 2.9. Method is pre/post fuel use monitoring. (No savings in natural gas sites.)||2.1|
|NYSERDA heat pump performance page. Chooses a summary efficiency quote for ASHPs based on the four studies above.||2.4|
|Aldrich Presentation to Passive House Conference (2018) reviews several of the other studies listed below.|
The Aldrich presentation listed above includes one slide (page 18) which makes strongly the point about how strongly heat pump performance can vary based on unique conditions. Observe not only the different performance results for the same heat pump in two different sites, but also the various observed Daily COPs within the same site at the same outside temperature (ODB).
Note re performance of combustion heating
While they generate carbon and other forms of pollution, fossil heating systems powered by oil, propane, or natural gas are more predictable in their performance than heat pumps:
- When replacing a boiler or furnace, one generally does not need to struggle to predict the performance of a new and possibly complex heat distribution system. The replacement will be fully compatible with the old heat distribution system (radiators, baseboards, ducts).
- Fossil burning generates the same amount of heat regardless of outside temperature. There is no complex judgment to make about performance across multiple temperatures. One just sizes the furnace or boiler to meet the maximum design temperature delta (outside to inside).
- Fossil fuel systems are expected to stop and start — the need to avoid cycling is much less urgent.
Good quality combustion heating equipment may exceed 90% rated efficiency. For the purpose of broad efficiency comparisons of new oil equipment to heat pumps, one professional has suggested 85% as a generally achievable efficiency ratio for new oil burners (btu’s delivered vs btu’s in the fuel). Older burners may have much lower efficiency and 80% as an overall average appears reasonable. Interested to hear from others with opinions on this.
Please enrich this post with additional data through the comment facility below!
The comment forms will accept standard html. Let me know if you would like to post graphics or attachments, and I can help you do that.