I’ve been a little obsessed by the question of how well heat pumps perform in real world installations. It turns out to be a complicated question and hard to quantify definitively. However, it seems very clear that we should generally expect seasonal coefficients of performance materially below rated performance. In practice, we should expect common seasonal COPs to run at 2.0 to 2.5 with wide variability both above and below that range.
The automobile analogy
Everyone who has ever tracked their gas mileage has experienced mileage below the EPA rated miles-per-gallon for a vehicle. Not that the rating is invalid — it sets a comparison standard. But real world conditions differ from testing conditions, and real world mpg is usually below rated mpg.
If one is driving very conservatively on a flat country road with no intersections and little wind, one can sometimes beat EPA rated mileage. But usually, driving on a freeway, one is going a little above the speed limit and one will not achieve EPA rated highway mileage. Similarly, congested city streets may be much more congested than the EPA tests assume. For hybrid vehicles the disparity between actual and rated mpg can become dramatic when weather chills their battery.
According to Consumer Reports, the EPA mileage ratings are closer to reality than they used to be. But the Consumer Reports “real world” test conditions still don’t match highway conditions. Consumer Reports tests at highway speeds of 65mph. For better or worse, most drivers drive somewhat above 65 mph unless there is heavy traffic or a visible law enforcement presence.
Analogous real vs. test discrepancies appear for heat pumps. There is a rating system for heat pumps and rebate programs require heat pumps to meet certain rating standards, but the installed performance of heat pumps will often be below the rating.
Heat Pump Efficiency Basics
MassCEC offers a basic explanation about how heat pumps work here. It’s also helpful to know some basic college thermodynamics. For understanding real vs. tested performance, the key points about heat pumps are the following:
- The purpose of a heat pump is to pump heat from a cold place to a hot place. That’s a counter-intuitive idea. An air conditioner moves heat from the cold inside to the hot outside. A heat pump can move heat from the cold outside to the warm inside. All modern heat pumps can actually run either way and provide heat or cooling.
- It takes mechanical work to pump heat. Usually, electricity is used to drive motors that do that work. In addition to the intrinsic pumping work, there are friction and other losses in the conversion of electrical energy into mechanical work.
- The efficiency of a heat pump is the ratio of the heat moved to the electrical energy it consumes to do the mechanical work of pumping. This is known as the “coefficient of performance” — the COP.
- Efficiency depends not only on the pump itself, but also on the heat distribution system inside the house. An outdoor heat pump unit supplies warm fluid to heat the home, but the heat in that fluid needs to be transferred efficiently around the home. That is usually done by an “air handler” that blows air over pipes that are warmed by the fluid from the pump. The air handler could be a wall mounted unit that circulates air within a room or it could receive and expel air through a duct system. Either way, if that air handler cannot circulate enough air past those warmed pipes, it will not pick up all the available heat in the fluid. In a ducted system, ducts that are too narrow may make it harder to circulate air and diminish the efficiency of the pump. Heat pumps are always rated in combination with an indoor air handler that is assumed to be properly installed in a room or supported by an adequate duct system.
- The greater the difference between the temperatures on the hot side and the cold side, the more energy it will take to pump heat per unit of heat. Obviously enough, when the heat pump is heating a home, the colder it is outside, then the more heat a heat pump will need to supply. But also, the colder it is outside, then the greater the hot-side/cold-side difference and therefore the less efficient the pump will be. So, electrical energy consumed by a heat pump in very cold weather goes up for two reasons — greater need and lower efficiency. Some heat pumps do much better than others in maintaining their efficiency at colder temperatures, and can handle Massachusetts’ climate, but for any heat pump system, there is an outside temperature below which it becomes inefficient.
- Efficiency also depends on the physical layout of the house. For example, suppose a wall-mounted air handler unit is installed in one room. If the room is large or the unit is expected to heat other rooms as well, then heat may not spread across the whole area. The air near the wall unit might get to 80 if the pump is run enough to heat the farther reaches to 70. But that means the air going into the wall unit is 80 and the pump is performing as if it were heating to 80 degrees. And that means the pump is less efficient because it is pumping across a larger temperature difference. So, energy is being wasted in two ways — a part of the house is excessively hot, but also the pump is running less efficiently. The same kinds of problems can occur in ducted systems if ducts are not well placed.
- Finally, efficiency is best when the pump operates steadily. There is mechanical energy involved in cycling on and off and also the heat in the fluid in the lines connecting the indoor and outdoor components can be wasted when the fluid is not moving. One wants a pump that is large enough to supply the home’s needs in very cold weather, but also a pump that can dial down its output when the weather is not quite as cold so that it keeps up but doesn’t have to cycle on and off. An over-sized pump, incapable of adequately lowering its output, is a common cause of inefficiency.
Performance rating metrics for heat pumps
Across a season, a heat pump system will perform at different efficiencies on different days. So, one would like to know an average coefficient of performance (COP) over a representative season.
The most commonly published seasonal performance rating is the heating seasonal performance factor (“HSPF”) — a variant of the COP that uses heat BTUs in the numerator and watt-hours in the denominator and represents an expected average performance across a season. If it were perfectly predictive of reality, the HSPF would equal the total BTUs of heat pumped into the house across the season divided by the watt-hours used by the heat pump across the season. To compute a predicted seasonal average COP from the the HSPF one divides by 3.412 to convert the BTUs in the numerator to the same unit as the denominator (watt hours) — 3.412 is the number of BTUs of energy in a watt-hour of energy.
To qualify for Mass Save incentives heat pump systems had to have an HSPF of 9.5 or better in 2022. The Mass Save qualified product list is an inventory of specific combinations of outdoor compressors and indoor air handlers that have been rated to meet this standard.
While the HSPF offers a single comparative number for particular heat pump equipment, professionals choosing equipment will look at multiple available numbers that offer a sense of how the equipment will perform at different temperatures. In particular, Northeast Energy Efficiency Partnerships has developed a database including coefficients of performance at different temperatures. The data is presented together with a performance model that includes local temperature data.
For 2023, there is a new metric, HSPF2, which is intended to better reflect real conditions by requiring that the air handlers push against more air resistance than in the previous HSPF test. The same pumps will score about 15% lower on the HSPF2 than on the HSPF. This is expected to bring rated performance closer to real performance. Under the new HSPF2 rating methodology, Mass Save is requiring pumps to hit 8.1 for ducted and 8.5 for ductless which translate to SCOPs of 2.37 and 2.49.
Performance rating methods for heat pumps
The HSPF’s reported for heat pumps are the results of tests performed in certain ideal conditions. Those conditions are precisely and elaborately specified in a standards document developed by the Air-Conditioning, Heating and Refrigeration Institute (AHRI): AHRI Standard 210/240, Performance Rating of Unitary Air-conditioning & Air-source Heat Pump Equipment.
Summarizing a very complex process for computing the HSPF of a combined system including a heat pump and an air handler: The system is tested running at a steady state under defined air handling conditions at each of several different “outdoor” and “indoor” temperatures and then the coefficients of performance at each temperature are averaged with weighting corresponding to the average temperature mix in the climatic region — so the HSPF is dependent on the climatic region (of which there are six in the standard).
Discrepancies between predicted and actual performance will emerge from several factors:
- The particular climate of the home will rarely match the broad regional averages used for the HSPF. Western Massachusetts is in Region V which runs up to northern Maine. Eastern Massachusetts is in Region IV which runs south to Washington DC. There is a big difference in climate between Boston and Washington DC.
- Each installation requires a difficult set of judgment calls about sizing based on imperfect estimates of the heating load for the home. The pump may not be able to run steadily as contemplated in the HSPF testing process.
- Similarly, a variable speed system will by design run at speeds different than those in the steady state testing.
- Additionally, the real world air handler may face different circulation challenges than the air handler in the test — it may have to press harder to circulate air through a duct system and so may circulate less air, resulting in less heat transfer.
- The heat pump may be serving a home in which the layout does not support the distribution of heat well away from the air handler.
- Temperature profiles vary from winter to winter and different homeowners set their thermostats differently, resulting not only in variable heating needs, but in varying heat-pump efficiency as discussed above.
- Homeowners may choose lower fan speeds for quiet or air stability and this may reduce heat transfer.
- Electric back up systems that use expensive resistance heat may kick-in.
The HSPF provides a rough method for ranking and comparing heat pumps, but it only very crudely predicts real world performance.
Real world heat pump performance data
Public agencies evaluating their heat pump incentive programs are rapidly expanding the historically thin collection of real-world studies performance of heat pumps. We will have better perspective in each of the coming years. So far, the studies seem to support a general expectation of performance between 2.0 to 2.5 with wide variability above and below that level.
| Study | Seasonal Average COP Found |
|---|---|
| Department of Energy (2015): SCOP (seasonal coefficient of performance) for ductless units in 2013-2014. This was a carefully done study, but included only 10 homes. Wide variability. | 2.0 |
| Department of Energy (2018): Only two heat pumps studied; notes that low fan speed may have depressed observed averages. | 2.5 |
| Massachusetts Energy Efficiency Advisory Council Study (2014-6). Over 100 homes homes with ductless installations. Wide COP range (<1 to > 5). (Values extracted from Figures 30 and 31 by Mike Duclos.) | 1.8 for cold winter; 2.4 for a very mild winter |
| Vermont Public Service (2017). “ccHPs operated at 88% of the average nameplate HSPF. In situ HSPF varied from 57% to 119% of nameplate HSPF.” Page 29. Many factors influenced in situ results. | 88% of nameplate |
| Canadian Standards Association (2020). Tested multiple heat pumps using a new test methodology designed to simulate real world variability and challenge the control algorithms of the heat pumps. Measured performance for all tested pumps dropped substantially from HSPF and the rank ordering of heat pumps by performance changed. | Well below nameplate |
| MassCEC/NYSERDA (Cadmus, 2022). Measured seasonal COP of 2.3 across 43 homes, similar results for whole-home and partial conversions. Slide 33. | 2.3 |
| NYSERDA (DNV, 2022). 8 ASHPs intensively monitored within much larger sample. Mostly ductless. Found performance near only slightly below rated. Main concern raised by study was lower-than-projected use of heat pumps resulting in dramatically reduced savings. | 3.3 |
| NYSERDA (Urban, multiple consultants, 2022). Seasonal COP of 2.4 across 19 sites. Method is pre/post fuel use monitoring. Mostly multi-head ductless sytems. (Found no cost-savings at natural gas sites. Across all sites, estimated costs per ton of installed capacity of $4,483. Estimated a per year marginal 592 pounds of carbon dioxide per ton installed. The emissions and installation cost estimates result in a cost of carbon emission saved of $981/MTCO2, assuming a 17 year life.) | 2.4 |
| NYSERDA (Hudson Valley, multiple consultants, 2022). 20 homes. Mostly ductless. Range from 1.7 to 2.9. Method is pre/post fuel use monitoring. (No savings in natural gas sites.) | 2.1 |
| NYSERDA heat pump performance page. Chooses a summary efficiency quote for ASHPs based on the four studies above. | 2.4 |
| Aldrich Presentation to Passive House Conference (2018) reviews several of the other studies listed below. |
The Aldrich presentation listed above includes one slide (page 18) which makes strongly the point about how strongly heat pump performance can vary based on unique conditions. Observe not only the different performance results for the same heat pump in two different sites, but also the various observed Daily COPs within the same site at the same outside temperature (ODB).
Note re performance of combustion heating
While they generate carbon and other forms of pollution, fossil heating systems powered by oil, propane, or natural gas are more predictable in their performance than heat pumps:
- When replacing a boiler or furnace, one generally does not need to struggle to predict the performance of a new and possibly complex heat distribution system. The replacement will be fully compatible with the old heat distribution system (radiators, baseboards, ducts).
- Fossil burning generates the same amount of heat regardless of outside temperature. There is no complex judgment to make about performance across multiple temperatures. One just sizes the furnace or boiler to meet the maximum design temperature delta (outside to inside).
- Fossil fuel systems are expected to stop and start — the need to avoid cycling is much less urgent.
Good quality combustion heating equipment may exceed 90% rated efficiency. For the purpose of broad efficiency comparisons of new oil equipment to heat pumps, one professional has suggested 85% as a generally achievable efficiency ratio for new oil burners (btu’s delivered vs btu’s in the fuel). Older burners may have much lower efficiency and 80% as an overall average appears reasonable. Interested to hear from others with opinions on this.
Please enrich this post with additional data through the comment facility below!
The comment forms will accept standard html. Let me know if you would like to post graphics or attachments, and I can help you do that.
Will, the Feds are now saying we should all get rid of gas stoves and convert to electric:
https://nypost.com/2023/01/09/biden-administration-reportedly-weighing-national-ban-on-gas-stoves/
This is right up your alley, Will.
I assume you like this since it involves going electric, and you always like to force people to do things in accordance with the Climate Change religion.
Electric cars
Heat Pumps
Electric Heat
Covid vaccinations and boosters ad infinitum
Masks and Zoom for kids even though it harms their education
Mandatory Flu vaccines for kids (as Governor Baker ordered or the kids can’t go to school)
“Woke” lessons for schoolkids.
Bike lanes that mess up Concord Avenue so people open car doors into traffic and get killed?
What next, Will?
You’ll think of something to force people to do what you want.
Maybe lower the new Millionaire’s tax to cover incomes above $100,000?
Since many homes in Belmont use radiant heat, not forced hot air, it will be important to offer pure electric solutions beside heat pumps. You can find “electric boilers” via Google. How about some real world data on the pros and cons of replacing a fossil fuel boiler with an electric one?
One wouldn’t want to use an electric boiler per se. Straight resistance heat is both costly and inefficient. What would be great is an efficient air to water heat pump that could replace a boiler in common hydronic systems (like those that use radiators for heat distribution). There are some who hope and expect the heat pump industry to bring out a product like this at some point, but it appears to be some ways off.
Helpfully informative. Thanks, Senator!
No amount of data on the performance or benefits of heat pumps (or electric cars, or electric stoves) makes me want to completely eliminate the use of natural gas in my house (or to get rid of my gasoline-powered vehicle, though a hybrid holds appeal) and become 100% reliant on electricity alone – and I was surprised when you did that in your house, Will. Perhaps you are trying to tell us we better follow your example, because you know the government intends to take our choices away from us. Well, good luck with it. (As someone who grew up in the Soviet bloc and remembers how people were affected by totalitarian policies, I cringe at that — and I think it’s only a matter of time before people start rebelling against excessive control over their lives.)
You mentioned before that in case of a power failure (in wintertime, I assume), you could last in your house for a week. I attribute it to the fact that you had your house gutted prior to moving in, and all your exterior walls and roof were “fattened” to accommodate an unusually thick layer of foam insulation (12 inches or something), and you had a sophisticated climate control system installed (which a house that tight needs to have on, and relying on electricity of course, all the time).
But the costly system your house has is not an option for most homeowners in your district or across the state. Even if some people can afford to do it that way (and don’t mind the tremendous disruption to their lives and altering of the home’s original architecture that installing that much insulation entails), we need to keep in mind that spray foam insulation is a petroleum product that has a significant environmental cost while it’s getting produced, applied, and disposed of. It cannot be recycled or composted. How are we going to be disposing of thousands or millions of tons of it when houses that have it need to be taken down, since nothing lasts forever? There is no way to get rid of it in an environmentally responsible way. Ditto for wind turbines and solar panels.
In our homes, once we are forced to rely exclusively on electricity for heat (incl. warm water) and food preparation, how are people going to cope when there are frequent blackouts (with current energy policies, very likely down the road), or if a very serious, long-lasting power failure occurs due to some unforeseen events (anything is possible – like the covid disaster that none of us saw coming). Also, what if the cost of electricity becomes so high that it starts crushing the budgets of regular people, retirees, low-wage workers, middle-class families living on single incomes?
With current population levels and energy needs, converting electricity production entirely to renewables is not even possible in the foreseeable future – you know that. The aggressive efforts to achieve that will be very expensive and have numerous harmful consequences (on wildlife, food supply, and environment). Consumers will be of course forced to pay for all of that via state and Biden administration’s new taxes on fossil fuels, and by a generally much higher cost of living. So how does it even make sense to champion policies that undermine the natural gas industry that so many of us rely on, and which is supposed to generate tax revenue for creating new “green” energy infrastructure?
For our country to remain safe and prosperous, we need to have a variety of sources of affordable energy available for years to come — otherwise the cost of maintaining national defense, as well as the population’s daily necessities, such as food, housing, and transportation will continue to skyrocket — effectively impoverishing the nation, and almost certainly leading to dire socio-political consequences. The US already has too many people who live paycheck to paycheck or worse (while we keep importing more and more). Things are really getting out of balance. Choosing between embracing some draconian climate measures, or food and shelter and ability to retire, people will always choose the latter.
As for global warming – yes, we can feel it; our winters have gotten milder (but extreme weather events of all kinds have always been happening – even when the media didn’t exist to report them). Humans are contributing to climate change, it would be hard to argue against that – but there is no proof that we are the only cause, or that we can stop those changes — because throughout our planet’s history, climate was always in flux – it changed many times and included periods of warm temperatures and rising ocean levels even when humans didn’t exist, or existed in small numbers and no one was burning fossil fuels.
I just read the other day (on 23andMe, while learning about ancient origins of my maternal line) about Doggerland – the area of land where humans lived for thousands of years, that is now submerged under the North Sea, after it was flooded by rising sea levels around 6500-6200 BCE. It happened when no one was driving cars or using gas stoves – and yet there was a global warming.
The bottom line is that we do not know everything there is to know about the Earth’s climate cycles, and what causes them. One thing is for sure: we live in the now, and we and our children have only our current lives to live. As climate gradually continues to change, it will force some behavioral and civilizational changes for sure – but we are not there yet, and it is misguided to be imposing economic and other hardships on people in the name of preventing some future cataclysm that may not be a cataclysm at all – because human population may start naturally shrinking, and people affected by rising sea levels will just move to higher grounds — just as those who used to live in Doggerland did. This will be happening gradually, over centuries, and this time, humanity will be assisted by a range of helpful technologies that will be making adapting to new conditions easier. Needless to say, I do not lose any sleep over this — and I don’t think we should be allowing climate alarmists and politicians who are aligned with them to be controlling our lives..
I’m definitely not telling people what to do, much less telling them to do as I have done (whole house heat pump; disconnect gas). I think I’ve been very clear that the economics are different for different homes and unfavorable for many. I do think these are issues that homeowners should be thinking about and I appreciate that you are thinking about them.
Will,
Very impressed with your presentation. As background, I am very well versed in the theory of heat pumps, having taken a number of thermodynamic courses at MIT. However, I am not really up-to-date on commercial offerings. With that disclaimer, let me add 3 pieces of additional information to your post that might be helpful.
1. A good analogy to understand what heat pumps do is to look at riding a bicycle on a hill. Gravity pulls you downhill. To go uphill takes lots of energy on your part. The steeper the hill, the more energy you expend. In nature, heat wants to flow from hot to cold (i.e., going downhill). Heat pumps move heat uphill, from cold to hot. This requires energy input, usually in the form of electricity. The bigger the temperature difference (i.e., the steeper the hill) the more energy that is required.
2. The way heat pumps work is by using a working fluid that goes through a thermodynamic cycle. For an air conditioner, the working fluid is called a refrigerant and the cycle is referred to a a refrigeration cycle. In the thermodynamic cycles of heat pumps, the working fluid gets compressed (which requires electricity) and expanded (which results in cooling). After compression there is heat exchange that expels heat. After expansion, there is heat exchange that captures the heat. The one thing you did not discuss in your post is the impact of working fluids. Working fluids are determined by the manufacturer, not the consumer. There are many options for working fluids. The type of working fluid in the heat pump has a big impact on its efficiency, as well as what temperature ranges it can effectively work in.
3. I want to mention geothermal heat pumps. They can have consistently high COPs, definitely over 3, I would guess around 3.5 (once again, I am not familiar with commercial offerings). There are several reasons for this. First, the ground is a constant temperature around 55F. This means smaller temperature differences compared to air-to-air heat pumps. Secondly, it is easier to optimize a thermodynamic cycle knowing you have that constant ground temperature. There is a cost and that is the installation of the underground piping system, which can be significant. There are at least two major options for the underground system. If you have a large enough lot, you can use a horizontal system. They have trenching devices to install the pipes that only minimally impact your lawn. Most lots around here are not big enough for this, so you will need a vertical system of wells.
Thanks, Howie.
Your point about the working fluid is very well taken. There is a trade off between performance and pollution risk — some of the commonly used fluids can make large contributions to global warming if released. This is an issue to which I want to give more attention.
Yes, ground source heat pumps are great. In the dense senate district that I represent, few lots will support a ground source heat pump. We shopped and couldn’t find anyone willing to bid a project on our small lot. Some ground source companies aren’t even bidding in our zip code.
The final slide of the Aldrich presentation mentioned in your blog post reports that 70 ductless heat pumps that were properly sized and installed in Vermont in 2015- 2017 had an average SCOP of 3.0. (HSPF of 10.2)
As you point out, seasonal performance of heat pumps is significantly better in warmer climates. The ASHRAE design temp for Burlington, VT is negative 4°F. While the ASHRAE design temp for Bedford, MA (the closest station to Belmont) is 8°F. That’s a full twelve degrees warmer than Burlington, VT. Air source heat pumps will perform significantly better in Belmont, MA than in Burlington, VT all other things being equal.
http://ashrae-meteo.info/v2.0/
Thank you, Mark. I agree that study suggests that in the right conditions it may be possible to realize seasonal COPs near 3. People that I’ve talked to about that study see it as reflecting particular conditions and still see 2.5 or lower as a more realistic overall field expectation.
That is the same Vermont study linked to in the table above and it seems to be a good study. It offers a wealth of data and it highlights the factors that contribute to the wide variability of heat pump performance.
As you say, in that sample of 77, they measured an HSPF a little over 10, actually 10.7. The measured HSPF is a complicated number created to be comparable to the manufacturer’s rated HSPF, constructed by “binning” hours and weighting performance observations in each bin. Like the rated HSPF, it excludes standby power consumption, but this has a modest impact. See discussion around Table 8 and compare Figure 32 in the study.
These were pumps that had very high manufacturer-rated performance. The rated HSPF’s of the sampled cold climate pumps averaged 11.9 — well above the required HSPF for eligibility for Mass Save incentives (9.5 for ducted and 10 for ductless). All the measured units were ductless, so duct sizing cannot affect their performance. Most of them were smaller units “rarely” used as the exclusive heating system. Figure 5 shows that only 23 of the 77 were models above 15kbtu. It’s also worth noting that the winters during which the measurements were made were mild, although they adjust for that.
In the context of overall performance estimation, I mainly take from this study two propositions: (a) actual performance is less than rated performance; (b) performance varies widely. Additionally, the study suggests that total energy may go up because people use the heat pumps to keep particular rooms warmer or cooler than before.
Hugely valuable post and data. I think real world performance is not well studied, and we may be setting ourselves up for a fall with a push to disconnect fuel heating.
I’m up in NH, built a new ultra high performance house and have no fuel delivery – just heat pumps. Real world performance of the heat pumps definitely leaves something to be desired.
I believe in northern climates mixed fuel likely makes the most sense, as you can certainly bend the cost curve and reduce carbon with the great performance of heat pumps above 30 degrees.
Found this post as I was debating GSHP vs ASHP and it was hugely valuable. Thank you Senator for taking the time to put this data together. At the extreme electricity prices we have now ($.45/kWh for us) the real installed COP matters. If you’re doing a conversion from oil, a gallon of oil releases 40 kWhs of heat. Dandelion claims installed COPs slightly above 4. If that’s true, (40/2.5 – 40/4) * .45 = $2.70 more per gallon oil equivalent of heat.
In fact, at those electric prices its hard to see how an ASHP could ever compete with oil even if the installation is free. Say an oil furnace is 85% efficient. That means for every gallon of oil you were burning in your furnace, your new heat pump is consuming 40 * 8.5 / 2.5 = 13.6 kWh of electricity. At $.45/kWh, oil would have to be above $6.12 to break even. Again, without any installation costs.
This points to a larger problem of electric scarcity. MA has overwhelmingly focused on generating consumer demand, but I hear zero about generating new electricity to meet that demand. If the state actually succeeds in moving all homes off fossil fuels, we’ll have created an enormous new need for electricity (in our case, even using geothermal will nearly double our total household use!) most of it during a part of the year when solar yields are negligible. What’s the plan to meet that need without just burning more fossil fuels? Wind is unreliable, and nuclear power is very expensive and takes a very long time to build.