June 5, 2018 by Maartjes Sevenster, Sevenster Environmental
Our guest-blogger today is Maartje Sevenster, Sevenster Environmental, who has followed and analysed the process leading to the recently published weighting method for the EU Product Environmental Footprint (PEF). Here she shares her serious reservations on the process and the results.
A weighting set for the EU Product Environmental Footprint (PEF) was published last month. The weighting factors have been developed by the Joint Research Centre via an elaborate approach that has attempted to separate value-based weighting into objective factors. Nevertheless, the result is a poor, semi-qualitative approximation that mixes characterization, distance-to-target weighting, panel weighting, and uncertainty. All in all, the approach comes across as a black box of flawed mathematical operations.
A PEF consists of a characterization result for each of 16 impact categories, a corresponding set of weighted normalised results, and one single-score result. The use of this weighting method and the resulting single-score will be a requirement in all PEF studies and is meant to facilitate interpretation.
The final weighting set is an average of three independently derived sets, with the average multiplied by a robustness factor. Two of three independent sets are derived via traditional panel weighting and the third is based on a hybrid ‘evidence-based’ approach.
The term evidence-based has a feel of objectivity about it, but in its first part the approach applies weighting to issues that could better be investigated by natural science, and in the second part – that necessarily must be based on subjective preferences – the chosen approach violates basic requirements for good valuation practice.
In the first part of the JRC approach, an expert panel was asked to score the below seven characteristics of impacts on a scale from 1 to 100:
- Spread of impact
- Time span of generated impact
- Reversibility of impact
- Level of impact compared to planetary boundary
- Severity of effect on human health
- Severity of effect on ecosystem quality
- Severity of effect on resources availability
It is immediately obvious that many of these factors, such as time span, are already covered by the commonly used LCA natural science based characterization models, even if they are not always made explicit in the end results. In fact, Annex 13 of the JRC weighting report includes a similar criticism by Mark Goedkoop: “using a panel to link mid to endpoint is really weird. This means we replace science by the verdict of panellist. I am quite aware about some of the uncertainties in the mid to end point factors, but I always thought we prefer science over the laymen’s view. Uncertain science is always better than no science at all.” Another example: the use of GWP100 for characterizing global climate change impacts implies a natural science based assessment of the timing of the impacts for comparison of greenhouse gas emissions. Is it then valid to subsequently allow an expert panel to assign a zero weight to time span as a weighting factor, which was theoretically possible for the experts in this approach?
Only two of the seven factors are not part of LCA characterization models, namely reversibility and level compared to planetary boundary. Reversibility is also the only factor that is intrinsically categorical and therefore an excellent illustration of the artificiality of the approach. Is it valid to say that an irreversible impact is (only) a 100 times worse than an impact that can be reversed by natural processes within one year? It would certainly be useful to dicuss these factors prior to determining a multi-criteria type panel weighting per impact category.
Factors such as reversibility and time span may well play a role in expert judgements of the severity of a certain impact as compared to others. However, the JRC approach first introduces a categorical scaling for each of those factors turning them into artificial ordinal variables. For instance, for time span the following categories are used:
- Momentary [less than 1 month] = score of 1
- Very short term [more than 1 month and less than 1 year] = score of 20
- Short term [1-3 years] = score of 40
- Medium term [4-30 years] = score of 60
- Long term [31 – 100 years] = score of 80
- Very long term [more than 100 years] = score of 100
Even though we know precisely that some impacts are instantaneous and others may be spread out over hundreds of thousands of years, such as those of radioactive radiation, the difference is here reduced to an arbitrary factor of 100. The bottom line is that most of the seven factors can be evaluated by natural science, albeit with considerable uncertainty, and do not need expert weighting. The scaling wipes out all scientific evidence and along with it any understanding of what a resulting indicator might really mean.
The second part of the JRC expert weighting procedure is a more traditional expert panel judgement of the relative importance of the seven factors. This leads us to another troubling aspect, which is that the seven factors are not completely independent as is required for proper evaluation of (compensatory) weights. Especially the “level compared to planetary boundary” overlaps with all other factors to at least some extent. Moreover, averaging categorical variables is mathematically meaningless, even when the categories appear to be “numerical”.
Finally, this weighting set from the expert panel is averaged with two other weighting sets derived via a different approach. This seriously undermines the transparency of the weighting, which should at all times be straightforward to interpret, not just a set of numbers to arrive at a single score. This is further aggravated by the use of a robustness factor to assess what is in essence uncertainty. Again, this factor involves three arbitrarily scaled ordinal variables that are averaged. The report shows some inconsistencies regarding the final choice for this robustness factor, which is apparently not considered very robust by JRC, since toxicity impacts have been excluded from the benchmark calculations in spite of already having a very low weighting due to their low estimated robustness. The semi-numerical approach gives a false sense of objectivity to this “uncertainty assessment”.
To summarize, the final weighting set is the result of so many mathematically questionable averaging, scaling, and multiplication steps that it is hard to take serious. To allow for proper interpretation of results, weighting sets should be based on clear and transparent principles. It is preferable to use a single-step conversion with a fairly limited but unambiguous perspective, such as weighting based on damage costs.
Previous blog-posts on PEF:
January 31, 2018 by Jannick H. Schmidt
The dog is said to be man’s best friend, but is it a climate enemy?
Dogs and cats are as fond of meat as are their owners. And therefore their CO2 emissions make up a substantial part of their owners emission totals. In fact, our calculation shows that a 10 kilo dog like the ones below on average emits up to 1.1 tonnes of CO2 annually, mainly through their meat consumption.
Last week I was called in as an expert for a radio programme: P1 – ‘Your dog, the climate enemy’. This segment is part of a series, ‘The climate testament’, that addresses climate issues and what each of us can do to alleviate some of the problems we face today.
The largest and most obvious problem with cats and dog lies with their meat consumption. As meat is produced, we see both land use effects and various greenhouse gas emissions, since the conversion from plant to animal protein is very inefficient, esp. for the bigger animals like pigs and cows.
For the radio programme, I had calculated emissions for three different dog sizes and I compared these to driving a car, where the emissions from small dog is equivalent to driving up to 1000 km in a average Danish car, while the big dog equals up to 5200 km driven, provided the dogs eat meat. I based these rough and average calculations on the current dietary recommendations for each of the weight classes given by professor Charlotte Bjørnvad. Bjørnvad is a professor in Internal Medicine at Department of Veterinary Clinical and Animal Sciences, University of Copenhagen.
Luckily for the environmentally conscious pet owner, Bjørnvad explained that a conversion to a more climate neutral diet is possible.
Dogs have had a long co-evolution with humans, eating the leftovers from our table. So in spite of their wolf ancestry, they can now thrive on a vegetarian diet. This is also a consequence of breeding, with all the genetic changes that this entails. Charlotte Bjørnvad explained how we could give them a protein base from e.g. beans combined with eggs, but let their main energy come from carbohydrates. In fact, a dog’s digestive system can handle carbohydrates and fibres from vegetables, provided they are heat-treated, just as is the case for humans.
So there is now enough knowledge to suggest it is safe to switch to a vegetarian diet for our canine friends. For cats, it remains a bit more tricky – a suggestion in the programme was to include insects as a source of animal protein for the cats, since they need a wider range of animal proteins in their diet, compared to both humans and dogs.
So starting from a rather polemic initial question; we arrived safely at the end of the radio programme at a solution that can be directly implemented in your dogs food bowl – next step is to push the pet food market in the right direction.
For those of you that understand Danish the podcast and Danish article is here.
December 11, 2017 by Bo Weidema
Last week I attended a meeting on the EU commission’s Product Environmental Footprint (PEF) arranged by the Danish Environmental Protection Agency. The PEF initiative was presented as providing a level playing field for competition on environmental claims, countering the current situation that many (allegedly 95% of all environmental claims) are misleading.
The PEF pilot phase is now coming to and end, and the Commission representative presented the result as a success story: “We now have a machine that works!” although it was acknowledged that the data to drive the system is still insufficient. The PEF “machine” was likened to a watch that shows the time in simple terms, while the (LCA) clockwork inside is intricate and complex.
In spite of the attempt to present the PEF as a success story, there was much criticism from the more than 100 representatives from Danish industries and NGOs that were present. Some were concerned about the increased costs, pointing out that the money would be better spent on making improvements than on documenting status quo to the consumers. Others questioned whether LCA is at all relevant for consumer information, and it was suggested that new technology is now making LCA information for this purpose out-dated (see also my blog-post on distributed ledger technology).
The PEF system relies on the development of a “Product Category Rule” (PCR) for each product group. These PCRs are developed on a voluntary basis, paid by the majority of the industries that produce the products in each product group, providing a golden business for consultants. During the pilot phase it has turned out that this results in different rules for different products, so that only products within the same product group can be compared (as with the existing eco-labels).
An example from the audience was raised, about the gifts that a speaker often receive as thanks for giving an otherwise unpaid talk: Would a bottle of French wine or a box of locally produced chocolate (with cocoa beans from Africa) be the best choice? Since these two products belong to different product groups, they will have different rules for their PEFs and should therefore not be compared.
This simple example illustrates one of the largest problems of the idea of Product Category Rules, namely that they do not further improvements in environmental performance across product categories. NGOs and scientific advisors have pointed out that even within product categories it is now questionable if the PEF LCA calculation rules further environmental improvements. We have lobbied for constraining the industry consensus on the PCRs to the original Commission Guideline (see also my interpretation guide to this) and the requirements of the ISO LCA standards that focus on environmental improvements, but are now mocked with not understanding that compromise on scientific validity is necessary to reach industry consensus.
Personally, I concluded that if the concern is unfair competition, PEF is not the answer, but rather a part of the problem. A more simple solution would be to enforce the existing legislation on misleading claims, a solution that has worked well in Denmark so far.
In my view, there is not much point in having an expensive watch – even with the simplest user interface – if it shows the wrong time.
October 30, 2017 by Bo Weidema
Since the UN Sustainable Development Goals (SDG) were published 2 years ago, much has been said on the difficulty in implementing them into business practice. Part of the difficulty comes from the wordings, which often appear better suited for governmental use than specifically for use in a business context. But the main difficulty comes from the sheer number of goals (17) and accompanying targets (169) and indicators (so far 230). While this should provide something for everyone, it also implies an obvious risk of cherry-picking and sub-optimised decision-making. These problems have been pointed out very eloquently by other bloggers, e.g. Nienke Palstra & Ruth Fuller from Bond.
The Business and Sustainable Development Commission have done a great job in pointing out the positive market opportunities that the SDGs open up for first-movers, and the UN Global Compact and the Global Reporting Initiative (GRI) have teamed up in an action platform to provide best practices for corporate reporting on the SDGs, with a first analysis report published last month and a “Practical Guide for Defining Priorities and Reporting” announced for January 2018.
So what can we add from an LCA perspective that has not already been said and is not already being done? Well, what is missing in the approaches mentioned above, and which LCA has always been focused on providing, is an overall framework can avoid shifting of responsibilities and avoid sub-optimised decision-making.
Therefore, we now launch the SDG club, a new crowd-funded project to place each of the indicators for 169 targets of the 17 SDGs into a comprehensive, quantified and operational impact pathway framework, linking forward to sustainable wellbeing (utility) as a comprehensive summary (endpoint) indicator for all social, ecosystem and economic impacts. At the same time, we will link each of the indicators for 169 targets back to company specific activities and product life cycles, using a global multi-regional input-output database with environmental and socio-economic extensions. Due to the use of a single endpoint, this framework will allow to differentiate major from minor impact pathways, to quantify trade-offs and synergies, and to compare business decisions, performance and improvement options, also across industry sectors. With this project, we wish to provide an actionable and rational method for businesses and governments to integrate the SDGs into decision making and monitoring.
This new project builds on and extends the impact assessment method developed by 2.-0 LCA consultants for social footprinting, which has been successfully tested for feasibility in global supply chain contexts. For example, a recent whitepaper from Nestlé appraised our method with these words:
“The great benefit of the Social Footprint method lies in the use of widely available background information from databases to assess social impacts top-down. As opposed to many other approaches, this means that some initial data is available for practically any specific case study, drastically reducing the overall cost”
We invite everyone to join the SDG club and thereby contribute to streamline and coordinate action and increase efficiency in implementing the 2030 Agenda.
See also a previous blogpost on sustainability indicators.