If you were to build an investment portfolio with sustainability as one of your main objectives, which of the following two companies would you rather invest in? (a) Tomra, a Norwegian maker of machines enabling the collection of plastic and glass bottles so that they can be recycled or (b) Altria, an American multinational making tobacco and nicotine-based products? If you follow one of the leading ESG research providers, you would rather invest in Altria since it gets a better ESG rating than Tomra. Put differently, a fund holding 2% in Altria shares would be deemed more sustainable than if the fund held 2% in Tomra shares.

ESG fund ratings are becoming more and more popular, with several of the leading ESG data providers having launched their own rating. However, are investors adequately scrutinizing the methodology and the data on which these ratings are based?

The demand by investors to have ESG dimensions taken into account by their fund managers has been rising constantly for several years. The Covid-19 crisis provides additional impetus for this, witnessed by the positive investment flows into many ESG funds since the start of the year. At the same time the perceived complexity and lack of standards that characterizes sustainable investing has made ESG fund ratings more attractive. These ratings  derive from the same principles that made fund ratings so popular, serving as a tool that allows investors to quickly narrow their choice amongst sometimes hundreds of potential funds.

However, the parallel between ESG ratings and fund ratings is misleading. Fund ratings are usually based on a quantitative analysis of funds historical risk-adjusted returns. Whilst different methodologies can be used to calculate such risk-adjusted returns, , all methodologies tend to be anchored in portfolio management theory. The raw data used is usually the historical  security closing prices, as quoted by a regulated marketplace or reputable security pricing aggregator. This data can be considered unambiguous, objective, and standardized across thousands of securities.

ESG ratings on the other hand, rely on large amount of quantitative, as well as narrative disclosures by companies in order to be calculated. And when no data is disclosed, they often resort to estimations.

First, there is no global standard defining what should be included in a given company’s ESG rating. Each ESG rating provider makes its own decision, based on its assessment of how material it deems each topic to be for each sector. Take the case of Boohoo  for example. Two of the largest ESG rating agencies provided different risk assessments – one rating the company highly while the other rated the firm as medium risk, ahead of the company being exposed for poor working practices.  

There have been attempts at standardizing the list of ESG topics to consider, such as the materiality map developed by the Sustainable Accounting Standard Board (SASB) or the reporting standards created by GRI and others. However to this day, most ESG rating providers have been defining their own materiality matrix in order to calculate their scores.

Then, for each ESG topic comes the question of how to measure it. Let’s say that climate change is deemed relevant for a given company. How does one assess the company’s exposure to the complex topic of climate change? By measuring its greenhouse gas emissions? If so, should one also include emissions from the company’s raw material suppliers or those emissions resulting from the use of the company’s products (called Scope 3 emissions)?

And should a rating agency  also subtract from a company’s own greenhouse gas emissions the emissions that the company’s products have allowed another company to avoid emitting? For example, if a steel manufacturer provides the masts to allow a power utility to install wind turbines, does that company become sustainable even though steel production is one of the most carbon intensive activities in the world?

After addressing the question of what to include in an ESG rating and how to measure it, then comes the issue around data availability. Whilst attempts exist at making certain ESG disclosures mandatory, as well as defining a common format for disclosure, there is currently no global standard adopted by companies that require them to disclose specific data. The closest we get to such standards are initiatives such as SASB, TCFD and GRI, or CDP which collects data on companies’ greenhouse gas emissions globally. Despite these efforts, a recent report by the US Government Accountability Office deemed ESG disclosures to still lack in detail, consistency and comparability.

The SASB materiality map contains 26 possible ESG topics, with hundreds of underlying ESG datapoints. The breadth and depth of information on companies provided by such data explains why most investors agree with the need to integrate ESG. The debates centers around which information to consider and how it should be integrated into investment decisions. Each ESG fund rating represents just one attempt at answering these questions, amongst all the possible combinations of available or estimated ESG data.

Going back to our example of Tomra vs Altria, and as seen with Boohoo, if another leading ESG rating provider had been used, it would have yielded the exact opposite investment decision, with Tomra getting the better ESG rating. This discrepancy between ESG analyst opinions does not invalidate ESG ratings per se. It instead reiterates the need for investors to understand underlying ESG rating methodologies, including their strengths and limitations.

On top of raising questions regarding data consistency across providers, ESG fund ratings only capture part of a fund’s ESG credentials. Important aspects related to how the fund manager engages with the companies, its voting track record, or whether certain sectors are excluded, are typically not covered by ESG fund rating assessments.

These ratings are incomplete, or can even become misleading if they are construed by investors as a comprehensive picture of funds’ sustainability or ESG credentials, and it is crucial that investors  recognize these limitations..