Ed. note: Advisory Board member Tamara Close provides this timely analysis as the EU and IOSCO announced they intend to regulate ESG data providers. This is a two part piece, with the second coming tomorrow that includes suggestions for investors on ways to manage the differences.
The ESG data market is booming. Depending on which source you look at, there are currently between 125[1] and 600[2] providers of ESG data, ratings, and rankings. This is not surprising given that investors are spending millions on ESG data. In 2019, users spent over $600 million[3] and according to some sources, this should reach $1 billion for 2021.
ESG data providers are not created equal and, as opposed to credit rating agencies, are currently unregulated. They tend to have their own proprietary rating and ranking methodologies. While there is overlap between providers, they can use different sources to gather underlying raw data for the same ESG issues. Some providers make their methodologies transparent, but most do not.
Differences in methodologies and data sourcing have caused widely discussed differences in ratings between providers and the correlations between ESG data providers can be very low. In an often-cited paper[4], these correlations average around 0.60 which is significantly lower than the correlations between credit rating agencies, such as Moody’s and S&P, which tend to remain close to 0.99[5].
Given differences in ESG ratings, investors would be better-off thinking of them as ESG data “opinions”. Similar to sell-side research opinions which can vary from firm to firm, ESG data providers should serve as an input to the investment process rather than a determination of the underlying issuers’ ESG quality.
In an upcoming academic paper, “Inconsistencies in methodologies of ESG data providers and green bond standards” authors Kim Schumacher and Tamara Close identify 11 major inconsistencies between ESG data providers based both on the type of data provider and their underlying methodologies. These include differences in:
- definitions of materiality,
- normalisation techniques,
- aggregation and weighting,
- survivorship bias and missing data,
- use of standards and metrics,
- creation of benchmarking and peer groups,
- sources and timing of data collection, and
- conduct vs product-based scoring methodologies.
The authors also point out how these inconsistencies can have a material impact on an investment portfolio. This includes potential cherry picking of ESG ratings, as well as factor, geography, and size biases.
Holistic and in-depth analysis required
Using ESG data in an investment process can strengthen the process and provide a wider lens as it encompasses an assessment of intangibles of an investment. However, to ensure credibility and integrity, ESG data needs to follow the same data management processes as other material investment data. This includes validation and data quality checks.
The quality of an ESG rating also depends on the comparability of its source data and methods used to analyse this data. For instance, data from developing countries, or countries with diverging regulatory standards, can be tainted by gaps or bias. Therefore, companies from the same sector may be assessed in “very different ways as the context in which the underlying data was produced was highly divergent.”[6]
In addition, simply taking these ratings/data at face value without a holistic view will potentially create unintended sustainability exposures in a portfolio and can actually create the opposite effects of what an investor originally intended.
For instance, if a company has a best practice diversity & inclusion, employee health and safety, or other type of ESG policy, but does not make the policy publicly available, certain ESG ratings providers will give that company a low rating (or the industry average). At the other extreme, companies that publish well written, comprehensive policies receive higher ratings even though these policies may not be followed at an operational level. Simply having a policy for an ESG issue does not mean the issue is being properly managed at a company[7].
Some ESG data providers can also be sector-neutral, meaning that companies even in sectors with significant ESG risks (such as the oil and gas sector) can still score high on ESG metrics. A high ESG rating therefore does not necessarily mean a company is more sustainable or takes “better care of the environment or society”[8]. Hence the peer group or benchmark that is used to determine the ESG score or metric becomes of paramount importance.
[1] Scm Direct. 2019. “Greenwashing: Misclassification and Mis-selling of Ethical Investments.”
[2] SustainAbility. 2020. “Rate the Raters 2020: Investor Survey and Interview Results”
[3] Opimas “Spending on ESG data could hit $1B in 2021” https://www.spglobal.com/marketintelligence/en/news-insights/latest-news-headlines/spending-on-esg-data-could-hit-1b-in-2021-8211-opimas-57525642
[4] Berg, Florian and Kölbel, Julian and Rigobon, Roberto, Aggregate Confusion: The Divergence of ESG Ratings (May 17, 2020). Available at SSRN: https://ssrn.com/abstract=3438533 or http://dx.doi.org/10.2139/ssrn.3438533
[5] https://www.bnnbloomberg.ca/conflicting-esg-ratings-are-confusing-sustainable-investors-1.1360887
[6] Source: Kim Schumacher, School of Environment and Society, Tokyo Institute of Technology, School of Geography and the Environment, University of Oxford, (schumacher.k.aa@m.titech.ac.jp)
[7] The Alliance for Corporate Transparency’s 2019 report assessed 1,000 European companies, and concluded that companies tend to report on policies and not on specific ESG data, citing that there is also a lack of ESG metrics and targets. It is difficult to determine clearly how a firm is performing if only policies are published, without specific metrics, targets and reporting on the implementation of those targets; https://allianceforcorporatetransparency.org/
[8] Environmental Finance. 2020. “Pitfalls in ESG ratings requires investor caution”
