Spotlight: Data Quality – Dimension 6, Consistency
An Interview Series with Clarity AI Executive Team on the 8 Dimensions of Data Quality
How does Clarity AI ensure its data is of the highest quality?
Clarity AI uses an 8-dimension framework to ensure data is of the highest quality. Those dimensions are coverage, freshness / timeliness, accuracy, data updates, explainability, consistency, point-in-time, and feedback. In this series of interviews with Clarity AI executives, each of these dimensions is explored and explained. Clarity AI’s expert team creates scientific- and evidence-based methodologies that then leverage powerful, scalable artificial intelligence (e.g., machine learning) to collect, clean, analyze and expand existing data sets to power its sustainability tech platform or to integrate directly into users’ existing workflows.
Dimension 6 – Consistency
Clarity AI’s VP of Product, Ángel Agudo, Head of Product Research & Innovation, Patricia Pina, Head of Data Strategy, Juan Diego Martin, and Head of Data Science, Ron Potok, discuss – with Chris Ciompi, Clarity AI’s Chief Marketing Officer – the critical dimension of consistency and its relationship to data quality.
Chris Ciompi: Hi, everyone. Welcome back for another discussion on data quality. Ángel, can you please define consistency as it relates to data quality?
Ángel Agudo: Consistency can be described from different angles, but for us, it is mainly about treating information in the same way over time and being able to provide visuals of how companies have been doing. It’s also related to the capability to compare, and one of the main challenges of data quality is to compare apples to apples. The accuracy of the definition of the metric should be exactly the same for every single company. The end goal is to be able to make comparisons, and to have consistency when you read the metric across all the companies. There’s also consistency in the way that we aggregate all the information and how we treat companies, subsidiaries, securities… So when you look at a portfolio, everything is going to be organized and aggregated in a consistent way across the different securities, companies, and funds.
Juan Diego Martín: Absolutely, I agree with Ángel, and from a client perspective, consistency is crucial because our customers are looking at the sustainability problem from different lenses. Some are looking at ESG, others are looking at impact, or using regulations. Consistency is essential to combine and aggregate different companies’ equities portfolios and extract comparisons, but customers also want to cross-check different perspectives. They need to understand portfolios, funds, and companies from different perspectives to have a better view, so they need all this consistency to be granted and provided.
Chris Ciompi: Okay, very good. Patricia, why is consistency important for consumers of sustainability data?
Patricia Pina: When we talk about consistency there are two different dimensions: first, consistency across sustainability-related analytics, and second, time consistency. Let’s first focus on the former, it refers to the same company having a metric value or a score that is different depending on the product. For example, the client would see a different value for the company’s carbon score in the climate product and in the ESG Risk score product. And there could be very good reasons for this discrepancy: different data update frequencies by product, different universes of coverage, by product that lead to different scores when calculating relative rankings and best-in-class scores, or different inheritance logics between parent and subsidiary companies depending on the use case. However, the first thing clients will notice is the inconsistency and they will very quickly lose trust in the data. Furthermore, clients are often mixing analytics from different products to build a comprehensive understanding of a company’s sustainability journey, inconsistencies across products makes it very difficult for them to know what the ground truth is and how they should think about that particular company.
With regards to time consistency, clients struggle with large unexpected changes between years; in this case it is about explainability and ensuring that those changes are justified in the context of a time series. In other words, ensuring that these large changes are due to actual changes of behavior and that they are not linked with the company changing the reporting boundaries or other methodological changes. The issue of companies restating their reported values from previous years is also another challenge because it results in the company having two different values for the same metric and year, which is another type of inconsistency. These instances need to be carefully assessed to understand what is driving the restatement and how it should be addressed based on the use case.
Chris Ciompi: Thank you, Patricia. Juan Diego, how does Clarity AI ensure that its data is consistent?
Juan Diego Martín: Our clients need to use the same data, and we need to make sure that we are using the same mapping and aggregating the information in a way that makes sense. The interpretation needs to be consistent because consistency is about providing a comprehensive view. They understand things better and are able to make better investment decisions.
Chris Ciompi: Okay, I understand. On “making better investment decisions”, this is exactly the thing they do…
Juan Diego Martín: Well, this is their magic.
Chris Ciompi: I think there is something here for us to emphasize. We’re talking about different dimensions around data quality, right? Consistency of data can help to shape their conviction. We’re not talking about making decisions for them. What do you think about that?
Juan Diego Martín: No, it’s trust, right? This conviction is also what has been mentioned previously about building trust. This is one additional element that makes them confident in regards to the Clarity AI tool and on the information therein, because they understand there is a logic behind it all.
Chris Ciompi: Okay, great. Ron, let’s talk about consistency and AI. How is consistency and Clarity AI influenced by artificial intelligence?
Ron Potok: Something we see very clearly in sustainability data is that it typically doesn’t change much year on year unless there’s an event that dramatically changes the course of the company. That’s one thing investors are mindful of. Companies don’t change very quickly, especially in manufacturing. So the expectation is that year on year, a worse performer will take a little while to evolve into the best in class. We have learned this pattern with artificial intelligence and apply it to ensure that our scores remain true to the nature of the business, which is often slow-changing. You can also see these patterns in social and government companies where board directors and employee turnover don’t change very often. We identify patterns and apply them to the data, and when we see exceptions, we take another look.
Chris Ciompi: So there’s a marriage of AI and human experts working together to identify patterns and apply them to the data. Is that correct?
Ron Potok: Yes, the AI identifies patterns, and we apply those patterns to the data. When we see exceptions, we take another look.
Chris Ciompi: Okay, good. Patricia, how does consistency help drive product innovation at Clarity AI?
Patricia Pina: The pursuit of consistency has made us create a platform that integrates multiple products all connected to a single source of truth. Such an architecture allows us to quickly propagate innovation across all of our products. For example, we are currently ingesting satellite data to estimate the activity level of the assets that a company owns, and this data will be available to our climate product as well as to our newly created biodiversity product. In climate, we will use it to estimate GHG data while in biodiversity, we will use it to quantify the share of a company’s operations that is exposed to nature-related risks.
Chris Ciompi: How does data consistency at Clarity AI influence the capabilities of the software app?
Ron Potok: One of our value propositions is that we provide all of your sustainability needs in one place, which has a significant benefit. We leverage the same base data for all sustainability metrics across different frameworks – like biodiversity impact metrics included in our dedicated biodiversity solution use the same data as the ones included in our SFDR solutions. This approach, coupled with consistent methodologies, guarantees a high level of overall consistency. This means that once you learn how we do things in one module, such as Net Zero, SFDR, or EU Taxonomy, it’s the same across all modules. This could be a valuable proposition, especially for small and medium-sized businesses. Instead of having to go to multiple places and worry about different coverages, inheritances, and other issues, they can rely on our platform for all their sustainability needs.
Ángel Agudo: Our platform provides traceability across all the different dimensions we mentioned, ensuring everything works consistently across them. Let me give you an example. When you upload a portfolio, you can see how it’s broken down into individual organizations or governments that populate the comprehensive view of the portfolio. This is spread across different lenses, and you can see that all those universes are replicated across the different modules. It’s important to keep traceability of the ones included or excluded within each of the different lenses to understand the displayed information. It’s also important to have good links between securities and organizations. Our software provides full transparency about how the different pieces are connected, and the definitions of metrics are consistent across modules. So you can trust the information you see.
Chris Ciompi: Thanks again, everyone. Great discussion!
