Red Hat and OS-Climate Data Commons: Helping financial services with data for informed decisions

In this post: 

• Effective data gathering and data processing is crucial for businesses looking to evaluate climate change risk and opportunity.

• Red Hat is helping build technology and data platforms needed to more fully integrate the impacts of climate change in global financial decision making and risk management.

Since joining OS-Climate (OS-C), Red Hat has been participating in an initiative to equip financial services institutions with the data they need to make informed decisions. At the 26th UN Climate Change Conference of the Parties (COP26), Red Hat speakers discussed ways financial institutions can more effectively use and share data to address climate change.

Climate finance was a central talking point among delegates at COP26. How can banks help address these goals of COP26? We’ll share some highlights of our conference presentation in this post.

Why effective data gathering and processing matters

In order to identify which industries and sectors to invest in, effective data gathering and data processing is crucial for businesses looking for a detailed understanding of policy and regulation, the emerging technologies which require investment, and an insight into market sentiment.

The problem is far from a lack of data. In fact, vast amounts of data are generated globally. However, because many financial institutions tend only to analyze in-house data, or data purchased from third-party vendors, data scientists and analysts are effectively working in silos. This data gap needs to be closed in order for financial service institutions (FSIs) to have the broadest possible view of investment opportunities and risk.

For climate-, and compliance-minded investors, data will need to be much more widely available, and more effectively analyzed. 

Creating a level playing field

OS-Climate (OS-C) is a Linux Foundation-backed open source project that intends to build technology and data platforms to integrate the impacts of climate change in global financial decision making and risk management.

As part of its membership, Red Hat has been providing technical acumen and engineering resources to help build the OS-C Data Commons, an open platform that aggregates a variety of structured and unstructured data into a single library of trusted data. (Not to be confused with the Data Commons project that aggregates public data for use by students, researchers and enthusiasts.)

Data Commons is focused on data availability, data comparability, and data reliability—and by building the platform collaboratively, the individual needs of contributing teams can be built in for use by all stakeholders. 

Data commons is not just another data platform. It is a collaborative project to adopt common data tooling, and to unite and federate all the existing data platforms together.

High-quality data from both public and proprietary sources can be collated, sanitized, and made available to enable developers to use this in open source analysis tools.

Why open source?

Open source principles promote transparency, participation, and collaboration. Red Hat has committed a team of architects, data engineers, and software engineers to bring our open source expertise in the upstream community to develop the platform.

If a group of data scientists are studying physical risk, for example, they collate geographic data and create a methodology for its analysis. Under open source principles, they then share that methodology with other groups working on similar projects who can now adopt it for their own work. 

As teams identify potential improvements to the platform, they share those ideas back to the group so all participants are still working to a single standard. This means that, as teams publish results or the data itself, other teams can be confident that the data analysis is being conducted through the same transparent processes they use themselves.

Red Hat has a long track record in the data science space, and has been working within the Open Data Hub initiative, a blueprint for running artificial intelligence and machine learning (AI/ML) platforms on Kubernetes. The potential scale of Data Commons provides an enterprise-grade use case for this next step in managing complex data ingestion. The project is also an opportunity for Red Hat to contribute upstream to the open source data science projects which are redefining how we ingest and process data.

Open source, governed data

A core aspect of the open source model is the confidence that the models are using the appropriate methodology because the source code is accessible and transparent. For many financial institutions however, the idea of sharing proprietary data could prove an obstacle to their participation.

Under Data Commons, the data isn't necessarily public or open in the way we think of open source code. Data providers can set, manage, and enforce compliance and security rules to dictate exactly which users will have access to their datasets. That could mean sharing some data to help build the model, other data with data scientists for environmental research, and more business-sensitive data with regulators only. 

The open, interoperable platform created and made available through Data Commons can also be used independently by financial institutions within their own workflows. FSIs know they have a transparent and trusted method to help meet new regulations or compliance requirements. They can also use it to build their own investment models to address climate change and employ it more generally in their own credit risk, market risk, and financial data. 

Tackling climate change together

Buy-in from the financial sector—and their technology and consulting partners—seems strong. Companies and FSIs including Amazon, Microsoft, Allianz, BNP Paribas, Goldman Sachs, and KPMG are already members of OS-Climate, and BNY Mellon, the world’s largest custodian bank and assets servicing company, announced it was joining at COP26. 

With the conversation frequently posed as a competition between the global economy and climate change, OS-Climate and the Data Commons initiative reframe the question as how the global economy will be a crucial tool in tackling climate change. 

Learn more about how we contribute to open source communities like OS-Climate.