Abonnez-vous au flux

The extraction, transformation and loading (ETL) of data is one of the most common processes used in enterprise organizations to deal with large amounts of data. It is a very effective method for preparing batch data for analysis, often requiring days from data capture to business insights. However, modern digital experiences delivered by enterprise organizations today put ETL and batch processing at risk, since it fails to deliver actionable results in minutes. 

Newer technologies like Apache Kafka appear to be great solutions to support and even replace ETL and batch processing. Apache Kafka has risen to become the preferred and proven open source technology for streaming data between sources and for processing data in minutes. Apache Kafka has been designed for capturing, ingesting and streaming large amounts of data with low overhead, providing the capability to deliver intelligent data in near real time. This eliminates the need for batch processing, large data storage and delays on message delivery. 

The value of Apache Kafka

Apache Kafka is a distributed streams processing platform that uses the publish/subscribe method to move data between microservices, cloud-native or traditional applications and other systems. This technology differentiates itself from others due to its ability to send and receive messages at a very fast rate, horizontally scale as the number of requests increases and retain the data even after messages have been received. All these benefits allow using the technology in innovative use cases that couldn’t be solved using traditional messaging or processing solutions.

Replace batch data with real-time processing

Batch data processing is the traditional method for managing data collection, processing and analysis. This method has been very successful for scenarios where time was not an issue, such as bank reports, billing and order fulfillment, but customers today expect faster response times.

Digital experiences have changed the way organizations ingest and process data, as the flow of data is now continuous and data-driven decisions need to be made in hours rather than days. Batch data processing prevents you from responding to changes in real time, as access to data is delayed by collection and analysis. Closing the gap between data processing and decision making or actionable insights is the most important part. 

Apache Kafka can support modernization of many traditional use cases that rely on batch processing, since it eliminates delays of data processing and delivers higher performance and better customer experiences. It is the data streaming technology of choice for businesses that depend on the ability to process data in real time and deliver a high-quality digital experience to customers. 

For example, businesses that require detecting patterns and gaining insights in near real time could use this technology to extract, store and transform data at a fast pace. Apache Kafka can support the analysis of continuous streams of data, eliminating the need for aggregating batches of historical data. This could lead to a cost-saving scenario since any data that is not needed can be deleted rather than stored in large data warehouses. Apache Kafka can support organizations that are looking into analyzing data-in-motion.

The Red Hat approach

Red Hat looks at messaging as an essential part of the application development process and as such provides a variety of solutions to support messaging communication between applications across hybrid cloud environments. Apache Kafka is one of the key technologies we are focusing on in this area. 

Red Hat OpenShift Streams for Apache Kafka is a fully managed cloud service for IT development teams that want to incorporate streaming data into applications to deliver real-time experiences. The service allows developers to focus on streaming data rather than on maintaining or configuring the infrastructure. OpenShift Streams for Apache Kafka, together with open source technologies like Kafka Streams and other Red Hat components like Red Hat OpenShift Connectors, support developers on ingesting, aggregating and transforming data for analysis. 

We invite you to check out our webinar series “Understanding Kafka in the enterprise”. This blog is part of a series that offers technical solutions to commonly known use cases, such as how intelligent applications can benefit from Apache Kafka,  streamlining application modernization and managing event-driven architectures.

For more information, visit the Red Hat OpenShift Streams for Apache Kafka page to learn more.

À propos de l'auteur

Jennifer Vargas is a marketer — with previous experience in consulting and sales — who enjoys solving business and technical challenges that seem disconnected at first. In the last five years, she has been working in Red Hat as a product marketing manager supporting the launch of a new set of cloud services. Her areas of expertise are AI/ML, IoT, Integration and Mobile Solutions.

Read full bio

Parcourir par canal

automation icon


Les dernières nouveautés en matière d'automatisation informatique pour les technologies, les équipes et les environnements

AI icon

Intelligence artificielle

Actualité sur les plateformes qui permettent aux clients d'exécuter des charges de travail d'IA sur tout type d'environnement

open hybrid cloud icon

Cloud hybride ouvert

Découvrez comment créer un avenir flexible grâce au cloud hybride

security icon


Les dernières actualités sur la façon dont nous réduisons les risques dans tous les environnements et technologies

edge icon

Edge computing

Actualité sur les plateformes qui simplifient les opérations en périphérie

Infrastructure icon


Les dernières nouveautés sur la plateforme Linux d'entreprise leader au monde

application development icon


À l’intérieur de nos solutions aux défis d’application les plus difficiles

Original series icon

Programmes originaux

Histoires passionnantes de créateurs et de leaders de technologies d'entreprise