RHEL AI 1.3 Docling context aware chunking: What you need to know

With the release of RHEL AI 1.3, we’re excited to introduce context aware chunking powered by the Docling project, a significant enhancement that expands the capabilities of taxonomy contributions, pushes the limits of synthetic data generation and offers enhanced document support (pdf and md).  Through our collaboration with IBM Research and adoption of Docling, RHEL AI 1.3 now features a new data ingestion pipeline.

This update enables seamless integration of PDF documents, marking a shift from the previous Markdown-only support in qna.yaml and a new chunking strategy for documents, context aware chunking which allows better representation of different document elements.

Fig 1: Docling Integration in RHEL AI brings improved chunking strategy and enhanced document support

Fig 2: Zoomed in view of data ingestion pipeline using Docling

What’s new?

PDF support

Contributors can now reference PDF documents directly in taxonomy submissions alongside Markdown files. This update eliminates the need to manually convert PDFs to Markdown, streamlining the contribution process.

With the support of pdf documents, end users can bring their personal/enterprise documents directly for model customization for their use cases. In RHEL AI 1.4, more document types such as word, pptx, docx, html will be supported allowing users to support a broad range of use cases.

Docling adoption/context aware chunking

With the adoption of Docling, the #1 open source document paper on GitHub, we are introducing a new context aware chunking capability. It intelligently recognizes and processes different document elements – from text and tables to figures, lists and columns. This means more accurate extraction and better understanding of your documents' structure and meaning. This is an improvement over naive chunking  being used in RHEL AI 1.2.

We've also enhanced our synthetic data generation (SDG) pipeline to leverage these new capabilities. We are continuing our collaboration with IBM Research to push the boundaries of context-aware document processing even further in future releases.

Docling parses PDFs and converts them into structured, context-aware chunks. The tool accurately represents critical semantic elements, including text, tables and images and enhances contextual understanding for better synthetic data generation.

Why this matters

The addition of PDF support overcomes the limitations of Markdown-only workflows, enabling contributors to include richer, more detailed documents in their submissions. Docling’s robust chunking capabilities ensure that PDFs are no longer a barrier to streamlined knowledge integration, making taxonomy contributions faster, easier, and more effective.

Naive chunking strategies often result in poor outputs for synthetic data generation and thereby finetuning of language models. Context aware chunking can result in reduced hallucinations involving complex document structures. This can facilitate seamless integration across various departments within an organization, each handling complex document representations.  Another capability we are working on is hierarchical context aware chunking that captures additional meta data such as headings/captions for better context.