Structured OCR for Newspapers: Using Yolox and Vision LLMS

1. Introduction

Building a OCR Structured for Newspapers is not a simple task. Unlike books or documents, messy newspapers are often noisy, tilted, and low resolution.

Traditional OCR tools struggle with so complex layouts.

Newspapers also do not follow the standard layout. They use several columns, information, mixed fonts, and articles that might jump across pages.

Therefore, tools like Tesseract often return mixed and unstructured text. These tools read row after row – without understanding the context.

But what if you need structured data such as the title, writer, date, or page number? Raw text is not enough.

To solve this, we will combine Yoloks to detect the layout block with Vision llm For intelligent text extraction.

This Modern OCR Pipe Turns a Scanned Pages to Clean and Structured Json – Each block labeled and ordered correctly.

This blog guides you through building A OCR Structured for Newspapers using modern AI tools.

Let’s dive.

2. Project Review: Structured OCR for Newspapers

This project helps extract structured content from the scanned newspaper page. The system detects the Layout Block – such as the title, information, and body of the article – and then read the text using AI.

This is how it works:

1. Users upload newspaper images.
2. The system detects blocks such as Title, sub -head, textAnd information using Yolox.
3. Each block is sent to the OCR engine:
   • Easyocr For simpler content
   • Vision llm for solid or complex areas
4. Extracted texts are grouped and labeled.
5. Clean, structured Json The file is returned.

This JSON can be used for research, digital archiving, or databases that can be sought. This can be read the machine and easy to understand.

Main component

• Yoloks – For object detection and layout analysis
• Easyocr / Vision LLM – For flexible text extraction
• Python 3.10 – With .env for fire key management

This system can run locally or on a small server. GPU helps, but it is not entirely needed for testing.

3. Yolox training for OCR structured in newspapers

Before running the pipe, you must practice a special Yolox model that can detect the type of newspaper block.

3.1 Create a virtual environment

Use Python 3.10.13:

python3.10 -m venv .venv
source .venv/bin/activate  # macOS/Linux
# .venv\Scripts\activate    # Windows

3.2 pairs of dependencies

First, increase the PIP and install all the packages needed:

pip install --upgrade pip
pip install -r requirements.txt

3.3 Make a special dataset newspaper for OCR

Make sure your dataset is notated Coco format with relevant classes such as:

• title
• subheading
• textblock
• caption
• author
• page_number

The folder structure should look like this:

datasets/
├── train2017/
├── val2017/
└── annotations/
    ├── instances_train2017.json
    └── instances_val2017.json

3.4 Configuring Yolox Experiment

Create an experimental file at:

exps/example/custom/newspaper_yolox.py

Set training parameters such as the number of classes, dataset paths, and batch size:

self.num_classes = 6
self.data_dir = "datasets"
self.train_ann = "annotations/instances_train2017.json"

3.5 Start training

Run this command to start training:

python tools/train.py -expn newspaper_yolox -d 1 -b 8 --fp16

• -expn: Your experimental name
• -d: Number of GPUs
• -b: Batch size
• --fp16: Activate the mixture precision (faster in the GPU)

3.6 Save the best model

After the training is complete, use the best inspection posts found at:

YOLOX_outputs/newspaper_yolox/best_ckpt.pth

4. How is the OCR structured for the newspaper to work

Let’s describe the complete pipe, from the layout detection to the structured output.

4.1 Detecting the Layout Block with Yolox

First, the image is passed through the trained Yolox model. This detects various layout components such as:

• Title and subpos
• Body text block
• Information and writer
• Illustration and page number

For each block, Yolox returns the box, label, and confidence score. These boxes are then cut to isolate the individual area.

4.2 Selecting the Right OCR Machine

Furthermore, each block that is cut is forwarded to the OCR engine. Based on the type and size of the block, we chose:

• Easyocr: Fast and accurate for clean text
• All llm: Stronger for noisy blocks, wrapped, or stylish

This decision can be made automatically using simple logic in your code.

4.3 Prompt engineering for better OCR output

To get maximum results from the vision language model, use special instructions for each type of block.

For example:

“The complete title extract of this picture. Don’t include the text or name of the author.”

This request helps LLM focus on what’s important. You can adjust the instructions functions.py For each type of content.

4.4 Arrange output

After the text is extracted, we group and label each block. This step includes:

• Sort the block from top to bottom and left-to-right
• Text that matches the illustration
• Connecting the author with the closest title

Finally, we made Json structured:

{
  "title": "New Discovery in AI",
  "author": "Jane Doe",
  "text": "Researchers at XYZ University...",
  "caption": "Illustration of the AI model."
}

With Yolox and LLM vision, you can finally make a reliable thing OCR Structured for Newspapers It provides a clean and labeled output.

5. Challenges in building structured OCR for newspapers

Building this system is not easy. Here are some of the real challenges that we face – and how we finish it.

5.1 Complex Layout

Newspapers do not follow the rules. Articles wrap advertisements. The title sitting next to an unrelated picture. To train Yolox well, we need many diverse examples.

Main Lesson: Annotation of various layouts and fonts to get consistent results.

5.2 OCR struggle with noisy scanning

Low -quality scanning is a real problem. Blurred text and ink stains confusing Easyocr.

Switch to Vision LLM for key blocks (such as titles or text) significantly increase results – but additional costs and latency.

5.3 Speed ​​and accuracy balanced

Vision is accurate, but slow and expensive. So, we add a switch to choose between Easyocr (FAST) and Vision LLM (accurate) based on the case of use.

In this way, users can balance performance and quality.

5.4 Dataset Annotation

The label block layout manually requires time – but that is important. We use tools like Studio label to speed up annotation.

In the future, the pre-training layout model can help reduce this workload.

5.5 suitable related areas

It is not always easy to connect the writer to their articles or information with illustrations. We use the rule of closeness to group the nearest blocks, but it’s not perfect.

Potential improvement can be used Layout graph or Document Parsing Model.

6. Conclusion

OCR for the newspaper is difficult – but it is not impossible. The standard tool will not cut it. You need awareness of layout, intelligent extraction, and structured output.

With training Yoloks In a special newspaper class, we detect meaningful regions such as the title, information, and writer. With Easyocr And Vision llmWe extract clean text – even from difficult scanning.

The final result? Structured and labeled JSON is ready for indexing, research, or digital archives.

Whether you digitize the archive or automate the editorial task, this OCR Structured for Newspapers Strong pipeline, can be discharged, and open source.

Thank you for reading! Try pipes, increase, and share the results. We want to see what you are building.