AI-based handwriting recognition

Challenge

The Städel Museum is considered the oldest and most prestigious museum foundation in Germany. To fulfill its educational mission beyond physical boundaries, it offers various digital art mediation formats. The centerpiece is the Digital Collection, whose curation and continuous development require extensive efforts. One challenge lies in the difficult readability of handwriting in sketchbooks and on hand drawings, which previously had to be manually recorded, slowing down the digitization process.

Approach

As AI experts, we are familiar with the various possibilities that artificial intelligence offers for digitizing content and processes. Therefore, together with the Städel Museum, we developed a solution that can reliably recognize even difficult-to-read handwriting automatically. By utilizing the Google Vision API, we could deliver convincing results quickly with minimal development effort:

• Recognition of handwriting in hundreds of images in less than 30 seconds
• OCR with subsequent post-processing of the texts
• Localization of texts including bounding boxes

To optimally integrate the developed solution into the workflow of the Digital Collection department, a scalable processing pipeline was implemented in the Google Cloud Platform. Thanks to the "serverless" architecture used, the solution scales cost-effectively with usage.

Result

The developed AI-based solution supports human work by localizing handwriting in sketchbooks and translating it into machine-readable text using OCR. This provides the experts at the Städel Museum with precise support in their work, allowing sketchbooks and hand drawings to be enriched with transcriptions much faster in the future. The transcriptions enable, for example, machine searches for keywords.