A SCALABLE, LOW-COST FRAMEWORK FOR MULTILINGUAL INTELLIGENT DOCUMENT PROCESSING FOR CONTINUITY OF CARE

Abstract

Paper-based prescriptions and reports constitute a major part of the medical health records across the globe. Accordingly, paper-based manual data recording is a common practice in the Community Health Centers (CHCs) in India. These records result in poor handling, fragmented information, inefficient data retrieval, sharing, and storage of clinical data. To address this gap, we present Intelligent Document Processing Application (IDPA), a low-cost, scalable data digitization pipeline combining Optical Character Recognition (OCR) and Vision-Language Models (VLMs) for converting bilingual (Hindi-English), handwritten, and numerical medical records into structured digital formats. IDPA comprises a two-stage pipeline, where Stage 1 employs table cell segmentation using OpenCV and Stage 2 uses OCR extraction with PaliGemma VLM. As a proof-of-concept, the application was tested using a dataset of 150 patient records of the Indian population, which exhibited prevalent data input issues including overwritten texts, obscured columns, and the application of whiteners. PaliGemma, refined using over 650 labelled table cell images, attained 74% accuracy and a 13% Character Error Rate (CER), outperforming other open-source VLM models in extracting the medical records. The extracted data is organized into structured dataframes, served through a FastAPI endpoint, and accessible through a Progressive Web App (PWA). The interface supports secure user authentication via Clerk API and enables real-time image upload, editable tabular outputs, and data export in CSV/PDF formats. Together, these digital tools offer an affordable, user-centric approach to improve healthcare data management in low-resource settings. They hold strong potential for integration with national health systems, improvement of continuity of care, enablement of longitudinal monitoring, and expansion into predictive analytics for clinical decision support. © This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)