Korean OCR model: fine-tuning, evaluation and release
I fine-tuned LightOnOCR, a 1-billion-parameter vision-language OCR model (first version 2-1B, then 3-1B with DPO), on Korean business reports and released it into a client's live document pipeline for a side-by-side test to replace HunyuanOCR, the model the pipeline used.
Demo and evidence
There is no public demo of this model, because it was trained on client documents. The client report and the model card are private.
Optional: a clean-room demo on Hugging Face Spaces (the public LightOnOCR-2-1B base on public or synthetic Korean pages, with a loop detector).
The problem
- The pipeline: the client turns uploaded PDFs into text for search and question answering.
- The old OCR step: HunyuanOCR made frequent character and digit errors on the client's Korean reports and took about 10 seconds per page on the live server.
- Hard pages: dense tables with empty cells, checkboxes, stamps, very small text, sideways scans and embedded photos.
- No fallback: the new model had to work alone, without the old model behind it.
- Shared hardware: it had to share GPUs with other live LLMs.
- Production settings: it had to hold up under the exact production request: 200 DPI JPEG pages, greedy decoding and a 4,096-token limit.
The system

What I did
- Training data. 4,555 training rows: 681 pages from 18 client documents, 1,443 pages of public Korean law PDFs, 785 public Korean and English pages, and 1,646 public Korean form and text-image samples from AI Hub. About a quarter are crops (names and regions) rather than full pages. The reference transcriptions were made with Gemma 4 31B, not by hand. Test documents were held out as whole documents, and 206 near-duplicates of test pages were removed from training.
- Supervised fine-tuning (SFT). LoRA on LightOnOCR-2-1B-base (rank 64, rsLoRA, about 77M trainable parameters, 2 epochs), each run on one RTX PRO 6000 GPU in 3.6 to 6.6 hours. Three runs (two learning rates, one with rotated pages added) were averaged into a model soup and merged into the base weights, so serving needs no adapter.
- LightOnOCR-3-1B. The same recipe on the newer base. Its first pick on the 195-page development split lost to the 2-1B model on the 223-page test split (7.93% against 6.40%); the 3-1B model soup, trained further with DPO, became the best model (results below).
- Preference and RL training. DPO on the 3-1B model soup with 1,409 preference pairs, the reference text against a looping version of it (LoRA rank 16, β 0.1, 71 minutes on one GPU). Against its starting point it halved digit errors (12.35% to 6.01%) and looping pages (6 to 3), and on 11 Nov it was released as a fourth OCR model for the client to test. I also tried GRPO with rule-based rewards: after four rounds no round beat its starting point, so DPO is the one carried forward.
- Evaluation. 418 pages from 14 held-out documents, scored by Korean character (jamo-level) and digit error rates with paired, document-level bootstrap intervals, all through the production engine and settings. The October model was picked among ten candidates on these pages (they tied within noise). From 31 Oct, models were picked on a development split (195 pages) and confirmed on a separate test split (223 pages).
- Serving and release. vLLM in Docker on a shared GPU, with Grafana monitoring and a stopping rule for looping output calibrated on 5,219 reference texts. A new gateway version serves three OCR models through one OpenAI-compatible API. The release had canary checks, a scripted rollback and about five minutes of downtime, and a replay showed the two existing models' outputs unchanged (12 of 12). The gateway changes were merged into the client's repository.
- Layout boxes. The client also tests bounding boxes, and fine-tuning for text had cut their quality. A second LoRA stage on human-labelled layouts (DocLayNet and AI Hub layout data), with text examples mixed in, brought box F1 back above the untuned base without hurting text accuracy (results below). Late in training a few pages started looping again; averaging two checkpoints removed it. A further stage for line-level boxes learned real lines but began answering block requests in line form, so it was not used. Trained and evaluated in Nov 2026, not deployed yet.
Part of the training data comes from AI Hub (aihub.or.kr). 이 연구는 한국지능정보사회진흥원의 사업결과입니다.
Results
| Measure (lower is better) | HunyuanOCR (in use) | LightOnOCR-2-1B, SFT | LightOnOCR-3-1B, SFT + DPO | 3-1B vs HunyuanOCR |
|---|---|---|---|---|
| Korean characters, all pages (418) | 10.22% | 6.45% | 5.49% | −46% |
| Digits, all pages (418) | 20.4% | 7.1% | 6.01% | −71% |
| Looping pages, of 418 | 7 2,048-token limit | 8 | 3 | |
| Korean characters, normal pages (381) | 7.15% | 4.14% | not measured | |
| Digits, normal pages (381) | 14.5% | 3.9% | not measured | |
| Mean time per page | 9.9 s | 2.0 s | not measured |
Layout boxes
| Box F1 (higher is better) | HunyuanOCR | Untuned base | After text fine-tuning (DPO) | After the box stage |
|---|---|---|---|---|
| LightOnOCR-bbox-bench, F1 at IoU 0.75 (855 pages) | 0.714 | 0.513 | 0.399 | 0.597 |
| OmniDocBench, F1 at IoU 0.5 (1,620 pages) | 0.580 | 0.515 | 0.325 | 0.557 |
Public box benchmarks, Nov 2026. The box stage beats the untuned base (bbox-bench +0.084, 95% interval +0.059 to +0.111) and keeps text accuracy (Korean character error 5.58%, the same as its starting point). HunyuanOCR still leads overall, especially on titles, figures and tight boxes; on text blocks the box stage is ahead (OmniDocBench text F1 0.482 against 0.461).
- Uncertainty: against HunyuanOCR, the 2-1B model's gain is clear: −3.8 points on Korean characters (95% interval −8.3 to −0.7) and −10.5 points on digits on normal pages (−17.8 to −5.2). The 3-1B model's further gain over the 2-1B model is −0.95 points (−1.86 to +0.10), within noise for Korean characters; its clearer gain is on looping pages, 8 down to 3.
- Speed (2-1B model): 2.0 s per page against 9.9 s, about 1.5 pages per second with the gateway's four parallel jobs. HunyuanOCR was timed on the live server under load and ours on a dedicated test server, so the 5× is approximate.
- Loop guard: the stopping rule plus a re-read at a slightly higher temperature lowered the error further, to 6.00% (digits 5.87%), with no document worse.
- English: not better than HunyuanOCR (1.49% against 1.38% on 18 English pages), so I claim gains for Korean only.
- Leakage check: 48 test pages shared boilerplate with training documents. Without them the gap grows to 4.7 points.
Example: a looping output, and the released model on a public form
Model output (looping)
| 항목 | 내용 | 비고 | | 1 | 현장 확인 | | | 2 | | | | 3 | | | | 4 | | | | 5 | | | … continues to the 4,096-token limit
After the stopping rule
| 항목 | 내용 | 비고 | | 1 | 현장 확인 | | | 2 | | | [cut: near-repeated rows] → page is re-read or kept with the cut
Synthetic illustration, not a client page.
On a public form
A blank government form from law.go.kr (주민등록법 시행규칙, 별지 제7호서식), rendered at 200 DPI and sent to the released model with the production decoding settings. The output is shown as it came back.
Model output for the same part
■ 주민등록법 시행규칙 [별지 제7호서식] <개정 2024. 12. 본인인 경우 정부24(www.gov.kr) 에서도 20.> 신청할 수 있습니다. 주민등록표 열람 또는 등 · 초본 교부 신청서 ※ 3쪽의 유의 사항을 읽고 작성하시기 바라며, 해당하는 내용 양의 [ ]에 √표를 합니다. (3쪽 중 1쪽) 성명 주민등록번호 (서명 또는 인) 주소 (시 · 도) (시 · 군 · 구) 신청인 ※ 시 · 도, 시 · 군 · 구까지만 작성 (상세 주소는 작성하지 않아도 됩니다) (개인) 대상자와의 관계 연락처 [ ] 수수료 면제를 신청함 ※ 수수료 면제 대상에 대해서는 3쪽 유의사항 3번을 참고하시기 바랍니다.
One wrong character in this part, marked: 앞 read as 양. The whole page has one more (중질지 read as 종질지 in the footer). On page 2, which is denser, four syllables were misread (고엽제 → 고업제, 참전 → 첨전, 임무 → 입무, 전자정부법 → 전자정보법) and two commas were dropped, out of about 1,240 characters. The labels in the left column, 신청인 and (개인), come out between the rows they sit beside. A clean, typed form is an easy case compared with the client's pages.
