← All projects

Korean OCR model: fine-tuning, evaluation and release

I fine-tuned LightOnOCR, a 1-billion-parameter vision-language OCR model (first version 2-1B, then 3-1B with DPO), on Korean business reports and released it into a client's live document pipeline for a side-by-side test to replace HunyuanOCR, the model the pipeline used.

Where
Enaible – Too Much AI (TMAI), Seoul · Aug–Nov 2026
My role
LLM engineer: training data, fine-tuning, evaluation design, serving and release of the OCR model
Team
8 people
Stack
SFT with LoRA, model soup, DPO, GRPO · PyTorch, Hugging Face Transformers and PEFT, vLLM, Docker, systemd, Prometheus, Loki, Grafana, Weights & Biases
Result
LightOnOCR-3-1B after SFT and DPO: 46% fewer Korean character errors and 71% fewer digit errors than HunyuanOCR on 418 held-out pages (5.49% against 10.22%; 6.01% against 20.4%)
Status
Released next to HunyuanOCR: the 2-1B model on 19 Oct 2026, the 3-1B model on 5 Nov and its DPO version on 11 Nov. The client is testing them side by side; the live pipeline still calls HunyuanOCR (Nov 2026).

Demo and evidence

There is no public demo of this model, because it was trained on client documents. The client report and the model card are private.

Optional: a clean-room demo on Hugging Face Spaces (the public LightOnOCR-2-1B base on public or synthetic Korean pages, with a loop detector).

The problem

The system

OCR serving path: the client document pipeline (upload, ingress, chunking, RAG index) sends page images to the OCR service (gateway and the fine-tuned LightOnOCR-2-1B on vLLM), with Prometheus, Loki and Grafana monitoring.
Where the model sits in the document pipeline. The gateway serves several OCR models behind one API, turns sideways pages upright and stops looping outputs.

What I did

Part of the training data comes from AI Hub (aihub.or.kr). 이 연구는 한국지능정보사회진흥원의 사업결과입니다.

Results

Error rates on 418 held-out pages: HunyuanOCR, LightOnOCR-2-1B and LightOnOCR-3-1BKorean character error: 10.22% (HunyuanOCR), 6.45% (LightOnOCR-2-1B, SFT), 5.49% (LightOnOCR-3-1B, SFT and DPO); digit error: 20.4%, 7.1% and 6.01%.HunyuanOCR (in use)LightOnOCR-2-1B, SFT (19 Oct)LightOnOCR-3-1B, SFT + DPO (best)0%5%10%15%20%25%Korean charactersall pages (418)HunyuanOCR (in use): 10.22%10.22%LightOnOCR-2-1B, SFT (19 Oct): 6.45%6.45%LightOnOCR-3-1B, SFT + DPO (best): 5.49%5.49%Digitsall pages (418)HunyuanOCR (in use): 20.4%20.4%LightOnOCR-2-1B, SFT (19 Oct): 7.1%7.1%LightOnOCR-3-1B, SFT + DPO (best): 6.01%6.01%
Error rates on 418 pages from 14 held-out client documents, with the production engine and settings; lower is better. HunyuanOCR and the 2-1B model were scored on 18 Oct, and the 3-1B model on 5 Nov under the same settings, where the 2-1B model scored exactly as before.
Measure (lower is better)Hunyuan­OCR (in use)LightOnOCR-2-1B, SFTLightOnOCR-3-1B, SFT + DPO3-1B vs Hunyuan­OCR
Korean characters, all pages (418)10.22%6.45%5.49%−46%
Digits, all pages (418)20.4%7.1%6.01%−71%
Looping pages, of 4187
2,048-token limit
83
Korean characters, normal pages (381)7.15%4.14%not measured
Digits, normal pages (381)14.5%3.9%not measured
Mean time per page9.9 s2.0 snot measured

Layout boxes

Box F1 (higher is better)Hunyuan­OCRUntuned baseAfter text fine-tuning (DPO)After the box stage
LightOnOCR-bbox-bench, F1 at IoU 0.75 (855 pages)0.7140.5130.3990.597
OmniDocBench, F1 at IoU 0.5 (1,620 pages)0.5800.5150.3250.557

Public box benchmarks, Nov 2026. The box stage beats the untuned base (bbox-bench +0.084, 95% interval +0.059 to +0.111) and keeps text accuracy (Korean character error 5.58%, the same as its starting point). HunyuanOCR still leads overall, especially on titles, figures and tight boxes; on text blocks the box stage is ahead (OmniDocBench text F1 0.482 against 0.461).

Example: a looping output, and the released model on a public form

Model output (looping)

| 항목 | 내용 | 비고 |
| 1 | 현장 확인 |  |
| 2 |  |  |
| 3 |  |  |
| 4 |  |  |
| 5 |  |  |
…  continues to the
   4,096-token limit

After the stopping rule

| 항목 | 내용 | 비고 |
| 1 | 현장 확인 |  |
| 2 |  |  |

[cut: near-repeated rows]
→ page is re-read or
  kept with the cut

Synthetic illustration, not a client page.

On a public form

A blank government form from law.go.kr (주민등록법 시행규칙, 별지 제7호서식), rendered at 200 DPI and sent to the released model with the production decoding settings. The output is shown as it came back.

Page 1, top half

Top half of a blank Korean government form, 주민등록표 열람 또는 등·초본 교부 신청서, with fields for name, resident registration number, address and contact details.

Model output for the same part

■ 주민등록법 시행규칙 [별지 제7호서식] <개정 2024. 12. 본인인 경우 정부24(www.gov.kr) 에서도
20.> 신청할 수 있습니다.

주민등록표 열람 또는 등 · 초본 교부 신청서

※ 3쪽의 유의 사항을 읽고 작성하시기 바라며, 해당하는 내용 양의 [ ]에 √표를 합니다. (3쪽 중 1쪽)
성명 주민등록번호
(서명 또는 인)

주소 (시 · 도) (시 · 군 · 구)
신청인
※ 시 · 도, 시 · 군 · 구까지만 작성 (상세 주소는 작성하지 않아도 됩니다)
(개인)
대상자와의 관계 연락처

[ ] 수수료 면제를 신청함
※ 수수료 면제 대상에 대해서는 3쪽 유의사항 3번을 참고하시기 바랍니다.

One wrong character in this part, marked: 앞 read as 양. The whole page has one more (중질지 read as 종질지 in the footer). On page 2, which is denser, four syllables were misread (고엽제 → 고업제, 참전 → 첨전, 임무 → 입무, 전자정부법 → 전자정보법) and two commas were dropped, out of about 1,240 characters. The labels in the left column, 신청인 and (개인), come out between the rows they sit beside. A clean, typed form is an easy case compared with the client's pages.

← All projects Next: Korean voice phishing detection →