| --- |
| language: en |
| license: mit |
| tags: |
| - chess |
| - ocr |
| - handwritten-text-recognition |
| - trocr |
| - transformers |
| - computer-vision |
| - image-to-text |
| datasets: |
| - handwritten-chess-notation |
| metrics: |
| - accuracy |
| - cer |
| widget: |
| - src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/chess_example.png |
| example_title: Chess Move Example |
| pipeline_tag: image-to-text |
| --- |
| |
| # Handwritten Chess Notation Recognition with TrOCR |
|
|
| ## Model Description |
|
|
| This is a fine-tuned version of Microsoft's TrOCR-Large model, specifically trained to recognize handwritten chess notation from images of chess scoresheets. The model can accurately transcribe chess moves in Standard Algebraic Notation (SAN) from handwritten images. |
|
|
| **Model Type:** Vision Encoder-Decoder |
| **Architecture:** TrOCR (Transformer-based Optical Character Recognition) |
| **Base Model:** `microsoft/trocr-large-handwritten` |
|
|
| ## Intended Uses & Limitations |
|
|
| ### Intended Use |
| - Transcription of handwritten chess moves from scoresheet images |
| - Digitization of historical chess games |
| - Chess notation recognition in mobile apps |
| - Educational tools for chess analysis |
|
|
| ### Limitations |
| - Works best with clear handwriting |
| - Trained specifically on chess notation (not general text) |
| - May struggle with extremely cursive handwriting |
| - Requires well-lit, focused images |
|
|
| ## Training Data |
|
|
| The model was trained on a custom dataset of 13,731 handwritten chess move images with corresponding text labels in Standard Algebraic Notation. |
|
|
| **Dataset Characteristics:** |
| - Format: PNG images with move text annotations |
| - Content: Chess moves in SAN format (e.g., "e4", "Nf3", "O-O", "Qxf7#") |
| - Split: 88% training, 12% validation |
| - Handwriting styles: Multiple variations |
|
|
| ## Training Procedure |
|
|
| ### Preprocessing |
| Images were resized to 384x384 pixels and converted to RGB format. Text was tokenized with chess-specific vocabulary and padded/truncated to a maximum length of 16 tokens. |
|
|
| ### Training Hyperparameters |
| - **Epochs:** 6 |
| - **Batch Size:** 1 (with gradient accumulation of 8) |
| - **Learning Rate:** 3e-5 |
| - **Optimizer:** AdamW |
| - **Weight Decay:** 0.01 |
| - **Warmup Steps:** 200 |
| - **Mixed Precision:** FP16 |
|
|
| ### Hardware |
| - **GPU:** NVIDIA (8+ GB VRAM recommended) |
| - **Training Time:** ~6 hours |
|
|
| ## Evaluation Results |
|
|
| | Metric | Value | |
| |--------|-------| |
| | Accuracy | ~92% | |
| | Character Error Rate (CER) | ~3% | |
| | Inference Speed | ~100 ms/image | |
|
|
| ## How to Use |
|
|
| ### Direct Inference |
| ```python |
| from transformers import TrOCRProcessor, VisionEncoderDecoderModel |
| from PIL import Image |
| import requests |
| |
| # Load model and processor |
| model = VisionEncoderDecoderModel.from_pretrained("username/trocr-chess-handwritten") |
| processor = TrOCRProcessor.from_pretrained("username/trocr-chess-handwritten") |
| |
| # Load and process image |
| image = Image.open("chess_move.png").convert("RGB") |
| pixel_values = processor(image, return_tensors="pt").pixel_values |
| |
| # Generate prediction |
| generated_ids = model.generate(pixel_values) |
| predicted_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| print(f"Predicted move: {predicted_text}") |
