The latest language models make it possible to decipher handwritten texts much more accurately than before. In our demo we use this to recognize handwritten license plates.
Start the demo, write a valid Dutch license plate code in the drawing field and press the “start” button.
Your handwriting is now automatically deciphered and converted into the six characters on a Dutch license plate. This license plate is then used to retrieve some additional information from the National Road Traffic Service (RDW).
Good luck!
Background
The model used in this demo is based on the TrOCR model developed by researchers at Microsoft. This model consists of a vision transformer in combination with a Roberta model to encode the license plate text. The image transformer splits the image with the handwritten text into a grid of blocks that serve as input to the transformer. Furthermore, this model uses the self-attention mechanism just like the text-based models shown elsewhere on this demo site.
This model has been trained on a corpus of millions of automatically generated plates and further fine-tuned specifically for the number plate recognition task on several tens of thousands of synthetic number plates. To make the model even more robust, this dataset has been expanded using data augmentation. The original images are distorted in different ways to prevent overfitting during training.
Practical applications
Practical application of these types of models naturally lie in the field of recognizing texts in images. This can be handwritten texts, but this is not necessary. Our model is also perfectly capable of correctly deciphering printed texts. This is useful, for example, when automatically processing completed forms. The software of this demo of recognizing handwritten license plates is used for the automatic processing of completed car damage forms.