Choose a Model¶
Deepparse offers pretrained models built on three embeddings, FastText, BPEmb and Model2Vec, each with or without an
attention mechanism, plus a lighter FastText version. You select one with the model_type (and attention_mechanism)
argument of AddressParser. All the figures below come from this documentation; their
source is given under each table.
Model Types¶
|
Alias |
Description |
|---|---|---|
|
|
Seq2Seq model using the French fastText pretrained embeddings. The
|
|
The |
|
|
|
The FastText model using the Magnitude embeddings
mapping, which reduces the memory footprint. The |
|
|
Seq2Seq model using the byte-pair multilingual subword pretrained
embeddings. The |
|
The |
|
|
|
Seq2Seq model using the Model2Vec
potion-base-8M static embeddings (256 dimensions).
The |
|
Seq2Seq model using the potion-base-32M static embeddings (512 dimensions). |
|
|
Seq2Seq model using the potion-multilingual-128M multilingual static embeddings (256 dimensions). |
|
|
The Model2Vec models with an attention mechanism. Require |
Source: the model_type argument of AddressParser and the model names accepted by the
parser (case-insensitive).
Resources and Speed¶
Model |
Embeddings on disk |
RAM (GB) |
GPU memory (GB) |
Time per address, CPU (batch of 1 / more than 1) (s) |
Time per address, GPU (batch of 1 / more than 1) (s) |
|---|---|---|---|---|---|
fastText |
6.8 GB |
~1.3 [1] |
~1 |
~0.0128 / ~0.0026 |
~0.0023 / ~0.0004 |
fastTextAttention |
6.8 GB |
~1.3 [1] |
~1.1 |
~0.0230 / ~0.0057 |
~0.0043 / ~0.0007 |
fastText-light |
3.3 GB |
~1 |
~1 |
~0.0170 / ~0.0030 |
~0.0028 / ~0.0037 |
BPEmb [2] |
116 MB |
~1 |
~1 |
~0.0179 / ~0.0044 |
~0.0055 / ~0.0015 |
BPEmbAttention [2] |
116 MB |
~1 |
~1.1 |
~0.0286 / ~0.0075 |
~0.0081 / ~0.0019 |
Model2Vec 8M [3] |
59 MB |
~1.0 |
~0.2 |
~0.0095 / ~0.0016 |
~0.0035 / ~0.0006 |
Model2Vec 32M [3] |
249 MB |
~1.2 |
~0.3 |
~0.0098 / ~0.0017 |
~0.0035 / ~0.0007 |
Model2Vec Multilingual [3] |
1 GB |
~2.8 |
~0.2 |
~0.0097 / ~0.0017 |
~0.0035 / ~0.0007 |
Sources:
Embeddings on disk: the note of
AddressParser(the FastText embeddings take 6.8 GB, the FastText-light embeddings 3.3 GB and BPEmb 116 MB). The pretrained weights take at most 44 MB in addition.RAM, GPU memory and time per address: the “Memory Usage and Time Performance” tables of the Parser page (Linux, Python 3.11, Torch 2.0 and CUDA 11.7, ~183,000 addresses, March 21, 2023). The attention mechanism takes about 100 MB more on GPU.
Accuracy and Multilingual Coverage¶
All the models were trained on addresses from the same 20 countries; the Pretrained Models Results page gives their accuracy on those countries and a zero-shot evaluation on 41 other countries.
FastText uses French word embeddings with character n-grams. On the 20 training countries (clean data), it reaches 98.9% to 99.97% accuracy (see Pretrained Models Results).
BPEmb uses multilingual subword embeddings, which cover the vocabulary of many languages. On the 20 training countries (clean data), it reaches 96.97% to 99.99% accuracy (see Pretrained Models Results).
Attention mechanism: it improves the accuracy by around 0.5% on the training countries, and it further improves the zero-shot accuracy on most of the 41 other countries (see the attention table of Pretrained Models Results).
Model2Vec uses static embeddings distilled from sentence transformers, so it needs neither a sub-word network nor large embeddings. Unlike the FastText and BPEmb models, the Model2Vec models were trained on the worldwide-addresses data, with clean and incomplete addresses of the same 20 countries, and their hidden size was selected on the validation split (see the Model2Vec section of Pretrained Models Results for their test accuracy). Since this data has no
OrientationnorGeneralDeliverytags, they never predict these two tags (the street orientation is part of theStreetName).FastText Light: on average, its results are about 2% lower on the training countries, but similar on the zero-shot countries (see the Parser page and our article).
When to Use Each Model¶
You parse addresses from the 20 training countries and want the quickest model:
"fasttext"("fastest"). Its embeddings take the most disk space (6.8 GB).You parse addresses from other countries (zero-shot): prefer a model with an attention mechanism (
"fasttext-attention"or"bpemb-attention") and check the country in the zero-shot tables of Pretrained Models Results, since the accuracy varies a lot between countries.Disk space is limited (e.g. a container image or a small node):
"bpemb"or"bpemb-attention", whose embeddings take 116 MB.Disk space and RAM are very limited (e.g. a serverless function or an edge device):
"model2vec-8m"("tiny"), whose embeddings take 59 MB, or"model2vec-32m"(249 MB). Note that they never predict theOrientationandGeneralDeliverytags.You want the default model:
"bpemb"("best").You want the FastText model with a lower memory footprint:
"fasttext-light"("lightest"). Note that a FastText Light model cannot be retrained nor tested, and cannot use data loading workers (num_workers> 0) on Windows (seeFastTextModelError).You need to retrain a model on new data or new tags: any model except
"fasttext-light"(see the Training Guide).