Embeddings Models

The embeddings models convert each word of an address into a vector that the Seq2Seq network then tags. The AddressParser creates the right embeddings model for its model_type, so you only need these classes to use the embeddings directly or to build your own pipeline:

• the FastText models ("fasttext" and "fasttext-attention") use FastTextEmbeddingsModel, which reads the French fastText .bin embeddings,

• the FastText Light models ("fasttext-light") use MagnitudeEmbeddingsModel, which reads the same embeddings in the Magnitude format,

• the BPEmb models ("bpemb" and "bpemb-attention") use BPEmbEmbeddingsModel, which uses the byte-pair multilingual subword embeddings.

The FastText .bin embeddings are read by FastTextBinReader, our own reader of the fastText binary format. It gives the same vectors as the fasttext package without depending on fasttext nor gensim, and its input matrix is memory-mapped: it loads in a few seconds, only the rows used are kept in RAM, and the data loading workers share its memory. It replaces the fasttext/gensim loader used up to version 0.11.0.

Interface

class deepparse.embeddings_models.EmbeddingsModel(verbose: bool = True)

Abstract (wrapper) class for callable embeddings network.

Implementations

class deepparse.embeddings_models.FastTextEmbeddingsModel(embeddings_path: str, verbose: bool = True)

FastText embeddings network from Enriching Word Vectors with Subword Information.

Parameters:

• embeddings_path (str) – Path to the bin embeddings vector (.bin).

• verbose (bool) – Either or not to make the loading of the embeddings verbose.

Note

The embeddings are read with our own reader of the fastText .bin format (FastTextBinReader), which gives the same vectors as the fasttext package without depending on it nor on gensim. The input matrix is memory-mapped, so the loading is fast and the data loading workers share its memory.

__call__(word: str) → ndarray

Callable method to get a word vector.

Parameters:

word (str) – Word to get vector.

Returns:

The FastText embedding for a word.

class deepparse.embeddings_models.MagnitudeEmbeddingsModel(embeddings_path: str, verbose: bool = True)

FastText embeddings network from Enriching Word Vectors with Subword Information using the magnitude mapping (Magnitude), which reduces the memory footprint.

Parameters:

• embeddings_path (str) – Path to the Magnitude embeddings vector (.magnitude).

• verbose (bool) – Either or not to make the loading of the embeddings verbose.

__call__(words: str) → ndarray

Callable method to get the word vector of a complete address.

Parameters:

words (str) – Address to get vector for words.

Returns:

The FastText embedding for a list of words.

class deepparse.embeddings_models.BPEmbEmbeddingsModel(cache_dir: str, verbose: bool = True)

BPEmb embeddings network from BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages. The arguments are the same as the BPEmb class

Parameters:

• cache_dir (str) – Path to the cache directory to the embeddings’ bin vector and the model.

• verbose (bool) – Whether or not to make the loading of the embeddings verbose.

__call__(word: str) → ndarray

Callable method to get a word vector.

Parameters:

word (str) – Word to get vector.

Returns:

The BP embedding for a word.

FastText Binary Reader

class deepparse.embeddings_models.FastTextBinReader(path: str)

Reader of the Facebook fastText .bin format (version 12, non-quantized, unsupervised models) that computes word vectors like FastText::getWordVector without the fasttext or gensim packages.

The input matrix is memory-mapped, so loading is fast, the RAM usage stays low, and the pages are shared between processes (e.g. DataLoader workers). When pickled, only the path and the parsed dictionary are kept, and the memory map is reopened on unpickling.

Parameters:

path (str) – Path to the fastText .bin file.

get_dimension() → int

Return the dimension of the word vectors (same interface as the fasttext model object).

get_word_id(word: str) → int

Return the id of a word in the dictionary, or -1 if the word is out of vocabulary.

get_subword_ids(word: str) → List[int]

Return the input matrix rows used to compute a word vector, like Dictionary::getSubwords: the word id (if the word is in the vocabulary) followed by the bucket rows of its hashed character n-grams.

get_word_vector(word: str) → ndarray

Compute the vector of a word as the mean of its subword rows, or a zero vector if it has none.

Factory

class deepparse.embeddings_models.EmbeddingsModelFactory

A factory for the creation of embeddings models.

create(embedding_model_type: str, cache_dir: str, verbose: bool = True) → EmbeddingsModel

Embeddings model creation method.

Parameters:

• embedding_model_type (str) – the type of the embeddings model to create. Valid options are "bpemb", "fasttext", "fasttext-light" (the Magnitude embeddings), "model2vec-8m", "model2vec-32m" and "model2vec-multilingual".

• cache_dir (str) – Path to the cache directory where the embeddings model exists or is to be downloaded.

• verbose (bool) – Whether or not to make the loading of the embeddings verbose.

Returns:

An EmbeddingsModel