Models Download

The AddressParser downloads the pretrained weights and the embeddings it needs the first time it is used (in ~/.cache/deepparse by default). The following functions download them ahead of time, for example before sending a model to a node without Internet access (then use offline=True in the AddressParser). The download_model command of the CLI does the same from the command line. They can be imported directly from deepparse (e.g. from deepparse import download_model).

Pretrained Models

deepparse.download_tools.download_model(model_type: str, saving_cache_path: Path | None = None) → None

Function to download a pretrained model. It will download its corresponding checkpoint and version file.

Parameters:

• model_type – The model type (i.e. fasttext or bpemb-attention).

• saving_cache_path – The path to the saving cache directory for the specified model.

deepparse.download_tools.download_models(saving_cache_path: Path | None = None) → None

Function to download all the pretrained models. It will download all the model’s checkpoints and version files.

Parameters:

saving_cache_path – The path to the saving cache directory for the specified model.

deepparse.download_tools.download_weights(model_type: str, saving_dir: str, verbose: bool = True, offline: bool = False) → str

Function to download the pretrained weights of one of our pre-trained base models. :param model_type: The network pretrained weights to load. :type model_type: str :param saving_dir: The path to the saving directory. :param verbose: Either or not to be verbose during the download of a model. The default value is True. :type verbose: bool :param offline: Whether the model is an offline or an online. :type offline: bool

Returns:

The model rep id (str) which can be used with hugging face’s from_pretrained method.

Embeddings

deepparse.download_tools.download_fasttext_embeddings(cache_dir: str, verbose: bool = True, offline: bool = False) → str

Simpler version of the download_model function from FastText to download pretrained common-crawl vectors from FastText’s website https://fasttext.cc/docs/en/crawl-vectors.html and save it in the saving directory (saving_dir).

Return the full path to the FastText embeddings.

deepparse.download_tools.download_fasttext_magnitude_embeddings(cache_dir: str, verbose: bool = True, offline: bool = False) → str

Function to download the magnitude pretrained FastText model.

Return the full path to the FastText embeddings.

Sample Data

A small sample of tagged addresses of one country can be downloaded from the worldwide-addresses dataset and used with a ListDatasetContainer (see Dataset Container).

deepparse.download_tools.download_worldwide_addresses_sample(country_code: str = 'ca', num_samples: int = 1000) → List[Tuple[str, List]]

Function to download a small sample of tagged addresses of one country from the worldwide-addresses HuggingFace dataset and convert it to the deepparse tag space. It is meant to give small, ready-to-use datasets for examples and quick experiments.

Only the needed part of the country’s last parquet file is read (the beginning of each country’s data mostly holds postal-code-only addresses, while the end holds complete street addresses). The tags are mapped to the deepparse tags as follows: Suburb and District become Municipality, and the words tagged County or Country are removed from the address. The addresses left empty by this removal are dropped.

It requires the pyarrow package (pip install pyarrow or pip install deepparse[all]).

Parameters:

• country_code (str) – The ISO 3166-1 alpha-2 code of the country to download (e.g. "ca" or "us"). The default value is "ca".

• num_samples (int) – The maximum number of tagged addresses to return. The default value is 1000.

Returns:

A list of tuples (address, tags) where the address is a string and the tags a list of deepparse tags, one per whitespace-separated word. The list can be used as is with a ListDatasetContainer.