NLLB Serve

该项目提供了一个Web界面和REST API，用于Meta的“无语言遗留”（NLLB）模型，该模型可以跨200种语言进行翻译。

设置

# Option 1: install as read only; recommended to use as is
pip install git+https://github.com/thammegowda/nllb-serve

# Option 2: install for editable mode; recommended if you'd like to modify code
git clone  https://github.com/thammegowda/nllb-serve
cd nllb-serve
pip install -e .


# Entry point: either one of these options should work
nllb-serve -h
python -m nllb_serve -h

启动服务器

# Either one of these should work
nllb-serve
# or
python -m nllb_serve

# Use CPU, ignore GPUs even if they exist
CUDA_VISIBLE_DEVICES= nllb-serve

# Use GPU device 0
CUDA_VISIBLE_DEVICES=0 nllb-serve

默认情况下，这将在一个https://:6060上的服务启动。

Cuda和Torch兼容性问题

# check 1:
$ python -c 'import torch; print(torch.cuda.is_available())'
True

# Check 2: match the version of installed cudatookit with the version for which torch bins were compiled
# if exactly matching the versions is not possible/difficult, try getting the versions as close as possible
$ python -c 'import torch; print(torch.version.cuda)'
11.7
$  nvidia-smi  | grep -o 'CUDA.*'
CUDA Version: 12.0

命令行选项

$ nllb-serve -h
usage: nllb-serve [-h] [-d] [-p PORT] [-ho HOST] [-b BASE] [-mi MODEL_ID] [-msl MAX_SRC_LEN]

Deploy NLLB model to a RESTful server

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Run Flask server in debug mode (default: False)
  -p PORT, --port PORT  port to run server on (default: 6060)
  -ho HOST, --host HOST
                        Host address to bind. (default: 0.0.0.0)
  -b BASE, --base BASE  Base prefix path for all the URLs. E.g., /v1 (default: None)
  -mi MODEL_ID, --model_id MODEL_ID
                        model ID; see https://hugging-face.cn/models?other=nllb (default: facebook/nllb-200-distilled-600M)
  -msl MAX_SRC_LEN, --max-src-len MAX_SRC_LEN
                        max source len; longer seqs will be truncated (default: 250)

REST API

• /translate端点接受以下参数的GET和POST请求
  • source -- 源文本。可以是单个字符串或一批（即字符串列表）
  • src_lang -- 源语言ID，例如，eng_Latn
  • tgt_lang -- 目标语言ID，例如，eng_Latn

HTTP客户端可以通过以下三种方式发送这些参数

1. 查询参数（GET）
   例如

   • http://0.0.0.0:6060/translate?source=I%20am%20testing&src_lang=eng_Latn&tgt_lang=fra_Latn
   • http://0.0.0.0:6060/translate?source=I%20am%20testing&src_lang=eng_Latn&tgt_lang=fra_Latn&source=another%20sentence

2. URL编码表单（POST）

 curl --data "source=Comment allez-vous?" --data "source=Bonne journée" \
 --data "src_lang=fra_Latn" --data "tgt_lang=eng_Latn" \
  https://:6060/translate

3. JSON体（POST）

$ curl -H "Content-Type: application/json" -X POST \
    https://:6060/translate \
   --data '{"source": ["Comment allez-vous?"], "src_lang": "fra_Latn", "tgt_lang": "kan_Knda"}'

语言代码列表：https://hugging-face.cn/facebook/nllb-200-distilled-600M/blob/main/special_tokens_map.json

NLLB-Batch

此CLI工具用于解码数据批。虽然REST API针对一次性翻译单个翻译进行了优化，但nllb-batch针对解码大型文件进行了优化。

$ nllb-batch  --help
INFO:root:torch device=cuda
usage: nllb-batch [-h] [-mi MODEL_ID] -sl SRC_LANG -tl TGT_LANG [-i INP] [-o OUT] [-msl MAX_SRC_CHARS] [-b BATCH_SIZE]

Serve NLLB model via command line

options:
  -h, --help            show this help message and exit
  -mi MODEL_ID, --model-id MODEL_ID
                        model ID; see https://hugging-face.cn/models?other=nllb (default: facebook/nllb-200-distilled-600M)
  -sl SRC_LANG, --src-lang SRC_LANG
                        source language identifier; eg: eng_Latn (default: None)
  -tl TGT_LANG, --tgt-lang TGT_LANG
                        Target language identifier; eg: eng_Latn (default: None)
  -i INP, --inp INP     Input file (default: <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>)
  -o OUT, --out OUT     Output file (default: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)
  -msl MAX_SRC_CHARS, --max-src-chars MAX_SRC_CHARS
                        max source chars len; longer seqs will be truncated (default: 512)
  -b BATCH_SIZE, --batch-size BATCH_SIZE
                        Batch size; number of sentences (default: 10)

许可证

代码和模型权重携带不同的许可证。此存储库中的代码通过Apache License 2.0分发。
但是模型权重是从Huggingface Hub获取的，并且适用原始模型权重的许可。在撰写本文时，NLLB模型的作者通过CC-BY-NC-4.0许可分发权重。更多信息请参阅 LICENSE.model.md 和 创意共享许可

参考文献

• https://research.facebook.com/publications/no-language-left-behind/
• https://hugging-face.cn/docs/transformers/main/en/model_doc/nllb
• https://ai.facebook.com/research/no-language-left-behind/
• https://github.com/facebookresearch/fairseq/tree/nllb/