用于开发Task Bots的包

导航

已验证细节

这些细节已由PyPI 验证
维护者
Avatar for zomux from gravatar.com zomux

未验证细节

这些细节 尚未 由PyPI验证
项目链接
元数据
  • 许可： MIT许可
  • 作者： Raphael Shu
  • 要求： Python >=3.8
分类

项目描述

Mask-Predict

下载模型

描述 数据集 模型
MASK-PREDICT [WMT14 English-German] 下载 (.tar.bz2)
MASK-PREDICT [WMT14 German-English] 下载 (.tar.bz2)
MASK-PREDICT [WMT16 English-Romanian] 下载 (.tar.bz2)
MASK-PREDICT [WMT16 Romanian-English] 下载 (.tar.bz2)
MASK-PREDICT [WMT17 English-Chinese] 下载 (.tar.bz2)
MASK-PREDICT [WMT17 Chinese-English] 下载 (.tar.bz2)

预处理

text=PATH_YOUR_DATA

output_dir=PATH_YOUR_OUTPUT

src=source_language

tgt=target_language

model_path=PATH_TO_MASKPREDICT_MODEL_DIR

python preprocess.py --source-lang src−−target−lang{tgt} --trainpref text/train−−validpreftext/valid --testpref text/test−−destdir{output_dir}/data-bin --workers 60 --srcdict Missing superscript or subscript argument{src}tgt/dict.{src}.txt --tgtdict ${model_path}/maskPredictMissing superscript or subscript argument{tgt}/dict.${tgt}.txt

训练

model_dir=PLACE_TO_SAVE_YOUR_MODEL

python train.py outputdir/data−bin−−archberttransformerseq2seq−−share−all−embeddings−−criterionlabelsmoothedlengthcrossentropy−−label−smoothing0.1−−lr5e−4−−warmup−init−lr1e−7−−min−lr1e−9−−lr−schedulerinversesqrt−−warmup−updates10000−−optimizeradam−−adam−betas′(0.9,0.999)′−−adam−eps1e−6−−tasktranslationself−−max−tokens8192−−weight−decay0.01−−dropout0.3−−encoder−layers6−−encoder−embed−dim512−−decoder−layers6−−decoder−embed−dim512−−fp16−−max−source−positions10000−−max−target−positions10000−−max−update300000−−seed0−−save−dir{model_dir}

评估

python generate_cmlm.py outputdir/data−bin−−path{model_dir}/checkpoint_best_average.pt --task translation_self --remove-bpe --max-sentences 20 --decoding-iterations 10 --decoding-strategy mask_predict

许可

MASK-PREDICT遵循CC-BY-NC 4.0许可。此许可证也适用于预训练模型。

引用

请引用如下

@inproceedings{ghazvininejad2019MaskPredict,
  title = {Mask-Predict: Parallel Decoding of Conditional Masked Language Models},
  author = {Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
}

支持者