跳转到主要内容

用于开发开放代理的软件包

项目描述

Mask-Predict

下载模型

描述 数据集 模型
MASK-PREDICT [WMT14 英语-德语] 下载 (.tar.bz2)
MASK-PREDICT [WMT14 德语-英语] 下载 (.tar.bz2)
MASK-PREDICT [WMT16 英语-罗马尼亚语] 下载 (.tar.bz2)
MASK-PREDICT [WMT16 罗马尼亚语-英语] 下载 (.tar.bz2)
MASK-PREDICT [WMT17 英语-中文] 下载 (.tar.bz2)
MASK-PREDICT [WMT17 中文-英语] 下载 (.tar.bz2)

预处理

text=PATH_YOUR_DATA

output_dir=PATH_YOUR_OUTPUT

src=source_language

tgt=target_language

model_path=PATH_TO_MASKPREDICT_MODEL_DIR

python preprocess.py --source-lang srctargetlang{tgt} --trainpref text/trainvalidpreftext/valid --testpref text/testdestdir{output_dir}/data-bin --workers 60 --srcdict Missing superscript or subscript argument{src}tgt/dict.{src}.txt --tgtdict ${model_path}/maskPredictMissing superscript or subscript argument{tgt}/dict.${tgt}.txt

训练

model_dir=PLACE_TO_SAVE_YOUR_MODEL

python train.py outputdir/databinarchberttransformerseq2seqshareallembeddingscriterionlabelsmoothedlengthcrossentropylabelsmoothing0.1lr5e4warmupinitlr1e7minlr1e9lrschedulerinversesqrtwarmupupdates10000optimizeradamadambetas(0.9,0.999)adameps1e6tasktranslationselfmaxtokens8192weightdecay0.01dropout0.3encoderlayers6encoderembeddim512decoderlayers6decoderembeddim512fp16maxsourcepositions10000maxtargetpositions10000maxupdate300000seed0savedir{model_dir}

评估

python generate_cmlm.py outputdir/databinpath{model_dir}/checkpoint_best_average.pt --task translation_self --remove-bpe --max-sentences 20 --decoding-iterations 10 --decoding-strategy mask_predict

许可证

MASK-PREDICT 是 CC-BY-NC 4.0。该许可证适用于预训练模型。

引用

请引用如下

@inproceedings{ghazvininejad2019MaskPredict,
  title = {Mask-Predict: Parallel Decoding of Conditional Masked Language Models},
  author = {Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
}

由以下支持

AWSAWS云计算和安全赞助商DatadogDatadog监控FastlyFastlyCDNGoogleGoogle下载分析MicrosoftMicrosoftPSF赞助商PingdomPingdom监控SentrySentry错误日志StatusPageStatusPage状态页面