[ACL 2021 Highlights] Neural Machine Translation 관련 논문

ACL 2021 Highlights 중 Neural Machine Translation 관련 논문 정리 목록 

https://www.paperdigest.org/2021/07/acl-2021-highlights/

Paper Digest: ACL 2021 Highlights – Paper Digest

위 사이트에서 발췌한 논문 내역입니다! ACL 2021 Highlights 전체 목록은 위 링크에서 확인해주세요~

 

• 16번 논문은 제가 KCS2021에 투고한 "문서 단위 기계 번역 성능 향상을 위한 데이터 증강"논문의 기본 모델으로 문서 단위 기계 번역 성능 향상을 크게 보이는 모델 구조를 제안함
• 22번 논문은 신경망 기계 번역이 주 내용은 아니지만 기계 번역 성능 향상을 보일 수 있는 방법론을 제시한 논문이어서 추가함 
• 46번 논문은 신경망 기계 번역에서 연구되던 방법론을 다국어 음성 번역에 적용한 부분이라 추가함 

 

1	Contrastive Learning for Many-to-many Multilingual Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this work, we aim to build a many-to-many translation system with an emphasis on the quality of non-English language directions.	Xiao Pan; Mingxuan Wang; Liwei Wu; Lei Li;
2	Understanding The Properties of Minimum Bayes Risk Decoding in Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search.	Mathias M?ller; Rico Sennrich;
3	Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To enable sequence-level parallelization of LSTMs, we approximate full LSTM context modelling by computing hidden states and gates with the current input and a simple bag-of-words representation of the preceding tokens context.	Hongfei Xu; Qiuhui Liu; Josef van Genabith; Deyi Xiong; Meng Zhang;
4	Learning Language Specific Sub-network for Multilingual Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose LaSS to jointly train a single unified multilingual MT model.	Zehui Lin; Liwei Wu; Mingxuan Wang; Lei Li;
5	Analyzing The Source and Target Contributions to Predictions in Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: We argue that this relative contribution can be evaluated by adopting a variant of Layerwise Relevance Propagation (LRP).	Elena Voita; Rico Sennrich; Ivan Titov;
6	Attention Calibration for Transformer in Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose to calibrate the attention weights by introducing a mask perturbation model that automatically evaluates each input’s contribution to the model outputs.	Yu Lu; Jiali Zeng; Jiajun Zhang; Shuangzhi Wu; Mu Li;
7	Crafting Adversarial Examples for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this work, we investigate veritable evaluations of NMT adversarial attacks, and propose a novel method to craft NMT adversarial examples.	Xinze Zhang; Junzhe Zhang; Zhenhua Chen; Kun He;
8	Glancing Transformer for Non-Autoregressive Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: We propose the Glancing Language Model (GLM) for single-pass parallel generation models.	Lihua Qian; Hao Zhou; Yu Bao; Mingxuan Wang; Lin Qiu; Weinan Zhang; Yong Yu; Lei Li;
9	Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this work, we propose to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data.	Wenxiang Jiao; Xing Wang; Zhaopeng Tu; Shuming Shi; Michael Lyu; Irwin King;
10	Breaking The Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To break the corpus bottleneck, in this paper we aim to improve context-aware NMT by taking the advantage of the availability of both large-scale sentence-level parallel dataset and source-side monolingual documents.	Linqing Chen; Junhui Li; Zhengxian Gong; Boxing Chen; Weihua Luo; Min Zhang; Guodong Zhou;
11	Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To address this problem, we introduce another decoder, called seer decoder, into the encoder-decoder framework during training, which involves future information in target predictions.	Yang Feng; Shuhao Gu; Dengji Guo; Zhengxin Yang; Chenze Shao;
12	Unsupervised Neural Machine Translation for Low-Resource Domains Via Meta-Learning Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To address this issue, this paper presents a novel meta-learning algorithm for unsupervised neural machine translation (UNMT) that trains the model to adapt to another domain by utilizing only a small amount of training data.	Cheonbok Park; Yunwon Tae; TaeHee Kim; Soyoung Yang; Mohammad Azam Khan; Lucy Park; Jaegul Choo;
13	Online Learning Meets Machine Translation Evaluation: Finding The Best Systems with The Least Human Effort Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To overcome the latter challenge, we propose a novel application of online learning that, given an ensemble of Machine Translation systems, dynamically converges to the best systems, by taking advantage of the human feedback available.	V?nia Mendon?a; Ricardo Rei; Luisa Coheur; Alberto Sardinha; Ana L?cia Santos;
14	From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this work, we adapt a state-of-the-art neural machine translation model to generate Hindi-English code-switched sentences starting from monolingual Hindi sentences.	Ishan Tarunesh; Syamantak Kumar; Preethi Jyothi;
15	Fast and Accurate Neural Machine Translation with Translation Memory Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose a fast and accurate approach to TM-based NMT within the Transformer framework: the model architecture is simple and employs a single bilingual sentence as its TM, leading to efficient training and inference; and its parameters are effectively optimized through a novel training criterion.	Qiuxiang He; Guoping Huang; Qu Cui; Li Li; Lemao Liu;
16	G-Transformer for Document-Level Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: As a solution, we propose G-Transformer, introducing locality assumption as an inductive bias into Transformer, reducing the hypothesis space of the attention from target to source.	Guangsheng Bao; Yue Zhang; Zhiyang Teng; Boxing Chen; Weihua Luo;
17	Prevent The Language Model from Being Overconfident in Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Based on the property, we propose a Margin-based Token-level Objective (MTO) and a Margin-based Sentence-level Objective (MSO) to maximize the Margin for preventing the LM from being overconfident.	Mengqi Miao; Fandong Meng; Yijin Liu; Xiao-Hua Zhou; Jie Zhou;
18	Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: This paper proposes a sophisticated neural architecture to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models.	Tong Zhang; Long Zhang; Wei Ye; Bo Li; Jinan Sun; Xiaoyu Zhu; Wen Zhao; Shikun Zhang;
19	Towards User-Driven Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To fill this gap, we introduce a novel framework called user-driven NMT.	Huan Lin; Liang Yao; Baosong Yang; Dayiheng Liu; Haibo Zhang; Weihua Luo; Degen Huang; Jinsong Su;
20	End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In particular, we focus on methods based on training the model with constraints provided as part of the input sequence.	Josef Jon; Jo?o Paulo Aires; Dusan Varis; Ondrej Bojar;
21	Reservoir Transformers Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Inspired by old and well-established ideas in machine learning, we explore a variety of non-linear reservoir layers interspersed with regular transformer layers, and show improvements in wall-clock compute time until convergence, as well as overall performance, on various machine translation and (masked) language modelling tasks.	Sheng Shen; Alexei Baevski; Ari Morcos; Kurt Keutzer; Michael Auli; Douwe Kiela;
22	SemFace: Pre-training Encoder and Decoder with A Semantic Interface for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose a better pre-training method for NMT by defining a semantic interface (SemFace) between the pre-trained encoder and the pre-trained decoder.	huo Ren; Long Zhou; Shujie Liu; Furu Wei; Ming Zhou; Shuai Ma;
23	Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To benefit from this observation, we train an energy-based model to mimic the behavior of the task measure (i.e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR).	Sumanta Bhattacharyya; Amirmohammad Rooshenas; Subhajit Naskar; Simeng Sun; Mohit Iyyer; Andrew McCallum;
24	On Compositional Generalization of Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we study NMT models from the perspective of compositional generalization by building a benchmark dataset, CoGnition, consisting of 216k clean and consistent sentence pairs.	Yafu Li; Yongjing Yin; Yulong Chen; Yue Zhang;
25	Rewriter-Evaluator Architecture for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To address this issue, we introduce a novel architecture of Rewriter-Evaluator.	Yangming Li; Kaisheng Yao;
26	Importance-based Neuron Allocation for Multilingual Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To solve these problems, we propose to divide the model neurons into general and language-specific parts based on their importance across languages.	Wanying Xie; Yang Feng; Shuhao Gu; Dong Yu;
27	Good for Misconceived Reasons: An Empirical Revisiting on The Need for Visual Context in Multimodal Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Upon further investigation, we discover that the improvements achieved by the multimodal models over text-only counterparts are in fact results of the regularization effect.	Zhiyong Wu; Lingpeng Kong; Wei Bi; Xiang Li; Ben Kao;
28	Selective Knowledge Distillation for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we design a novel protocol that can effectively analyze the different impacts of samples by comparing various samples’ partitions.	Fusheng Wang; Jianhao Yan; Fandong Meng; Jie Zhou;
29	Measuring and Increasing Context Usage in Context-Aware Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we introduce a new metric, conditional cross-mutual information, to quantify usage of context by these models.	Patrick Fernandes; Kayo Yin; Graham Neubig; Andr? F. T. Martins;
30	Mid-Air Hand Gestures for Post-Editing of Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Here, we present the first study that investigates the usefulness of mid-air hand gestures in combination with the keyboard (GK) for text editing in PE of MT. Guided by a gesture elicitation study with 14 freelance translators, we develop a prototype supporting mid-air hand gestures for cursor placement, text selection, deletion, and reordering.	Rashad Albo Jamara; Nico Herbig; Antonio Kr?ger; Josef van Genabith;
31	Beyond Noise: Mitigating The Impact of Fine-grained Semantic Divergences on Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Based on these findings, we introduce a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences, improving both translation quality and model calibration on EN-FR tasks.	leftheria Briakou; Marine Carpuat;
32	Discriminative Reranking for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Since such a discriminator contains hundreds of millions of parameters, we improve its generalization using pre-training and data augmentation techniques.	Ann Lee; Michael Auli; Marc?Aurelio Ranzato;
33	Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: This paper presents the first large-scale meta-evaluation of machine translation (MT).	Benjamin Marie; Atsushi Fujita; Raphael Rubino;
34	Neural Machine Translation with Monolingual Translation Memory Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In contrast to existing work that uses bilingual corpus as TM and employs source-side similarity search for memory retrieval, we propose a new framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner.	Deng Cai; Yan Wang; Huayang Li; Wai Lam; Lemao Liu;
35	Vocabulary Learning Via Optimal Transport for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: This paper aims to figure out what is a good vocabulary and whether we can find the optimal vocabulary without trial training.	ingjing Xu; Hao Zhou; Chun Gan; Zaixiang Zheng; Lei Li;
36	Difficulty-Aware Machine Translation Evaluation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose a novel difficulty-aware MT evaluation metric, expanding the evaluation dimension by taking translation difficulty into consideration	Runzhe Zhan; Xuebo Liu; Derek F. Wong; Lidia S. Chao;
37	Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: We investigate architectures and techniques commonly used to speed up decoding in Transformer-based models, such as greedy search, quantization, average attention networks (AANs) and shallow decoder models and show their effect on gendered noun translation.	Adithya Renduchintala; Denise Diaz; Kenneth Heafield; Xian Li; Mona Diab;
38	Machine Translation Into Low-resource Language Varieties Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: We propose a general framework to rapidly adapt MT systems to generate language varieties that are close to, but different from, the standard target language, using no parallel (source-variety) data.	Sachin Kumar; Antonios Anastasopoulos; Shuly Wintner; Yulia Tsvetkov;
39	Multilingual Agreement for Multilingual Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this work, we propose a novel agreement-based method to encourage multilingual agreement among different translation directions, which minimizes the differences among them.	Jian Yang; Yuwei Yin; Shuming Ma; Haoyang Huang; Dongdong Zhang; Zhoujun Li; Furu Wei;
40	Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose to efficiently increase the capacity for multilingual NMT by increasing the cardinality.	Hongfei Xu; Qiuhui Liu; Josef van Genabith; Deyi Xiong;
41	Adaptive Nearest Neighbor Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose Adaptive kNN-MT to dynamically determine the number of k for each target token.	Xin Zheng; Zhirui Zhang; Junliang Guo; Shujian Huang; Boxing Chen; Weihua Luo; Jiajun Chen;
42	On Orthogonality Constraints for Transformers Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To fill this gap, this paper studies orthogonality constraints for transformers, showing the effectiveness with empirical evidence from ten machine translation tasks and two dialogue generation tasks.	Aston Zhang; Alvin Chan; Yi Tay; Jie Fu; Shuohang Wang; Shuai Zhang; Huajie Shao; Shuochao Yao; Roy Ka-Wei Lee;
43	Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we propose a novel bilingual mutual information (BMI) based adaptive objective, which measures the learning difficulty for each target token from the perspective of bilingualism, and assigns an adaptive weight accordingly to improve token-level adaptive training.	Yangyifan Xu; Yijin Liu; Fandong Meng; Jiajun Zhang; Jinan Xu; Jie Zhou;
44	When Is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we present an in-depth comparison between character-based and subword-based NMT systems under three settings: translating to typologically diverse languages, training with low resource, and adapting to unseen domains.	Jiahuan Li; Yutong Shen; Shujian Huang; Xinyu Dai; Jiajun Chen;
45	Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: In this paper, we instead tackle a more challenging setup consisting of domain-specific corpora with much longer n-gram and highly specialized terms.	Gyubok Lee; Seongjun Yang; Edward Choi;
46	Lightweight Adapter Tuning for Multilingual Speech Translation Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: While adapter tuning was investigated for multilingual neural machine translation, this paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST).	Hang Le; Juan Pino; Changhan Wang; Jiatao Gu; Didier Schwab; Laurent Besacier;
47	BERTTune: Fine-Tuning Neural Machine Translation with BERTScore Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: To amend this form of overfitting, in this paper we propose fine-tuning the models with a novel training objective based on the recently-proposed BERTScore evaluation metric.	Inigo Jauregi Unanue; Jacob Parnell; Massimo Piccardi;
48	Don’t Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data Related Papers   Related Patents   Related Grants   Related Orgs   Related Experts   Details Highlight: Here, we present a data collection strategy for MT which, in contrast, is cheap and simple, as it does not require bilingual speakers.	Rajat Bhatnagar; Ananya Ganesh; Katharina Kann;