DL之Encoder-Decoder：Encoder-Decoder结构的相关论文、设计思路、关键步骤等配图集合之详细攻略

蜜糖 2022-09-19 16:22:52  64145

分类专栏：资讯

DL之Encoder-Decoder：Encoder-Decoder模型的相关论文、设计思路、关键步骤等配图集合之详细攻略

Encoder-Decoder模型的相关论文

1、Encoder-Decoder 结构做机器翻译任务的更多细节，可以参考原始论文《Learning Phrase Representations using RNN Encoder– Decoder for Statistical Machine Translation》
论文地址：https://arxiv.org/pdf/1406.1078.pdf

Encoder-Decoder模型的设计思路

Abstract：In this paper, we propose a novel neural network model called RNN Encoder– Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixedlength vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder–Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.

1、An illustration of the proposed RNN Encoder–Decoder.

2、An illustration of the proposed hidden activation function. The update gate z selects whether the hidden state is to be updated with a new hidden state h˜. The reset gate r decides whether the previous hidden state is ignored. See Eqs. (5)–(8) for the detailed equations of r, z, h and h˜.

3、: BLEU scores computed on the development and test sets using different combinations of approaches. WP denotes a word penalty, where we penalizes the number of unknown words to neural networks.

4、2–D embedding of the learned word representation. The left one shows the full embedding space, while the right one shows a zoomed-in view of one region (color–coded). For more plots, see the supplementary material.

5、2–D embedding of the learned phrase representation. The top left one shows the full representation space (5000 randomly selected points), while the other three figures show the zoomed-in view of specific regions (color–coded).