What's your reasoning? Google's latest published research is analogously parsimonious with this approach. They are doing decoding using a simple beam search through a single neural network. See https://ai.googleblog.com/2019/03/an-all-neural-on-device-sp...