The latest improvements for RNN are flowing in, rolling in accuracy and performance gains "orders of magnitude" better than current state of the art. RNN's are particularly useful for speech recognition and natural language processing. This new improvement, the 'Transformer' is able to optimise the sequential learning steps when translating a sentence by homing in on particular words in a sentence that most significantly affect it's meaning (hence require more 'attention').
we show that the Transformer outperforms both recurrent and convolutional models on academic English to German and English to French translation benchmarks. On top of higher translation quality, the Transformer requires less computation to train and is a much better fit for modern machine learning hardware, speeding up training by up to an order of magnitude.