|Jürgen Schmidhuber||Swiss AI Lab, IDSIA (USI & SUPSI) - NNAISENSE|
The authors combine Recurrent Neural Networks with Tensor Product Representations tolearn combinatorial representations of sequential data.
We combine Recurrent Neural Networks with Tensor Product Representations tolearn combinatorial representations of sequential data. This improves symbolicinterpretation and systematic generalisation. Our architecture is trained end-to-endthrough gradient descent on a variety of simple natural language reasoning tasks,significantly outperforming the latest state-of-the-art models in single-task andall-tasks settings. We also augment a subset of the data such that training and testdata exhibit large systematic differences and show that our approach generalisesbetter than the previous state-of-the-art.