Learning To Reason With Third Order Tensor Products

Authors:
Imanol Schlag IDSIA
Jürgen Schmidhuber Swiss AI Lab, IDSIA (USI & SUPSI) - NNAISENSE

Introduction:

The authors combine Recurrent Neural Networks with Tensor Product Representations tolearn combinatorial representations of sequential data.

Abstract:

We combine Recurrent Neural Networks with Tensor Product Representations tolearn combinatorial representations of sequential data. This improves symbolicinterpretation and systematic generalisation. Our architecture is trained end-to-endthrough gradient descent on a variety of simple natural language reasoning tasks,significantly outperforming the latest state-of-the-art models in single-task andall-tasks settings. We also augment a subset of the data such that training and testdata exhibit large systematic differences and show that our approach generalisesbetter than the previous state-of-the-art.

You may want to know: