Sequence to Sequence


Analytics Zoo provides Seq2seq model which is a general-purpose encoder-decoder framework that can be used for Chatbot, Machine Translation and more. The model could be fed into NNFrames or BigDL Optimizer directly for training.


Build a Seq2seq Model

Before build Seq2seq Model, you need build Encoder, Decoder. And Bridge if you want to do some transformation before passing encoder states to decoder.

Build an Encoder

Currently we only support RNNEncoder which enables you to put RNN layers into encoder. You can call the following API in Scala and Python respectively to create a RNNEncoder.

Scala

val encoder = RNNEncoder(rnnType, numLayer, hiddenSize, embedding)

You can also define RNN layers yourself

val encoder = RNNEncoder(rnns, embedding, inputShape)

Python

encoder = RNNEncoder.initialize(rnn_type, nlayers, hidden_size, embedding)

Or

encoder = RNNEncoder(rnns, embedding, input_shape)

Build a Decoder

Similar to Encoder, we only support RNNDecoder and API is pretty much the same with RNNEncoder

Scala

val decoder = RNNDecoder(rnnType, numLayers, hiddenSize, embedding)

You can also define RNN layers yourself

val decoder = RNNDecoder(rnns, embedding, inputShape)

Python

encoder = RNNDecoder.initialize(rnn_type, nlayers, hidden_size, embedding):

Or

decoder = RNNDecoder(rnns, embedding, input_shape)

Build a Bridge

By default, encoder states are directly fed into decoder. In this case, you don't need build a Bridge. But if you want to do some transformation before feed encoder states to decoder, please use following API to create a Bridge.

Scala

val bridge = Bridge(bridgeType, decoderHiddenSize)

You can also specify various keras layers as a Bridge

val bridge = Bridge(bridge)

Python

bridge = Bridge.initialize(bridge_type, decoder_hidden_size)

Or

bridge = Bridge.initialize_from_keras_layer(bridge)

Build a Seq2seq

Scala

val seq2seq = Seq2seq(encoder,
    decoder,
    inputShape,
    outputShape,
    bridge,
    generator)

See here for the Scala example that trains the Seq2seq model and uses the model to do prediction.

Python

seq2seq = Seq2seq(encoder, decoder, input_shape, output_shape, bridge,
                 generator)