Text Matching

Analytics Zoo provides a pre-defined KNRM model that can be used for text matching (e.g. question answering). For training, you can use Keras-Style API methods or alternatively feed the model into NNFrames and BigDL Optimizer. More text matching models will be supported in the future.

Build a KNRM Model

Kernel-pooling Neural Ranking Model with RBF kernel. See here for more details.

You can call the following API in Scala and Python respectively to create a KNRM with pre-trained GloVe word embeddings.


val knrm = KNRM(text1Length, text2Length, embeddingFile, wordIndex = null, trainEmbed = true, kernelNum = 21, sigma = 0.1, exactSigma = 0.001, targetMode = "ranking")

See here for the Scala example that trains a KNRM model on WikiQA dataset.


knrm = KNRM(text1_length, text2_length, embedding_file, word_index=None, train_embed=True, kernel_num=21, sigma=0.1, exact_sigma=0.001, target_mode="ranking")

See here for the Python example that trains a KNRM model on WikiQA dataset.

Save Model

After building and training a KNRM model, you can save it for future use.


knrm.saveModel(path, weightPath = null, overWrite = false)


knrm.save_model(path, weight_path=None, over_write=False)

Load Model

To load a KNRM model (with weights) saved above:


KNRM.loadModel(path, weightPath = null)


KNRM.load_model(path, weight_path=None)