A part of our system does a limited rule-based word labeling (kinda WSD). Basically it takes a dependency tree of a sentence, finds words or POS tags of interest and assigns to each a label/sense based on the word itself, it's POS tag, position in sentence and so forth.

I was wondering if ML methods in general and Tensorflow in particular can be used for the task, and if so which ones to try. I'm familiar with theoretical basics of ML, but never actually used it in work, so I'd grateful for any pointers.

Basically, our input is a ConLL representation of a sentence that we can turn into an matrix:

 ['cat', 'NN', 'nsubj']
 ['jumped', 'VBD', 'root']
 ['over', 'IN', 'case']
 ['fox', 'NN', 'nmod:over']

we can encode words and labels to numbers and pad sentence to max length of say 60 words, which gives the final input 60x3 matrix of integers. Our output then is a vector of 60 numbers with codes for labels for corresponding input words.

So far my research points to using bidirectional RNN for the task, but all tutorials I can find use them either for classification (as with MNIST) or finding the next best word as in the Tensorflow's RNN tutorial, whereas I want to label each word/input.

So, my questions are:

  1. Is a bidirection RNN a good choice for the task? And if so, what should the composition be for it? How many steps and hidden layers would you recommend?
  2. If RNN isn't a good choice, what else would you recommend?
  3. Would using word embeddings add to the accuracy if our labeling doesn't care for word meanings/similarities?

One solution to this problem can be found in Grammar as a Foreign Language. The paper outlines a system that "translates" English sentences into syntactic constituency parse trees using a sequence-to-sequence LSTM (Long Short-Term Memory model). A TensorFlow implementation can be found here.


Your Answer

