{"push_id":908161749,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"c8ddefc3c4ca56ac198340c14be2e220c02b9fbc","before":"77b2e0fc6f8304d06a14d742409fba03b818bea2","commits":[{"sha":"c8ddefc3c4ca56ac198340c14be2e220c02b9fbc","author":{"email":"3c5c24de8a0c3f767f86229b75979fdef7ebbdcc@yahoo-inc.com","name":"Bobby Jaros"},"message":"Prepare data for consumption by SeqToSeq LSTMs.\n\nStarts with the output of nnparse.exe, two paired files each with this format:\n         p1 s1 w1\n         p2 s2 w2\n         p3 s3 w3\n         p4 s4 w4\n         p5 s5 w5\n         p6 s6 w6\n\n(For SeqToSeq we assume each line contains one sentence, so the paragraphid\n(the first column) denotes the sentence and sentenceid (the second column)\nis always ignored).\n\nThe two parsed sentence IMats are paired line-by-line:  the ith line of the\nsrc IMat corresponds to the ith line of the dst IMat.\n\nProduces two paired SMat's of the following form:\n         w00  w01  w02  w03  w04  w05  ...\n         w10  w11  w12  w13  w14  w15P ...\n         w20  w21  w22  w23P w24  w25P ...\n         w30  w31P w32                 ...\n         w40P w32P w33                 ...\n\nwhere\n   wij is the dictionary index of the i'th word in the j'th sentence and\n   words with a P suffix are padding symbols.\n\nThe columns of the two output SMat's are still paired:  column j of the\nsrc output SMat and column j of the dst output SMat correspond to line j\nof the src input and line j of the dst input respectively.\n\nFurthermore, the sentences are collated into batches of similar lengths.\n\nThe minibatches are randomly permuted after collation to avoid training bias.\n\nSee in-file docs for additional options.","distinct":true,"url":"https://api.github.com/repos/bobbyjaros/BIDMach/commits/c8ddefc3c4ca56ac198340c14be2e220c02b9fbc"}]}