{"action":"created","issue":{"url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/803","labels_url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/803/labels{/name}","comments_url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/803/comments","events_url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/803/events","html_url":"https://github.com/JohnLangford/vowpal_wabbit/issues/803","id":108349041,"number":803,"title":"In pyvw how do I predict an LDA example without learning it?","user":{"login":"jmizgajski","id":3886340,"avatar_url":"https://avatars.githubusercontent.com/u/3886340?v=3","gravatar_id":"","url":"https://api.github.com/users/jmizgajski","html_url":"https://github.com/jmizgajski","followers_url":"https://api.github.com/users/jmizgajski/followers","following_url":"https://api.github.com/users/jmizgajski/following{/other_user}","gists_url":"https://api.github.com/users/jmizgajski/gists{/gist_id}","starred_url":"https://api.github.com/users/jmizgajski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmizgajski/subscriptions","organizations_url":"https://api.github.com/users/jmizgajski/orgs","repos_url":"https://api.github.com/users/jmizgajski/repos","events_url":"https://api.github.com/users/jmizgajski/events{/privacy}","received_events_url":"https://api.github.com/users/jmizgajski/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-25T15:36:37Z","updated_at":"2015-09-26T15:40:00Z","closed_at":null,"body":"vw uses an online LDA algorithm. By this virtue learning an example changes the model. In many cases this is undesired at prediction time (learning the same document multiple times can skew the model considerably).\r\n\r\nWhen using pyvw how do I predict the topic distribution without learning the predicted example?\r\n\r\nConsider this code as a template:\r\n\r\n```\r\nimport pyvw\r\nfrom itertools import imap\r\nimport array\r\nimport numpy as np\r\n\r\n\r\nclass VwLda(object):\r\n\r\n    @classmethod\r\n    def make_line(cls, bow):\r\n        if issubclass(type(bow), dict):\r\n            return \"|\" + \" \".join(imap(lambda (w, c): \"%s:%s\" % (w, c), bow.iteritems()))\r\n        if issubclass(type(bow), list) or issubclass(type(bow), array) or issubclass(type(bow), set):\r\n            return \"|\" + \" \".join(imap(lambda w: \"%s:1\" % w, bow))\r\n\r\n    def __init__(self, k):\r\n        self.vw = pyvw.vw(lda=k)\r\n        self.k = k\r\n\r\n    def learn(self, docs):\r\n        for d in docs:\r\n            ex = self.vw.example(self.make_line(d))\r\n            ex.learn()\r\n            ex.finish()\r\n\r\n    def predict(self, doc):\r\n        ex = self.vw.example(self.make_line(doc))\r\n\r\n        #FIXME: !!! I don't know how to predict an example without actualy learning it\r\n        ex.learn()\r\n        topic_probabilities = np.array([ex.get_topic_prediction(i) for i in xrange(self.k)])\r\n        return topic_probabilities / sum(topic_probabilities)\r\n\r\n```\r\nIf I simply remove ex.learn() the notebook kernel crashes. I couldn't find any method inside the pyvw source that would be equivalent to ex.learn() but have no influence on the model."},"comment":{"url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/comments/143464478","html_url":"https://github.com/JohnLangford/vowpal_wabbit/issues/803#issuecomment-143464478","issue_url":"https://api.github.com/repos/JohnLangford/vowpal_wabbit/issues/803","id":143464478,"user":{"login":"JohnLangford","id":79590,"avatar_url":"https://avatars.githubusercontent.com/u/79590?v=3","gravatar_id":"","url":"https://api.github.com/users/JohnLangford","html_url":"https://github.com/JohnLangford","followers_url":"https://api.github.com/users/JohnLangford/followers","following_url":"https://api.github.com/users/JohnLangford/following{/other_user}","gists_url":"https://api.github.com/users/JohnLangford/gists{/gist_id}","starred_url":"https://api.github.com/users/JohnLangford/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JohnLangford/subscriptions","organizations_url":"https://api.github.com/users/JohnLangford/orgs","repos_url":"https://api.github.com/users/JohnLangford/repos","events_url":"https://api.github.com/users/JohnLangford/events{/privacy}","received_events_url":"https://api.github.com/users/JohnLangford/received_events","type":"User","site_admin":false},"created_at":"2015-09-26T15:40:00Z","updated_at":"2015-09-26T15:40:00Z","body":"Apparently, you set ec->test_only:\n\n|https://github.com/JohnLangford/vowpal_wabbit/blob/master/python/pylibvw.cc#L107\nhttps://github.com/JohnLangford/vowpal_wabbit/blob/master/python/pylibvw.cc#L488\n\nHal, it might be better to have a predict() function since that is: (a) \nless modal and (b) more directly guaranteed to do the right thing.\n|\n-John\n\nOn 09/25/2015 11:36 AM, jmizgajski wrote:\n>\n> vw uses an online LDA algorithm. By this virtue learning an example \n> changes the model. In many cases this is undesired at prediction time \n> (learning the same document multiple times can skew the model \n> considerably).\n>\n> When using pyvw how do I predict the topic distribution without \n> learning the predicted example?\n>\n> Consider this code as a template:\n>\n> |import pyvw\n> from itertools import imap\n> import array\n> import numpy as np\n>\n>\n> class VwLda(object):\n>\n>      @classmethod\n>      def make_line(cls, bow):\n>          if issubclass(type(bow), dict):\n>              return \"|\" + \" \".join(imap(lambda (w, c): \"%s:%s\" % (w, c), bow.iteritems()))\n>          if issubclass(type(bow), list) or issubclass(type(bow), array) or issubclass(type(bow), set):\n>              return \"|\" + \" \".join(imap(lambda w: \"%s:1\" % w, bow))\n>\n>      def __init__(self, k):\n>          self.vw = pyvw.vw(lda=k)\n>          self.k = k\n>\n>      def learn(self, docs):\n>          for d in docs:\n>              ex = self.vw.example(self.make_line(d))\n>              ex.learn()\n>              ex.finish()\n>\n>      def predict(self, doc):\n>          ex = self.vw.example(self.make_line(doc))\n>\n>          #FIXME: !!! I don't know how to predict an example without actualy learning it\n>          ex.learn()\n>          topic_probabilities = np.array([ex.get_topic_prediction(i) for i in xrange(self.k)])\n>          return topic_probabilities / sum(topic_probabilities)\n>\n> |\n>\n> If I simply remove ex.learn() the notebook kernel crashes. I couldn't \n> find any method inside the pyvw source that would be equivalent to \n> ex.learn() but have no influence on the model.\n>\n> —\n> Reply to this email directly or view it on GitHub \n> <https://github.com/JohnLangford/vowpal_wabbit/issues/803>.\n>\n\n"}}