{"action":"created","comment":{"url":"https://api.github.com/repos/oppia/oppia/pulls/comments/41600873","id":41600873,"diff_hunk":"@@ -0,0 +1,534 @@\n+# coding: utf-8\n+#\n+# Copyright 2015 The Oppia Authors. All Rights Reserved.\n+#\n+# Licensed under the Apache License, Version 2.0 (the \"License\");\n+# you may not use this file except in compliance with the License.\n+# You may obtain a copy of the License at\n+#\n+#      http://www.apache.org/licenses/LICENSE-2.0\n+#\n+# Unless required by applicable law or agreed to in writing, software\n+# distributed under the License is distributed on an \"AS-IS\" BASIS,\n+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n+# See the License for the specific language governing permissions and\n+# limitations under the License.\n+\n+import copy\n+import operator\n+\n+import numpy\n+\n+\n+class StringClassifier(object):\n+    \"\"\"\n+    A classifier that uses supervised learning to match free-form text answers\n+    to answer groups. The classifier trains on answers that exploration editors\n+    have assigned to an answer group. Given a new answer, it predicts the\n+    answer group using Latent Dirichlet Allocation\n+    (https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation) with Gibbs\n+    Sampling.\n+\n+    Below is an example workflow for running a batch job that builds a new\n+    classifier and outputs it to a dictionary.\n+\n+        # Examples are formatted as a list. Each element is a doc followed by a\n+        # list of labels.\n+        examples = [\n+            ['i eat fish and vegetables', ['food']],\n+            ['fish are pets', ['pets']],\n+            ['my kitten eats fish', ['food', 'pets']]\n+        ]\n+        string_classifier = StringClassifier()\n+        string_classifier.load_examples(examples)\n+        classifier_dict = string_classifier.to_dict()\n+        save_to_data_store(classifier_dict)\n+\n+    Below is an example workflow for using an existing classifier to predict a\n+    document's label.\n+\n+        # A list of docs to classify.\n+        prediction_examples = [\n+            'i only eat fish and vegetables'\n+        ]\n+        classifier_dict = load_from_data_store()\n+        string_classifier = StringClassifier()\n+        string_classifier.from_dict(classifier_dict)\n+        doc_ids = string_classifier.add_examples_for_predicting(\n+            prediction_examples)\n+        label = string_classifier.predict_label_for_doc(doc_ids[0])\n+        print label\n+\n+    Below are some concepts used in this class.\n+    doc - A student free form response, represented as a string of arbitrary\n+        non-whitespace characters (a \"word\") separated by single spaces.\n+    word - A string of arbitrary non-whitespace characters.\n+    word id - A unique word. Each unique word has one word id that is used\n+        to represent the word in the classifier model.\n+    word instance - An instance of a word id. Each word id can have multiple\n+        instances corresponding to its occurrences in all docs.\n+    label - An answer group that the doc should correspond to. Labels should\n+        not begin with a '_' (underscore). If a doc is being added to train\n+        a model, labels are provided. If a doc is being added for prediction\n+        purposes, no labels are provided.\n+    label bit vector - A bit vector corresponding to a doc, indexed by label\n+        id. A one in the vector means the label id matches some word instance\n+        in the doc; a zero in the vector means the label id does not match any\n+        word instance in the doc.\n+\n+    This class uses index notation with the format \"_V_XY\", where V is the\n+    element of an array, and X and Y are the indices used to measure V.\n+    https://en.wikipedia.org/wiki/Index_notation\n+\n+    The following maps index notation letters to their meanings:\n+    b - boolean value for whether Y is set in X\n+    c - count of Y's in X\n+    p - position of a value V in X\n+    l - label id\n+    w - word id\n+    d - doc id\n+\n+    Internal model representation:\n+    _w_dp - lists of word ids, where each list represents a doc\n+    _b_dl - lists of booleans, where each boolean represents whether a\n+        label is contained within a doc.\n+    _l_dp - lists of label ids, where each list represents what label\n+        is assigned to a word instance.\n+    _c_dl - lists of counts, where each list represents the number of\n+        word instances assigned to each label in a doc.\n+    _c_lw - lists of counts, where each list represents the number of\n+        word instances assigned to each label for a given word id (aggregated\n+        across all docs).\n+    _c_l - a list of counts, where each count is the number of times\n+        a label was assigned (aggregated over all word instances in all docs).\n+\n+    There are two internal learning rates, _DEFAULT_ALPHA and _DEFAULT_BETA.\n+    These are initialized to Wikipedia's recommendations. Do not change these\n+    unless you know what you're doing.\n+\n+    It is possible for a word instance in a doc to not have an explicit label\n+    assigned to it. This is characterized by assigning _DEFAULT_LABEL to the\n+    word instance.\n+    \"\"\"\n+\n+    _DEFAULT_ALPHA = 0.1\n+    _DEFAULT_BETA = 0.001\n+\n+    _DEFAULT_TRAINING_ITERATIONS = 25\n+    _DEFAULT_PREDICTION_ITERATIONS = 5\n+\n+    _DEFAULT_PREDICTION_THRESHOLD = 0.5\n+\n+    _DEFAULT_LABEL = '_default'\n+\n+    def __init__(self):\n+        \"\"\"\n+        Initializes constants for the classifier. Setting a seed ensures\n+        that results are deterministic. There is nothing special about the\n+        value 4. These should not be changed unless you know what you're doing.\n+        \"\"\"\n+        numpy.random.seed(seed=4)\n+\n+        self._alpha = self._DEFAULT_ALPHA\n+        self._beta = self._DEFAULT_BETA\n+\n+        self._training_iterations = self._DEFAULT_TRAINING_ITERATIONS\n+        self._prediction_iterations = self._DEFAULT_PREDICTION_ITERATIONS\n+\n+        self._prediction_threshold = self._DEFAULT_PREDICTION_THRESHOLD\n+\n+    def _get_word_id(self, word):\n+        \"\"\"\n+        Returns a word's id if it exists, otherwise assigns\n+        a new id to the word and returns it.\n+        \"\"\"\n+        if word not in self._word_to_id:\n+            self._word_to_id[word] = self._num_words\n+            self._num_words += 1\n+        return self._word_to_id[word]\n+\n+    def _get_label_id(self, label):\n+        \"\"\"\n+        Returns a label's id if it exists, otherwise assigns\n+        a new id to the label and returns it.\n+        \"\"\"\n+        if label not in self._label_to_id:\n+            self._label_to_id[label] = self._num_labels\n+            self._num_labels += 1\n+        return self._label_to_id[label]\n+\n+    def _get_label_name(self, l):\n+        \"\"\"\n+        Returns a label's string name given its internal id.\n+        If the id does not have a corresponding name, an exception is raised.\n+        \"\"\"\n+        for label_name, label_id in self._label_to_id.iteritems():\n+            if label_id == l:\n+                return label_name\n+\n+        raise Exception('Label id %d does not exist.' % l)\n+\n+    def _get_doc_with_label_vector(self, d):\n+        \"\"\"\n+        Given a doc id, return the doc and its label bit vector.\n+        \"\"\"\n+        return self._w_dp[d], self._b_dl[d]\n+\n+    def _get_label_vector(self, labels):\n+        \"\"\"\n+        Given a list of labels, return the label bit vector.\n+        An empty label list is an alias for all labels being set. This is\n+        useful because otherwise when a new label is introduce to the model,\n+        none of the input data needs to be altered.\n+        \"\"\"\n+        if len(labels) == 0:\n+            return numpy.ones(self._num_labels)\n+        label_vector = numpy.zeros(self._num_labels)\n+        for label in labels:\n+            label_vector[self._get_label_id(label)] = 1\n+        label_vector[self._label_to_id[self._DEFAULT_LABEL]] = 1\n+        return label_vector\n+\n+    def _update_counting_matrices(self, d, w, l, val):\n+        \"\"\"\n+        Updates counting matrices (ones that begin with _c) when a label\n+        is assigned and unassigned to a word.\n+        \"\"\"\n+        self._c_dl[d, l] += val\n+        self._c_lw[l, w] += val\n+        self._c_l[l] += val\n+\n+    def _increment_counting_matrices(self, d, w, l):\n+        \"\"\"\n+        Assign a label to a word in a doc.\n+        \"\"\"\n+        self._update_counting_matrices(d, w, l, 1)\n+\n+    def _decrement_counting_matrices(self, d, w, l):\n+        \"\"\"\n+        Unassign a label from a word in a doc.\n+        \"\"\"\n+        self._update_counting_matrices(d, w, l, -1)\n+\n+    def _get_doc_ids(self, doc_ids):\n+        \"\"\"\n+        Helper function for methods that iterate over doc ids.\n+        If no doc ids are provided, then all doc ids are iterated over.\n+        This simplifies logic closer to the client.\n+        \"\"\"\n+        if doc_ids is None:\n+            doc_ids = xrange(self._num_docs)\n+        return doc_ids\n+\n+    def _assign_initial_labels_to_word_instances(self, doc_ids=None):\n+        \"\"\"\n+        Uniformly assigns labels to word instances in a new document. This\n+        prevents the model from converging to a non-useful state when training.\n+        \"\"\"\n+        doc_ids = self._get_doc_ids(doc_ids)\n+\n+        for d in doc_ids:\n+            doc, labels = self._get_doc_with_label_vector(d)\n+            l_p = [\n+                numpy.random.multinomial(1, labels / labels.sum()).argmax()\n+                for _ in xrange(len(doc))\n+            ]\n+            self._l_dp.append(l_p)\n+            for w, l in zip(doc, l_p):\n+                self._increment_counting_matrices(d, w, l)\n+\n+    def _run_gibbs_sampling(self, doc_ids):\n+        \"\"\"\n+        Runs one iteration of Gibbs sampling on the provided docs.\n+        The statez variable is used for debugging, and possibly convergence\n+        testing in the future.\n+        \"\"\"\n+        doc_ids = self._get_doc_ids(doc_ids)\n+\n+        statez = {\n+            'updates': 0,\n+            'computes': 0\n+        }\n+\n+        for d in doc_ids:\n+            doc, labels = self._get_doc_with_label_vector(d)\n+            for p, w in enumerate(doc):\n+                l = self._l_dp[d][p]\n+\n+                self._decrement_counting_matrices(d, w, l)\n+\n+                coeff_a = 1.0 / (\n+                    self._c_dl[d].sum() + self._num_labels * self._alpha)\n+                coeff_b = 1.0 / (\n+                    self._c_lw.sum(axis=1) + self._num_words * self._beta)\n+                label_probabilities = (\n+                    labels *\n+                    coeff_a * (self._c_dl[d] + self._alpha) *\n+                    coeff_b * (self._c_lw[:, w] + self._beta))\n+                new_label = numpy.random.multinomial(\n+                    1,\n+                    label_probabilities / label_probabilities.sum()).argmax()\n+\n+                statez['computes'] += 1\n+                if l != new_label:\n+                    statez['updates'] += 1\n+\n+                self._l_dp[d][p] = new_label\n+                self._increment_counting_matrices(d, w, new_label)\n+\n+        return statez\n+\n+    def _get_label_probabilities(self, d):\n+        \"\"\"\n+        Returns a list of label probabilities for a given doc, indexed by\n+        label id.\n+        \"\"\"\n+        probs_with_label_id = self._c_dl[d] + (self._b_dl[d] * self._alpha)\n+        probs_with_label_id = (probs_with_label_id /\n+            probs_with_label_id.sum(axis=0)[numpy.newaxis])\n+        return probs_with_label_id\n+\n+    def _get_prediction_report_for_doc(self, d):\n+        \"\"\"Generates and returns a prediction report for a given doc.\n+\n+        The prediction report is a dict with the following keys:\n+        - 'prediction_label': the document's predicted label\n+        - 'prediction_confidence': the prediction confidence. This is\n+        Prob(the doc should be assigned this label |\n+        the doc is not assigned _DEFAULT_LABEL).\n+        - 'all_predictions': a dict mapping each label to\n+        Prob(the doc should be assigned to this label).\n+\n+        Because _DEFAULT_LABEL is a special label that captures unspecial\n+        tokens (how ironic), its probability is not a good predicting\n+        indicator. For the current normalization process, it has on\n+        average a higher probability than other labels. To combat this,\n+        all other labels are normalized as if the default label did not\n+        exist. For example, if we have two labels with probability 0.2\n+        and 0.3 with the default label having probability 0.5, the\n+        normalized probability for the two labels without the default\n+        label is 0.4 and 0.6, respectively.\n+\n+        The prediction threshold is currently defined at the classifier level.\n+        A higher prediction threshold indicates that the predictor needs\n+        more confidence prior to making a prediction, otherwise it will\n+        predict _DEFAULT_LABEL. This will make non-default predictions more\n+        accurate, but result in fewer of them.\n+        \"\"\"\n+        default_label_id = self._get_label_id(self._DEFAULT_LABEL)\n+        prediction_label_id = default_label_id\n+        prediction_confidence = 0\n+        probs_with_label_id = self._get_label_probabilities(d)\n+        default_prob = probs_with_label_id[default_label_id]\n+        default_prob_comp = 1.0 - default_prob\n+\n+        for l, prob in enumerate(probs_with_label_id):\n+            if (l != default_label_id and\n+                    prob / default_prob_comp > self._prediction_threshold):","path":"core/domain/classifier_services.py","position":327,"original_position":327,"commit_id":"43c8acc9478eeb3a2cf485f9beb8d7e9fda31738","original_commit_id":"43c8acc9478eeb3a2cf485f9beb8d7e9fda31738","user":{"login":"seanlip","id":10575562,"avatar_url":"https://avatars.githubusercontent.com/u/10575562?v=3","gravatar_id":"","url":"https://api.github.com/users/seanlip","html_url":"https://github.com/seanlip","followers_url":"https://api.github.com/users/seanlip/followers","following_url":"https://api.github.com/users/seanlip/following{/other_user}","gists_url":"https://api.github.com/users/seanlip/gists{/gist_id}","starred_url":"https://api.github.com/users/seanlip/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seanlip/subscriptions","organizations_url":"https://api.github.com/users/seanlip/orgs","repos_url":"https://api.github.com/users/seanlip/repos","events_url":"https://api.github.com/users/seanlip/events{/privacy}","received_events_url":"https://api.github.com/users/seanlip/received_events","type":"User","site_admin":false},"body":"(Also, there should probably be a test to catch this.)","created_at":"2015-10-09T06:28:42Z","updated_at":"2015-10-09T06:28:42Z","html_url":"https://github.com/oppia/oppia/pull/1008#discussion_r41600873","pull_request_url":"https://api.github.com/repos/oppia/oppia/pulls/1008","_links":{"self":{"href":"https://api.github.com/repos/oppia/oppia/pulls/comments/41600873"},"html":{"href":"https://github.com/oppia/oppia/pull/1008#discussion_r41600873"},"pull_request":{"href":"https://api.github.com/repos/oppia/oppia/pulls/1008"}}},"pull_request":{"url":"https://api.github.com/repos/oppia/oppia/pulls/1008","id":46720584,"html_url":"https://github.com/oppia/oppia/pull/1008","diff_url":"https://github.com/oppia/oppia/pull/1008.diff","patch_url":"https://github.com/oppia/oppia/pull/1008.patch","issue_url":"https://api.github.com/repos/oppia/oppia/issues/1008","number":1008,"state":"open","locked":false,"title":"StringClassifier","user":{"login":"stephenchiang","id":3666806,"avatar_url":"https://avatars.githubusercontent.com/u/3666806?v=3","gravatar_id":"","url":"https://api.github.com/users/stephenchiang","html_url":"https://github.com/stephenchiang","followers_url":"https://api.github.com/users/stephenchiang/followers","following_url":"https://api.github.com/users/stephenchiang/following{/other_user}","gists_url":"https://api.github.com/users/stephenchiang/gists{/gist_id}","starred_url":"https://api.github.com/users/stephenchiang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stephenchiang/subscriptions","organizations_url":"https://api.github.com/users/stephenchiang/orgs","repos_url":"https://api.github.com/users/stephenchiang/repos","events_url":"https://api.github.com/users/stephenchiang/events{/privacy}","received_events_url":"https://api.github.com/users/stephenchiang/received_events","type":"User","site_admin":false},"body":"Supervised string classifier to be used as a black box to predict student free form responses.\r\n\r\n/cc @seanlip @BenHenning \r\n","created_at":"2015-10-04T07:00:13Z","updated_at":"2015-10-09T06:28:42Z","closed_at":null,"merged_at":null,"merge_commit_sha":"ef5413d1294fb23aad81b683590a5965eac1e00b","assignee":{"login":"seanlip","id":10575562,"avatar_url":"https://avatars.githubusercontent.com/u/10575562?v=3","gravatar_id":"","url":"https://api.github.com/users/seanlip","html_url":"https://github.com/seanlip","followers_url":"https://api.github.com/users/seanlip/followers","following_url":"https://api.github.com/users/seanlip/following{/other_user}","gists_url":"https://api.github.com/users/seanlip/gists{/gist_id}","starred_url":"https://api.github.com/users/seanlip/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seanlip/subscriptions","organizations_url":"https://api.github.com/users/seanlip/orgs","repos_url":"https://api.github.com/users/seanlip/repos","events_url":"https://api.github.com/users/seanlip/events{/privacy}","received_events_url":"https://api.github.com/users/seanlip/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/oppia/oppia/pulls/1008/commits","review_comments_url":"https://api.github.com/repos/oppia/oppia/pulls/1008/comments","review_comment_url":"https://api.github.com/repos/oppia/oppia/pulls/comments{/number}","comments_url":"https://api.github.com/repos/oppia/oppia/issues/1008/comments","statuses_url":"https://api.github.com/repos/oppia/oppia/statuses/43c8acc9478eeb3a2cf485f9beb8d7e9fda31738","head":{"label":"stephenchiang:classifier-services","ref":"classifier-services","sha":"43c8acc9478eeb3a2cf485f9beb8d7e9fda31738","user":{"login":"stephenchiang","id":3666806,"avatar_url":"https://avatars.githubusercontent.com/u/3666806?v=3","gravatar_id":"","url":"https://api.github.com/users/stephenchiang","html_url":"https://github.com/stephenchiang","followers_url":"https://api.github.com/users/stephenchiang/followers","following_url":"https://api.github.com/users/stephenchiang/following{/other_user}","gists_url":"https://api.github.com/users/stephenchiang/gists{/gist_id}","starred_url":"https://api.github.com/users/stephenchiang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stephenchiang/subscriptions","organizations_url":"https://api.github.com/users/stephenchiang/orgs","repos_url":"https://api.github.com/users/stephenchiang/repos","events_url":"https://api.github.com/users/stephenchiang/events{/privacy}","received_events_url":"https://api.github.com/users/stephenchiang/received_events","type":"User","site_admin":false},"repo":{"id":43627434,"name":"oppia","full_name":"stephenchiang/oppia","owner":{"login":"stephenchiang","id":3666806,"avatar_url":"https://avatars.githubusercontent.com/u/3666806?v=3","gravatar_id":"","url":"https://api.github.com/users/stephenchiang","html_url":"https://github.com/stephenchiang","followers_url":"https://api.github.com/users/stephenchiang/followers","following_url":"https://api.github.com/users/stephenchiang/following{/other_user}","gists_url":"https://api.github.com/users/stephenchiang/gists{/gist_id}","starred_url":"https://api.github.com/users/stephenchiang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stephenchiang/subscriptions","organizations_url":"https://api.github.com/users/stephenchiang/orgs","repos_url":"https://api.github.com/users/stephenchiang/repos","events_url":"https://api.github.com/users/stephenchiang/events{/privacy}","received_events_url":"https://api.github.com/users/stephenchiang/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/stephenchiang/oppia","description":"Tool for collaboratively building interactive tutors.","fork":true,"url":"https://api.github.com/repos/stephenchiang/oppia","forks_url":"https://api.github.com/repos/stephenchiang/oppia/forks","keys_url":"https://api.github.com/repos/stephenchiang/oppia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/stephenchiang/oppia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/stephenchiang/oppia/teams","hooks_url":"https://api.github.com/repos/stephenchiang/oppia/hooks","issue_events_url":"https://api.github.com/repos/stephenchiang/oppia/issues/events{/number}","events_url":"https://api.github.com/repos/stephenchiang/oppia/events","assignees_url":"https://api.github.com/repos/stephenchiang/oppia/assignees{/user}","branches_url":"https://api.github.com/repos/stephenchiang/oppia/branches{/branch}","tags_url":"https://api.github.com/repos/stephenchiang/oppia/tags","blobs_url":"https://api.github.com/repos/stephenchiang/oppia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/stephenchiang/oppia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/stephenchiang/oppia/git/refs{/sha}","trees_url":"https://api.github.com/repos/stephenchiang/oppia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/stephenchiang/oppia/statuses/{sha}","languages_url":"https://api.github.com/repos/stephenchiang/oppia/languages","stargazers_url":"https://api.github.com/repos/stephenchiang/oppia/stargazers","contributors_url":"https://api.github.com/repos/stephenchiang/oppia/contributors","subscribers_url":"https://api.github.com/repos/stephenchiang/oppia/subscribers","subscription_url":"https://api.github.com/repos/stephenchiang/oppia/subscription","commits_url":"https://api.github.com/repos/stephenchiang/oppia/commits{/sha}","git_commits_url":"https://api.github.com/repos/stephenchiang/oppia/git/commits{/sha}","comments_url":"https://api.github.com/repos/stephenchiang/oppia/comments{/number}","issue_comment_url":"https://api.github.com/repos/stephenchiang/oppia/issues/comments{/number}","contents_url":"https://api.github.com/repos/stephenchiang/oppia/contents/{+path}","compare_url":"https://api.github.com/repos/stephenchiang/oppia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/stephenchiang/oppia/merges","archive_url":"https://api.github.com/repos/stephenchiang/oppia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/stephenchiang/oppia/downloads","issues_url":"https://api.github.com/repos/stephenchiang/oppia/issues{/number}","pulls_url":"https://api.github.com/repos/stephenchiang/oppia/pulls{/number}","milestones_url":"https://api.github.com/repos/stephenchiang/oppia/milestones{/number}","notifications_url":"https://api.github.com/repos/stephenchiang/oppia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/stephenchiang/oppia/labels{/name}","releases_url":"https://api.github.com/repos/stephenchiang/oppia/releases{/id}","created_at":"2015-10-04T06:32:57Z","updated_at":"2015-10-04T06:33:01Z","pushed_at":"2015-10-09T00:19:36Z","git_url":"git://github.com/stephenchiang/oppia.git","ssh_url":"git@github.com:stephenchiang/oppia.git","clone_url":"https://github.com/stephenchiang/oppia.git","svn_url":"https://github.com/stephenchiang/oppia","homepage":"https://www.oppia.org","size":37717,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":false,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"oppia:develop","ref":"develop","sha":"cbacb052cfca450e94adbdad51685d8b5663dd04","user":{"login":"oppia","id":11620230,"avatar_url":"https://avatars.githubusercontent.com/u/11620230?v=3","gravatar_id":"","url":"https://api.github.com/users/oppia","html_url":"https://github.com/oppia","followers_url":"https://api.github.com/users/oppia/followers","following_url":"https://api.github.com/users/oppia/following{/other_user}","gists_url":"https://api.github.com/users/oppia/gists{/gist_id}","starred_url":"https://api.github.com/users/oppia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oppia/subscriptions","organizations_url":"https://api.github.com/users/oppia/orgs","repos_url":"https://api.github.com/users/oppia/repos","events_url":"https://api.github.com/users/oppia/events{/privacy}","received_events_url":"https://api.github.com/users/oppia/received_events","type":"Organization","site_admin":false},"repo":{"id":40687563,"name":"oppia","full_name":"oppia/oppia","owner":{"login":"oppia","id":11620230,"avatar_url":"https://avatars.githubusercontent.com/u/11620230?v=3","gravatar_id":"","url":"https://api.github.com/users/oppia","html_url":"https://github.com/oppia","followers_url":"https://api.github.com/users/oppia/followers","following_url":"https://api.github.com/users/oppia/following{/other_user}","gists_url":"https://api.github.com/users/oppia/gists{/gist_id}","starred_url":"https://api.github.com/users/oppia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oppia/subscriptions","organizations_url":"https://api.github.com/users/oppia/orgs","repos_url":"https://api.github.com/users/oppia/repos","events_url":"https://api.github.com/users/oppia/events{/privacy}","received_events_url":"https://api.github.com/users/oppia/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/oppia/oppia","description":"Tool for collaboratively building interactive tutors.","fork":false,"url":"https://api.github.com/repos/oppia/oppia","forks_url":"https://api.github.com/repos/oppia/oppia/forks","keys_url":"https://api.github.com/repos/oppia/oppia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/oppia/oppia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/oppia/oppia/teams","hooks_url":"https://api.github.com/repos/oppia/oppia/hooks","issue_events_url":"https://api.github.com/repos/oppia/oppia/issues/events{/number}","events_url":"https://api.github.com/repos/oppia/oppia/events","assignees_url":"https://api.github.com/repos/oppia/oppia/assignees{/user}","branches_url":"https://api.github.com/repos/oppia/oppia/branches{/branch}","tags_url":"https://api.github.com/repos/oppia/oppia/tags","blobs_url":"https://api.github.com/repos/oppia/oppia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/oppia/oppia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/oppia/oppia/git/refs{/sha}","trees_url":"https://api.github.com/repos/oppia/oppia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/oppia/oppia/statuses/{sha}","languages_url":"https://api.github.com/repos/oppia/oppia/languages","stargazers_url":"https://api.github.com/repos/oppia/oppia/stargazers","contributors_url":"https://api.github.com/repos/oppia/oppia/contributors","subscribers_url":"https://api.github.com/repos/oppia/oppia/subscribers","subscription_url":"https://api.github.com/repos/oppia/oppia/subscription","commits_url":"https://api.github.com/repos/oppia/oppia/commits{/sha}","git_commits_url":"https://api.github.com/repos/oppia/oppia/git/commits{/sha}","comments_url":"https://api.github.com/repos/oppia/oppia/comments{/number}","issue_comment_url":"https://api.github.com/repos/oppia/oppia/issues/comments{/number}","contents_url":"https://api.github.com/repos/oppia/oppia/contents/{+path}","compare_url":"https://api.github.com/repos/oppia/oppia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/oppia/oppia/merges","archive_url":"https://api.github.com/repos/oppia/oppia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/oppia/oppia/downloads","issues_url":"https://api.github.com/repos/oppia/oppia/issues{/number}","pulls_url":"https://api.github.com/repos/oppia/oppia/pulls{/number}","milestones_url":"https://api.github.com/repos/oppia/oppia/milestones{/number}","notifications_url":"https://api.github.com/repos/oppia/oppia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/oppia/oppia/labels{/name}","releases_url":"https://api.github.com/repos/oppia/oppia/releases{/id}","created_at":"2015-08-14T00:16:14Z","updated_at":"2015-10-06T09:18:56Z","pushed_at":"2015-10-09T06:26:53Z","git_url":"git://github.com/oppia/oppia.git","ssh_url":"git@github.com:oppia/oppia.git","clone_url":"https://github.com/oppia/oppia.git","svn_url":"https://github.com/oppia/oppia","homepage":"https://www.oppia.org","size":43589,"stargazers_count":44,"watchers_count":44,"language":"Python","has_issues":true,"has_downloads":false,"has_wiki":true,"has_pages":false,"forks_count":54,"mirror_url":null,"open_issues_count":177,"forks":54,"open_issues":177,"watchers":44,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/oppia/oppia/pulls/1008"},"html":{"href":"https://github.com/oppia/oppia/pull/1008"},"issue":{"href":"https://api.github.com/repos/oppia/oppia/issues/1008"},"comments":{"href":"https://api.github.com/repos/oppia/oppia/issues/1008/comments"},"review_comments":{"href":"https://api.github.com/repos/oppia/oppia/pulls/1008/comments"},"review_comment":{"href":"https://api.github.com/repos/oppia/oppia/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/oppia/oppia/pulls/1008/commits"},"statuses":{"href":"https://api.github.com/repos/oppia/oppia/statuses/43c8acc9478eeb3a2cf485f9beb8d7e9fda31738"}}}}