{"action":"created","issue":{"url":"https://api.github.com/repos/pymc-devs/pymc/issues/16","labels_url":"https://api.github.com/repos/pymc-devs/pymc/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/pymc-devs/pymc/issues/16/comments","events_url":"https://api.github.com/repos/pymc-devs/pymc/issues/16/events","html_url":"https://github.com/pymc-devs/pymc/issues/16","id":64771990,"number":16,"title":"Weigths of categorical variables","user":{"login":"erwindassen","id":10025190,"avatar_url":"https://avatars.githubusercontent.com/u/10025190?v=3","gravatar_id":"","url":"https://api.github.com/users/erwindassen","html_url":"https://github.com/erwindassen","followers_url":"https://api.github.com/users/erwindassen/followers","following_url":"https://api.github.com/users/erwindassen/following{/other_user}","gists_url":"https://api.github.com/users/erwindassen/gists{/gist_id}","starred_url":"https://api.github.com/users/erwindassen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erwindassen/subscriptions","organizations_url":"https://api.github.com/users/erwindassen/orgs","repos_url":"https://api.github.com/users/erwindassen/repos","events_url":"https://api.github.com/users/erwindassen/events{/privacy}","received_events_url":"https://api.github.com/users/erwindassen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"fonnesbeck","id":81476,"avatar_url":"https://avatars.githubusercontent.com/u/81476?v=3","gravatar_id":"","url":"https://api.github.com/users/fonnesbeck","html_url":"https://github.com/fonnesbeck","followers_url":"https://api.github.com/users/fonnesbeck/followers","following_url":"https://api.github.com/users/fonnesbeck/following{/other_user}","gists_url":"https://api.github.com/users/fonnesbeck/gists{/gist_id}","starred_url":"https://api.github.com/users/fonnesbeck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fonnesbeck/subscriptions","organizations_url":"https://api.github.com/users/fonnesbeck/orgs","repos_url":"https://api.github.com/users/fonnesbeck/repos","events_url":"https://api.github.com/users/fonnesbeck/events{/privacy}","received_events_url":"https://api.github.com/users/fonnesbeck/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2015-03-27T14:23:05Z","updated_at":"2015-03-28T23:12:38Z","closed_at":null,"body":"I'm having a problem with Categorical variables and I trying to find out if it a bug of mine or pymc's. It seems that categorical variables are being unevenly weighted. This can be demonstraded in the following program. It models the well-known Monty Hall [example].(http://en.wikipedia.org/wiki/Monty_Hall_problem) See the explanation below.\r\n\r\n```\r\nimport numpy as np\r\nimport pymc as pm\r\n\r\ngRange = 3\r\npRange = 3\r\nmRange = 3\r\n\r\ng = pm.Categorical('guest', np.ones(gRange)/gRange, value=2, observed=True)\r\np = pm.Categorical('prize', np.ones(pRange)/pRange)\r\n\r\n# Make a CPT table as an array of CPTColumns\r\nmCPT_shifts = {'guest': 3, 'prize': 1}\r\nmCPT = np.array((  0, 0, 0, 0, 0.5, 1, 0, 1, 0.5,\r\n                 0.5, 0, 1, 0,   0, 0, 1, 0, 0.5,\r\n                 0.5, 1, 0, 1, 0.5, 0, 0, 0,   0), dtype=float)\r\nmCPT.shape = (mRange, pRange*gRange)\r\n\r\nparents = {'guest': g, 'prize': p}\r\n\r\n# Make a node for variable monty\r\ndef m_dist(value=1, CPT_shifts=mCPT_shifts, CPT=mCPT, **parents):\r\n    col = sum(CPT_shifts[par]*parents[par] for par in parents)\r\n    return pm.categorical_like(value, p=CPT[:, col])\r\n\r\nm = pm.Stochastic(m_dist, \"Monty's door\", 'monty', parents, dtype=int,\r\n                  observed=True, value=1)\r\nmodel = pm.MCMC([g, p, m])\r\n```\r\n\r\nThe idea is to have two nodes (the prize door and the guest door) as `Categorical` variables with uniform distribution. And the third (Monty's door), depends on the first two. I implement this as a custom `Stochastic` that uses the states (values) of the parents to select a column of the *CPT* (Conditional Probability Table). It then returns a loglikelyhood for a value according to this table column.\r\n\r\nBy setting the guest evidence at 2 (door C) and Monty's door at 1 (door B) and tracing the prize door we expect 2/3 ot the prize to be at door A and only 1/3 of the prize to be at door C. Using the snippet:\r\n\r\n```\r\ntotal = 20000\r\nmodel.sample(iter=total, burn=0, thin=1)\r\ncounts = Counter(p.trace[:])\r\nratio = {var: counts[var]/total for var in counts}\r\n```\r\n\r\nI obtain the following results:\r\n\r\n`{0: 0.4979, 2: 0.5021}`\r\n\r\nEven stranger, if I just select door A as a guest and keep door B for Monty I obtain different ratios (the example should be symmetric here as can be seen from the `mCPT`)!\r\n\r\n`{0: 0.20695, 2: 0.79305}`\r\n\r\nThese numbers are stable under different settings of sample, burning, and thinning. If this is my mistake I welcome corrections. If not, it seems like pymc is drawing from the wrong distribution...\r\n"},"comment":{"url":"https://api.github.com/repos/pymc-devs/pymc/issues/comments/87327933","html_url":"https://github.com/pymc-devs/pymc/issues/16#issuecomment-87327933","issue_url":"https://api.github.com/repos/pymc-devs/pymc/issues/16","id":87327933,"user":{"login":"fonnesbeck","id":81476,"avatar_url":"https://avatars.githubusercontent.com/u/81476?v=3","gravatar_id":"","url":"https://api.github.com/users/fonnesbeck","html_url":"https://github.com/fonnesbeck","followers_url":"https://api.github.com/users/fonnesbeck/followers","following_url":"https://api.github.com/users/fonnesbeck/following{/other_user}","gists_url":"https://api.github.com/users/fonnesbeck/gists{/gist_id}","starred_url":"https://api.github.com/users/fonnesbeck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fonnesbeck/subscriptions","organizations_url":"https://api.github.com/users/fonnesbeck/orgs","repos_url":"https://api.github.com/users/fonnesbeck/repos","events_url":"https://api.github.com/users/fonnesbeck/events{/privacy}","received_events_url":"https://api.github.com/users/fonnesbeck/received_events","type":"User","site_admin":false},"created_at":"2015-03-28T23:12:38Z","updated_at":"2015-03-28T23:12:38Z","body":"I'm not clear about a couple of things here: what do you mean by \"unevenly weighted\"? The probabilities for both Categoricals, as you have set them, is array([ 0.33333333,  0.33333333,  0.33333333]), so they are evenly-weighted in that sense. Moreover, its easy to check whether the distribution is working by checking the log-probabilities for each candidate value:\r\n\r\n    In [15]: p.value = 0\r\n\r\n    In [16]: p.logp\r\n    Out[16]: -1.0986122886681098\r\n\r\n    In [17]: p.value = 1\r\n\r\n    In [18]: p.logp\r\n    Out[18]: -1.0986122886681098\r\n\r\n    In [19]: p.value = 2\r\n\r\n    In [20]: p.logp\r\n    Out[20]: -1.0986122886681098\r\n\r\nSo, the Categorical appears to be functioning as expected.\r\n\r\nOne tip: you can use the `@observed` decorator to turn any function that returns a log-probability into an observed stochastic. Thus, your code becomes:\r\n\r\n    @pm.observed\r\n    def m(value=1, CPT_shifts=mCPT_shifts, CPT=mCPT, **parents):\r\n        col = sum(CPT_shifts[par]*parents[par] for par in parents)\r\n        return pm.categorical_like(value, p=CPT[:, col])\r\n\r\nHowever, are you certain this is a valid probability function?"}}