{"action":"created","issue":{"url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/64","labels_url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/64/comments","events_url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/64/events","html_url":"https://github.com/JuliaOpt/Convex.jl/issues/64","id":56837968,"number":64,"title":"Problem with solution of maximum entropy problem (or request for documentation)","user":{"login":"gajomi","id":244887,"avatar_url":"https://avatars.githubusercontent.com/u/244887?v=3","gravatar_id":"","url":"https://api.github.com/users/gajomi","html_url":"https://github.com/gajomi","followers_url":"https://api.github.com/users/gajomi/followers","following_url":"https://api.github.com/users/gajomi/following{/other_user}","gists_url":"https://api.github.com/users/gajomi/gists{/gist_id}","starred_url":"https://api.github.com/users/gajomi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gajomi/subscriptions","organizations_url":"https://api.github.com/users/gajomi/orgs","repos_url":"https://api.github.com/users/gajomi/repos","events_url":"https://api.github.com/users/gajomi/events{/privacy}","received_events_url":"https://api.github.com/users/gajomi/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-02-06T17:00:41Z","updated_at":"2015-02-08T16:06:00Z","closed_at":"2015-02-08T16:06:00Z","body":"I thought it would be nice to contribute a pair of examples for finding exponential family distributions (http://en.wikipedia.org/wiki/Exponential_family) with a given set of features and expectation values. The first example would consist of maximizing the relative entropy between the target distribution and a base measure under constraints n the expectation values. The second example would be the same thing, but where the maximization occurs on a much lower dimensional dual space (the Langrange multipliers that fix expectation values for a set of chosen features), which presumably is the easier problem. The point of the example would be to show that the two methods yield the same result.\r\n\r\nHere is a bit of code that (at least initially), appeared to solve the first part of the problem for the case where the solution should be a Binomial distribution\r\n\r\n```julia\r\n\r\nusing Gadfly\r\nusing Distributions\r\nusing Convex, SCS\r\n\r\n#space and probalities\r\nN = 32\r\nX = 0:N\r\np = Variable(length(X))\r\nprobability_contraints = [0 <= p, p <= 1, sum(p) == 1];\r\n#base measure\r\nh = pdf(Binomial(N,1/2))\r\n#feature expectation values\r\nq = .3\r\nμ =N*q\r\nfs = [identity]\r\nη = [μ]\r\nC = hcat([map(f,X) for f in fs]...)'\r\nfeature_constraints = C * p == η\r\n#definte optimization problem and solve\r\nproblem = maximize(entropy(p)+dot(p,log(h)),probability_contraints...,feature_constraints)\r\nsolve!(problem, SCSSolver(verbose=0))\r\n#plot solution\r\nplot(layer(x=X,y=p.value,Geom.point),\r\n     layer(x=X,y=pdf(Binomial(N,q)),Geom.bar),Scale.x_discrete)\r\n```\r\n![example1](https://cloud.githubusercontent.com/assets/244887/6083265/92c3580e-ae27-11e4-9a29-fb4356961411.png)\r\n\r\nHowever, trying out a larger number of states (N=64) results in:\r\n\r\n![image](https://cloud.githubusercontent.com/assets/244887/6083400/8cea6890-ae28-11e4-97cb-42b51ddd3045.png)\r\n\r\nIn both cases problem.solution.status is :Optimal. However, from the figure above it is clear that the solution is considerably far from the linear constraint. So, is there a bug (in convergence or constraint enforcement) at larger N values or do I just have the wrong idea about the range of final states that yield :Optimal convergence status? If the later is the case, is there some documentation one sensitivity parameters to ensure one is near the constraint subspace?\r\n\r\nP.S.\r\n\r\nThis is my first time looking into Convex.jl and this whole thing about automatic convexity detection within such a rich modeling language seems very magical. Its great :)\r\n\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/comments/73417208","html_url":"https://github.com/JuliaOpt/Convex.jl/issues/64#issuecomment-73417208","issue_url":"https://api.github.com/repos/JuliaOpt/Convex.jl/issues/64","id":73417208,"user":{"login":"gajomi","id":244887,"avatar_url":"https://avatars.githubusercontent.com/u/244887?v=3","gravatar_id":"","url":"https://api.github.com/users/gajomi","html_url":"https://github.com/gajomi","followers_url":"https://api.github.com/users/gajomi/followers","following_url":"https://api.github.com/users/gajomi/following{/other_user}","gists_url":"https://api.github.com/users/gajomi/gists{/gist_id}","starred_url":"https://api.github.com/users/gajomi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gajomi/subscriptions","organizations_url":"https://api.github.com/users/gajomi/orgs","repos_url":"https://api.github.com/users/gajomi/repos","events_url":"https://api.github.com/users/gajomi/events{/privacy}","received_events_url":"https://api.github.com/users/gajomi/received_events","type":"User","site_admin":false},"created_at":"2015-02-08T16:06:00Z","updated_at":"2015-02-08T16:06:00Z","body":"@karanveerm- Thanks. This fixes my problem. @mlubin- Yeah, I can put together a little notebook about all this. I should have some time next week."}}