{"action":"created","issue":{"url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/33","labels_url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/33/labels{/name}","comments_url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/33/comments","events_url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/33/events","html_url":"https://github.com/sisl/POMDPs.jl/issues/33","id":108506333,"number":33,"title":"POMDP belief representations vs solver/belief updater belief representations","user":{"login":"ebalaban","id":13439168,"avatar_url":"https://avatars.githubusercontent.com/u/13439168?v=3","gravatar_id":"","url":"https://api.github.com/users/ebalaban","html_url":"https://github.com/ebalaban","followers_url":"https://api.github.com/users/ebalaban/followers","following_url":"https://api.github.com/users/ebalaban/following{/other_user}","gists_url":"https://api.github.com/users/ebalaban/gists{/gist_id}","starred_url":"https://api.github.com/users/ebalaban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ebalaban/subscriptions","organizations_url":"https://api.github.com/users/ebalaban/orgs","repos_url":"https://api.github.com/users/ebalaban/repos","events_url":"https://api.github.com/users/ebalaban/events{/privacy}","received_events_url":"https://api.github.com/users/ebalaban/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ebalaban","id":13439168,"avatar_url":"https://avatars.githubusercontent.com/u/13439168?v=3","gravatar_id":"","url":"https://api.github.com/users/ebalaban","html_url":"https://github.com/ebalaban","followers_url":"https://api.github.com/users/ebalaban/followers","following_url":"https://api.github.com/users/ebalaban/following{/other_user}","gists_url":"https://api.github.com/users/ebalaban/gists{/gist_id}","starred_url":"https://api.github.com/users/ebalaban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ebalaban/subscriptions","organizations_url":"https://api.github.com/users/ebalaban/orgs","repos_url":"https://api.github.com/users/ebalaban/repos","events_url":"https://api.github.com/users/ebalaban/events{/privacy}","received_events_url":"https://api.github.com/users/ebalaban/received_events","type":"User","site_admin":false},"milestone":null,"comments":9,"created_at":"2015-09-27T01:12:24Z","updated_at":"2015-09-27T06:37:01Z","closed_at":null,"body":"Here is an issue with our current approach to belief management:\r\n\r\n- initial_belief() or create_belief() return a belief representation that's problem-specific (i.e. POMDP-specific). For the purposes of explanation, let's assume that initial_belief(my_pomdp) returns a 10-element vector of particles that is then stored in current_belief.\r\n\r\n- Now, in my [online] solver, I'd like to use a higher-fidelity 500-particle belief representation. Furthermore, I'd also like to use the same 500-particle representation for maintaining the belief state in the simulation loop (via our belief() function).\r\n\r\n- It would, obviously, be undesirable to try to convert back and forth between the 10-particle (POMDP) and 500-particle (solver/belief updater) representations.\r\n\r\nHere are a few possible solutions:\r\n\r\n1. During the first iteration of the simulation, the belief structure is converted from 10 to 500 particles inside action(), then, again, inside belief() (current_belief goes into belief() as a 10-particle vector, then comes out as a 500-particle updated_belief). Both current_belief and updated_belief then remain 500-particle vectors until the end of the simulation. Kind of ugly, especially since we'd have to ensure that conversions inside action() and belief() are done in an exactly the same way (alternatively, we need to return the converted belief vector from action() and pass it into belief()).\r\n\r\n2. Use a wrapper type, like what Zach uses, and change the actual representation within that wrapper structure. This, to me, is not all that different from the first option and also requires the user to know about some MyBeliefWrapper type and construct an instance of it before calling action() and belief().\r\n\r\n3. Add initial_belief() and create_belief() versions that take the solver and the belief updater as arguments and return a belief structure that's tailored to solver/belief updater needs from the very beginning. We would still retain the current versions of  initial_belief() and create_belief(), of course:\r\n\r\ncreate_belief(pomdp::POMDP)\r\ncreate_belief(pomdp::POMDP, solver::Solver, bu::BeliefUpdater)\r\n\r\ninitial_belief(pomdp::POMDP, belief = create_belief(pomdp))\r\ninitial_belief(pomdp::POMDP, solver::Solver, bu::BeliefUpdater, belief = create_belief(pomdp))\r\n\r\nThe third option is my preferred solution (assuming that we go ahead and implement the BeliefUpdater type, see #32), but I am open to other ideas, of course.\r\n"},"comment":{"url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/comments/143523945","html_url":"https://github.com/sisl/POMDPs.jl/issues/33#issuecomment-143523945","issue_url":"https://api.github.com/repos/sisl/POMDPs.jl/issues/33","id":143523945,"user":{"login":"zsunberg","id":4240491,"avatar_url":"https://avatars.githubusercontent.com/u/4240491?v=3","gravatar_id":"","url":"https://api.github.com/users/zsunberg","html_url":"https://github.com/zsunberg","followers_url":"https://api.github.com/users/zsunberg/followers","following_url":"https://api.github.com/users/zsunberg/following{/other_user}","gists_url":"https://api.github.com/users/zsunberg/gists{/gist_id}","starred_url":"https://api.github.com/users/zsunberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zsunberg/subscriptions","organizations_url":"https://api.github.com/users/zsunberg/orgs","repos_url":"https://api.github.com/users/zsunberg/repos","events_url":"https://api.github.com/users/zsunberg/events{/privacy}","received_events_url":"https://api.github.com/users/zsunberg/received_events","type":"User","site_admin":false},"created_at":"2015-09-27T06:37:01Z","updated_at":"2015-09-27T06:37:01Z","body":"@ebalaban, in response to your first comment, if we want our solvers to work with arbitrary problems, we will have no choice but to do the lossy conversion. If I was to write `initial_beilief(problem::POMDP, solver::POMCPSolver)`, I would use `initial_belief(problem)` inside my function so that I can get the initial distribution for any pomdp that someone might throw at me, then I would have to sample this belief to create particles for my filter."}}