{"action":"created","issue":{"url":"https://api.github.com/repos/compose/transporter/issues/33","labels_url":"https://api.github.com/repos/compose/transporter/issues/33/labels{/name}","comments_url":"https://api.github.com/repos/compose/transporter/issues/33/comments","events_url":"https://api.github.com/repos/compose/transporter/issues/33/events","html_url":"https://github.com/compose/transporter/issues/33","id":53065455,"number":33,"title":"Enhancement: Add the ability to save and restore the state of an adaptor","user":{"login":"jipperinbham","id":829493,"avatar_url":"https://avatars.githubusercontent.com/u/829493?v=3","gravatar_id":"","url":"https://api.github.com/users/jipperinbham","html_url":"https://github.com/jipperinbham","followers_url":"https://api.github.com/users/jipperinbham/followers","following_url":"https://api.github.com/users/jipperinbham/following{/other_user}","gists_url":"https://api.github.com/users/jipperinbham/gists{/gist_id}","starred_url":"https://api.github.com/users/jipperinbham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jipperinbham/subscriptions","organizations_url":"https://api.github.com/users/jipperinbham/orgs","repos_url":"https://api.github.com/users/jipperinbham/repos","events_url":"https://api.github.com/users/jipperinbham/events{/privacy}","received_events_url":"https://api.github.com/users/jipperinbham/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/compose/transporter/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2014-12-29T19:33:50Z","updated_at":"2015-01-02T03:53:48Z","closed_at":null,"body":"Currently, if transporter fails, it is not able to start back where it left off during a copy/tail. In order to support this, each adaptor will want to save/persist the most recent document it has processed. We should be able to support multiple types of persistent stores using a simple interface like so:\r\n\r\n```go\r\ntype SessionStore interface {\r\n\tSet(path string, msg *message.Msg) error\r\n\tGet(path string) (string, int64, error)\r\n}\r\n```\r\n\r\nThe path would typically be a combination of the Transporter key and the node path. When retrieving the last known State, we will only return the last `_id` and the timestamp of the operation. It may be necessary in the future to support retuning the entire message.Msg. \r\n\r\nIdeally, an implementing class will not constantly write the last operation but have the ability to \"flush\" on an interval which can be defined in the config.yaml.\r\n\r\nThe beginning of this can be seen in [adaptor-state](https://github.com/compose/transporter/tree/adaptor-state) branch but it is incomplete. It currently introduces a sessionTicker where the Pipeline will call the `Set` func for each Node. As of right now, I have not added the ability to retrieve/get the state during startup/initialization. \r\n\r\nThoughts and feedback?"},"comment":{"url":"https://api.github.com/repos/compose/transporter/issues/comments/68507642","html_url":"https://github.com/compose/transporter/issues/33#issuecomment-68507642","issue_url":"https://api.github.com/repos/compose/transporter/issues/33","id":68507642,"user":{"login":"jipperinbham","id":829493,"avatar_url":"https://avatars.githubusercontent.com/u/829493?v=3","gravatar_id":"","url":"https://api.github.com/users/jipperinbham","html_url":"https://github.com/jipperinbham","followers_url":"https://api.github.com/users/jipperinbham/followers","following_url":"https://api.github.com/users/jipperinbham/following{/other_user}","gists_url":"https://api.github.com/users/jipperinbham/gists{/gist_id}","starred_url":"https://api.github.com/users/jipperinbham/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jipperinbham/subscriptions","organizations_url":"https://api.github.com/users/jipperinbham/orgs","repos_url":"https://api.github.com/users/jipperinbham/repos","events_url":"https://api.github.com/users/jipperinbham/events{/privacy}","received_events_url":"https://api.github.com/users/jipperinbham/received_events","type":"User","site_admin":false},"created_at":"2015-01-02T03:53:48Z","updated_at":"2015-01-02T03:53:48Z","body":"Another part of this that has yet to be discussed is how to actually configure a SessionStore. My thought is to have something like the following in the config.yaml:\r\n\r\n```yaml\r\nsessions:\r\n  type: \"filestore\",\r\n  interval: 10s\r\n```\r\n\r\nIf the sessions section is missing, no SessionStore will be used."}}