{"action":"closed","issue":{"url":"https://api.github.com/repos/fusionapp/entropy/issues/4","labels_url":"https://api.github.com/repos/fusionapp/entropy/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/fusionapp/entropy/issues/4/comments","events_url":"https://api.github.com/repos/fusionapp/entropy/issues/4/events","html_url":"https://github.com/fusionapp/entropy/issues/4","id":44252793,"number":4,"title":"Implement migration between backends","user":{"login":"mithrandi","id":856430,"avatar_url":"https://avatars.githubusercontent.com/u/856430?v=3","gravatar_id":"","url":"https://api.github.com/users/mithrandi","html_url":"https://github.com/mithrandi","followers_url":"https://api.github.com/users/mithrandi/followers","following_url":"https://api.github.com/users/mithrandi/following{/other_user}","gists_url":"https://api.github.com/users/mithrandi/gists{/gist_id}","starred_url":"https://api.github.com/users/mithrandi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mithrandi/subscriptions","organizations_url":"https://api.github.com/users/mithrandi/orgs","repos_url":"https://api.github.com/users/mithrandi/repos","events_url":"https://api.github.com/users/mithrandi/events{/privacy}","received_events_url":"https://api.github.com/users/mithrandi/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/fusionapp/entropy/labels/In%20Progress","name":"In Progress","color":"ddffdd"},{"url":"https://api.github.com/repos/fusionapp/entropy/labels/Medium","name":"Medium","color":"ffdddd"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2014-09-29T02:51:01Z","updated_at":"2015-10-27T09:53:34Z","closed_at":"2015-10-27T09:53:34Z","body":"[See also for more implementation details: bug #814366, bug #815495]\n\nWhen a new backend is configured, any objects that are created subsequent to configuring the new backend will be uploaded to that backend via the upload scheduler, as normal. However, any objects created prior to this will not be migrated replicated to the new backend. There should be a way to make this happen en masse, and as a resumable process.\n\nCurrently, all objects are stored in the local content store, so the local store can always be used as the \"source\" of the migration. However, the migration API should allow for migrations using any pair of source and destination stores, as we are moving away from having the local store as a hardcoded-always-there backend.\n\nThe plan is to add a migrateTo method to IContentStore; to initiate a migration, call sourceStore.migrateTo(destinationStore). This would be required to be implemented such that once the migration completes, all objects stored in the sourceStore at the time of the migrateTo() call (and only those objects) will have been uploaded to destinationStore. The state of the migration would be tracked by an IMigration powerup; this would hold any state and behaviour specific to the IContentStore's implementation of migrateTo, as well as some generic methods for querying the status of the migration, pausing it, resuming it, cancelling it, and so on.\n\nThe state of the migration would not be replicated in any way beyond the local Entropy node; it is assumed that starting the migration from scratch on another node will be an acceptable option for recovery in the event of a catastrophic failure of the node that was managing the original migration attempt. This would necessarily involve migrating some objects that have already been uploaded to the new backend by the upload scheduler, but hopefully this will be a relatively small fraction of objects by comparison, and content stores are supposed to be idempotent with respect to the creation of objects, so the consequences should merely be a minor increase in the total cost (network/CPU/time) of the migration as a whole.\n\n------------------------------------\nImported from Launchpad using lp2gh.\n\n * date created: 2011-07-22T00:14:32Z\n * owner: mithrandi\n * the launchpad url was https://bugs.launchpad.net/bugs/814359\n"}}