{"action":"created","issue":{"url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1259","labels_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1259/labels{/name}","comments_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1259/comments","events_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1259/events","html_url":"https://github.com/couchbase/sync_gateway/issues/1259","id":114913957,"number":1259,"title":"[Distributed Index] Panic when resharding DCP feed","user":{"login":"sethrosetter","id":1365736,"avatar_url":"https://avatars.githubusercontent.com/u/1365736?v=3","gravatar_id":"","url":"https://api.github.com/users/sethrosetter","html_url":"https://github.com/sethrosetter","followers_url":"https://api.github.com/users/sethrosetter/followers","following_url":"https://api.github.com/users/sethrosetter/following{/other_user}","gists_url":"https://api.github.com/users/sethrosetter/gists{/gist_id}","starred_url":"https://api.github.com/users/sethrosetter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sethrosetter/subscriptions","organizations_url":"https://api.github.com/users/sethrosetter/orgs","repos_url":"https://api.github.com/users/sethrosetter/repos","events_url":"https://api.github.com/users/sethrosetter/events{/privacy}","received_events_url":"https://api.github.com/users/sethrosetter/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/distributed-index","name":"distributed-index","color":"5319e7"},{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/in%20progress","name":"in progress","color":"ededed"},{"url":"https://api.github.com/repos/couchbase/sync_gateway/labels/release-prep","name":"release-prep","color":"eb6420"}],"state":"open","locked":false,"assignee":{"login":"tleyden","id":296876,"avatar_url":"https://avatars.githubusercontent.com/u/296876?v=3","gravatar_id":"","url":"https://api.github.com/users/tleyden","html_url":"https://github.com/tleyden","followers_url":"https://api.github.com/users/tleyden/followers","following_url":"https://api.github.com/users/tleyden/following{/other_user}","gists_url":"https://api.github.com/users/tleyden/gists{/gist_id}","starred_url":"https://api.github.com/users/tleyden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tleyden/subscriptions","organizations_url":"https://api.github.com/users/tleyden/orgs","repos_url":"https://api.github.com/users/tleyden/repos","events_url":"https://api.github.com/users/tleyden/events{/privacy}","received_events_url":"https://api.github.com/users/tleyden/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/couchbase/sync_gateway/milestones/12","html_url":"https://github.com/couchbase/sync_gateway/milestones/1.2.0","labels_url":"https://api.github.com/repos/couchbase/sync_gateway/milestones/12/labels","id":771891,"number":12,"title":"1.2.0","description":"","creator":{"login":"jessliu","id":4998909,"avatar_url":"https://avatars.githubusercontent.com/u/4998909?v=3","gravatar_id":"","url":"https://api.github.com/users/jessliu","html_url":"https://github.com/jessliu","followers_url":"https://api.github.com/users/jessliu/followers","following_url":"https://api.github.com/users/jessliu/following{/other_user}","gists_url":"https://api.github.com/users/jessliu/gists{/gist_id}","starred_url":"https://api.github.com/users/jessliu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jessliu/subscriptions","organizations_url":"https://api.github.com/users/jessliu/orgs","repos_url":"https://api.github.com/users/jessliu/repos","events_url":"https://api.github.com/users/jessliu/events{/privacy}","received_events_url":"https://api.github.com/users/jessliu/received_events","type":"User","site_admin":false},"open_issues":111,"closed_issues":136,"state":"open","created_at":"2014-08-29T16:34:15Z","updated_at":"2015-12-17T19:27:19Z","due_on":null,"closed_at":null},"comments":24,"created_at":"2015-11-03T21:25:16Z","updated_at":"2015-12-18T01:15:43Z","closed_at":null,"body":"Using binary built from source - d760caa\r\n\r\nThe test consists of 4 sync_gateways. While adding docs via bulks and individual doc puts to sync_gateways 2 and 3, sync_gateway 1 is taken off line. This results in a sporadic crashing of the sync_gateways after issuing a /db/_changes request.\r\n\r\n```\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [ABC], vbNo=701, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 2 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [ABC], vbNo=701, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 1 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 3 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [NBC], vbNo=700, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [ABC], vbNo=701, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 4 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [ABC], vbNo=802, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 2 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache+: Add to channel index [NBC], vbNo=700, isRemoval:false\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 7 is 0\r\n2015-11-03T12:39:25.080-08:00 DCache: block index for minSequence 3 is 0\r\npanic: Update attempted to set clock to earlier sequence.  Vb: 542, currentSequence: 4, newSequence: 2\r\n\r\ngoroutine 22289 [running]:\r\ngithub.com/couchbase/sync_gateway/base.(*SequenceClockImpl).UpdateWithClock(0xc2086eac30, 0x7fb18e762d28, 0xc2090d6420)\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/base/sequence_clock.go:276 +0x29e\r\ngithub.com/couchbase/sync_gateway/db.(*kvChannelIndex).writeClockCas(0xc208174480, 0x7fb18e762d28, 0xc2090d6420, 0x0, 0x0)\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/kv_channel_index.go:378 +0x6f\r\ngithub.com/couchbase/sync_gateway/db.(*kvChannelIndex).AddSet(0xc208174480, 0xc2083f2000, 0x196, 0x7d0, 0x0, 0x0)\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/kv_channel_index.go:91 +0x304\r\ngithub.com/couchbase/sync_gateway/db.(*kvChangeIndex).addSetToChannelIndex(0xc2080c8000, 0xc2081fb6b3, 0x3, 0xc2083f2000, 0x196, 0x7d0)\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/kv_change_index.go:839 +0xdc\r\ngithub.com/couchbase/sync_gateway/db.func·037(0xc2081fb6b3, 0x3, 0xc2083f2000, 0x196, 0x7d0)\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/kv_change_index.go:654 +0x84\r\ncreated by github.com/couchbase/sync_gateway/db.(*kvChangeIndex).indexEntries\r\n\t/home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/kv_change_index.go:656 +0x5fe\r\n```\r\n\r\nHere are the full logs - https://s3.amazonaws.com/sync-gateway-logs/2015-11-03-12-53-18-sglogs.zip"},"comment":{"url":"https://api.github.com/repos/couchbase/sync_gateway/issues/comments/165630719","html_url":"https://github.com/couchbase/sync_gateway/issues/1259#issuecomment-165630719","issue_url":"https://api.github.com/repos/couchbase/sync_gateway/issues/1259","id":165630719,"user":{"login":"tleyden","id":296876,"avatar_url":"https://avatars.githubusercontent.com/u/296876?v=3","gravatar_id":"","url":"https://api.github.com/users/tleyden","html_url":"https://github.com/tleyden","followers_url":"https://api.github.com/users/tleyden/followers","following_url":"https://api.github.com/users/tleyden/following{/other_user}","gists_url":"https://api.github.com/users/tleyden/gists{/gist_id}","starred_url":"https://api.github.com/users/tleyden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tleyden/subscriptions","organizations_url":"https://api.github.com/users/tleyden/orgs","repos_url":"https://api.github.com/users/tleyden/repos","events_url":"https://api.github.com/users/tleyden/events{/privacy}","received_events_url":"https://api.github.com/users/tleyden/received_events","type":"User","site_admin":false},"created_at":"2015-12-18T01:15:43Z","updated_at":"2015-12-18T01:15:43Z","body":"@steveyen moving the conversation over to https://github.com/couchbase/sync_gateway/issues/1371"}}