{"action":"created","issue":{"url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/9","labels_url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/9/comments","events_url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/9/events","html_url":"https://github.com/rethinkdb/elasticsearch-river-rethinkdb/issues/9","id":75858456,"number":9,"title":"River does not restart after crashing/erroring","user":{"login":"kishorevarma","id":1729506,"avatar_url":"https://avatars.githubusercontent.com/u/1729506?v=3","gravatar_id":"","url":"https://api.github.com/users/kishorevarma","html_url":"https://github.com/kishorevarma","followers_url":"https://api.github.com/users/kishorevarma/followers","following_url":"https://api.github.com/users/kishorevarma/following{/other_user}","gists_url":"https://api.github.com/users/kishorevarma/gists{/gist_id}","starred_url":"https://api.github.com/users/kishorevarma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kishorevarma/subscriptions","organizations_url":"https://api.github.com/users/kishorevarma/orgs","repos_url":"https://api.github.com/users/kishorevarma/repos","events_url":"https://api.github.com/users/kishorevarma/events{/privacy}","received_events_url":"https://api.github.com/users/kishorevarma/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-13T05:52:07Z","updated_at":"2015-05-18T20:02:03Z","closed_at":null,"body":"We have a setup of a RethinkDB cluster being read from an Elasticsearch cluster using this plugin. Recently when we updated RethinkDB - which caused downtime on RethinkDB - the plugin failed to find the tables and bailed out as in the logs below.\r\n\r\nSo the documents between ES and RethinkDB went out of sync - and only when we went into the logs did we notice that the plugin had completely bailed out. Restarting the ES node where the plugin was running fixes this - but it would be great if the plugin would attempt restarting after failure like ES itself.\r\n\r\nLogs:\r\n```\r\n[2015-04-07 11:56:48,630][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 450 documents\r\n[2015-04-07 18:14:03,286][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 460 documents\r\n[2015-04-08 04:02:28,360][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 470 documents\r\n[2015-04-08 06:19:23,834][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 480 documents\r\n[2015-04-08 12:13:20,714][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 490 documents\r\n[2015-04-08 12:29:34,956][INFO ][cluster.metadata         ] [cortex-elasticsearch2] [cortex] update_mapping [layouts] (dynamic)\r\n[2015-04-08 13:55:32,769][INFO ][cluster.metadata         ] [cortex-elasticsearch2] [cortex] update_mapping [layouts] (dynamic)\r\n[2015-04-08 13:58:30,165][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 500 documents\r\n[2015-04-08 22:07:51,018][WARN ][monitor.jvm              ] [cortex-elasticsearch2] [gc][young][4331504][511] duration [1.4s], collections [1]/[2.4s], total [1.4s]/[2.5m], memory [567.9mb]->[40mb]/[3.9gb], all_pools {[young] [531.5mb]->[3.5mb]/[532.5mb]}{[survivor] [1.4mb]->[1.2mb]/[66.5mb]}{[old] [34.8mb]->[35.3mb]/[3.3gb]}\r\n[2015-04-14 19:56:46,340][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 510 documents\r\n[2015-04-14 22:28:37,068][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 520 documents\r\n[2015-04-18 14:15:29,601][WARN ][monitor.jvm              ] [cortex-elasticsearch2] [gc][young][5166964][660] duration [1.2s], collections [1]/[1.5s], total [1.2s]/[2.6m], memory [569.6mb]->[43.2mb]/[3.9gb], all_pools {[young] [532.5mb]->[6.2mb]/[532.5mb]}{[survivor] [31.1kb]->[33kb]/[66.5mb]}{[old] [37mb]->[37mb]/[3.3gb]}\r\n[2015-04-20 18:56:07,735][INFO ][river.rethinkdb.feedworker] [cortex.layouts] Synced 530 documents\r\n[2015-04-28 15:57:42,071][ERROR][river.rethinkdb.feedworker] [cortex.symlinks] Worker has a problem: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n[2015-04-28 15:57:42,075][INFO ][river.rethinkdb.feedworker] [cortex.symlinks] This probably isn't recoverable, bailing.\r\n[2015-04-28 15:57:42,075][ERROR][river.rethinkdb.feedworker] [cortex.symlinks] failed due to exception\r\ncom.rethinkdb.RethinkDBException: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n\tat com.rethinkdb.response.DBResultFactory.convert(DBResultFactory.java:25)\r\n\tat com.rethinkdb.Cursor.loadNextBatch(Cursor.java:62)\r\n\tat com.rethinkdb.Cursor.next(Cursor.java:85)\r\n\tat org.elasticsearch.river.rethinkdb.FeedWorker.run(FeedWorker.java:77)\r\n\tat java.lang.Thread.run(Thread.java:744)\r\n[2015-04-28 15:57:42,075][INFO ][river.rethinkdb.feedworker] [cortex.symlinks] thread shutting down\r\n[2015-04-28 15:57:42,075][ERROR][river.rethinkdb.feedworker] [cortex.templates] Worker has a problem: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n[2015-04-28 15:57:42,078][ERROR][river.rethinkdb.feedworker] [cortex.layouts] Worker has a problem: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n[2015-04-28 15:57:42,080][INFO ][river.rethinkdb.feedworker] [cortex.layouts] This probably isn't recoverable, bailing.\r\n[2015-04-28 15:57:42,080][INFO ][river.rethinkdb.feedworker] [cortex.templates] This probably isn't recoverable, bailing.\r\n[2015-04-28 15:57:42,080][ERROR][river.rethinkdb.feedworker] [cortex.layouts] failed due to exception\r\ncom.rethinkdb.RethinkDBException: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n\tat com.rethinkdb.response.DBResultFactory.convert(DBResultFactory.java:25)\r\n\tat com.rethinkdb.Cursor.loadNextBatch(Cursor.java:62)\r\n\tat com.rethinkdb.Cursor.next(Cursor.java:85)\r\n\tat org.elasticsearch.river.rethinkdb.FeedWorker.run(FeedWorker.java:77)\r\n\tat java.lang.Thread.run(Thread.java:744)\r\n[2015-04-28 15:57:42,080][INFO ][river.rethinkdb.feedworker] [cortex.layouts] thread shutting down\r\n[2015-04-28 15:57:42,080][ERROR][river.rethinkdb.feedworker] [cortex.templates] failed due to exception\r\ncom.rethinkdb.RethinkDBException: RUNTIME_ERROR: Changefeed aborted (table unavailable).\r\n\tat com.rethinkdb.response.DBResultFactory.convert(DBResultFactory.java:25)\r\n\tat com.rethinkdb.Cursor.loadNextBatch(Cursor.java:62)\r\n\tat com.rethinkdb.Cursor.next(Cursor.java:85)\r\n\tat org.elasticsearch.river.rethinkdb.FeedWorker.run(FeedWorker.java:77)\r\n\tat java.lang.Thread.run(Thread.java:744)\r\n[2015-04-28 15:57:42,081][INFO ][river.rethinkdb.feedworker] [cortex.templates] thread shutting down\r\n```"},"comment":{"url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/comments/103193715","html_url":"https://github.com/rethinkdb/elasticsearch-river-rethinkdb/issues/9#issuecomment-103193715","issue_url":"https://api.github.com/repos/rethinkdb/elasticsearch-river-rethinkdb/issues/9","id":103193715,"user":{"login":"deontologician","id":1366,"avatar_url":"https://avatars.githubusercontent.com/u/1366?v=3","gravatar_id":"","url":"https://api.github.com/users/deontologician","html_url":"https://github.com/deontologician","followers_url":"https://api.github.com/users/deontologician/followers","following_url":"https://api.github.com/users/deontologician/following{/other_user}","gists_url":"https://api.github.com/users/deontologician/gists{/gist_id}","starred_url":"https://api.github.com/users/deontologician/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deontologician/subscriptions","organizations_url":"https://api.github.com/users/deontologician/orgs","repos_url":"https://api.github.com/users/deontologician/repos","events_url":"https://api.github.com/users/deontologician/events{/privacy}","received_events_url":"https://api.github.com/users/deontologician/received_events","type":"User","site_admin":false},"created_at":"2015-05-18T20:02:03Z","updated_at":"2015-05-18T20:02:03Z","body":"@kishorevarma I would be open to pull requests on this, but at the moment the idea is to deprecate this river (since rivers are deprecated by ElasticSearch), and move to the [logstash input](https://github.com/rethinkdb/logstash-input-rethinkdb). Once RethinkDB has resumable changefeeds, we'll add that capability to the logstash plugin and formally deprecate the river."}}