{"action":"created","issue":{"url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/102","labels_url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/102/labels{/name}","comments_url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/102/comments","events_url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/102/events","html_url":"https://github.com/logsearch/logsearch-boshrelease/issues/102","id":57375186,"number":102,"title":"Is there a way of specifying multiple redis queues for ingestors and parsers?","user":{"login":"Vincent978","id":10964368,"avatar_url":"https://avatars.githubusercontent.com/u/10964368?v=3","gravatar_id":"","url":"https://api.github.com/users/Vincent978","html_url":"https://github.com/Vincent978","followers_url":"https://api.github.com/users/Vincent978/followers","following_url":"https://api.github.com/users/Vincent978/following{/other_user}","gists_url":"https://api.github.com/users/Vincent978/gists{/gist_id}","starred_url":"https://api.github.com/users/Vincent978/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Vincent978/subscriptions","organizations_url":"https://api.github.com/users/Vincent978/orgs","repos_url":"https://api.github.com/users/Vincent978/repos","events_url":"https://api.github.com/users/Vincent978/events{/privacy}","received_events_url":"https://api.github.com/users/Vincent978/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-02-11T20:50:28Z","updated_at":"2015-02-12T08:42:38Z","closed_at":null,"body":"Is there a way of having multiple redis instances configured and have the ingestor failover to use another redis instance if the current redis queue fails. \r\nThe functionality needed is as described in logstash redis output: http://logstash.net/docs/1.4.2/outputs/redis\r\n\r\nIdeally the config is something like this for the redis output on the ingester:\r\n```\r\noutput {\r\n  redis {\r\n    host => [ \"10.0.0.10\", \"10.0.0.11\" ]\r\n    shuffle_hosts => true\r\n    data_type => \"list\"\r\n    key => \"logstash\"\r\n  }\r\n}\r\n```\r\nThe \"host\" field specifies an array of redis instances.  \r\nThe \"shuffle_hosts\" will shuffle the list of hosts when Logstash starts. \r\nThis means Logstash will try one of these Redis instances and if it connects it will send events to that instance. If the connection fails it will try the next instance in the list and so on until it either finds an instance that receives events or runs out of instances and fails.\r\n\r\nCurrently the ingestor and parser config seems to only accept a single string instead of an array of strings for redis queue instances.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/comments/74035854","html_url":"https://github.com/logsearch/logsearch-boshrelease/issues/102#issuecomment-74035854","issue_url":"https://api.github.com/repos/logsearch/logsearch-boshrelease/issues/102","id":74035854,"user":{"login":"mrdavidlaing","id":227505,"avatar_url":"https://avatars.githubusercontent.com/u/227505?v=3","gravatar_id":"","url":"https://api.github.com/users/mrdavidlaing","html_url":"https://github.com/mrdavidlaing","followers_url":"https://api.github.com/users/mrdavidlaing/followers","following_url":"https://api.github.com/users/mrdavidlaing/following{/other_user}","gists_url":"https://api.github.com/users/mrdavidlaing/gists{/gist_id}","starred_url":"https://api.github.com/users/mrdavidlaing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrdavidlaing/subscriptions","organizations_url":"https://api.github.com/users/mrdavidlaing/orgs","repos_url":"https://api.github.com/users/mrdavidlaing/repos","events_url":"https://api.github.com/users/mrdavidlaing/events{/privacy}","received_events_url":"https://api.github.com/users/mrdavidlaing/received_events","type":"User","site_admin":false},"created_at":"2015-02-12T08:42:38Z","updated_at":"2015-02-12T08:42:38Z","body":"@Vincent978 - sounds like a good idea for adressing the current redis SPOF (single point of failure).\r\n\r\nDear future reader,\r\n\r\nWe're well aware that redis is a glaring SPOF; and we believe that the longterm solution is to switch to a \"proper\" distributed queue like Kafka.  We started a spike to evalute Kafka - https://github.com/logsearch/logsearch-boshrelease/issues/73 - but decided that the (current) complexities of running it outweighed the benefits of replacing the redis SPOF.\r\n\r\nRedis has been amazingly stable for us in production; as long as it has sufficient RAM and disk to handle large queues.\r\n\r\nFor now, we're waiting for Kafka 1.0 to land, and then we will re-evaluate"}}