{"action":"created","issue":{"url":"https://api.github.com/repos/mperham/sidekiq/issues/2474","labels_url":"https://api.github.com/repos/mperham/sidekiq/issues/2474/labels{/name}","comments_url":"https://api.github.com/repos/mperham/sidekiq/issues/2474/comments","events_url":"https://api.github.com/repos/mperham/sidekiq/issues/2474/events","html_url":"https://github.com/mperham/sidekiq/issues/2474","id":99032734,"number":2474,"title":"Missing jobs in batch ","user":{"login":"Saicheg","id":624999,"avatar_url":"https://avatars.githubusercontent.com/u/624999?v=3","gravatar_id":"","url":"https://api.github.com/users/Saicheg","html_url":"https://github.com/Saicheg","followers_url":"https://api.github.com/users/Saicheg/followers","following_url":"https://api.github.com/users/Saicheg/following{/other_user}","gists_url":"https://api.github.com/users/Saicheg/gists{/gist_id}","starred_url":"https://api.github.com/users/Saicheg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Saicheg/subscriptions","organizations_url":"https://api.github.com/users/Saicheg/orgs","repos_url":"https://api.github.com/users/Saicheg/repos","events_url":"https://api.github.com/users/Saicheg/events{/privacy}","received_events_url":"https://api.github.com/users/Saicheg/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mperham/sidekiq/labels/pro","name":"pro","color":"e10c02"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":31,"created_at":"2015-08-04T18:41:27Z","updated_at":"2015-12-14T15:09:06Z","closed_at":null,"body":"As i described on gitter. I realized a feature which uses sidekiq batches under the hood. Everythere were pretty fine on development and test. But in production after i missed some jobs on the 1st worker launch. I mean COMPLETE missed. Can't find them anywhere, in queue, failures, process.\r\n\r\nHere is screenshot of how it looks: https://yadi.sk/i/zjnHLByriGMWz\r\n6 hours after after couple full sidekiq restarts it is still there: https://yadi.sk/i/MVr1C7gSiGf9h\r\n\r\nMy sidekiq config:\r\n\r\n```\r\nrequire 'sidekiq-pro'\r\n\r\nraw_yaml = YAML.load(File.read(File.join(File.expand_path('../../config/redis.yml', \"config/redis.yml\"))))\r\nenv = ENV['RAILS_ENV'] || 'development'\r\nredis_config = raw_yaml[env]\r\n\r\nsidekiq_config = YAML.load(File.read(File.join(File.expand_path('../../config/sidekiq.yml', \"config/sidekiq.yml\"))))\r\n\r\nSidekiq.configure_server do |config|\r\n  config.reliable_fetch!\r\n  config.reliable_scheduler!\r\n\r\n  config.redis = {\r\n      url: \"redis://#{redis_config['host']}:#{redis_config['port']}/\",\r\n      namespace: 'sidekiq_clickcast'\r\n  }\r\n  ActiveRecord::Base.configurations[env]['pool'] = sidekiq_config[:concurrency].to_i + 2\r\n  ActiveRecord::Base.establish_connection\r\nend\r\n\r\nSidekiq.configure_client do |config|\r\n  config.redis = {\r\n      url: \"redis://#{redis_config['host']}:#{redis_config['port']}/\",\r\n      namespace: 'sidekiq_clickcast'\r\n  }\r\nend\r\n\r\nSidekiq::Client.reliable_push! if env == 'staging' || env == 'production'\r\n```\r\n\r\nGems:\r\n```\r\ngem 'sidekiq', '~> 3.4.2'\r\ngem 'sidekiq-client-cli', '~> 0.1.7'\r\ngem 'sidekiq-failures', github: 'mhfs/sidekiq-failures'                                                                                                       gem 'sidekiq-unique-jobs', '~> 3.0.2'\r\ngem 'sidekiq-limit_fetch', '~> 2.2.6'\r\nsource <STRIPPED> do\r\n  gem 'sidekiq-pro'             \r\nend \r\n```\r\n\r\nI can't reproduce it anymore. All other launches went fine.                                                                                                                        "},"comment":{"url":"https://api.github.com/repos/mperham/sidekiq/issues/comments/164461373","html_url":"https://github.com/mperham/sidekiq/issues/2474#issuecomment-164461373","issue_url":"https://api.github.com/repos/mperham/sidekiq/issues/2474","id":164461373,"user":{"login":"hicran","id":118728,"avatar_url":"https://avatars.githubusercontent.com/u/118728?v=3","gravatar_id":"","url":"https://api.github.com/users/hicran","html_url":"https://github.com/hicran","followers_url":"https://api.github.com/users/hicran/followers","following_url":"https://api.github.com/users/hicran/following{/other_user}","gists_url":"https://api.github.com/users/hicran/gists{/gist_id}","starred_url":"https://api.github.com/users/hicran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hicran/subscriptions","organizations_url":"https://api.github.com/users/hicran/orgs","repos_url":"https://api.github.com/users/hicran/repos","events_url":"https://api.github.com/users/hicran/events{/privacy}","received_events_url":"https://api.github.com/users/hicran/received_events","type":"User","site_admin":false},"created_at":"2015-12-14T15:09:06Z","updated_at":"2015-12-14T15:09:06Z","body":"Could this be related to what @jschairb mentioned here: https://github.com/mperham/sidekiq/issues/2201#issuecomment-162718449\r\n\r\nOur application does not use redis directly, all of our workers are written the same way I had pasted here.\r\n"}}