{"action":"created","issue":{"url":"https://api.github.com/repos/fail2ban/fail2ban/issues/1074","labels_url":"https://api.github.com/repos/fail2ban/fail2ban/issues/1074/labels{/name}","comments_url":"https://api.github.com/repos/fail2ban/fail2ban/issues/1074/comments","events_url":"https://api.github.com/repos/fail2ban/fail2ban/issues/1074/events","html_url":"https://github.com/fail2ban/fail2ban/issues/1074","id":89286539,"number":1074,"title":"fail2ban cannot handle hundreds log entries","user":{"login":"squidetc","id":4160640,"avatar_url":"https://avatars.githubusercontent.com/u/4160640?v=3","gravatar_id":"","url":"https://api.github.com/users/squidetc","html_url":"https://github.com/squidetc","followers_url":"https://api.github.com/users/squidetc/followers","following_url":"https://api.github.com/users/squidetc/following{/other_user}","gists_url":"https://api.github.com/users/squidetc/gists{/gist_id}","starred_url":"https://api.github.com/users/squidetc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/squidetc/subscriptions","organizations_url":"https://api.github.com/users/squidetc/orgs","repos_url":"https://api.github.com/users/squidetc/repos","events_url":"https://api.github.com/users/squidetc/events{/privacy}","received_events_url":"https://api.github.com/users/squidetc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-06-18T12:54:57Z","updated_at":"2015-07-03T15:32:52Z","closed_at":null,"body":"Hello. Could you please help (or advice) us to solve problem with fail2ban and varnish logs. \r\n\r\nProblem: We configured and tested using fail2ban-regex filter for banning 403 varnish responses - it working fine. When we moved it to production - it was working fine as well, but after 6-13 hours it just stopped to ban IPs using provided filter. After restart - it goes back and start work property for next 6-24 hours and stuck again.... Other filter working without problems (ssh). \r\n\r\nWhat we try to do: \r\n- Changed  backed to polling,auto,gamin,pyinotify\r\n- Redirect all 403 responses to separate file (it increasing ban time to 20-30 hours but stuck after...)\r\n- Update fail2ban to latest version (0.8.13)\r\nNothing helped.... \r\n\r\nAdditional info:\r\nOS: Ubuntu 12.04\r\nVarnish writing about 800 log entries per second.\r\n \r\nCould you please advise how to improve banning in such case? \r\nLogs sample:\r\n####################\r\n109.96.146.77           -       2015-06-10 12:03:12 +0300       196             0.002336        miss    403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:12 +0300       196             0.000458        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000330        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000486        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000412        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000524        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000415        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000420        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n109.96.146.77           -       2015-06-10 12:03:13 +0300       196             0.000426        hit     403     GET     http://somesite.com/linkname   \"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\"\r\n####################\r\n\r\nFilter config:\r\n####################\r\n[Definition]\r\nfailregex = ^<HOST>.*miss\\s*403\\s*GET.*\"$\r\n\r\nignoreregex =\r\n####################\r\n "},"comment":{"url":"https://api.github.com/repos/fail2ban/fail2ban/issues/comments/118375356","html_url":"https://github.com/fail2ban/fail2ban/issues/1074#issuecomment-118375356","issue_url":"https://api.github.com/repos/fail2ban/fail2ban/issues/1074","id":118375356,"user":{"login":"leeclemens","id":1250267,"avatar_url":"https://avatars.githubusercontent.com/u/1250267?v=3","gravatar_id":"","url":"https://api.github.com/users/leeclemens","html_url":"https://github.com/leeclemens","followers_url":"https://api.github.com/users/leeclemens/followers","following_url":"https://api.github.com/users/leeclemens/following{/other_user}","gists_url":"https://api.github.com/users/leeclemens/gists{/gist_id}","starred_url":"https://api.github.com/users/leeclemens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leeclemens/subscriptions","organizations_url":"https://api.github.com/users/leeclemens/orgs","repos_url":"https://api.github.com/users/leeclemens/repos","events_url":"https://api.github.com/users/leeclemens/events{/privacy}","received_events_url":"https://api.github.com/users/leeclemens/received_events","type":"User","site_admin":false},"created_at":"2015-07-03T15:32:52Z","updated_at":"2015-07-03T15:32:52Z","body":"I would recommend restarting fail2ban as least times as makes sense for your environment - same rule with any service (and OS). If you need to restart frequently, there is probably something else wrong or lacking with your system or application."}}