{"action":"created","issue":{"url":"https://api.github.com/repos/facebook/hhvm/issues/4386","labels_url":"https://api.github.com/repos/facebook/hhvm/issues/4386/labels{/name}","comments_url":"https://api.github.com/repos/facebook/hhvm/issues/4386/comments","events_url":"https://api.github.com/repos/facebook/hhvm/issues/4386/events","html_url":"https://github.com/facebook/hhvm/issues/4386","id":51044391,"number":4386,"title":"socket file descriptor leak","user":{"login":"apaloma","id":9606231,"avatar_url":"https://avatars.githubusercontent.com/u/9606231?v=3","gravatar_id":"","url":"https://api.github.com/users/apaloma","html_url":"https://github.com/apaloma","followers_url":"https://api.github.com/users/apaloma/followers","following_url":"https://api.github.com/users/apaloma/following{/other_user}","gists_url":"https://api.github.com/users/apaloma/gists{/gist_id}","starred_url":"https://api.github.com/users/apaloma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apaloma/subscriptions","organizations_url":"https://api.github.com/users/apaloma/orgs","repos_url":"https://api.github.com/users/apaloma/repos","events_url":"https://api.github.com/users/apaloma/events{/privacy}","received_events_url":"https://api.github.com/users/apaloma/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/facebook/hhvm/labels/webserver","name":"webserver","color":"22CCAA"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2014-12-04T23:33:44Z","updated_at":"2015-02-23T23:37:27Z","closed_at":null,"body":"Under some condition, HHVM will leak handles to sockets. After about a day of uptime, our production server stops serving requests. We get the occasional \"Too many open files\" request in the hhvm error log. Other requests just die. Looking at /proc/pid/fd shows thousands of lines for sockets.\r\n\r\n```\r\n$ sudo ls -l /proc/32169/fd | wc\r\n  32766  360418 2481333\r\n```\r\n\r\nIt's filled with sockets\r\n\r\n```\r\nlr-x------ 1 www-data www-data 64 Dec  3 22:56 0 -> /dev/null\r\nl-wx------ 1 www-data www-data 64 Dec  3 22:56 1 -> /dev/null\r\nlrwx------ 1 www-data www-data 64 Dec  3 22:56 10 -> socket:[3596582261]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 100 -> socket:[3597758510]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 1000 -> socket:[3605493219]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10000 -> socket:[3680948810]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10001 -> socket:[3678904959]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10002 -> socket:[3679435921]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10003 -> socket:[3681017174]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10004 -> socket:[3678952537]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10005 -> socket:[3678812944]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10006 -> socket:[3679018814]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10007 -> socket:[3681546683]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10008 -> socket:[3678877815]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10009 -> socket:[3678739319]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 1001 -> socket:[3604949673]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10010 -> socket:[3678055448]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10011 -> socket:[3677663437]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10012 -> socket:[3681492015]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10013 -> socket:[3678659403]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10014 -> socket:[3678429483]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10015 -> socket:[3677905669]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10016 -> socket:[3678179934]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10017 -> socket:[3683564723]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10018 -> socket:[3678138796]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10019 -> socket:[3678291679]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 1002 -> socket:[3608512492]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10020 -> socket:[3678042296]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10021 -> socket:[3678267390]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10022 -> socket:[3679512225]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10023 -> socket:[3678372093]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10024 -> socket:[3679972353]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10025 -> socket:[3679285769]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10026 -> socket:[3678047516]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10027 -> socket:[3678681504]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10028 -> socket:[3678422862]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10029 -> socket:[3678271286]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 1003 -> socket:[3605956097]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10030 -> socket:[3680417526]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10031 -> socket:[3679205947]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10032 -> socket:[3680241473]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10033 -> socket:[3678911797]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10034 -> socket:[3678134073]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10035 -> socket:[3678826980]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10036 -> socket:[3678192320]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10037 -> socket:[3678598995]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10038 -> socket:[3678412123]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 10039 -> socket:[3678885452]\r\nlrwx------ 1 www-data www-data 64 Dec  4 15:15 1004 -> socket:[3606091696]\r\n<snip>\r\n```\r\n\r\nAfter restarting HHVM, there are much fewer file descriptors open hovering at 300-400.\r\n\r\nSorry I don't have more information. Let me know if there's something I can do next time it happens to gather more information."},"comment":{"url":"https://api.github.com/repos/facebook/hhvm/issues/comments/75662260","html_url":"https://github.com/facebook/hhvm/issues/4386#issuecomment-75662260","issue_url":"https://api.github.com/repos/facebook/hhvm/issues/4386","id":75662260,"user":{"login":"paulbiss","id":1039792,"avatar_url":"https://avatars.githubusercontent.com/u/1039792?v=3","gravatar_id":"","url":"https://api.github.com/users/paulbiss","html_url":"https://github.com/paulbiss","followers_url":"https://api.github.com/users/paulbiss/followers","following_url":"https://api.github.com/users/paulbiss/following{/other_user}","gists_url":"https://api.github.com/users/paulbiss/gists{/gist_id}","starred_url":"https://api.github.com/users/paulbiss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paulbiss/subscriptions","organizations_url":"https://api.github.com/users/paulbiss/orgs","repos_url":"https://api.github.com/users/paulbiss/repos","events_url":"https://api.github.com/users/paulbiss/events{/privacy}","received_events_url":"https://api.github.com/users/paulbiss/received_events","type":"User","site_admin":false},"created_at":"2015-02-23T23:37:27Z","updated_at":"2015-02-23T23:37:27Z","body":"We had a rather nasty socket leak internally that was fixed by 0e12db8597d9271f3dde918a7d7abbf9a73f932f but that was in 3.5.0 so if the fix is new in 3.5.1 I'm not sure what did the trick, I don't see any potentially related cherry-picks."}}