{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla/olympia/issues/1026","labels_url":"https://api.github.com/repos/mozilla/olympia/issues/1026/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/olympia/issues/1026/comments","events_url":"https://api.github.com/repos/mozilla/olympia/issues/1026/events","html_url":"https://github.com/mozilla/olympia/issues/1026","id":120045881,"number":1026,"title":"Stop spawning persistent spidermonkey processes from celery","user":{"login":"kumar303","id":55398,"avatar_url":"https://avatars.githubusercontent.com/u/55398?v=3","gravatar_id":"","url":"https://api.github.com/users/kumar303","html_url":"https://github.com/kumar303","followers_url":"https://api.github.com/users/kumar303/followers","following_url":"https://api.github.com/users/kumar303/following{/other_user}","gists_url":"https://api.github.com/users/kumar303/gists{/gist_id}","starred_url":"https://api.github.com/users/kumar303/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kumar303/subscriptions","organizations_url":"https://api.github.com/users/kumar303/orgs","repos_url":"https://api.github.com/users/kumar303/repos","events_url":"https://api.github.com/users/kumar303/events{/privacy}","received_events_url":"https://api.github.com/users/kumar303/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mozilla/olympia/labels/AWS","name":"AWS","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-02T22:02:51Z","updated_at":"2015-12-02T23:17:28Z","closed_at":null,"body":"There is a feature of the validator where it opens a spidermonkey subprocess (to do javascript reflection) and [assigns it to a class variable for persistence](https://github.com/mozilla/amo-validator/blob/master/validator/testcases/javascript/jsshell.py#L87). \r\n\r\nWe would like to fix this to address celery instability we're seeing related to the validator.\r\n\r\nSome details: the validator assigns the process to a class variable and relies on `__del__` to terminate the instance but there are several things possibly wrong with this.\r\n\r\n* references are very sticky in python so `__del__` might not be called as often as expected\r\n* when a process is terminated, it is still held open (as defunct) because the parent process is still running. The parent process in this case would be the celery worker which runs forever (until restarted)\r\n\r\nExample of a celery worker machine filling up with processes:\r\n````\r\n[root@ip-172-31-27-84 ~]# ps -ef | grep js\r\nolympia  21907 21796  0 19:40 ?        00:00:01 [js] <defunct>\r\nolympia  22405 21799  0 20:08 ?        00:00:00 [js] <defunct>\r\nolympia  22660 21797  0 20:22 ?        00:00:00 [js] <defunct>\r\nolympia  22751 21793  0 20:27 ?        00:00:00 [js] <defunct>\r\nolympia  24086 21798  0 21:43 ?        00:00:00 [js] <defunct>\r\n````\r\n"},"comment":{"url":"https://api.github.com/repos/mozilla/olympia/issues/comments/161464323","html_url":"https://github.com/mozilla/olympia/issues/1026#issuecomment-161464323","issue_url":"https://api.github.com/repos/mozilla/olympia/issues/1026","id":161464323,"user":{"login":"kmaglione","id":125842,"avatar_url":"https://avatars.githubusercontent.com/u/125842?v=3","gravatar_id":"","url":"https://api.github.com/users/kmaglione","html_url":"https://github.com/kmaglione","followers_url":"https://api.github.com/users/kmaglione/followers","following_url":"https://api.github.com/users/kmaglione/following{/other_user}","gists_url":"https://api.github.com/users/kmaglione/gists{/gist_id}","starred_url":"https://api.github.com/users/kmaglione/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmaglione/subscriptions","organizations_url":"https://api.github.com/users/kmaglione/orgs","repos_url":"https://api.github.com/users/kmaglione/repos","events_url":"https://api.github.com/users/kmaglione/events{/privacy}","received_events_url":"https://api.github.com/users/kmaglione/received_events","type":"User","site_admin":false},"created_at":"2015-12-02T23:17:28Z","updated_at":"2015-12-02T23:17:28Z","body":"Manually killing the process should work. Or just leaving the instance around for additional validation runs.\r\n\r\nUsing a separate instance of Spidermonkey for every snippet that's parsed, like we used to do, isn't really an option, though. It nearly doubles the runtime of the validator."}}