{"action":"created","issue":{"url":"https://api.github.com/repos/facebook/hhvm/issues/4676","labels_url":"https://api.github.com/repos/facebook/hhvm/issues/4676/labels{/name}","comments_url":"https://api.github.com/repos/facebook/hhvm/issues/4676/comments","events_url":"https://api.github.com/repos/facebook/hhvm/issues/4676/events","html_url":"https://github.com/facebook/hhvm/issues/4676","id":54851759,"number":4676,"title":"Death flock on .hhvm.hhbc","user":{"login":"GreenRover","id":512850,"avatar_url":"https://avatars.githubusercontent.com/u/512850?v=3","gravatar_id":"","url":"https://api.github.com/users/GreenRover","html_url":"https://github.com/GreenRover","followers_url":"https://api.github.com/users/GreenRover/followers","following_url":"https://api.github.com/users/GreenRover/following{/other_user}","gists_url":"https://api.github.com/users/GreenRover/gists{/gist_id}","starred_url":"https://api.github.com/users/GreenRover/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GreenRover/subscriptions","organizations_url":"https://api.github.com/users/GreenRover/orgs","repos_url":"https://api.github.com/users/GreenRover/repos","events_url":"https://api.github.com/users/GreenRover/events{/privacy}","received_events_url":"https://api.github.com/users/GreenRover/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/facebook/hhvm/labels/needs%20more%20info","name":"needs more info","color":"f7c6c7"},{"url":"https://api.github.com/repos/facebook/hhvm/labels/no%20isolated%20repro","name":"no isolated repro","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-01-20T07:28:11Z","updated_at":"2015-01-23T18:20:55Z","closed_at":null,"body":"Hello, i have server with sever cron jobs runing by hhvm.\r\n\r\nAfter a couple of hours, one process dont gives the flock on .hhvm.hhbc free.\r\n\r\n```\r\nHipHop VM 3.6.0-dev (rel)\r\nCompiler: heads/master-0-g8fa595dcc71ba2de63fb73b15e61dc9fd50a0971\r\nRepo schema: d8ec4ce9760f167994ab9f19161c09a7534ff14f\r\n```\r\n\r\nHere a strace ofa new process\r\n\r\n```\r\nfstat(4, {st_mode=S_IFREG|0644, st_size=92, ...}) = 0\r\nmmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a72f35000\r\nread(4, \"# The \\\"order\\\" line is only used \"..., 4096) = 92\r\nread(4, \"\", 4096)                       = 0\r\nclose(4)                                = 0\r\nmunmap(0x7f6a72f35000, 4096)            = 0\r\nfutex(0x7f6a6bbea3d0, FUTEX_WAKE_PRIVATE, 2147483647) = 0\r\nlseek(3, 0, SEEK_SET)                   = 0\r\nread(3, \"127.0.0.1\\tlocalhost\\n127.0.1.1\\tpo\"..., 4096) = 246\r\nread(3, \"\", 4096)                       = 0\r\nuname({sys=\"Linux\", node=\"portal3-demo\", ...}) = 0\r\nstat(\"/etc/resolv.conf\", {st_mode=S_IFREG|0644, st_size=19, ...}) = 0\r\nlseek(3, 0, SEEK_SET)                   = 0\r\nread(3, \"127.0.0.1\\tlocalhost\\n127.0.1.1\\tpo\"..., 4096) = 246\r\nread(3, \"\", 4096)                       = 0\r\nopen(\"/etc/ld.so.cache\", O_RDONLY|O_CLOEXEC) = 4\r\nfstat(4, {st_mode=S_IFREG|0644, st_size=58633, ...}) = 0\r\nmmap(NULL, 58633, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f6a72eca000\r\nclose(4)                                = 0\r\naccess(\"/etc/ld.so.nohwcap\", F_OK)      = -1 ENOENT (No such file or directory)\r\nopen(\"/lib/x86_64-linux-gnu/libnss_dns.so.2\", O_RDONLY|O_CLOEXEC) = 4\r\nread(4, \"\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0\\0\\21\\0\\0\\0\\0\\0\\0\"..., 832) = 832\r\nfstat(4, {st_mode=S_IFREG|0644, st_size=22952, ...}) = 0\r\nmmap(NULL, 2117896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f6a603ed000\r\nmprotect(0x7f6a603f2000, 2093056, PROT_NONE) = 0\r\nmmap(0x7f6a605f1000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x7f6a605f1000\r\nclose(4)                                = 0\r\nmprotect(0x7f6a605f1000, 4096, PROT_READ) = 0\r\nmunmap(0x7f6a72eca000, 58633)           = 0\r\nstat(\"/etc/resolv.conf\", {st_mode=S_IFREG|0644, st_size=19, ...}) = 0\r\nsocket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 4\r\nconnect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(\"8.8.8.8\")}, 16) = 0\r\npoll([{fd=4, events=POLLOUT}], 1, 0)    = 1 ([{fd=4, revents=POLLOUT}])\r\nsendto(4, \"\\rU\\1\\0\\0\\1\\0\\0\\0\\0\\0\\0\\fportal3-demo\\0\\0\\34\\0\\1\", 30, MSG_NOSIGNAL, NULL, 0) = 30\r\npoll([{fd=4, events=POLLIN}], 1, 5000)  = 1 ([{fd=4, revents=POLLIN}])\r\nioctl(4, FIONREAD, [105])               = 0\r\nrecvfrom(4, \"\\rU\\201\\203\\0\\1\\0\\0\\0\\1\\0\\0\\fportal3-demo\\0\\0\\34\\0\\1\\0\\0\"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(\"8.8.8.8\")}, [16]) = 105\r\nclose(4)                                = 0\r\nrt_sigaction(SIGQUIT, {0x8b1e80, [QUIT], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\nrt_sigaction(SIGILL, {0x8b1e80, [ILL], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\nrt_sigaction(SIGFPE, {0x8b1e80, [FPE], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\nrt_sigaction(SIGSEGV, {0x8b1e80, [SEGV], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\nrt_sigaction(SIGBUS, {0x8b1e80, [BUS], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\nrt_sigaction(SIGABRT, {0x8b1e80, [ABRT], SA_RESTORER|SA_RESTART, 0x7f6a6b85cd40}, {SIG_DFL, [], 0}, 8) = 0\r\ngetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0\r\nopen(\"/proc/self/maps\", O_RDONLY|O_CLOEXEC) = 4\r\ngetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0\r\nfstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0\r\nmmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a72f35000\r\nread(4, \"00400000-02bcd000 r-xp 00000000 \"..., 1024) = 1024\r\nread(4, \"   /lib/x86_64-linux-gnu/libnss_\"..., 1024) = 1024\r\nread(4, \"7000 fc:00 786761               \"..., 1024) = 1024\r\nread(4, \" /usr/lib/x86_64-linux-gnu/libhx\"..., 1024) = 1024\r\nread(4, \"ibwind.so.0.0.0\\n7f6a64684000-7f6\"..., 1024) = 1024\r\nread(4, \"inux-gnu/libXdmcp.so.6.0.0\\n7f6a6\"..., 1024) = 1024\r\nread(4, \"          /usr/lib/x86_64-linux-\"..., 1024) = 1024\r\nread(4, \"xp 00000000 fc:00 788245        \"..., 1024) = 1024\r\nread(4, \"          /usr/lib/x86_64-linux-\"..., 1024) = 1024\r\nread(4, \"-gnu/libgpg-error.so.0.10.0\\n7f6a\"..., 1024) = 1024\r\nread(4, \"        /usr/lib/x86_64-linux-gn\"..., 1024) = 1024\r\nread(4, \"0 fc:00 788767                  \"..., 1024) = 1024\r\nread(4, \"0 fc:00 68                      \"..., 1024) = 1024\r\nread(4, \"ib/x86_64-linux-gnu/libunwind.so\"..., 1024) = 1024\r\nread(4, \"7f6a67494000 rwxp 00028000 fc:00\"..., 1024) = 1024\r\nread(4, \"00 787723                     /u\"..., 1024) = 1024\r\nread(4, \"4-linux-gnu/libgcrypt.so.11.8.2\\n\"..., 1024) = 1024\r\nread(4, \"nu/libidn.so.11.6.11\\n7f6a6829700\"..., 1024) = 1024\r\nread(4, \"       /usr/lib/x86_64-linux-gnu\"..., 1024) = 1024\r\nread(4, \"fc:00 950                       \"..., 1024) = 1024\r\nread(4, \"xp 00011000 fc:00 792058        \"..., 1024) = 1024\r\nread(4, \"u/libfftw3.so.3.3.2\\n7f6a6ae75000\"..., 1024) = 1024\r\nread(4, \"     /usr/lib/x86_64-linux-gnu/l\"..., 1024) = 1024\r\nread(4, \"             /lib/x86_64-linux-g\"..., 1024) = 1024\r\nread(4, \"             /lib/x86_64-linux-g\"..., 1024) = 1024\r\nread(4, \"9.so\\n7f6a6c626000-7f6a6c62a000 r\"..., 1024) = 1024\r\nread(4, \"x86_64-linux-gnu/libboost_system\"..., 1024) = 1024\r\nread(4, \"b000 r-xp 00000000 fc:00 790286 \"..., 1024) = 1024\r\nread(4, \"nu/libcrypt-2.19.so\\n7f6a6d2d3000\"..., 1024) = 1024\r\nread(4, \"0-7f6a6d943000 ---p 0000d000 fc:\"..., 1024) = 1024\r\nread(4, \"00 fc:00 801455                 \"..., 1024) = 1024\r\nread(4, \"/lib/x86_64-linux-gnu/libexpat.s\"..., 1024) = 1024\r\nread(4, \"0 787221                     /us\"..., 1024) = 1024\r\nread(4, \"6a6ec3b000-7f6a6ee3a000 ---p 000\"..., 1024) = 1024\r\nread(4, \"4-linux-gnu/libssl.so.1.0.0\\n7f6a\"..., 1024) = 1024\r\nread(4, \"-2.0.so.5.1.9\\n7f6a6f8eb000-7f6a6\"..., 1024) = 1024\r\nread(4, \"st_program_options.so.1.54.0\\n7f6\"..., 1024) = 1024\r\nread(4, \"000-7f6a701bd000 rwxp 00018000 f\"..., 1024) = 1024\r\nread(4, \"86_64-linux-gnu/libcurl.so.4.3.0\"..., 1024) = 1024\r\nread(4, \"dab000-7f6a70dac000 rwxp 0000000\"..., 1024) = 1024\r\nread(4, \"7f6a714e7000-7f6a715e8000 r-xp 0\"..., 1024) = 1024\r\nread(4, \"  /usr/lib/x86_64-linux-gnu/libv\"..., 1024) = 1024\r\nread(4, \"7f6a722df000 r-xp 00043000 fc:00\"..., 1024) = 1024\r\nread(4, \"u/libMagickWand.so.5.0.0\\n7f6a72a\"..., 1024) = 1024\r\nread(4, \"nux-gnu/ld-2.19.so\\n7f6a72f3a000-\"..., 1024) = 419\r\nclose(4)                                = 0\r\nmunmap(0x7f6a72f35000, 4096)            = 0\r\nsched_getaffinity(13325, 32, {3, 0, 0, 0}) = 32\r\ngetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0\r\nrt_sigaction(SIGVTALRM, {0x848230, [], SA_RESTORER|SA_NODEFER|SA_SIGINFO, 0x7f6a6c41c340}, NULL, 8) = 0\r\nuname({sys=\"Linux\", node=\"portal3-demo\", ...}) = 0\r\ngetcwd(\"/root\", 4096)                   = 6\r\ngettid()                                = 13325\r\ntimer_create(CLOCK_REALTIME, {0x7f6a63347608, SIGVTALRM, SIGEV_SIGNAL, {...}}, {0}) = 0\r\ntimer_settime(0, 0, {it_interval={0, 0}, it_value={0, 0}}, {it_interval={0, 0}, it_value={0, 0}}) = 0\r\ntimer_settime(0, 0, {it_interval={0, 0}, it_value={90000, 0}}, NULL) = 0\r\nopen(\"/tmp/tcHvgZwF\", O_RDWR|O_CREAT|O_EXCL, 0600) = 4\r\nunlink(\"/tmp/tcHvgZwF\")                 = 0\r\nftruncate(4, 16777216)                  = 0\r\nmmap(NULL, 67108864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a5c3ed000\r\nmmap(0x7f6a5f3ed000, 16777216, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 4, 0) = 0x7f6a5f3ed000\r\nmmap(NULL, 524288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a72e59000\r\nreadlink(\"/proc/self/exe\", \"/usr/bin/hhvm\", 4096) = 13\r\nreadlink(\"/proc/self/exe\", \"/usr/bin/hhvm\", 4096) = 13\r\nuname({sys=\"Linux\", node=\"portal3-demo\", ...}) = 0\r\nmadvise(0x7f6a63034000, 3149824, MADV_DONTNEED) = 0\r\nfutex(0x7f6a6ea126a8, FUTEX_WAKE_PRIVATE, 2147483647) = 0\r\nbrk(0x11000000)                         = 0x11000000\r\nmprotect(0x4400000, 197132288, PROT_READ|PROT_WRITE|PROT_EXEC) = 0\r\nopen(\"/tmp/perf-13325.map\", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 5\r\nreadlink(\"/proc/self/exe\", \"/usr/bin/hhvm\", 4096) = 13\r\nopen(\"/usr/bin/hhvm\", O_RDONLY)         = 6\r\nfcntl(6, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)\r\npread(6, \"\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\2\\0>\\0\\1\\0\\0\\0\\340\\352}\\0\\0\\0\\0\\0\"..., 64, 0) = 64\r\npread(6, \"\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\"..., 9984, 48050376) = 9984\r\npread(6, \"\\0.symtab\\0.strtab\\0.shstrtab\\0.inte\"..., 2356, 48048014) = 2356\r\nclose(6)                                = 0\r\nopen(\"/usr/bin/hhvm\", O_RDONLY)         = 6\r\nlseek(6, 47543694, SEEK_SET)            = 47543694\r\nmmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a5bfed000\r\nmunmap(0x7f6a5bfed000, 4194304)         = 0\r\nmmap(NULL, 8384512, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a5bbee000\r\nmunmap(0x7f6a5bbee000, 73728)           = 0\r\nmunmap(0x7f6a5c000000, 4116480)         = 0\r\nread(6, \"<?hh\\n// {@}generated\\n\\nnamespace \"..., 504320) = 504320\r\nclose(6)                                = 0\r\nstat(\"/var/www/.hhvm.hhbc\", {st_mode=S_IFREG|0644, st_size=29316096, ...}) = 0\r\nopen(\"/var/www/.hhvm.hhbc\", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 6\r\nfstat(6, {st_mode=S_IFREG|0644, st_size=29316096, ...}) = 0\r\nlseek(6, 0, SEEK_SET)                   = 0\r\nread(6, \"SQLite format 3\\0\\4\\0\\1\\1\\0@  \\0\\2)x\\0\\0o\\325\"..., 100) = 100\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 0}, NULL)                 = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 1000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 2000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 3000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 4000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 5000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 6000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 7000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 8000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 9000000}, NULL)           = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 10000000}, NULL)          = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 11000000}, NULL)          = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\nnanosleep({0, 12000000}, NULL)          = 0\r\nfcntl(6, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EAGAIN (Resource temporarily unavailable)\r\n```\r\n\r\nA lsof to see the lock\r\n```\r\nlsof | grep hhvm.hhbc\r\nhhvm        668       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        668       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        668       www-data   23u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        669       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        669       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        669       www-data   24u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        670       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        670       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        670       www-data   25u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        776       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm        777       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1174       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1174       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1174       www-data   26u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1175       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1175       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1175       www-data   27u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1176       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1176       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1176       www-data   28u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1252       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1252       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1253       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1254       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1330       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1335       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       1341       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3540       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3540       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3540       www-data   29u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3541       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3541       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3541       www-data   30u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3542       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3542       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3542       www-data   31u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3600       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3601       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3891       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3891       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3891       www-data   32u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3892       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3892       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3892       www-data   33u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3893       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3893       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3893       www-data   34u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3978       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       3980       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4291       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4291       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4291       www-data   35u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4292       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4292       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4292       www-data   36u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4325       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4325       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4325       www-data   37u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4326       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4326       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4326       www-data   38u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4327       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4327       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4327       www-data   39u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4389       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4403       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4715       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4715       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4715       www-data   40u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4716       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4716       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4716       www-data   41u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4717       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4717       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4717       www-data   42u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4769       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       4771       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5131       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5131       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5131       www-data   43u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5132       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5132       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5132       www-data   44u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5133       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5133       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5133       www-data   45u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5196       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5197       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5485       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5485       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5485       www-data   46u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5486       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5486       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5486       www-data   47u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5487       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5487       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5487       www-data   48u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5548       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5551       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       5552       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7742       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7742       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7742       www-data   49u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7743       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7743       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7743       www-data   50u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7744       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7744       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7744       www-data   51u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7793       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       7797       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8080       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8080       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8080       www-data   52u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8081       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8081       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8081       www-data   53u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8082       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8082       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8082       www-data   54u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8165       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8170       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8529       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8529       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8529       www-data   55u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8530       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8530       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8530       www-data   56u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8531       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8531       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8531       www-data   57u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8586       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8591       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8899       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8899       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8899       www-data   58u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8900       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8900       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8900       www-data   59u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8901       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8901       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8901       www-data   60u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8963       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       8972       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9315       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9315       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9315       www-data   61u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9316       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9316       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9316       www-data   62u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9317       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9317       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9317       www-data   63u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9379       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9381       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9670       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9670       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9670       www-data   64u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9671       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9671       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9671       www-data   65u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9672       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9672       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9672       www-data   66u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9730       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9743       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm       9744       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10631       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10631       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10631       www-data   67u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10632       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10632       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10632       www-data   68u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10633       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10633       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10633       www-data   69u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10676       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      10679       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12324       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12324       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12324       www-data   70u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12325       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12325       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12325       www-data   71u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12326       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12326       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12326       www-data   72u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12409       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12413       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12756       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12756       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12756       www-data   73u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12757       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12757       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12757       www-data   74u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12758       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12758       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12758       www-data   75u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12830       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      12832       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13145       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13145       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13145       www-data   76u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13146       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13146       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13146       www-data   77u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13147       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13147       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13147       www-data   78u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13336       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      13342       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      30994       www-data    6ur     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      30994       www-data   10ur     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      30994       www-data   13ur     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31169       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31169       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31170       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31170       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31171       www-data    6uw     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31171       www-data    9uw     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31171       www-data   10uw     REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31171       www-data   12u      REG              252,0     1544 276442 /var/www/.hhvm.hhbc-journal\r\nhhvm      31329       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31329       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31329       www-data   14u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31330       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31330       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31330       www-data   15u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31331       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31331       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31331       www-data   16u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31457       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31462       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31743       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31743       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31743       www-data   17u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31744       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31744       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31744       www-data   18u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31745       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31745       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31745       www-data   19u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31829       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      31838       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32203       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32203       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32203       www-data   20u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32204       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32204       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32204       www-data   21u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32205       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32205       www-data    9u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32205       www-data   22u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32314       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\nhhvm      32320       www-data    6u      REG              252,0 29316096 284931 /var/www/.hhvm.hhbc\r\n```\r\n\r\nThe strace of the process having the lock\r\n\r\n```\r\nStrace of prcess: 31171\r\n\r\nstrace -p 31171\r\nProcess 31171 attached\r\nrestart_syscall(<... resuming interrupted call ...>) = 0\r\nnanosleep({3, 576000000}, NULL)         = 0\r\nnanosleep({3, 577000000}, NULL)         = 0\r\nnanosleep({3, 578000000}, NULL)         = 0\r\nnanosleep({3, 579000000}, NULL)         = 0\r\nnanosleep({3, 580000000}, NULL)         = 0\r\nnanosleep({3, 581000000}, NULL)         = 0\r\nnanosleep({3, 582000000}, NULL)         = 0\r\nnanosleep({3, 583000000}, \r\n```\r\n\r\nps from this process\r\n```\r\nwww-data 30994  0.0  1.5 662924 94356 ?        S    06:35   0:03  |       \\_ /usr/bin/hhvm -c /etc/hhvm/php.ini /var/www/portal3/cli.php energymanager_importTrendData --mpm --fast_run\r\nwww-data 31169  0.0  1.1 646540 71012 ?        S    06:35   0:00  |           \\_ /usr/bin/hhvm -c /etc/hhvm/php.ini /var/www/portal3/cli.php energymanager_importTrendData --mpm --fast_run\r\nwww-data 31170  0.0  1.1 646540 71132 ?        S    06:35   0:00  |           \\_ /usr/bin/hhvm -c /etc/hhvm/php.ini /var/www/portal3/cli.php energymanager_importTrendData --mpm --fast_run\r\nwww-data 31171  0.0  1.1 646540 70344 ?        S    06:35   0:00  |           \\_ /usr/bin/hhvm -c /etc/hhvm/php.ini /var/www/portal3/cli.php energymanager_importTrendData --mpm --fast_run\r\n```\r\n\r\nHow can i provide more usefull data?\r\nI am actualy not able to reproduce this issue.\r\nOnly by let the server run a couple of hours.\r\n"},"comment":{"url":"https://api.github.com/repos/facebook/hhvm/issues/comments/71238453","html_url":"https://github.com/facebook/hhvm/issues/4676#issuecomment-71238453","issue_url":"https://api.github.com/repos/facebook/hhvm/issues/4676","id":71238453,"user":{"login":"jwatzman","id":575697,"avatar_url":"https://avatars.githubusercontent.com/u/575697?v=3","gravatar_id":"","url":"https://api.github.com/users/jwatzman","html_url":"https://github.com/jwatzman","followers_url":"https://api.github.com/users/jwatzman/followers","following_url":"https://api.github.com/users/jwatzman/following{/other_user}","gists_url":"https://api.github.com/users/jwatzman/gists{/gist_id}","starred_url":"https://api.github.com/users/jwatzman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jwatzman/subscriptions","organizations_url":"https://api.github.com/users/jwatzman/orgs","repos_url":"https://api.github.com/users/jwatzman/repos","events_url":"https://api.github.com/users/jwatzman/events{/privacy}","received_events_url":"https://api.github.com/users/jwatzman/received_events","type":"User","site_admin":false},"created_at":"2015-01-23T18:20:55Z","updated_at":"2015-01-23T18:20:55Z","body":"http://pastebin.com/JvK7XjVs is really interesting -- it looks like we were doing a sqlite operation, took a signal, ended up in `bt_handler`, which then tried to do another sqlite operation. My suspicion is that the latter hung waiting for the operation before it took the signal to complete, which of course it never can.\r\n\r\nThe signal handler only gets called on a few \"bad\" exceptions, so it seems likely that some other crash is the root cause."}}