{"action":"created","issue":{"url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/659","labels_url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/659/labels{/name}","comments_url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/659/comments","events_url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/659/events","html_url":"https://github.com/nexcess/magento-turpentine/issues/659","id":46931077,"number":659,"title":"Concurrency/load issues","user":{"login":"craigcarnell","id":1188869,"avatar_url":"https://avatars.githubusercontent.com/u/1188869?v=3","gravatar_id":"","url":"https://api.github.com/users/craigcarnell","html_url":"https://github.com/craigcarnell","followers_url":"https://api.github.com/users/craigcarnell/followers","following_url":"https://api.github.com/users/craigcarnell/following{/other_user}","gists_url":"https://api.github.com/users/craigcarnell/gists{/gist_id}","starred_url":"https://api.github.com/users/craigcarnell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/craigcarnell/subscriptions","organizations_url":"https://api.github.com/users/craigcarnell/orgs","repos_url":"https://api.github.com/users/craigcarnell/repos","events_url":"https://api.github.com/users/craigcarnell/events{/privacy}","received_events_url":"https://api.github.com/users/craigcarnell/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"miguelbalparda","id":3997682,"avatar_url":"https://avatars.githubusercontent.com/u/3997682?v=3","gravatar_id":"","url":"https://api.github.com/users/miguelbalparda","html_url":"https://github.com/miguelbalparda","followers_url":"https://api.github.com/users/miguelbalparda/followers","following_url":"https://api.github.com/users/miguelbalparda/following{/other_user}","gists_url":"https://api.github.com/users/miguelbalparda/gists{/gist_id}","starred_url":"https://api.github.com/users/miguelbalparda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miguelbalparda/subscriptions","organizations_url":"https://api.github.com/users/miguelbalparda/orgs","repos_url":"https://api.github.com/users/miguelbalparda/repos","events_url":"https://api.github.com/users/miguelbalparda/events{/privacy}","received_events_url":"https://api.github.com/users/miguelbalparda/received_events","type":"User","site_admin":false},"milestone":null,"comments":6,"created_at":"2014-10-27T17:02:30Z","updated_at":"2015-07-07T14:43:39Z","closed_at":"2015-07-07T14:43:39Z","body":"Hi,\r\n\r\nI have a store that is running amazingly fast (thanks to turpentine etc) for single users. But when I hit with say Gatling and 200 users concurrently it grinds to a halt.\r\n\r\nThe setup is Varnish Cache 4.0.1 with Nginx + HHVM + Redis Cache + Redis Session. Server setup is 32 cores, 120gb of RAM, 40 GB SSD.\r\n\r\nGatling\r\n```\r\nva.util.concurrent.TimeoutException: Request timed out to xxx of 60000 ms\r\n16:48:07.807 [WARN ] i.g.h.a.AsyncHandlerActor - Request 'request_1' failed: java.util.concurrent.TimeoutException: Request timed out to xxx of 60000 ms\r\n```\r\n\r\nVarnish \r\n```\r\n/usr/sbin/varnishd -P /var/run/varnishd.pid -a 0.0.0.0:80 -T 0.0.0.0:6082 -t 120 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,2G -p feature=+esi_ignore_other_elements -p vcc_allow_inline_c=on -p thread_pools=2 -p syslog_cli_traffic=off -p cli_buffer=24000\r\n```\r\n\r\n/etc/varnish/default.vcl\r\n```\r\nvcl 4.0;\r\n# Nexcess.net Turpentine Extension for Magento\r\n# Copyright (C) 2012  Nexcess.net L.L.C.\r\n#\r\n# This program is free software; you can redistribute it and/or modify\r\n# it under the terms of the GNU General Public License as published by\r\n# the Free Software Foundation; either version 2 of the License, or\r\n# (at your option) any later version.\r\n#\r\n# This program is distributed in the hope that it will be useful,\r\n# but WITHOUT ANY WARRANTY; without even the implied warranty of\r\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\r\n# GNU General Public License for more details.\r\n#\r\n# You should have received a copy of the GNU General Public License along\r\n# with this program; if not, write to the Free Software Foundation, Inc.,\r\n# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.\r\n\r\n## Nexcessnet_Turpentine Varnish v4 VCL Template\r\n\r\n## Custom C Code\r\n\r\nC{\r\n#include <stdlib.h>\r\n#include <stdio.h>\r\n#include <time.h>\r\n#include <pthread.h>\r\n\r\nstatic pthread_mutex_t lrand_mutex = PTHREAD_MUTEX_INITIALIZER;\r\n\r\nvoid generate_uuid(char* buf) {\r\n    pthread_mutex_lock(&lrand_mutex);\r\n    long a = lrand48();\r\n    long b = lrand48();\r\n    long c = lrand48();\r\n    long d = lrand48();\r\n    pthread_mutex_unlock(&lrand_mutex);\r\n    // SID must match this regex for Kount compat /^\\w{1,32}$/\r\n    sprintf(buf, \"frontend=%08lx%04lx%04lx%04lx%04lx%08lx\",\r\n        a,\r\n        b & 0xffff,\r\n        (b & ((long)0x0fff0000) >> 16) | 0x4000,\r\n        (c & 0x0fff) | 0x8000,\r\n        (c & (long)0xffff0000) >> 16,\r\n        d\r\n    );\r\n    return;\r\n}\r\n\r\n}C\r\n\r\n## Imports\r\n\r\nimport std;\r\n\r\n## Custom VCL Logic\r\n\r\n# Vax Custom Include - VCL V4\r\n\r\n# Additional includes for logging\r\nC{\r\n#include <stdio.h>\r\n#include <stdlib.h>\r\n#include <syslog.h>\r\n#include <stddef.h>\r\n#include <sys/time.h>\r\n#include <time.h>\r\n}C\r\n\r\nsub vcl_recv {\r\n    #   Add X-Request-Start header so we can track queue times in New Relic RPM beginning at Varnish.\r\n    #if (req.restarts == 0) {\r\n        C{\r\n    \t     /*struct timeval detail_time;\r\n            gettimeofday(&detail_time,NULL);\r\n            char start[20];\r\n            sprintf(start, \"t=%lu%06lu\", detail_time.tv_sec, detail_time.tv_usec);\r\n            static const struct gethdr_s VGC_HDR_REQ_VARNISH_FAKED_SESSION = { HDR_REQ, \"\\030X-Varnish-Faked-Session:\"};\r\n            VRT_SetHdr(ctx, &VGC_HDR_REQ_VARNISH_FAKED_SESSION, \"\\020X-Request-Start:\", start, vrt_magic_string_end);*/\r\n        }C\r\n    #}\r\n\r\n    # Bypass registration form\r\n    if (req.url ~ \"^/registration/form\") {\r\n        return (pass);\r\n    }\r\n}\r\n\r\nsub vcl_backend_error {\r\n    if (beresp.status >= 500) {\r\n        C{\r\n            /*FILE *fp;\r\n            char ft[256];\r\n            struct tm *tmp;\r\n            time_t curtime;\r\n\r\n            fp = fopen(\"/var/log/varnish/error_log\", \"a\");\r\n            time(&curtime);\r\n            tmp = localtime(&curtime);\r\n            strftime(ft, 256, \"%D - %T\", tmp);\r\n\r\n            if(fp != NULL) {\r\n                fprintf(fp, \"%s: Error (%s) (%s) (%s)\\n\",\r\n                ft, VRT_r_req_url(sp), VRT_r_obj_response(sp), VRT_r_req_xid(sp));\r\n\r\n                fclose(fp);\r\n            } else {\r\n                syslog(LOG_INFO, \"Error (%s) (%s) (%s)\",\r\n                VRT_r_req_url(sp), VRT_r_obj_response(sp), VRT_r_req_xid(sp));\r\n            }*/\r\n        }C\r\n    }\r\n\r\n    synthetic({\"\r\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\r\n \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n <html>\r\n   <head>\r\n     <title>\"} + beresp.status + \" \" + beresp.reason + {\"</title>\r\n     <style type=\"text/css\">\r\n        * {\r\n            margin: 0;\r\n            padding: 0;\r\n        }\r\n        .error-layout { width: 100%; max-width: 1600px; height: 100%; margin: 0 auto; padding: 0; }\r\n        .error-layout .main-container { height: 100%; }\r\n        .error-layout .main { background: url(\"/errors/default/images/error_background.jpg\") no-repeat #fff; min-height: 590px; color: #1D2B33; float: left; }\r\n        .error-layout .col-main { width: 100%; }\r\n        .error-layout .std { margin: 100px 0 0 400px; padding-left: 270px; background: url(\"/errors/default/images/warning_icon.png\") no-repeat 10px 0; height: 200px; }\r\n        .error-layout h3 { font-size: 400%; margin-bottom: 10px; }\r\n        .error-layout p { font-size: 180%; }\r\n        .error-layout .back { float: left; background: url(\"/skin/frontend/vax/uk/images/icons/left_arrow.png\") no-repeat 8px center #1D2B33; height: 18px; padding: 5px 8px 5px 33px; color: #FFF; font-size: 13px; line-height: 18px; text-decoration: none; margin-top: 10px; }\r\n    /* ======================================================================================= */\r\n    </style>\r\n   </head>\r\n   <body>\r\n     <div class=\"page error-layout\">\r\n        <div class=\"main-container\">\r\n            <div class=\"main\">\r\n                <div class=\"col-main\">\r\n                    <div class=\"std\"><h3>Oops, something went wrong</h3>\r\n                        <!--<h1>Error \"} + beresp.status + \" \" + beresp.reason + {\"</h1>\r\n                        <p>\"} + beresp.reason + {\"</p>\r\n                        <h3>Guru Meditation:</h3>\r\n                        <p>XID: \"} + bereq.xid + {\"</p>\r\n                        <hr>-->\r\n                        <p><a class=\"back\" title=\"Go Back\" href=\"javascript: history.go(-1);\">Go Back</a></p>\r\n                    </div>\r\n                 </div>\r\n            </div>\r\n      </div>\r\n    </body>\r\n </html>\r\n \"});\r\n     return (deliver);\r\n}\r\n\r\n# customized vcl_deliver to allow use of cross sub-domain cookies\r\nsub vcl_deliver {\r\n    if (req.http.X-Varnish-Faked-Session) {\r\n        # need to set the set-cookie header since we just made it out of thin air\r\n        call generate_session_expires;\r\n        set resp.http.Set-Cookie = req.http.X-Varnish-Faked-Session +\r\n            \"; expires=\" + resp.http.X-Varnish-Cookie-Expires + \"; path=/\";\r\n        if (req.http.Host) {\r\n\t    \tif (req.http.User-Agent ~ \"^(?:ApacheBench/.*|.*Googlebot.*|JoeDog/.*Siege.*|magespeedtest\\.com|Nexcessnet_Turpentine/.*)$\") {\r\n\t                # it's a crawler, no need to share cookies\r\n\t                set resp.http.Set-Cookie = resp.http.Set-Cookie +\r\n                    \"; domain=\" + regsub(req.http.Host, \":\\d+$\", \"\");\r\n\t        } else {\r\n\t            # it's a real user, allow sharing of cookies between support etc\r\n\t            if(req.http.Host ~ \"^(www\\.|spares\\.|support\\.)?vax\\.co\\.uk$\") {\r\n            \t\tset resp.http.Set-Cookie = resp.http.Set-Cookie +\r\n                    \t\"; domain=.vax.co.uk\";\r\n            \t} else {\r\n                \tset resp.http.Set-Cookie = resp.http.Set-Cookie +\r\n                    \t\"; domain=\" + regsub(req.http.Host, \":\\d+$\", \"\");\r\n            \t}\r\n\t        }\r\n        }\r\n        set resp.http.Set-Cookie = resp.http.Set-Cookie + \"; httponly\";\r\n        unset resp.http.X-Varnish-Cookie-Expires;\r\n    }\r\n    if (req.http.X-Varnish-Esi-Method == \"ajax\" && req.http.X-Varnish-Esi-Access == \"private\") {\r\n        set resp.http.Cache-Control = \"no-cache\";\r\n    }\r\n    if (false || client.ip ~ debug_acl) {\r\n        # debugging is on, give some extra info\r\n        set resp.http.X-Varnish-Hits = obj.hits;\r\n        set resp.http.X-Varnish-Esi-Method = req.http.X-Varnish-Esi-Method;\r\n        set resp.http.X-Varnish-Esi-Access = req.http.X-Varnish-Esi-Access;\r\n        set resp.http.X-Varnish-Currency = req.http.X-Varnish-Currency;\r\n        set resp.http.X-Varnish-Store = req.http.X-Varnish-Store;\r\n    } else {\r\n        # remove Varnish fingerprints\r\n        unset resp.http.X-Varnish;\r\n        unset resp.http.Via;\r\n        unset resp.http.X-Powered-By;\r\n        unset resp.http.Server;\r\n        unset resp.http.X-Turpentine-Cache;\r\n        unset resp.http.X-Turpentine-Esi;\r\n        unset resp.http.X-Turpentine-Flush-Events;\r\n        unset resp.http.X-Turpentine-Block;\r\n        unset resp.http.X-Varnish-Session;\r\n        unset resp.http.X-Varnish-Host;\r\n        unset resp.http.X-Varnish-URL;\r\n        # this header indicates the session that originally generated a cached\r\n        # page. it *must* not be sent to a client in production with lax\r\n        # session validation or that session can be hijacked\r\n        unset resp.http.X-Varnish-Set-Cookie;\r\n    }\r\n}\r\n\r\n## Backends\r\n\r\nbackend default {\r\n    .host = \"127.0.0.1\";\r\n    .port = \"8080\";\r\n   .first_byte_timeout = 300s;\r\n   .between_bytes_timeout = 300s;\r\n}\r\n\r\n\r\nbackend admin {\r\n    .host = \"127.0.0.1\";\r\n    .port = \"8080\";\r\n   .first_byte_timeout = 21600s;\r\n   .between_bytes_timeout = 21600s;\r\n}\r\n\r\n\r\n## ACLs\r\n\r\n#acl crawler_acl {\r\n#    \"127.0.0.1\";\r\n#}\r\n\r\nacl debug_acl {\r\n    \"127.0.0.1\";\r\n}\r\n\r\n## Custom Subroutines\r\n\r\nsub generate_session {\r\n    # generate a UUID and add `frontend=$UUID` to the Cookie header, or use SID\r\n    # from SID URL param\r\n    if (req.url ~ \".*[&?]SID=([^&]+).*\") {\r\n        set req.http.X-Varnish-Faked-Session = regsub(\r\n            req.url, \".*[&?]SID=([^&]+).*\", \"frontend=\\1\");\r\n    } else {\r\n        C{\r\n            char uuid_buf [50];\r\n            generate_uuid(uuid_buf);\r\n\t        static const struct gethdr_s VGC_HDR_REQ_VARNISH_FAKED_SESSION =\r\n            { HDR_REQ, \"\\030X-Varnish-Faked-Session:\"};\r\n            VRT_SetHdr(ctx,\r\n                &VGC_HDR_REQ_VARNISH_FAKED_SESSION,\r\n                uuid_buf,\r\n                vrt_magic_string_end\r\n            );\r\n        }C\r\n    }\r\n    if (req.http.Cookie) {\r\n        # client sent us cookies, just not a frontend cookie. try not to blow\r\n        # away the extra cookies\r\n        std.collect(req.http.Cookie);\r\n        set req.http.Cookie = req.http.X-Varnish-Faked-Session +\r\n            \"; \" + req.http.Cookie;\r\n    } else {\r\n        set req.http.Cookie = req.http.X-Varnish-Faked-Session;\r\n    }\r\n}\r\n\r\nsub generate_session_expires {\r\n    # sets X-Varnish-Cookie-Expires to now + esi_private_ttl in format:\r\n    #   Tue, 19-Feb-2013 00:14:27 GMT\r\n    # this isn't threadsafe but it shouldn't matter in this case\r\n    C{\r\n        time_t now = time(NULL);\r\n        struct tm now_tm = *gmtime(&now);\r\n        now_tm.tm_sec += 14400;\r\n        mktime(&now_tm);\r\n        char date_buf [50];\r\n        strftime(date_buf, sizeof(date_buf)-1, \"%a, %d-%b-%Y %H:%M:%S %Z\", &now_tm);\r\n\t    static const struct gethdr_s VGC_HDR_RESP_COOKIE_EXPIRES =\r\n        { HDR_RESP, \"\\031X-Varnish-Cookie-Expires:\"};\r\n        VRT_SetHdr(ctx,\r\n            &VGC_HDR_RESP_COOKIE_EXPIRES,\r\n            date_buf,\r\n            vrt_magic_string_end\r\n        );\r\n    }C\r\n}\r\n\r\n## Varnish Subroutines\r\n\r\nsub vcl_recv {\r\n    # this always needs to be done so it's up at the top\r\n    if (req.restarts == 0) {\r\n        if (req.http.X-Forwarded-For) {\r\n            set req.http.X-Forwarded-For =\r\n                req.http.X-Forwarded-For + \", \" + client.ip;\r\n        } else {\r\n            set req.http.X-Forwarded-For = client.ip;\r\n        }\r\n    }\r\n\r\n    # We only deal with GET and HEAD by default\r\n    # we test this here instead of inside the url base regex section\r\n    # so we can disable caching for the entire site if needed\r\n    if (!true || req.http.Authorization ||\r\n        req.method !~ \"^(GET|HEAD)$\" ||\r\n        req.http.Cookie ~ \"varnish_bypass=1\") {\r\n        return (pipe);\r\n    }\r\n\r\n    # remove double slashes from the URL, for higher cache hit rate\r\n    set req.url = regsuball(req.url, \"(.*)//+(.*)\", \"\\1/\\2\");\r\n\r\n    if (req.http.Accept-Encoding) {\r\n        if (req.http.Accept-Encoding ~ \"gzip\") {\r\n            set req.http.Accept-Encoding = \"gzip\";\r\n        } else if (req.http.Accept-Encoding ~ \"deflate\") {\r\n            set req.http.Accept-Encoding = \"deflate\";\r\n        } else {\r\n            # unkown algorithm\r\n            unset req.http.Accept-Encoding;\r\n        }\r\n    }\r\n\r\n    #if (req.http.User-Agent ~ \"(?i)(ads|google|bing|msn|yandex|baidu|ro|career|)bot\" ||\r\n\t#\treq.http.User-Agent ~ \"(?i)(baidu|jike|symantec)spider\" ||\r\n\t#\treq.http.User-Agent ~ \"(?i)scanner\" ||\r\n\t#\treq.http.User-Agent ~ \"(?i)(web)crawler\") {\r\n\t#\tset req.http.X-Normalized-User-Agent = \"bot\";\r\n    #} else {\r\n    #    set req.http.X-Normalized-User-Agent = \"other\";\r\n    #}\r\n\r\n    # check if the request is for part of magento\r\n    if (req.url ~ \"^(/media/|/skin/|/js/|/)(?:(?:index|litespeed)\\.php/)?\") {\r\n        # set this so Turpentine can see the request passed through Varnish\r\n        set req.http.X-Turpentine-Secret-Handshake = \"1\";\r\n        # use the special admin backend and pipe if it's for the admin section\r\n        if (req.url ~ \"^(/media/|/skin/|/js/|/)(?:(?:index|litespeed)\\.php/)?admin\") {\r\n            set req.backend_hint = admin;\r\n            return (pipe);\r\n        }\r\n        if (req.http.Cookie ~ \"\\bcurrency=\") {\r\n            set req.http.X-Varnish-Currency = regsub(\r\n                req.http.Cookie, \".*\\bcurrency=([^;]*).*\", \"\\1\");\r\n        }\r\n        if (req.http.Cookie ~ \"\\bstore=\") {\r\n            set req.http.X-Varnish-Store = regsub(\r\n                req.http.Cookie, \".*\\bstore=([^;]*).*\", \"\\1\");\r\n        }\r\n        # looks like an ESI request, add some extra vars for further processing\r\n        if (req.url ~ \"/turpentine/esi/get(?:Block|FormKey)/\") {\r\n            set req.http.X-Varnish-Esi-Method = regsub(\r\n                req.url, \".*/method/(\\w+)/.*\", \"\\1\");\r\n            set req.http.X-Varnish-Esi-Access = regsub(\r\n                req.url, \".*/access/(\\w+)/.*\", \"\\1\");\r\n\r\n            # throw a forbidden error if debugging is off and a esi block is\r\n            # requested by the user (does not apply to ajax blocks)\r\n            if (req.http.X-Varnish-Esi-Method == \"esi\" && req.esi_level == 0 &&\r\n                    !(false || client.ip ~ debug_acl)) {\r\n                return (synth(403, \"External ESI requests are not allowed\"));\r\n            }\r\n        }\r\n        # no frontend cookie was sent to us\r\n        if (req.http.Cookie !~ \"frontend=\") {\r\n            #if (client.ip ~ crawler_acl ||\r\n            #        req.http.User-Agent ~ \"^(?:ApacheBench/.*|.*Googlebot.*|JoeDog/.*Siege.*|magespeedtest\\.com|Nexcessnet_Turpentine/.*)$\") {\r\n                # it's a crawler, give it a fake cookie\r\n            #    set req.http.Cookie = \"frontend=crawler-session\";\r\n            #} else {\r\n                # it's a real user, make up a new session for them\r\n                call generate_session;\r\n            #}\r\n        }\r\n        if (true &&\r\n                req.url ~ \".*\\.(?:css|js|jpe?g|png|gif|ico|swf)(?=\\?|&|$)\") {\r\n            # don't need cookies for static assets\r\n            unset req.http.Cookie;\r\n            unset req.http.X-Varnish-Faked-Session;\r\n            return (hash);\r\n        }\r\n        # this doesn't need a enable_url_excludes because we can be reasonably\r\n        # certain that cron.php at least will always be in it, so it will\r\n        # never be empty\r\n        if (req.url ~ \"^(/media/|/skin/|/js/|/)(?:(?:index|litespeed)\\.php/)?(?:admin|api|cron\\.php|registration/form|oauth|site|scripts)\" ||\r\n                # user switched stores. we pipe this instead of passing below because\r\n                # switching stores doesn't redirect (302), just acts like a link to\r\n                # another page (200) so the Set-Cookie header would be removed\r\n                req.url ~ \"\\?.*__from_store=\") {\r\n            return (pipe);\r\n        }\r\n        if (true &&\r\n                req.url ~ \"(?:[?&](?:__SID|XDEBUG_PROFILE)(?=[&=]|$))\") {\r\n            # TODO: should this be pass or pipe?\r\n            return (pass);\r\n        }\r\n        if (req.url ~ \"[?&](utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=\") {\r\n            # Strip out Google related parameters\r\n            set req.url = regsuball(req.url, \"(?:(\\?)?|&)(?:utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=[^&]+\", \"\\1\");\r\n            set req.url = regsuball(req.url, \"(?:(\\?)&|\\?$)\", \"\\1\");\r\n        }\r\n\r\n        # everything else checks out, try and pull from the cache\r\n        return (hash);\r\n    }\r\n    # else it's not part of magento so do default handling (doesn't help\r\n    # things underneath magento but we can't detect that)\r\n}\r\n\r\nsub vcl_pipe {\r\n    # since we're not going to do any stuff to the response we pretend the\r\n    # request didn't pass through Varnish\r\n    unset bereq.http.X-Turpentine-Secret-Handshake;\r\n    set bereq.http.Connection = \"close\";\r\n}\r\n\r\n# sub vcl_pass {\r\n#     return (pass);\r\n# }\r\n\r\nsub vcl_hash {\r\n    hash_data(req.url);\r\n    if (req.http.Host) {\r\n        hash_data(req.http.Host);\r\n    } else {\r\n        hash_data(server.ip);\r\n    }\r\n    hash_data(req.http.Ssl-Offloaded);\r\n    if (req.http.X-Normalized-User-Agent) {\r\n        hash_data(req.http.X-Normalized-User-Agent);\r\n    }\r\n    if (req.http.Accept-Encoding) {\r\n        # make sure we give back the right encoding\r\n        hash_data(req.http.Accept-Encoding);\r\n    }\r\n    if (req.http.X-Varnish-Store || req.http.X-Varnish-Currency) {\r\n        # make sure data is for the right store and currency based on the *store*\r\n        # and *currency* cookies\r\n        hash_data(\"s=\" + req.http.X-Varnish-Store + \"&c=\" + req.http.X-Varnish-Currency);\r\n    }\r\n\r\n    if (req.http.X-Varnish-Esi-Access == \"private\" &&\r\n            req.http.Cookie ~ \"frontend=\") {\r\n        hash_data(regsub(req.http.Cookie, \"^.*?frontend=([^;]*);*.*$\", \"\\1\"));\r\n        \r\n\r\n    }\r\n    return (lookup);\r\n}\r\n\r\nsub vcl_hit {\r\n    # this seems to cause cache object contention issues so removed for now\r\n    # TODO: use obj.hits % something maybe\r\n    # if (obj.hits > 0) {\r\n    #     set obj.ttl = obj.ttl + s;\r\n    # }\r\n}\r\n\r\n# sub vcl_miss {\r\n#     return (fetch);\r\n# }\r\n\r\nsub vcl_backend_response {\r\n    # set the grace period\r\n    set beresp.grace = 15s;\r\n\r\n    # Store the URL in the response object, to be able to do lurker friendly bans later\r\n    set beresp.http.X-Varnish-Host = bereq.http.host;\r\n    set beresp.http.X-Varnish-URL = bereq.url;\r\n\r\n    # if it's part of magento...\r\n    if (bereq.url ~ \"^(/media/|/skin/|/js/|/)(?:(?:index|litespeed)\\.php/)?\") {\r\n        # we handle the Vary stuff ourselves for now, we'll want to actually\r\n        # use this eventually for compatibility with downstream proxies\r\n        # TODO: only remove the User-Agent field from this if it exists\r\n        unset beresp.http.Vary;\r\n        # we pretty much always want to do this\r\n        set beresp.do_gzip = true;\r\n\r\n        if (beresp.status != 200 && beresp.status != 404) {\r\n            # pass anything that isn't a 200 or 404\r\n            set beresp.ttl = 15s;\r\n            set beresp.uncacheable = true;\r\n            return (deliver);\r\n        } else {\r\n            # if Magento sent us a Set-Cookie header, we'll put it somewhere\r\n            # else for now\r\n            if (beresp.http.Set-Cookie) {\r\n                set beresp.http.X-Varnish-Set-Cookie = beresp.http.Set-Cookie;\r\n                unset beresp.http.Set-Cookie;\r\n            }\r\n            # we'll set our own cache headers if we need them\r\n            unset beresp.http.Cache-Control;\r\n            unset beresp.http.Expires;\r\n            unset beresp.http.Pragma;\r\n            unset beresp.http.Cache;\r\n            unset beresp.http.Age;\r\n\r\n            if (beresp.http.X-Turpentine-Esi == \"1\") {\r\n                set beresp.do_esi = true;\r\n            }\r\n            if (beresp.http.X-Turpentine-Cache == \"0\") {\r\n                set beresp.ttl = 15s;\r\n                set beresp.uncacheable = true;\r\n                return (deliver);\r\n            } else {\r\n                if (true &&\r\n                        bereq.url ~ \".*\\.(?:css|js|jpe?g|png|gif|ico|swf)(?=\\?|&|$)\") {\r\n                    # it's a static asset\r\n                    set beresp.ttl = 2592000s;\r\n                    set beresp.http.Cache-Control = \"max-age=2592000\";\r\n                } elseif (bereq.http.X-Varnish-Esi-Method) {\r\n                    # it's a ESI request\r\n                    if (bereq.http.X-Varnish-Esi-Access == \"private\" &&\r\n                            bereq.http.Cookie ~ \"frontend=\") {\r\n                        # set this header so we can ban by session from Turpentine\r\n                        set beresp.http.X-Varnish-Session = regsub(bereq.http.Cookie,\r\n                            \"^.*?frontend=([^;]*);*.*$\", \"\\1\");\r\n                    }\r\n                    if (bereq.http.X-Varnish-Esi-Method == \"ajax\" &&\r\n                            bereq.http.X-Varnish-Esi-Access == \"public\") {\r\n                        set beresp.http.Cache-Control = \"max-age=\" + regsub(\r\n                            bereq.url, \".*/ttl/(\\d+)/.*\", \"\\1\");\r\n                    }\r\n                    set beresp.ttl = std.duration(\r\n                        regsub(\r\n                            bereq.url, \".*/ttl/(\\d+)/.*\", \"\\1s\"),\r\n                        300s);\r\n                    if (beresp.ttl == 0s) {\r\n                        # this is probably faster than bothering with 0 ttl\r\n                        # cache objects\r\n                        set beresp.ttl = 15s;\r\n                        set beresp.uncacheable = true;\r\n                        return (deliver);\r\n                    }\r\n                } else {\r\n                    set beresp.ttl = 3600s;\r\n                }\r\n            }\r\n        }\r\n        # we've done what we need to, send to the client\r\n        return (deliver);\r\n    }\r\n    # else it's not part of Magento so use the default Varnish handling\r\n}\r\n\r\n```\r\n\r\nnginx 1.6.2 - nginx.conf\r\n```\r\nuser www-data;\r\nworker_processes 32;\r\nworker_rlimit_nofile 100000;\r\nerror_log  /var/log/nginx/error.log;\r\npid        /var/run/nginx.pid;\r\n\r\nevents {\r\n  worker_connections 4000;\r\n      multi_accept on;\r\n        use epoll;\r\n  }\r\n\r\nhttp {\r\n  include       /etc/nginx/mime.types;\r\n  default_type  application/octet-stream;\r\n\r\n  access_log  off;\r\n\r\n  sendfile    on;\r\n  tcp_nopush on;\r\n\r\n  # Add Support for Real IPs if Varnish is Enabled\r\n  \r\n  set_real_ip_from 127.0.0.1;\r\n  real_ip_header X-Cluster-Client-Ip;\r\n   \r\n  server_tokens off;\r\n\r\n  types_hash_max_size 2048;\r\n  types_hash_bucket_size 512;\r\n\r\n  server_names_hash_bucket_size 64;\r\n  server_names_hash_max_size 512;\r\n\r\n  keepalive_timeout 15;\r\n  tcp_nodelay on;\r\n  \r\n  gzip         on;\r\n  gzip_disable \"MSIE [1-6]\\.(?!.*SV1)\";\r\n  gzip_types   text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;\r\n\r\n  include /etc/nginx/conf.d/*.conf;\r\n  include /etc/nginx/sites-enabled/*;\r\n}\r\n```\r\n\r\nnginx site.conf\r\n```\r\nserver {\r\n  listen                *:8080;\r\n\r\n  server_name           xx.co.uk www.xx.co.uk;\r\n\r\n  root /var/www/vhosts/magento-capistrano/current;\r\n    index  index.html index.htm index.php;\r\n\r\n\r\n  access_log            off;\r\n\r\n  error_log             /var/log/nginx/xx.co.uk.error.log;    # Base Location\r\n    location / {\r\n    \t\t\t        \r\n        index index.html index.php;    # Display Static File\r\n        try_files $uri $uri/ @handler; # Pass URI to Front Handler\r\n        expires 30d;                   # Cache Assets\r\n    }\r\n\r\n    # Denied Locations\r\n    location ^~ /app/                { deny all; }\r\n    location ^~ /includes/           { deny all; }\r\n    location ^~ /lib/                { deny all; }\r\n    location ^~ /media/downloadable/ { deny all; }\r\n    location ^~ /pkginfo/            { deny all; }\r\n    location ^~ /report/config.xml   { deny all; }\r\n    location ^~ /var/                { deny all; }\r\n\r\n    # Turn off Logs for Media\r\n    location ~* \"^.+\\.(jpg|jpeg|gif|css|png|js|ico|pdf|zip|tar|t?gz|mp3|wav|swf|mp4|webm)$\" {\r\n        expires    max;\r\n        add_header Cache-Control public;\r\n        access_log off;\r\n    }\r\n   \r\n   # Hidden Files\r\n    location  /. {\r\n        return 404;\r\n    }\r\n\r\n    # Rewrite Internal Requests\r\n    location @handler {\r\n        rewrite / /index.php;\r\n    }\r\n\r\n    # Rewrite PHP Requests\r\n    location ~ .php/ {\r\n        rewrite ^(.*.php)/ $1 last;\r\n    }\r\n\r\n    # Magento Location\r\n    location ~ \\.php$ {\r\n        # Catch 404s\r\n        if (!-e $request_filename) {\r\n            rewrite / /index.php last;\r\n        }\r\n\r\n        # Do not cache dynamic content\r\n        expires        off;\r\n        \r\n        fastcgi_read_timeout 900s;\r\n        \r\n        fastcgi_pass   unix:/var/run/hhvm/hhvm.sock; # Increase net.core.somaxconn=500 in /etc/sysctl.conf\r\n\r\n        # Increase timeout for longer running scripts\r\n        location ~ (/index.php/admin/|/admin/).*$ {\r\n            fastcgi_param PHP_VALUE \"max_execution_time = 600\";\r\n        }\r\n\r\n        fastcgi_param  SCRIPT_FILENAME           $document_root$fastcgi_script_name;\r\n        fastcgi_param  MAGE_RUN_CODE             xx;\r\n        fastcgi_param  MAGE_RUN_TYPE             store;\r\n        \r\n        include        fastcgi_params;\r\n    }\r\n}\r\n```\r\n\r\nhhvm\r\n```\r\nHipHop VM 3.4.0-dev~nightly.2014.10.23 (rel)\r\nCompiler: heads/master-0-g0b46339509ebd9612623d52c296fdcc8e6f93dff\r\nRepo schema: 008a6ee43591a5f00ada353a546f193c91f3a4b9\r\nExtension API: 20140829\r\n```\r\n\r\n/etc/sysctl.conf\r\n```\r\nnet.core.netdev_max_backlog=4096\r\n\r\nnet.core.somaxconn=4096\r\n\r\nnet.ipv4.tcp_max_syn_backlog=4096\r\n\r\nnet.netfilter.nf_conntrack_max=131072\r\n\r\nnet.ipv4.netfilter.ip_conntrack_generic_timeout=120\r\n\r\nnet.ipv4.netfilter.ip_conntrack_tcp_timeout_established=54000\r\n\r\nvm.overcommit_memory=1\r\n```\r\n\r\n/etc/security/limits.conf\r\n```\r\n* hard memlock 200000\r\n* hard nofile 300000\r\n* soft memlock 82000\r\n* soft nofile 131072\r\n```\r\n\r\nulimit -a\r\n```\r\ncore file size          (blocks, -c) 0\r\ndata seg size           (kbytes, -d) unlimited\r\nscheduling priority             (-e) 0\r\nfile size               (blocks, -f) unlimited\r\npending signals                 (-i) 966731\r\nmax locked memory       (kbytes, -l) 82000\r\nmax memory size         (kbytes, -m) unlimited\r\nopen files                      (-n) 131072\r\npipe size            (512 bytes, -p) 8\r\nPOSIX message queues     (bytes, -q) 819200\r\nreal-time priority              (-r) 0\r\nstack size              (kbytes, -s) 8192\r\ncpu time               (seconds, -t) unlimited\r\nmax user processes              (-u) 966731\r\nvirtual memory          (kbytes, -v) unlimited\r\nfile locks                      (-x) unlimited\r\n```"},"comment":{"url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/comments/119225316","html_url":"https://github.com/nexcess/magento-turpentine/issues/659#issuecomment-119225316","issue_url":"https://api.github.com/repos/nexcess/magento-turpentine/issues/659","id":119225316,"user":{"login":"miguelbalparda","id":3997682,"avatar_url":"https://avatars.githubusercontent.com/u/3997682?v=3","gravatar_id":"","url":"https://api.github.com/users/miguelbalparda","html_url":"https://github.com/miguelbalparda","followers_url":"https://api.github.com/users/miguelbalparda/followers","following_url":"https://api.github.com/users/miguelbalparda/following{/other_user}","gists_url":"https://api.github.com/users/miguelbalparda/gists{/gist_id}","starred_url":"https://api.github.com/users/miguelbalparda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miguelbalparda/subscriptions","organizations_url":"https://api.github.com/users/miguelbalparda/orgs","repos_url":"https://api.github.com/users/miguelbalparda/repos","events_url":"https://api.github.com/users/miguelbalparda/events{/privacy}","received_events_url":"https://api.github.com/users/miguelbalparda/received_events","type":"User","site_admin":false},"created_at":"2015-07-07T14:43:39Z","updated_at":"2015-07-07T14:43:39Z","body":"This seems to be fixed by #470, closing the issue."}}