{"action":"created","issue":{"url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/1","labels_url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/1/comments","events_url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/1/events","html_url":"https://github.com/otherchirps/nsw_gov_docs/issues/1","id":83784822,"number":1,"title":"scraper being killed after requirements install","user":{"login":"otherchirps","id":385384,"avatar_url":"https://avatars.githubusercontent.com/u/385384?v=3","gravatar_id":"","url":"https://api.github.com/users/otherchirps","html_url":"https://github.com/otherchirps","followers_url":"https://api.github.com/users/otherchirps/followers","following_url":"https://api.github.com/users/otherchirps/following{/other_user}","gists_url":"https://api.github.com/users/otherchirps/gists{/gist_id}","starred_url":"https://api.github.com/users/otherchirps/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/otherchirps/subscriptions","organizations_url":"https://api.github.com/users/otherchirps/orgs","repos_url":"https://api.github.com/users/otherchirps/repos","events_url":"https://api.github.com/users/otherchirps/events{/privacy}","received_events_url":"https://api.github.com/users/otherchirps/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"otherchirps","id":385384,"avatar_url":"https://avatars.githubusercontent.com/u/385384?v=3","gravatar_id":"","url":"https://api.github.com/users/otherchirps","html_url":"https://github.com/otherchirps","followers_url":"https://api.github.com/users/otherchirps/followers","following_url":"https://api.github.com/users/otherchirps/following{/other_user}","gists_url":"https://api.github.com/users/otherchirps/gists{/gist_id}","starred_url":"https://api.github.com/users/otherchirps/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/otherchirps/subscriptions","organizations_url":"https://api.github.com/users/otherchirps/orgs","repos_url":"https://api.github.com/users/otherchirps/repos","events_url":"https://api.github.com/users/otherchirps/events{/privacy}","received_events_url":"https://api.github.com/users/otherchirps/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2015-06-02T01:45:37Z","updated_at":"2015-06-02T17:41:59Z","closed_at":null,"body":"# Scraper error\r\n\r\nScraper appears to be killed immediately after requirement installation, with the message:\r\n\r\n> /start: line 24:    21 Killed                  setuidgid scraper $(eval echo ${command})\r\n\r\nThis \"start\" script is not part of this scraper's codebase.  \r\n\r\nAlso, the scraper itself wasn't altered before this started happening.  So perhaps the morph framework has altered lately, and as a side effect, this scraper is no longer playing as nicely as it should?  \r\n\r\nOr maybe it's starting to hit a memory limit?  Don't know yet.\r\n\r\n# Oddness\r\n\r\nSomething seems to run at some point though, since the morph run status log shows things like:\r\n\r\n```\r\nManually ran revision 97c8157a and failed 2015-06-01.\r\nrun time 213.1 s\r\n533 records added, 533 records removed in the database\r\n11 pages scraped \r\n```\r\n\r\nThe number of records added appears a bit random at the moment, but if it was being killed before running, you'd think this would be zero.\r\n\r\n# Install logs\r\n\r\nHere is the output of the scraper install & attempt to run.  Might need to add more debug logs to see if it actually does get to run at all or not:\r\n\r\n```\r\nInjecting configuration and compiling...\r\n\u001B[1G-----> Python app detected\r\n\u001B[1G-----> Stack changed, re-installing runtime\r\n\u001B[1G-----> Installing runtime (python-2.7.6)\r\n\u001B[1G-----> Installing dependencies with pip\r\n\u001B[1G       Collecting scraperwiki==0.4.1 (from -r requirements.txt (line 1))\r\n\u001B[1G       Downloading scraperwiki-0.4.1.tar.gz\r\n\u001B[1G       Collecting Scrapy==0.24.6 (from -r requirements.txt (line 2))\r\n\u001B[1G       Downloading Scrapy-0.24.6-py2-none-any.whl (444kB)\r\n\u001B[1G       Collecting service-identity==14.0.0 (from -r requirements.txt (line 3))\r\n\u001B[1G       Downloading service_identity-14.0.0-py2.py3-none-any.whl\r\n\u001B[1G       Collecting requests (from scraperwiki==0.4.1->-r requirements.txt (line 1))\r\n\u001B[1G       Downloading requests-2.7.0-py2.py3-none-any.whl (470kB)\r\n\u001B[1G       Collecting sqlalchemy (from scraperwiki==0.4.1->-r requirements.txt (line 1))\r\n\u001B[1G       Downloading SQLAlchemy-1.0.4.tar.gz (4.6MB)\r\n\u001B[1G       Collecting alembic (from scraperwiki==0.4.1->-r requirements.txt (line 1))\r\n\u001B[1G       Downloading alembic-0.7.6.tar.gz (623kB)\r\n\u001B[1G       Collecting cssselect>=0.9 (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading cssselect-0.9.1.tar.gz\r\n\u001B[1G       Collecting queuelib (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading queuelib-1.2.2-py2.py3-none-any.whl\r\n\u001B[1G       Collecting pyOpenSSL (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading pyOpenSSL-0.15.1-py2.py3-none-any.whl (102kB)\r\n\u001B[1G       Collecting w3lib>=1.8.0 (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading w3lib-1.11.0-py2.py3-none-any.whl\r\n\u001B[1G       Collecting lxml (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading lxml-3.4.4.tar.gz (3.5MB)\r\n\u001B[1G       Building lxml version 3.4.4.\r\n\u001B[1G       Building without Cython.\r\n\u001B[1G       Using build configuration of libxslt 1.1.28\r\n\u001B[1G       /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'\r\n\u001B[1G       warnings.warn(msg)\r\n\u001B[1G       Collecting Twisted>=10.0.0 (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading Twisted-15.2.1.tar.bz2 (4.6MB)\r\n\u001B[1G       Collecting six>=1.5.2 (from Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading six-1.9.0-py2.py3-none-any.whl\r\n\u001B[1G       Collecting characteristic>=14.0.0 (from service-identity==14.0.0->-r requirements.txt (line 3))\r\n\u001B[1G       Downloading characteristic-14.3.0-py2.py3-none-any.whl\r\n\u001B[1G       Collecting pyasn1-modules (from service-identity==14.0.0->-r requirements.txt (line 3))\r\n\u001B[1G       Downloading pyasn1-modules-0.0.5.tar.gz\r\n\u001B[1G       Collecting pyasn1 (from service-identity==14.0.0->-r requirements.txt (line 3))\r\n\u001B[1G       Downloading pyasn1-0.1.7.tar.gz (68kB)\r\n\u001B[1G       Collecting Mako (from alembic->scraperwiki==0.4.1->-r requirements.txt (line 1))\r\n\u001B[1G       Downloading Mako-1.0.1.tar.gz (473kB)\r\n\u001B[1G       Collecting cryptography>=0.7 (from pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading cryptography-0.9.tar.gz (302kB)\r\n\u001B[1G       Collecting zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading zope.interface-4.1.2.tar.gz (919kB)\r\n\u001B[1G       Collecting MarkupSafe>=0.9.2 (from Mako->alembic->scraperwiki==0.4.1->-r requirements.txt (line 1))\r\n\u001B[1G       Downloading MarkupSafe-0.23.tar.gz\r\n\u001B[1G       Collecting idna (from cryptography>=0.7->pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading idna-2.0-py2.py3-none-any.whl (61kB)\r\n\u001B[1G       Collecting enum34 (from cryptography>=0.7->pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading enum34-1.0.4.tar.gz\r\n\u001B[1G       Collecting ipaddress (from cryptography>=0.7->pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading ipaddress-1.0.7-py27-none-any.whl\r\n\u001B[1G       Collecting cffi>=0.8 (from cryptography>=0.7->pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading cffi-1.1.0.tar.gz (323kB)\r\n\u001B[1G       Collecting pycparser (from cffi>=0.8->cryptography>=0.7->pyOpenSSL->Scrapy==0.24.6->-r requirements.txt (line 2))\r\n\u001B[1G       Downloading pycparser-2.13.tar.gz (299kB)\r\n\u001B[1G       Installing collected packages: pycparser, cffi, ipaddress, enum34, idna, MarkupSafe, zope.interface, cryptography, Mako, pyasn1, pyasn1-modules, characteristic, six, Twisted, lxml, w3lib, pyOpenSSL, queuelib, cssselect, alembic, sqlalchemy, requests, service-identity, Scrapy, scraperwiki\r\n\u001B[1G       Running setup.py install for pycparser\r\n\u001B[1G       Build the lexing/parsing tables\r\n\u001B[1G       Running setup.py install for cffi\r\n\u001B[1G       building '_cffi_backend' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/app/.heroku/python/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/c/_cffi_backend.o -L/app/.heroku/python/lib -lffi -lpython2.7 -o build/lib.linux-x86_64-2.7/_cffi_backend.so\r\n\u001B[1G       \r\n\u001B[1G       Running setup.py install for enum34\r\n\u001B[1G       \r\n\u001B[1G       Running setup.py install for MarkupSafe\r\n\u001B[1G       building 'markupsafe._speedups' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c markupsafe/_speedups.c -o build/temp.linux-x86_64-2.7/markupsafe/_speedups.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/markupsafe/_speedups.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/markupsafe/_speedups.so\r\n\u001B[1G       Running setup.py install for zope.interface\r\n\u001B[1G       building 'zope.interface._zope_interface_coptimizations' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.7/src/zope/interface/_zope_interface_coptimizations.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/zope/interface/_zope_interface_coptimizations.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/zope/interface/_zope_interface_coptimizations.so\r\n\u001B[1G       Skipping installation of /app/.heroku/python/lib/python2.7/site-packages/zope/__init__.py (namespace package)\r\n\u001B[1G       Installing /app/.heroku/python/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-nspkg.pth\r\n\u001B[1G       Running setup.py install for cryptography\r\n\u001B[1G       zip_safe flag not set; analyzing archive contents...\r\n\u001B[1G       six: module references __path__\r\n\u001B[1G       Installed /tmp/pip-build-VOb276/cryptography/.eggs/six-1.9.0-py2.7.egg\r\n\u001B[1G       Searching for pyasn1\r\n\u001B[1G       Reading https://pypi.python.org/simple/pyasn1/\r\n\u001B[1G       Best match: pyasn1 0.1.7\r\n\u001B[1G       Downloading https://pypi.python.org/packages/2.7/p/pyasn1/pyasn1-0.1.7-py2.7.egg#md5=15f079cabee01402bf86ca8b83356469\r\n\u001B[1G       Processing pyasn1-0.1.7-py2.7.egg\r\n\u001B[1G       Moving pyasn1-0.1.7-py2.7.egg to /tmp/pip-build-VOb276/cryptography/.eggs\r\n\u001B[1G       Installed /tmp/pip-build-VOb276/cryptography/.eggs/pyasn1-0.1.7-py2.7.egg\r\n\u001B[1G       building '_Cryptography_cffi_e7d09016xc302a38b' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_e7d09016xc302a38b.c -o build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_e7d09016xc302a38b.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_e7d09016xc302a38b.o -L/app/.heroku/python/lib -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/cryptography/_Cryptography_cffi_e7d09016xc302a38b.so\r\n\u001B[1G       building '_Cryptography_cffi_590da19fxffc7b1ce' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_590da19fxffc7b1ce.c -o build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_590da19fxffc7b1ce.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_590da19fxffc7b1ce.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/cryptography/_Cryptography_cffi_590da19fxffc7b1ce.so\r\n\u001B[1G       building '_Cryptography_cffi_26cb75b8x62b488b1' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_26cb75b8x62b488b1.c -o build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_26cb75b8x62b488b1.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_26cb75b8x62b488b1.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/cryptography/_Cryptography_cffi_26cb75b8x62b488b1.so\r\n\u001B[1G       Running setup.py install for Mako\r\n\u001B[1G       Installing mako-render script to /app/.heroku/python/bin\r\n\u001B[1G       Running setup.py install for pyasn1\r\n\u001B[1G       Running setup.py install for pyasn1-modules\r\n\u001B[1G       \r\n\u001B[1G       \r\n\u001B[1G       Running setup.py install for Twisted\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c conftest.c -o conftest.o\r\n\u001B[1G       building 'twisted.runner.portmap' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/runner/portmap.c -o build/temp.linux-x86_64-2.7/twisted/runner/portmap.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/runner/portmap.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/twisted/runner/portmap.so\r\n\u001B[1G       building 'twisted.test.raiser' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/test/raiser.c -o build/temp.linux-x86_64-2.7/twisted/test/raiser.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/test/raiser.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/twisted/test/raiser.so\r\n\u001B[1G       building 'twisted.python.sendmsg' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/python/sendmsg.c -o build/temp.linux-x86_64-2.7/twisted/python/sendmsg.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/python/sendmsg.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/twisted/python/sendmsg.so\r\n\u001B[1G       changing mode of build/scripts-2.7/pyhtmlizer from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/tap2rpm from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/manhole from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/tap2deb from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/trial from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/twistd from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/tkconch from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/conch from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/ckeygen from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/cftp from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/mailmail from 644 to 755\r\n\u001B[1G       changing mode of build/scripts-2.7/lore from 644 to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/tkconch to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/conch to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/pyhtmlizer to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/tap2rpm to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/manhole to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/ckeygen to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/tap2deb to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/mailmail to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/trial to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/twistd to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/lore to 755\r\n\u001B[1G       changing mode of /app/.heroku/python/bin/cftp to 755\r\n\u001B[1G       Running setup.py install for lxml\r\n\u001B[1G       Building lxml version 3.4.4.\r\n\u001B[1G       Building without Cython.\r\n\u001B[1G       Using build configuration of libxslt 1.1.28\r\n\u001B[1G       /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'\r\n\u001B[1G       warnings.warn(msg)\r\n\u001B[1G       building 'lxml.etree' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-VOb276/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -L/app/.heroku/python/lib -lxslt -lexslt -lxml2 -lz -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/lxml/etree.so\r\n\u001B[1G       building 'lxml.objectify' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-VOb276/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -w\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -L/app/.heroku/python/lib -lxslt -lexslt -lxml2 -lz -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/lxml/objectify.so\r\n\u001B[1G       \r\n\u001B[1G       \r\n\u001B[1G       \r\n\u001B[1G       Running setup.py install for cssselect\r\n\u001B[1G       Running setup.py install for alembic\r\n\u001B[1G       Installing alembic script to /app/.heroku/python/bin\r\n\u001B[1G       Running setup.py install for sqlalchemy\r\n\u001B[1G       building 'sqlalchemy.cprocessors' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c lib/sqlalchemy/cextension/processors.c -o build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/processors.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/processors.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/sqlalchemy/cprocessors.so\r\n\u001B[1G       building 'sqlalchemy.cresultproxy' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c lib/sqlalchemy/cextension/resultproxy.c -o build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/resultproxy.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/resultproxy.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/sqlalchemy/cresultproxy.so\r\n\u001B[1G       building 'sqlalchemy.cutils' extension\r\n\u001B[1G       gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c lib/sqlalchemy/cextension/utils.c -o build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/utils.o\r\n\u001B[1G       gcc -pthread -shared build/temp.linux-x86_64-2.7/lib/sqlalchemy/cextension/utils.o -L/app/.heroku/python/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/sqlalchemy/cutils.so\r\n\u001B[1G       \r\n\u001B[1G       \r\n\u001B[1G       \r\n\u001B[1G       Running setup.py install for scraperwiki\r\n\u001B[1G       Successfully installed Mako-1.0.1 MarkupSafe-0.23 Scrapy-0.24.6 Twisted-15.2.1 alembic-0.7.6 cffi-1.1.0 characteristic-14.3.0 cryptography-0.9 cssselect-0.9.1 enum34-1.0.4 idna-2.0 ipaddress-1.0.7 lxml-3.4.4 pyOpenSSL-0.15.1 pyasn1-0.1.7 pyasn1-modules-0.0.5 pycparser-2.13 queuelib-1.2.2 requests-2.7.0 scraperwiki-0.4.1 service-identity-14.0.0 six-1.9.0 sqlalchemy-1.0.4 w3lib-1.11.0 zope.interface-4.1.2\r\n\u001B[1G       \r\n\u001B[1G-----> Discovering process types\r\n\u001B[1G       Procfile declares types -> scraper\r\nInjecting scraper and running...\r\n/start: line 24:    21 Killed                  setuidgid scraper $(eval echo ${command})\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/comments/108027692","html_url":"https://github.com/otherchirps/nsw_gov_docs/issues/1#issuecomment-108027692","issue_url":"https://api.github.com/repos/otherchirps/nsw_gov_docs/issues/1","id":108027692,"user":{"login":"otherchirps","id":385384,"avatar_url":"https://avatars.githubusercontent.com/u/385384?v=3","gravatar_id":"","url":"https://api.github.com/users/otherchirps","html_url":"https://github.com/otherchirps","followers_url":"https://api.github.com/users/otherchirps/followers","following_url":"https://api.github.com/users/otherchirps/following{/other_user}","gists_url":"https://api.github.com/users/otherchirps/gists{/gist_id}","starred_url":"https://api.github.com/users/otherchirps/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/otherchirps/subscriptions","organizations_url":"https://api.github.com/users/otherchirps/orgs","repos_url":"https://api.github.com/users/otherchirps/repos","events_url":"https://api.github.com/users/otherchirps/events{/privacy}","received_events_url":"https://api.github.com/users/otherchirps/received_events","type":"User","site_admin":false},"created_at":"2015-06-02T17:41:58Z","updated_at":"2015-06-02T17:41:58Z","body":"Nope. Script is being killed externally.  Exception handler not getting a bite of the cherry. \r\n\r\nThe scraper does fetch data for a little while before being killed.\r\n\r\nImplies it's hitting a resource limit, I guess.  Memory leak?"}}