{"push_id":913989149,"size":1,"distinct_size":1,"ref":"refs/heads/diffbot-testing","head":"9147d6bb0287c436bba941f7bb7cdce6549a5093","before":"6f9f177d7c3f7bef75088d24ad7c51bf1c204eac","commits":[{"sha":"9147d6bb0287c436bba941f7bb7cdce6549a5093","author":{"email":"baee4bdce10f53c7f30b31ee3edd2d3a4d59287c@gigablast.com","name":"Matt Wells"},"message":"fix some diffbot crawls.\ndo not spider pages at the hopcount limit\nwhen 'only spider urls if new' is enabled.\nmeaning only spider each url once. (unless there is\na temporary error)\nfix malformed url bug some more.\nadded some commented out code for indexing spider replies\n(gbss docs) for certain fatal/critical errors, in which\ncase they are not being indexed.","distinct":true,"url":"https://api.github.com/repos/gigablast/open-source-search-engine/commits/9147d6bb0287c436bba941f7bb7cdce6549a5093"}]}