{"push_id":682710613,"size":32,"distinct_size":32,"ref":"refs/heads/master","head":"a691e7ca59355248dec0f1247de6c40ad293eab9","before":"d1c28b2fe7876fc71c9cbaf3942b4e784773c3ad","commits":[{"sha":"c49cf131ddb16157ea1a1663eef63133efe0068d","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Implement read operations for queue.file\n\ntilequeue/queue/file.py\n        -Implement the `read()` and `job_done()` methods.\n        -Make `close()` remove all `read()` elements from the queue.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/c49cf131ddb16157ea1a1663eef63133efe0068d"},{"sha":"90b14d4bc28220aa2e5b8e125e351eb9fd65e7cf","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add a timestamp attribute to CoordMessage.\n\ntilequeue/tile.py\n\t-In `tilequeue/worker.py:325`, the `sqs_handle` attribute of a\n\t`CoordMessage` instance is expected to have a bunch of\n\tproperties specific to the messages received from SQS (ie the\n\treal thing). To stub it out properly, the timestamp should be\n\tpassed to the `CoordMessage` constructor and made an instance\n\tvariable instead.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/90b14d4bc28220aa2e5b8e125e351eb9fd65e7cf"},{"sha":"2fe0fba4f8d18c60938b877d422321297f13e79a","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add multithreaded support to queue.file\n\ntilequeue/queue/file.py\n\t-Use an `RLock` in all of the file manipulation functions, since\n\tmultiple readers might be using the same file queue and would\n\totherwise corrupt it.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/2fe0fba4f8d18c60938b877d422321297f13e79a"},{"sha":"489e503c458ad2d95f16945c63e1f4a5fc54fae5","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Make CoordMessage() tilestamp parameter optional.\n\ntilequeue/(tile, queue/file).py\n\t-Since the stubbed out queues (`queue.memory`, `queue.file`)\n\tcreates CoordMessage with simply `None` timestamps, it's a good\n\tidea to just make it optional.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/489e503c458ad2d95f16945c63e1f4a5fc54fae5"},{"sha":"d49af5e2aff4b37f8d584ed63ec85a0284bdaea0","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Check if queue_printer_thread_stop exists.\n\ntilequeue/command.py\n\t-If `log-queue-sizes` is disabled in the config, then\n\t`queue_printer_thread_stop` will be set to `None` and an\n\texception will be thrown when `queue_printer_thread_stop.set()`\n\tis called.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/d49af5e2aff4b37f8d584ed63ec85a0284bdaea0"},{"sha":"d89fd2348fb1b9fbd2d92d02e47e6b2cf936a302","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add message timestamp to the metadata dict.\n\ntilequeue/worker.py\n\t-90b14d4 removed the expectation of `sqs_handle` containing a\n\tmethod that allows you to retrieve its \"sent to queue\"\n\ttimestamp, to allow SQS to be stubbed out with something like\n\t`queue.file`, and added a timestamp instance variable to\n\t`CoordMessage` to allow it to be set on a per-queue basis.\n\t`SqsQueueWriter` is still getting the `SentTimestamp` via\n\t`sqs_handle`, rather than from the initially received\n\t`CoordMessage`'s `timestamp` attribute. Add it to the `metadata`\n\tdict inside `SqsQueueReader` instead, and use that inside\n\t`SqsQueueWriter`.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/d89fd2348fb1b9fbd2d92d02e47e6b2cf936a302"},{"sha":"4417b3c701e44fbf94fb7375a7a3f148f1ee6112","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Rename lock to _lock to imply that it's private.\n\ntilequeue/queue/file.py\n\t-The `lock` instance variable shouldn't be used outside of the\n\t`OutputFileQueue`'s methods.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/4417b3c701e44fbf94fb7375a7a3f148f1ee6112"},{"sha":"6bfd2d0cb6a92322391d7f5d5348594268e305b4","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Revert \"Rename lock to _lock to imply that it's private.\"\n\ntilequeue/queue/file.py\n\t-On second thought, the convention of prefixing private instance\n\tvariables with an underscore isn't consistently adhered to\n\telsewhere in the codebase, so don't bother using it, or we'll\n\tend up with a mix of classes that do and don't use it.\n\nThis reverts commit 4417b3c701e44fbf94fb7375a7a3f148f1ee6112.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/6bfd2d0cb6a92322391d7f5d5348594268e305b4"},{"sha":"cb023d1b4ec1f8f7c200c9560e4b4c402a0a5dcd","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Use the current time if no timestamp was provided.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/cb023d1b4ec1f8f7c200c9560e4b4c402a0a5dcd"},{"sha":"01a8fcb70ea75d854aaf16547b837d861750c160","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Use readline() instead of next() to detect changes.\n\ntilequeue/queue/file.py\n\t-`readline()` will pick up new lines appended to the file,\n\twhereas `next()` will not since the iterator will just hit\n\t`StopIteration` and stop generating new lines. Use `readline()`\n\tinstead, then, since it might be desirable to append something\n\tto the queue file and have tilequeue detect the new input\n\twithout having to restart.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/01a8fcb70ea75d854aaf16547b837d861750c160"},{"sha":"aa74d228044599e8fcc7088e7de5fccca176fb2d","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Turn TileFile into TileDirectory.\n\ntilequeue/store.py\n\t-It's better to write the tiles to individual tiles, since\n\tthat makes them easier to inspect and removes the need for\n\tlocking the file pointer (even though there are multiple writer\n\tthreads).","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/aa74d228044599e8fcc7088e7de5fccca176fb2d"},{"sha":"01b511b1f337f00eb72530692eec202611599c5a","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Fix a bug in OutputFileQueue.close().\n\ntilequeue/queue/file.py\n\t-01a8fcb made `OutputFileQueue.read()` use `readline()` instead\n\tof `next()`, but didn't update `OutputFileQueue.close()`, which\n\tuses a list comprehension to grab the rest of the file. Since\n\t`.read()` no longer uses the iteration protocol, `.close()` will\n\tstart iterating from the beginning of the file. Use `.read()`\n\tinstead of a list comprehension to only grab everything after\n\twhat `.readline()` already picked up.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/01b511b1f337f00eb72530692eec202611599c5a"},{"sha":"4e53fcef25248b3e35e7000c5b7567827a13839e","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add documentation comments to queue.file and store.\n\ntilequeue/(queue/file, store).py\n\t-Add some API-style doc comments for clarity.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/4e53fcef25248b3e35e7000c5b7567827a13839e"},{"sha":"811c0da6ee8ba9d04420b5cbd010523f75198a4b","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Call .close() on SqsQueueReader.sqs_queue.\n\ntilequeue/worker.py\n\t-It wasn't getting called before, which isn't a problem for\n\t`queue.sqs` (which has a simple `pass` method), but *needs* to\n\tbe called for `queue.file` (which needs to close its file\n\tpointer before the process exits).","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/811c0da6ee8ba9d04420b5cbd010523f75198a4b"},{"sha":"7ce4f18c15d7c57cb245f6358a7f2f0ac5a1cd37","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Make aws.{s3,sqs} top-level store/queue keys.\n\nconfig.yaml.sample, tilequeue/config.py\n\t-Since `s3` and `sqs` now have drop-in file/directory based\n\treplacements for local development, the configuration for both\n\t`tilequeue`'s queue and store should be contained inside more\n\tgenerically named properties than `aws.s3` and `aws.sqs` in the\n\tconfig, like simply `queue` and `store`.\n\t-Change the sample config file and `config.py` to support them.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/7ce4f18c15d7c57cb245f6358a7f2f0ac5a1cd37"},{"sha":"8e53b553ee61cb4a3953b7f4eada0e0c9488ed21","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Use peripherals.queue instead of make_queue().\n\ntilequeue/command.py\n\t-Most/all of the `tilequeue_*` methods create their own queue\n\tvia `make_queue()`, even though `tilequeue_main()`, which calls\n\tthem, creates a queue as well and passes it to them inside the\n\t`peripherals` object. Use that one instead of recreating one\n\twith `make_queue()`.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/8e53b553ee61cb4a3953b7f4eada0e0c9488ed21"},{"sha":"3fd253855bfd311bc4c9a61c333293bb2b70f8d9","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add support for the new local queue/store.\n\ntilequeue/command.py\n\t-The past few commits finalized a file-based queue\n\t(`tilequeue.queue.file.OutputFileQueue`) and a\n\tdirectory/file-based store (`tilequeue.store.TileDirectory`).\n\t-Integrate them in `tilequeue.command`.\n\t-Define a `make_store()` function, similar to `make_queue()`,\n\twhich is responsible for creating either an s3 or directory\n\tstore.\n\t-Add some logic to `make_queue()` to ensure that the desired\n\tpath for the file-queue does not exist/is a file.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/3fd253855bfd311bc4c9a61c333293bb2b70f8d9"},{"sha":"14e6dd5d93ef2807d27654d653eba55f88621c20","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Fix failing tests related to the config.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/14e6dd5d93ef2807d27654d653eba55f88621c20"},{"sha":"2a25e583933325c032c53e885b6dcac04e8061bb","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add unit tests for queue.file\n\ntests/queue/test_file.py\n\t-Add a bunch of `unittest` unit tests for most/all of the\n\tfunctions in `tilequeue.queue.file`, as per all of the recent\n\twork there.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/2a25e583933325c032c53e885b6dcac04e8061bb"},{"sha":"b1be1e21c18ffe97a854d903fe74ee6e49a7e45c","author":{"email":"8880fa4052a08f7310bf7f1d36b1b72ebb969012@gmail.com","name":"Severyn Kozak"},"message":"Add tests for tilequeue.store.TileDirectory.\n\ntests/store.py\n\t-Add a tests file for `tilequeue.store`.\n\t-Add a test suite for the `TileDirectory` class, with a single\n\tunit test.","distinct":true,"url":"https://api.github.com/repos/mapzen/tilequeue/commits/b1be1e21c18ffe97a854d903fe74ee6e49a7e45c"}]}