{"push_id":571884392,"size":2,"distinct_size":2,"ref":"refs/heads/refactor-export","head":"cfbfe1555cccdf684faefedd533d8919b542c2b1","before":"4577228ac12079237ceb73f7e0841e0a482eb8c5","commits":[{"sha":"bdbaf18a8de2bbf60e492380efaeed4f8da9d6bd","author":{"email":"e9d71f5ee7c92d6dc9e92ffdad17b8bd49418f98@devel.io","name":"Byron Ruth"},"message":"Refactor formatter and exporter APIs to reduce overhead\n\nA call to formatter(...) was doing a lot of work for every row being\nprocessed. The primary slow spots are:\n\n- Format methods being dynamically accessed via getattr() on every loop\n- Unpacking keyword arguments (**kwargs) in the method call\n- Building and moving OrderedDict values\n- Logging a message when a value failed be coerced\n\nTo remediate these issues, targeted format methods are now processed\nat initialization rather that process time. This requires the list of\nformats to be passed into the constructor and not in the __call__ method.\n\nKeyword arguments passed in during a call are no longer unpacked in the\nmethod call, but are passed as a keyword argument itself, `kwargs`.\n\nUse of OrderedDict has been removed altogether. These structures take up\na lot of memory and have proven to add little value during processing.\nThis required adding a method for getting the header/field data for the\nvalues being processed. This is also a desirable change for addressing\n\nLogging has been addressed by only doing while in DEBUG mode. Since the\nexception is skipped, there is no point in logging the errors while in\nproduction.\n\nThe exporters have been updated to accommodate this changes. In addition,\nnew read methods have been implemented for performance testing including\ncached and threaded versions. The previous read method have been renamed\nto manual_read since it performs manual limit and offsetting and distinct\nchecking.\n\nTo accommodate the new read method, the `write` on all exporters have been\nchanged to not call `read` itself. This change requires the passed iterable\nto already contain the formatted rows.\n\nNote: This refactor requires backwards incompatible changes to get the\nperformance gains.\n\nSigned-off-by: Byron Ruth <b@devel.io>","distinct":true,"url":"https://api.github.com/repos/chop-dbhi/avocado/commits/bdbaf18a8de2bbf60e492380efaeed4f8da9d6bd"},{"sha":"cfbfe1555cccdf684faefedd533d8919b542c2b1","author":{"email":"e9d71f5ee7c92d6dc9e92ffdad17b8bd49418f98@devel.io","name":"Byron Ruth"},"message":"Fix (new) flake8 issues\n\nSigned-off-by: Byron Ruth <b@devel.io>","distinct":true,"url":"https://api.github.com/repos/chop-dbhi/avocado/commits/cfbfe1555cccdf684faefedd533d8919b542c2b1"}]}