{"push_id":638945566,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"634dbcc236643c8f8a561c21540f97f8a63bc6c2","before":"bcd2d84e5afaf8ca04d60f5d2a392ce7ad46e174","commits":[{"sha":"634dbcc236643c8f8a561c21540f97f8a63bc6c2","author":{"email":"79e2475f81a6317276bf7cbb3958b20d289b78df@gerg.ca","name":"Greg Ward"},"message":"#17445: difflib: add diff_bytes(), to compare bytes rather than str\n\nSome applications (e.g. traditional Unix diff, version control\nsystems) neither know nor care about the encodings of the files they\nare comparing. They are textual, but to the diff utility they are just\nbytes. This worked fine under Python 2, because all of the hardcoded\nstrings in difflib.py are ASCII, so could safely be combined with\nold-style u'' strings. But it stopped working in 3.x.\n\nThe solution is to use surrogate escapes for a lossless\nbytes->str->bytes roundtrip. That means {unified,context}_diff() can\ncontinue to just handle strings without worrying about bytes. Callers\nwho have to deal with bytes will need to change to using diff_bytes().\n\nUse case: Mercurial's test runner uses difflib to compare current hg\noutput with known good output. But Mercurial's output is just bytes,\nsince it can contain:\n  * file contents (arbitrary unknown encoding)\n  * filenames (arbitrary unknown encoding)\n  * usernames and commit messages (usually UTF-8, but not guaranteed\n    because old versions of Mercurial did not enforce it)\n  * user messages (locale encoding)\n\nSince the output of any given hg command can include text in multiple\nencodings, it is hopeless to try to treat it as decodable Unicode\ntext. It's just bytes, all the way down.\n\nThis is an elaboration of a patch by Terry Reedy.","distinct":true,"url":"https://api.github.com/repos/python/cpython/commits/634dbcc236643c8f8a561c21540f97f8a63bc6c2"}]}