{"action":"created","issue":{"url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/64","labels_url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/64/comments","events_url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/64/events","html_url":"https://github.com/sciunto-org/python-bibtexparser/pull/64","id":74044850,"number":64,"title":"New parser based on pyparsing.","user":{"login":"omangin","id":1171989,"avatar_url":"https://avatars.githubusercontent.com/u/1171989?v=3","gravatar_id":"","url":"https://api.github.com/users/omangin","html_url":"https://github.com/omangin","followers_url":"https://api.github.com/users/omangin/followers","following_url":"https://api.github.com/users/omangin/following{/other_user}","gists_url":"https://api.github.com/users/omangin/gists{/gist_id}","starred_url":"https://api.github.com/users/omangin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/omangin/subscriptions","organizations_url":"https://api.github.com/users/omangin/orgs","repos_url":"https://api.github.com/users/omangin/repos","events_url":"https://api.github.com/users/omangin/events{/privacy}","received_events_url":"https://api.github.com/users/omangin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-05-07T16:25:06Z","updated_at":"2015-12-01T22:06:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/pulls/64","html_url":"https://github.com/sciunto-org/python-bibtexparser/pull/64","diff_url":"https://github.com/sciunto-org/python-bibtexparser/pull/64.diff","patch_url":"https://github.com/sciunto-org/python-bibtexparser/pull/64.patch"},"body":"Hi,\r\n\r\nLooking for a fix for issue #47 (that at some point disappeared and then came back), I found that the current parser was quite difficult to modify and its behavior difficult to understand. I thus implemented an other parser based on [pyparsing](https://pyparsing.wikispaces.com/), that enables the definition of the parser through its grammar.\r\n\r\nThis comes with a little overhead but the resulting parser is greatly simplified and in my opinion easier to maintain because its behavior (and the corresponding implementation choices) are clearer.\r\n\r\nI found it quite difficult to find good references on the bibtex format (which seems to be mainly reverse-engineered from the bibtex binary), however since [maverick.inria.fr/~Xavier.Decoret/...](http://maverick.inria.fr/~Xavier.Decoret/resources/xdkbibtex/bibtex_summary.html) was cited in the tests and documentation, I based my work on it.\r\n\r\nSo, the new parser is completely integrated in the Bibtexparser class and the switch should be transparent. It also pass all tests but three, on which some choices have to be taken that I wanted to first submit to the community. In order to make it pass the other tests, I had to fix some behaviors as detailed below.\r\n\r\n- I updated the tests to remove unquoted values such as `jan`, except if they are meant to be interpreted as strings.\r\n\r\n- In particular common strings are often pre-defined in bibtex such as month short names. I thus included a mechanism to add such common strings in `bibdatabase.py` although it is not active by default.\r\n\r\n- Similarly `bwriter.py` has been updated so that preambles are quoted (otherwise it conflicts with the use of strings in preambles).\r\n\r\n- The handling of strings and their interpolation has changed since string definitions and expressions are now parsed correctly.\r\n\r\n- Maybe the definition of standard types (that I moved to a global variable for clarity) should go to `bibdatabase.py`.\r\n\r\n- In the current implementation, `@comment` are handled by selecting everything between the braces. It seems that it differs from [what Decoret describes](http://maverick.inria.fr/~Xavier.Decoret/resources/xdkbibtex/bibtex_summary.html#comment) (as being bibtex' behavior), that is to only include what is between `@comment` and the end of current line. I implemented the latter behavior but left two tests in `test_comments.py` as they were (this makes two of the three non-passing tests). Which behavior should be kept?\r\n\r\n- In `test_traps` the newline between author names is removed. This is a strange expected behavior at the bibtex parser level that do not occur any more (but I left the non-passing test). Authors are correctly handled by the author customization function later.\r\n\r\nA last comment on strings. I kept the previous behavior of interpolating strings on the go. Anyway it may be beneficial to represent bibtex strings with a custom object so that interpolation is delayed which would enable bwriter to declare again the parsed strings.    "},"comment":{"url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/comments/161110520","html_url":"https://github.com/sciunto-org/python-bibtexparser/pull/64#issuecomment-161110520","issue_url":"https://api.github.com/repos/sciunto-org/python-bibtexparser/issues/64","id":161110520,"user":{"login":"omangin","id":1171989,"avatar_url":"https://avatars.githubusercontent.com/u/1171989?v=3","gravatar_id":"","url":"https://api.github.com/users/omangin","html_url":"https://github.com/omangin","followers_url":"https://api.github.com/users/omangin/followers","following_url":"https://api.github.com/users/omangin/following{/other_user}","gists_url":"https://api.github.com/users/omangin/gists{/gist_id}","starred_url":"https://api.github.com/users/omangin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/omangin/subscriptions","organizations_url":"https://api.github.com/users/omangin/orgs","repos_url":"https://api.github.com/users/omangin/repos","events_url":"https://api.github.com/users/omangin/events{/privacy}","received_events_url":"https://api.github.com/users/omangin/received_events","type":"User","site_admin":false},"created_at":"2015-12-01T22:06:45Z","updated_at":"2015-12-01T22:06:45Z","body":"@gpoo First the test are now slower (although I added some but this does not account for the difference). The other way is simply to time the parsing time for a big bibtex file."}}