{"action":"created","issue":{"url":"https://api.github.com/repos/jflex-de/jflex/issues/188","labels_url":"https://api.github.com/repos/jflex-de/jflex/issues/188/labels{/name}","comments_url":"https://api.github.com/repos/jflex-de/jflex/issues/188/comments","events_url":"https://api.github.com/repos/jflex-de/jflex/issues/188/events","html_url":"https://github.com/jflex-de/jflex/pull/188","id":67805989,"number":188,"title":"Lintifier of JFLex grammar with PMD","user":{"login":"regisd","id":838356,"avatar_url":"https://avatars.githubusercontent.com/u/838356?v=3","gravatar_id":"","url":"https://api.github.com/users/regisd","html_url":"https://github.com/regisd","followers_url":"https://api.github.com/users/regisd/followers","following_url":"https://api.github.com/users/regisd/following{/other_user}","gists_url":"https://api.github.com/users/regisd/gists{/gist_id}","starred_url":"https://api.github.com/users/regisd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/regisd/subscriptions","organizations_url":"https://api.github.com/users/regisd/orgs","repos_url":"https://api.github.com/users/regisd/repos","events_url":"https://api.github.com/users/regisd/events{/privacy}","received_events_url":"https://api.github.com/users/regisd/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-04-11T16:44:22Z","updated_at":"2015-04-11T19:36:09Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/jflex-de/jflex/pulls/188","html_url":"https://github.com/jflex-de/jflex/pull/188","diff_url":"https://github.com/jflex-de/jflex/pull/188.diff","patch_url":"https://github.com/jflex-de/jflex/pull/188.patch"},"body":"Hello Gerwin, hello team!\r\n\r\nIn issue #183, Gerwin suggested to warn users about incorrect usage of Unicode characters.\r\n\r\nThe easy fix is to change the code and output a warning on stderr.\r\n\r\nGrammars can be complex, and I wanted to try a different and more generic approach, based on a *lintifier*, such as Java lint or JavaScript lint. Luckily PMD can be extended for any language :-)\r\n\r\nThat's what I do in this proof-of-concept. I added a parser for JFlex grammars, and a UnicodeRule that warns you if you use `\\u1234x` if `x` is in [A-F0-9].\r\n\r\nThe good things:\r\n* PMD is easy to integrate with Maven / ant / many continuous integration tools\r\n* PMD has a plugin architecture, this pull request is to discuss `flex-pmd-plugin`\r\n* I'm glad JFLex has it's own grammar, I didn't have to write a tokenizer ;-) I managed to get the list of tokens of the grammar, with `LexScan`.\r\n\r\nWhere I struggle a bit:\r\n* Ideally the PMD Rules work on the AST. I'm a bit confused… How can I get the AST of JFlex grammar I just tokenized? I got really confused with `LexParse` (I'm really ashamed of asking this question, but I'm sure you'll have a quick answer)\r\n* When I have a token, I'd like to know the starting line, ending line, starting column, ending column (this is for PMD to highlight the error). How do I get those (particularly in multiline scenarios)?\r\n"},"comment":{"url":"https://api.github.com/repos/jflex-de/jflex/issues/comments/91911641","html_url":"https://github.com/jflex-de/jflex/pull/188#issuecomment-91911641","issue_url":"https://api.github.com/repos/jflex-de/jflex/issues/188","id":91911641,"user":{"login":"regisd","id":838356,"avatar_url":"https://avatars.githubusercontent.com/u/838356?v=3","gravatar_id":"","url":"https://api.github.com/users/regisd","html_url":"https://github.com/regisd","followers_url":"https://api.github.com/users/regisd/followers","following_url":"https://api.github.com/users/regisd/following{/other_user}","gists_url":"https://api.github.com/users/regisd/gists{/gist_id}","starred_url":"https://api.github.com/users/regisd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/regisd/subscriptions","organizations_url":"https://api.github.com/users/regisd/orgs","repos_url":"https://api.github.com/users/regisd/repos","events_url":"https://api.github.com/users/regisd/events{/privacy}","received_events_url":"https://api.github.com/users/regisd/received_events","type":"User","site_admin":false},"created_at":"2015-04-11T19:36:09Z","updated_at":"2015-04-11T19:36:09Z","body":"If you use XML output, you even have the column numbers.\r\n\r\n\t\tCLASSPATH=/Users/regisd/workspace_jflex/jflex-regisd/jflex-pmd-plugin/target/jflex-pmd-plugin-1.7.0-SNAPSHOT.jar ; /Users/regisd/Downloads/pmd-bin-5.3.0/bin/pmd.sh pmd -auxclasspath $CLASSPATH -R jflex-basic -dir jflex-pmd-plugin/src/test/resources/ -f xml\r\n\r\nOutputs:\r\n\t\r\n\t\t<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n\t\t<pmd version=\"5.3.0\" timestamp=\"2015-04-11T21:33:50.111\">\r\n\t\t  <file name=\"/Users/regisd/workspace_jflex/jflex-regisd/jflex-pmd-plugin/src/test/resources/UnicodeRule.bad.jflex\">\r\n\t\t    <violation beginline=\"9\" endline=\"9\" begincolumn=\"22\" endcolumn=\"23\" rule=\"MistypedUnicodeCharacter\" ruleset=\"Basic XML\" priority=\"5\">\r\n\t\t        \\u116C is followed by character 0\r\n\t\t    </violation>\r\n\t\t    <violation beginline=\"9\" endline=\"9\" begincolumn=\"30\" endcolumn=\"31\" rule=\"MistypedUnicodeCharacter\" ruleset=\"Basic XML\" priority=\"5\">\r\n\t\t        \\u116C is followed by character 9\r\n\t\t    </violation>\r\n\t\t</file>\r\n\t\t</pmd>\r\n\t\r\nAs I said previously, I'm not sure how to handle multiline yytext(), though"}}