{"action":"created","issue":{"url":"https://api.github.com/repos/sirthias/pegdown/issues/170","labels_url":"https://api.github.com/repos/sirthias/pegdown/issues/170/labels{/name}","comments_url":"https://api.github.com/repos/sirthias/pegdown/issues/170/comments","events_url":"https://api.github.com/repos/sirthias/pegdown/issues/170/events","html_url":"https://github.com/sirthias/pegdown/pull/170","id":100687487,"number":170,"title":"fix parsing of sub-lists, sub-sub-lists with code, para and blank lin…","user":{"login":"vsch","id":10299086,"avatar_url":"https://avatars.githubusercontent.com/u/10299086?v=3","gravatar_id":"","url":"https://api.github.com/users/vsch","html_url":"https://github.com/vsch","followers_url":"https://api.github.com/users/vsch/followers","following_url":"https://api.github.com/users/vsch/following{/other_user}","gists_url":"https://api.github.com/users/vsch/gists{/gist_id}","starred_url":"https://api.github.com/users/vsch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vsch/subscriptions","organizations_url":"https://api.github.com/users/vsch/orgs","repos_url":"https://api.github.com/users/vsch/repos","events_url":"https://api.github.com/users/vsch/events{/privacy}","received_events_url":"https://api.github.com/users/vsch/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-08-13T04:31:20Z","updated_at":"2015-08-14T04:18:56Z","closed_at":"2015-08-14T04:18:56Z","pull_request":{"url":"https://api.github.com/repos/sirthias/pegdown/pulls/170","html_url":"https://github.com/sirthias/pegdown/pull/170","diff_url":"https://github.com/sirthias/pegdown/pull/170.diff","patch_url":"https://github.com/sirthias/pegdown/pull/170.patch"},"body":"List parsing fixed\r\n-----------------------\r\nI also took the liberty of prettying up the `<li>` tags when they contain children and added a `\\n` before printing of the `<p>` tag so that the generated HTML hierarchy is easier to follow. \r\n\r\nFixes issues: #57 and #123 \r\n\r\nList rendering beyond the first and second levels was off for all but compact items and in need of a little TLC.\r\n\r\nThe printout of the test follows. The raw markdown section gives the markdown source, the markdown section will be converted to HTML by GitHub, the HTML section is the output of the corrected pegdown parser and raw HTML is the parser output for those that prefer to render HTML in their head from source.\r\n\r\nAs you can see GitHub and pegdown are in agreement. \r\n\r\n#### License\r\nThis fix is my original work and I share it under the same license as the pegdown licence.\r\n\r\nCheers,\r\nVlaidimir.\r\n\r\nP.S. Please run reformat on the source of the parser in IntelliJ so that it has consistent format and gets rid of the trailing spaces. Took a while to discard all the extraneous chunks that had trailing spaces removed.\r\n\r\n-- raw markdown -----------------------------------------------\r\n\r\n~~~~~markdown\r\n\r\n1. B 1\r\n\r\n    1. A 1\r\n    2. A 2\r\n\r\n\r\n    B 1's paragraph\r\n\r\n<!-- end list -->\r\n2. XYZ 1\r\n\r\n    XYZ 1's paragraph\r\n\r\n    1. A 1\r\n    2. A 2\r\n\r\n    3. A 3\r\n\r\n        A 3's paragraph\r\n\r\n        - XYZ\r\n            1. abc\r\n            1. abc\r\n            1. abc\r\n        - XYZ\r\n            - abc\r\n\r\n                ```\r\n                fenced code block of sub-sub-list abc                \r\n                ```\r\n            - abc\r\n            - abc\r\n        - XYZ\r\n\r\n        Lorem ipsum...\r\n\r\n2. XYZ 2\r\n\r\n    1. A 2\r\n        - XYZ 2\r\n\r\n<!-- end list -->\r\n\r\n 1. List 1 Item 1\r\n\r\n    - sub list item 1\r\n        sub list item 1 continuation line\r\n\r\n            sub list item 1 code block 1\r\n\r\n        sub list item 1 paragraph\r\n\r\n            sub list item 1 code block 2\r\n\r\n    - sub list item 2\r\n\r\n    List 1 Item 1 paragraph\r\n\r\n        List 1 Item 1 code block\r\n\r\n    List 1 Item 1 paragraph or main code block\r\n\r\n\r\n~~~~~\r\n\r\n-- markdown -----------------------------------------------\r\n\r\n1. B 1\r\n\r\n    1. A 1\r\n    2. A 2\r\n\r\n\r\n    B 1's paragraph\r\n\r\n<!-- end list -->\r\n2. XYZ 1\r\n\r\n    XYZ 1's paragraph\r\n\r\n    1. A 1\r\n    2. A 2\r\n\r\n    3. A 3\r\n\r\n        A 3's paragraph\r\n\r\n        - XYZ\r\n            1. abc\r\n            1. abc\r\n            1. abc\r\n        - XYZ\r\n            - abc\r\n\r\n                ```\r\n                fenced code block of sub-sub-list abc                \r\n                ```\r\n            - abc\r\n            - abc\r\n        - XYZ\r\n\r\n        Lorem ipsum...\r\n\r\n2. XYZ 2\r\n\r\n    1. A 2\r\n        - XYZ 2\r\n\r\n<!-- end list -->\r\n\r\n 1. List 1 Item 1\r\n\r\n    - sub list item 1\r\n        sub list item 1 continuation line\r\n\r\n            sub list item 1 code block 1\r\n\r\n        sub list item 1 paragraph\r\n\r\n            sub list item 1 code block 2\r\n\r\n    - sub list item 2\r\n\r\n    List 1 Item 1 paragraph\r\n\r\n        List 1 Item 1 code block\r\n\r\n    List 1 Item 1 paragraph or main code block\r\n\r\n\r\n-- HTML ---------------------------------------------------\r\n<ol>\r\n  <li>\r\n    <p>B 1</p>\r\n    <ol>\r\n      <li>A 1</li>\r\n      <li>A 2</li>\r\n    </ol>\r\n    <p>B 1’s paragraph</p>\r\n  </li>\r\n</ol>\r\n<!-- end list -->\r\n<ol>\r\n  <li>\r\n    <p>XYZ 1</p>\r\n    <p>XYZ 1’s paragraph</p>\r\n    <ol>\r\n      <li>A 1</li>\r\n      <li>A 2</li>\r\n      <li>\r\n        <p>A 3</p>\r\n        <p>A 3’s paragraph</p>\r\n        <ul>\r\n          <li>XYZ\r\n            <ol>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n            </ol>\r\n          </li>\r\n          <li>XYZ\r\n            <ul>\r\n              <li>\r\n                <p>abc</p>\r\n                <p><code>fenced code block of sub-sub-list abc</code></p>\r\n              </li>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n            </ul>\r\n          </li>\r\n          <li>XYZ</li>\r\n        </ul>\r\n        <p>Lorem ipsum…</p>\r\n      </li>\r\n    </ol>\r\n  </li>\r\n  <li>\r\n    <p>XYZ 2</p>\r\n    <ol>\r\n      <li>A 2\r\n        <ul>\r\n          <li>XYZ 2</li>\r\n        </ul>\r\n      </li>\r\n    </ol>\r\n  </li>\r\n</ol>\r\n<!-- end list -->\r\n<ol>\r\n  <li>\r\n    <p>List 1 Item 1</p>\r\n    <ul>\r\n      <li>\r\n        <p>sub list item 1<br/>sub list item 1 continuation line</p>\r\n        <pre><code>sub list item 1 code block 1\r\n</code></pre>\r\n        <p>sub list item 1 paragraph</p>\r\n        <pre><code>sub list item 1 code block 2\r\n</code></pre>\r\n      </li>\r\n      <li>sub list item 2</li>\r\n    </ul>\r\n    <p>List 1 Item 1 paragraph</p>\r\n    <pre><code>List 1 Item 1 code block\r\n</code></pre>\r\n    <p>List 1 Item 1 paragraph or main code block</p>\r\n  </li>\r\n</ol>\r\n-- raw html -----------------------------------------------\r\n\r\n~~~~~\r\n<ol>\r\n  <li>\r\n    <p>B 1</p>\r\n    <ol>\r\n      <li>A 1</li>\r\n      <li>A 2</li>\r\n    </ol>\r\n    <p>B 1’s paragraph</p>\r\n  </li>\r\n</ol>\r\n<!-- end list -->\r\n<ol>\r\n  <li>\r\n    <p>XYZ 1</p>\r\n    <p>XYZ 1’s paragraph</p>\r\n    <ol>\r\n      <li>A 1</li>\r\n      <li>A 2</li>\r\n      <li>\r\n        <p>A 3</p>\r\n        <p>A 3’s paragraph</p>\r\n        <ul>\r\n          <li>XYZ\r\n            <ol>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n            </ol>\r\n          </li>\r\n          <li>XYZ\r\n            <ul>\r\n              <li>\r\n                <p>abc</p>\r\n                <p><code>fenced code block of sub-sub-list abc</code></p>\r\n              </li>\r\n              <li>abc</li>\r\n              <li>abc</li>\r\n            </ul>\r\n          </li>\r\n          <li>XYZ</li>\r\n        </ul>\r\n        <p>Lorem ipsum…</p>\r\n      </li>\r\n    </ol>\r\n  </li>\r\n  <li>\r\n    <p>XYZ 2</p>\r\n    <ol>\r\n      <li>A 2\r\n        <ul>\r\n          <li>XYZ 2</li>\r\n        </ul>\r\n      </li>\r\n    </ol>\r\n  </li>\r\n</ol>\r\n<!-- end list -->\r\n<ol>\r\n  <li>\r\n    <p>List 1 Item 1</p>\r\n    <ul>\r\n      <li>\r\n        <p>sub list item 1<br/>sub list item 1 continuation line</p>\r\n        <pre><code>sub list item 1 code block 1\r\n</code></pre>\r\n        <p>sub list item 1 paragraph</p>\r\n        <pre><code>sub list item 1 code block 2\r\n</code></pre>\r\n      </li>\r\n      <li>sub list item 2</li>\r\n    </ul>\r\n    <p>List 1 Item 1 paragraph</p>\r\n    <pre><code>List 1 Item 1 code block\r\n</code></pre>\r\n    <p>List 1 Item 1 paragraph or main code block</p>\r\n  </li>\r\n</ol>\r\n~~~~~\r\n\r\n-----------------------------------------------------------\r\n"},"comment":{"url":"https://api.github.com/repos/sirthias/pegdown/issues/comments/130960269","html_url":"https://github.com/sirthias/pegdown/pull/170#issuecomment-130960269","issue_url":"https://api.github.com/repos/sirthias/pegdown/issues/170","id":130960269,"user":{"login":"vsch","id":10299086,"avatar_url":"https://avatars.githubusercontent.com/u/10299086?v=3","gravatar_id":"","url":"https://api.github.com/users/vsch","html_url":"https://github.com/vsch","followers_url":"https://api.github.com/users/vsch/followers","following_url":"https://api.github.com/users/vsch/following{/other_user}","gists_url":"https://api.github.com/users/vsch/gists{/gist_id}","starred_url":"https://api.github.com/users/vsch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vsch/subscriptions","organizations_url":"https://api.github.com/users/vsch/orgs","repos_url":"https://api.github.com/users/vsch/repos","events_url":"https://api.github.com/users/vsch/events{/privacy}","received_events_url":"https://api.github.com/users/vsch/received_events","type":"User","site_admin":false},"created_at":"2015-08-14T04:18:56Z","updated_at":"2015-08-14T04:18:56Z","body":"There are other issues associated with this fix. Now compound lists are handled correctly but all sub-lists are compact unless they contain a paragraph. I have a fix for this and also fixed regression test failures because definitions and lists share rules.\r\n\r\nThe crux of the issue is that a list item's children can be an arbitrary set of indented blocks (one or more indents). All of these should be collected into a single chunk to be parsed recursively because its sub-items can also be interleaved with blank lines, code blocks, etc. If you break a list-item's child blocks into individual parts then one of its sub-list's items may not have the correct contents because the sub-list may be prematurely closed at the end of the recursive parsing of one of the text child blocks. \r\n\r\nThe list item rule now takes all following blocks that are part of its children and parses it recursively as one text stream. The results are much better. You get compact or loose items and regression tests pass for all except the pegdown.spec: AstText.ast. The issue is that you cannot include the trailing blank lines after a list item because the next item needs them to determine whether it is loose or tight. So a list item now takes leading blank lines but leaves trailing ones for the next item. This causes the blank line with two spaces at the end of the list in the test to be excluded from the BulletListNode leading to a discrepancy of 3 characters (2 spaces + `\\n`). I changed the expectation file for the test to match the current results. \r\n\r\nFor now I would not go as far as testing all the combinations and permutations for lists because I suspect that other issues will be discovered. I simply don't have the time to fix them all as they arise or write all the tests. If I create tests that fail, I'll need to validate that they are not false negative, etc. As one co-worker of mine used to say: \"It's a career decision.\" which I am not prepared to make. ;)\r\n\r\nI separated the definition block list parsing from the list block parsing because the definition tests expect the old behaviour and I didn't get to validating that part of the parser. For now I am willing to take it at face value. The definitions rules are a copy of the old ListItem rule and dependent rules that diverge for lists.\r\n\r\nI'll close this PR and open a new one with the fixes that pass the regression tests, but without adding new tests for the bug fixes. I will post another PR with new tests tomorrow or the day after."}}