{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/roslyn/issues/3617","labels_url":"https://api.github.com/repos/dotnet/roslyn/issues/3617/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/roslyn/issues/3617/comments","events_url":"https://api.github.com/repos/dotnet/roslyn/issues/3617/events","html_url":"https://github.com/dotnet/roslyn/issues/3617","id":90138876,"number":3617,"title":"VB parsing string-interpolation in erroneous code has poor error recovery","user":{"login":"ljw1004","id":3316258,"avatar_url":"https://avatars.githubusercontent.com/u/3316258?v=3","gravatar_id":"","url":"https://api.github.com/users/ljw1004","html_url":"https://github.com/ljw1004","followers_url":"https://api.github.com/users/ljw1004/followers","following_url":"https://api.github.com/users/ljw1004/following{/other_user}","gists_url":"https://api.github.com/users/ljw1004/gists{/gist_id}","starred_url":"https://api.github.com/users/ljw1004/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ljw1004/subscriptions","organizations_url":"https://api.github.com/users/ljw1004/orgs","repos_url":"https://api.github.com/users/ljw1004/repos","events_url":"https://api.github.com/users/ljw1004/events{/privacy}","received_events_url":"https://api.github.com/users/ljw1004/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dotnet/roslyn/labels/Area-Compilers","name":"Area-Compilers","color":"5319e7"},{"url":"https://api.github.com/repos/dotnet/roslyn/labels/Bug","name":"Bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"AnthonyDGreen","id":10539978,"avatar_url":"https://avatars.githubusercontent.com/u/10539978?v=3","gravatar_id":"","url":"https://api.github.com/users/AnthonyDGreen","html_url":"https://github.com/AnthonyDGreen","followers_url":"https://api.github.com/users/AnthonyDGreen/followers","following_url":"https://api.github.com/users/AnthonyDGreen/following{/other_user}","gists_url":"https://api.github.com/users/AnthonyDGreen/gists{/gist_id}","starred_url":"https://api.github.com/users/AnthonyDGreen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AnthonyDGreen/subscriptions","organizations_url":"https://api.github.com/users/AnthonyDGreen/orgs","repos_url":"https://api.github.com/users/AnthonyDGreen/repos","events_url":"https://api.github.com/users/AnthonyDGreen/events{/privacy}","received_events_url":"https://api.github.com/users/AnthonyDGreen/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/dotnet/roslyn/milestones/3","html_url":"https://github.com/dotnet/roslyn/milestones/1.1","labels_url":"https://api.github.com/repos/dotnet/roslyn/milestones/3/labels","id":937401,"number":3,"title":"1.1","description":"This is our 1.1 Milestone.\r\n\r\n<!---\r\n@huboard:{\"order\":1.5}\r\n-->\r\n","creator":{"login":"jaredpar","id":146967,"avatar_url":"https://avatars.githubusercontent.com/u/146967?v=3","gravatar_id":"","url":"https://api.github.com/users/jaredpar","html_url":"https://github.com/jaredpar","followers_url":"https://api.github.com/users/jaredpar/followers","following_url":"https://api.github.com/users/jaredpar/following{/other_user}","gists_url":"https://api.github.com/users/jaredpar/gists{/gist_id}","starred_url":"https://api.github.com/users/jaredpar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaredpar/subscriptions","organizations_url":"https://api.github.com/users/jaredpar/orgs","repos_url":"https://api.github.com/users/jaredpar/repos","events_url":"https://api.github.com/users/jaredpar/events{/privacy}","received_events_url":"https://api.github.com/users/jaredpar/received_events","type":"User","site_admin":false},"open_issues":365,"closed_issues":77,"state":"open","created_at":"2015-01-15T20:48:41Z","updated_at":"2015-06-26T21:47:08Z","due_on":null,"closed_at":null},"comments":10,"created_at":"2015-06-22T15:53:13Z","updated_at":"2015-06-26T21:49:25Z","closed_at":null,"body":"Watch this video:\r\n![string-interpolation-parsing](https://cloud.githubusercontent.com/assets/3316258/8285960/a243c030-18bb-11e5-9901-aef71f59b687.gif)\r\n\r\nI get into this situation often when I'm copying+pasting code around, or breaking a method up into smaller methods by deleting \"sub\" and then re-adding a bunch of \"sub\" and \"end sub\" statements where appropriate.\r\n\r\n\r\nWHAT I OBSERVE: When there's a valid statement, regular strings and interpolated string parse fine.\r\n\r\nWhen there's an invalid statement, regular strings parse fine, but interpolated strings don't recognize their terminating quote, so the remainder of the entire file gets \"stringed out\".\r\n\r\n\r\nWHAT I EXPECT: interpolated strings should believe themselves terminated at the terminating quote, just like regular strings do."},"comment":{"url":"https://api.github.com/repos/dotnet/roslyn/issues/comments/115900166","html_url":"https://github.com/dotnet/roslyn/issues/3617#issuecomment-115900166","issue_url":"https://api.github.com/repos/dotnet/roslyn/issues/3617","id":115900166,"user":{"login":"AnthonyDGreen","id":10539978,"avatar_url":"https://avatars.githubusercontent.com/u/10539978?v=3","gravatar_id":"","url":"https://api.github.com/users/AnthonyDGreen","html_url":"https://github.com/AnthonyDGreen","followers_url":"https://api.github.com/users/AnthonyDGreen/followers","following_url":"https://api.github.com/users/AnthonyDGreen/following{/other_user}","gists_url":"https://api.github.com/users/AnthonyDGreen/gists{/gist_id}","starred_url":"https://api.github.com/users/AnthonyDGreen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AnthonyDGreen/subscriptions","organizations_url":"https://api.github.com/users/AnthonyDGreen/orgs","repos_url":"https://api.github.com/users/AnthonyDGreen/repos","events_url":"https://api.github.com/users/AnthonyDGreen/events{/privacy}","received_events_url":"https://api.github.com/users/AnthonyDGreen/received_events","type":"User","site_admin":false},"created_at":"2015-06-26T21:49:25Z","updated_at":"2015-06-26T21:49:25Z","body":"@AdamSpeight2008, I'm not sure what you mean by symbol/syntactic - do you mean lexical (at the token level)?\r\n\r\nTo be clear the issue is not that there is a bug anywhere. The parser is written to parse incomplete code in a particular way and we're discussing changing that way based on a refined view on what the user probably intended in this particular circumstance. You'll see the same behavior here with XML literals:\r\n\r\n```VB.NET\r\nModule Module1\r\n    Sub Main()\r\n        Dim x\r\n        Console.WriteLine(<xml><%= Math.Abs(x %></xml>)\r\n    End Sub\r\nEnd Module\r\n```\r\n\r\nBecause of the missing closing parenthesis the XML is never considered closed and consumes all text until the end of the file. What's happening in this case is that like XML literals an interpolated string is a special embedded language in VB. Normally in VB a double quote character can only begin a string-literal-token. But, when the scanner encounters a dollar sign and double quote character back to back in VB's normal expression parsing context it creates a dollar-sign-double-quote token-which triggers the parser to enter a special mode just for interpolated strings. In this mode a double quote character should produce a double-quote-token instead and terminate the string. Like in XML, the interpolations themselves cause an interleaving of scanner modes where the parser has to switch back and forth between standard VB expression rules and interpolated string rules. Recovering at the boundary is the issue. When the parser parses x it looks ahead for something that could continue an expression plus-token (or any other binary operator). In looking ahead it causes the scanner to read the program at that point using VB standard rules. It sees a double quote character, assumes it's starting a string, and reads to the end of the string (which it doesn't find), attaches the malformed string as garbage, and then the inner invocation expression, the interpolation, the interpolated string itself, the outer invocation, the sub, and the module all look for the valid tokens/statements to close themselves but instead find the end of file and so they all parse as malformed. This is what it's designed to do, right now. The case where things are broken outside of method bodies is because we don't parse things as executable statements outside of method bodies. There are slightly different rules. For example, the open < that starts an attribute or the : after an attribute target specifier are different than if they were just inside of a method body. Dim x is a field and not a local declaration. We need to come up with a different approach.\r\n\r\nInterestingly, the problem is exacerbated by multi-line string literals. In prior versions string lexing would give up at a newline, so the corruption would end on that line and at least the sub and module could close normally. I guess if we made error recovery more lazy so that instead of greedily consuming garbage tokens it left them for its parents to handle we'd be in a better shape. But that could cause other scenarios to become worse - we'll need to think it through.\r\n\r\nThought through a bit. Take this code as an example:\r\n\r\n```VB.NET\r\nModule Module1\r\n    Sub Main()\r\n        Console.WriteLine($\"{Math.Abs(1 2, 3)}\")\r\n    End Sub\r\nEnd Module\r\n```\r\n\r\nIn this case when ParseExpression gets 1 it looks ahead for valid continuation, finds a 2, greedily consumes it as garbage and resyncs to right before the comma - this leaves things in a great state for ParseArgumentList because it has _skipped_ the 2 and continues parsing the arg list correctly, the intepolation, interpolated string, outer invocation, sub, and module are all now well-formed. If the recovery were lazy and didn't skip the 2 everyone would fail and the closing quote would still get tokenized as a string start. Now I guess if we made a special recovery scanning mode it at least would always tokenized skipped strings as single-line - then the corruption would be limited to the line, which is probably for the best. It's at least much better than what we're doing now.\r\n\r\nAlternately, we could maintain a stack of pending scanner states and when resyncing (skipping over tokens to get back to a good place) we could try each state in tern to see if we see any tokens that any of our parents could continue or close on in LILO priority order. So in Lucian's second example after grabbing the x it would peak ahead in standard VB mode and see a }, recognize that it could't do anything with the } and try rescanning in interpolated string mode still get a } and see that it could recover and bail, the invocation would repeat this and get the same answer and ParseInterpolation would recover fine and so the closing quote would scan correctly. That's a bit more state to track though, a much more invasive solution.\r\n\r\nThere might be a scenario where a parent further out might be a better place to resync than inside? Like does the inner array literal or the outer interpolation get the }? I don't know - but that's a different problem than what we're discussing and I'm willing to cross that bridge when we get there.\r\n\r\n+@JohnHamby who understands the madness (the concept of the compiler knowing when to give up).\r\n+@cston with whom I discussed recovery options for XML literals in similar situations for countless hours.\r\n+@VSadov who helped write the VB scanner and will try to talk us out of my crazy schemes."}}