{"action":"created","issue":{"url":"https://api.github.com/repos/arduino/Arduino/issues/4170","labels_url":"https://api.github.com/repos/arduino/Arduino/issues/4170/labels{/name}","comments_url":"https://api.github.com/repos/arduino/Arduino/issues/4170/comments","events_url":"https://api.github.com/repos/arduino/Arduino/issues/4170/events","html_url":"https://github.com/arduino/Arduino/issues/4170","id":117342209,"number":4170,"title":"Issues with Print::print() and partial writes","user":{"login":"cmaglie","id":423515,"avatar_url":"https://avatars.githubusercontent.com/u/423515?v=3","gravatar_id":"","url":"https://api.github.com/users/cmaglie","html_url":"https://github.com/cmaglie","followers_url":"https://api.github.com/users/cmaglie/followers","following_url":"https://api.github.com/users/cmaglie/following{/other_user}","gists_url":"https://api.github.com/users/cmaglie/gists{/gist_id}","starred_url":"https://api.github.com/users/cmaglie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmaglie/subscriptions","organizations_url":"https://api.github.com/users/cmaglie/orgs","repos_url":"https://api.github.com/users/cmaglie/repos","events_url":"https://api.github.com/users/cmaglie/events{/privacy}","received_events_url":"https://api.github.com/users/cmaglie/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/arduino/Arduino/labels/Component:%20Core","name":"Component: Core","color":"009800"}],"state":"open","locked":false,"assignee":{"login":"cmaglie","id":423515,"avatar_url":"https://avatars.githubusercontent.com/u/423515?v=3","gravatar_id":"","url":"https://api.github.com/users/cmaglie","html_url":"https://github.com/cmaglie","followers_url":"https://api.github.com/users/cmaglie/followers","following_url":"https://api.github.com/users/cmaglie/following{/other_user}","gists_url":"https://api.github.com/users/cmaglie/gists{/gist_id}","starred_url":"https://api.github.com/users/cmaglie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmaglie/subscriptions","organizations_url":"https://api.github.com/users/cmaglie/orgs","repos_url":"https://api.github.com/users/cmaglie/repos","events_url":"https://api.github.com/users/cmaglie/events{/privacy}","received_events_url":"https://api.github.com/users/cmaglie/received_events","type":"User","site_admin":false},"milestone":null,"comments":8,"created_at":"2015-11-17T12:13:39Z","updated_at":"2015-11-25T19:21:10Z","closed_at":null,"body":"quoting @bperrybap here \r\nhttp://forum.arduino.cc/index.php?topic=357312.msg2481770#msg2481770\r\n\r\n```\r\nWhile there was an issue in the i2c library that was triggered by this new Print\r\nclass code, there is a still a fundamental issue with new new Print class code.\r\nThe issue is that the Print class code is actually composed of two parts.\r\nThe low level functions like read()/write() and the higher level output formatting\r\nfunctions like print(). And both layers support partial/incomplete i/o.\r\n(outputting less than the maximum asked to output)\r\nI believe that if print() is allowed to support partial/incomplete i/o, that we will be\r\nfighting related issues for years.\r\n\r\nWhat seems to have happened is that in 1.6.6. write() was updated to properly\r\nsupport incomplete i/o and print() is now seeing issues because it doesn't\r\nsupport incomplete i/o.\r\n(An error is essentially a single type of incomplete i/o)\r\n\r\nYes it was a coding error in a library in this case, but it brings up a larger issue\r\nrelated to partial/incomplete i/o and the print() functions.\r\n\r\nCurrently, I don't believe that the write() API is mandated to always swallow all\r\nthe data provided unless there is an error.\r\n\r\nSo the issues becomes, how to deal with print() when write() does\r\nincomplete/partial i/o?\r\n\r\nLooking forward, unless the write() API mandates blocking when low level \r\nbuffers are full or updating the Print class print() functions to ensure blocking, \r\nwhen write() returns after incomplete i/o, I think we will see more issues in \r\nthe future.\r\n\r\nWhile it is ok for functions like write() to support and potentially even require \r\nsupporting incomplete i/o, it is not appropriate for functions like print() to force\r\nthis same requirement back up to its callers.\r\nThis is because callers of write() know the full contents of what is being output.\r\nThe callers of print() do not know the full contents of what is being output since\r\nprint() is performing output formatting.\r\nIn other words suppose I do something like:\r\n\r\nfoo.print(bar);\r\n\r\nand the variable bar is an big integer number and is is 1234567\r\n\r\nNow suppose that the low level i/o library buffer is full and rather than block\r\n(spin) waiting for enough room to take all the requested output,  decides to \r\ntake only what it can.\r\nSuppose that was only 4 bytes.\r\nSo the low level i/o library write() will return 4 and then foo.print() will now \r\nreturn 4. What the heck is a sketch or any other potentially caller of print() \r\ncode, supposed to do with that?\r\nIt has no way of resuming the output nor does it even know how many total \r\noutput characters there were.\r\n\r\nThe point is that if the API is going to support incomplete i/o, it MUST have \r\na option to enable/disable this mechanism for functions like print() to not \r\nreturn until all output characters have been handed off.\r\n(The only exception is a fatal error where the i/o would never complete)\r\nI argued this point (enable/disable incomplete/async i/o) a year or so ago \r\nwhen we were looking at and arguing over the buffer available function calls.\r\n\r\nThere must be a way to allow users of the Print class, specifically the print()\r\nfunctions to have a mechanism to be assured that when the print() function \r\nreturns that all the characters have been handed off.\r\nThis is an absolute requirement since only print() knows what characters \r\nare to be output when formatting is done. The caller has no idea.\r\n\r\nIt is unacceptable to require all users of the print() calls to have to support \r\nincomplete i/o. And even if they do support incomplete i/o, there is no way \r\nto complete the i/o for output that was formatted by print() like number output.\r\n\r\nSuppose somebody writes an i/o library that decides to do very limited \r\noutput buffering and simply returns 0 when the output buffer is full vs \r\nblocking? This is allowed by the API but a sketch would never be able to \r\nprint numbers using print() calls when the numbers were more digits than \r\noutput buffer could hold, even if the sketch code add in all the complexities \r\nto support incomplete i/o for print() functions.\r\n\r\nI'm strongly suggesting that print() use blocking i/o.\r\nIf non blocking print() i/o behavior is supported by print() (which I don't even \r\nthink is possible since print() does output formatting), then there needs to be \r\nan API to enable incomplete i/o for an object (disabled by default) so that \r\nsketches can get the existing blocking behavior by default which preserves\r\nbackward compatibility and keeps user sketches simple.\r\nBut I'd still recommend that print() be kept simple and always be blocking \r\nsince there is no real technical way to support incomplete i/o with print() \r\nfunctions. Return on errors, yes, but not incomplete i/o.\r\n\r\nWith regard to errors vs partial i/o. That brings up another API issue.\r\nThe write() API should be able to distinguish between errors and incomplete \r\ni/o. for example, if the hardware is broken and it can't transmit, how can that\r\nbe indicated?\r\nCurrently, if you return zero, then that could mean the low level buffers are \r\nfull (incomplete i/o) or it could mean an actual error.\r\nA negative value would be better to indicate an error.\r\n```"},"comment":{"url":"https://api.github.com/repos/arduino/Arduino/issues/comments/159706212","html_url":"https://github.com/arduino/Arduino/issues/4170#issuecomment-159706212","issue_url":"https://api.github.com/repos/arduino/Arduino/issues/4170","id":159706212,"user":{"login":"bperrybap","id":1288729,"avatar_url":"https://avatars.githubusercontent.com/u/1288729?v=3","gravatar_id":"","url":"https://api.github.com/users/bperrybap","html_url":"https://github.com/bperrybap","followers_url":"https://api.github.com/users/bperrybap/followers","following_url":"https://api.github.com/users/bperrybap/following{/other_user}","gists_url":"https://api.github.com/users/bperrybap/gists{/gist_id}","starred_url":"https://api.github.com/users/bperrybap/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bperrybap/subscriptions","organizations_url":"https://api.github.com/users/bperrybap/orgs","repos_url":"https://api.github.com/users/bperrybap/repos","events_url":"https://api.github.com/users/bperrybap/events{/privacy}","received_events_url":"https://api.github.com/users/bperrybap/received_events","type":"User","site_admin":false},"created_at":"2015-11-25T19:21:10Z","updated_at":"2015-11-25T19:21:10Z","body":"matthijskooijman, I still don't understand your use case for print() so it is hard to respond to your comments.\r\n\r\nI still don't follow how returning partial write information from print() helps with padding and alignment.\r\nWhen doing padding and alignment the padding and alignment spacing (space characters) is not all on the end. If it was, there would be no need for alignment padding characters.\r\n\r\nFor example, suppose there is a number and the number should be right justified within a character output area/field.\r\nHow is partial writes being supported by the print() function going to help know how many spaces need to be output prior to sending the number to print()?\r\nWhile the caller of print() may know the field width, the caller of print() has no idea how many digits will be output by the print formatting in print()\r\n\r\nOr in another case where you send a string to print() and you want space padding between  words to be done to ensure that words don't wrap across a line. If the device returns an \"error\" or simply a count that is lower than the total output size requested when attempting to write beyond the end of the line and then print() returns the number of characters written on the line, how does that help with the padding to prevent word wrap. The partial word up the wrap point has already been output.\r\n\r\nCan you be very specific on what you mean by alignment/spacing and provide an exact example of how print() supporting fractional output helps?\r\n\r\nIn terms of return values, what I mentioned earlier was that the return value of print() is not really tied to the return value of write(). print() could be smarter and treat it differently since it knows if all the characters were successfully output and could return a status based on that information, and that is what I was proposing.\r\n\r\nThe real issue is that within the current write() API definition there is no documented way to distinguish between a partial write and an error.\r\nTechnically the current write() API does not define how to communicate an error, since it defines the return value as the number of characters written and is silent on how to handle errors.\r\n\r\nWhat you have proposed is the same as what I proposed when partial writes were not allowed and a non-zero return for print() was used vs a boolean.\r\nThat was one of the options I proposed earlier depending on how partial writes were handled by write() - in your example they are not allowed.\r\n\r\nIf partial writes without an error are going to be allowed by write() - then the write() API must be clear on how to support that and clearly indicate the difference between a partial write and error that occurred mid way through the processing.\r\n\r\nMy biggest concern and focus is return status for print().\r\nprint() can be smart enough to return  true/false or 0/non-zero regardless of the write() behavior and how/if partial writes are supported.\r\nAnd for that reason I'd like to decouple their return values from an API perspective.\r\ni.e. I don't think the return value from print() should have to depend on or be tied to the behavior of write() and write() returning zero vs non-zero on partial output or an error.\r\nWhile it is easier for the print() code if write() mimics the needed print() return status, it is not required.\r\n\r\nThe reason I want to separate them is that in my view print() is pretty much a  simple \"fire and forget\" interface that may block but yet you might like to know if all your output was successful.\r\n\r\nwrite() on the other hand is different beast and callers of write() may want and need more sophisticated things and so things like support for partial writes and better status information may be necessary.\r\n\r\nThe real issue is that when this stuff was tossed in back in 1.0 the the Print class API functions didn't include support for errors.\r\nIn real systems things like printf() and write() use negative values to indicate an error.\r\n\r\n\r\nI think print() is easy. \r\nIt can return 0 on errors, and either non zero (a byte count) or true on success and be backward compatible. Regardless of the final write() API and behavior print() could and should have a defined behavior that does not need to change including if write() is ever altered or updated in the future.\r\n\r\nwrite() is the potential challenge, particularly if partial writes are supported as that can affect backward compatibility.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n"}}