{"action":"created","issue":{"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/60","labels_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/60/labels{/name}","comments_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/60/comments","events_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/60/events","html_url":"https://github.com/jupyter-incubator/sparkmagic/issues/60","id":120507233,"number":60,"title":"Return well formated string/error from Livy to user","user":{"login":"aggFTW","id":654204,"avatar_url":"https://avatars.githubusercontent.com/u/654204?v=3","gravatar_id":"","url":"https://api.github.com/users/aggFTW","html_url":"https://github.com/aggFTW","followers_url":"https://api.github.com/users/aggFTW/followers","following_url":"https://api.github.com/users/aggFTW/following{/other_user}","gists_url":"https://api.github.com/users/aggFTW/gists{/gist_id}","starred_url":"https://api.github.com/users/aggFTW/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aggFTW/subscriptions","organizations_url":"https://api.github.com/users/aggFTW/orgs","repos_url":"https://api.github.com/users/aggFTW/repos","events_url":"https://api.github.com/users/aggFTW/events{/privacy}","received_events_url":"https://api.github.com/users/aggFTW/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-04T23:28:37Z","updated_at":"2015-12-09T00:44:44Z","closed_at":null,"body":"Right now, magics return the result from Livy without being aware of whether the string is a result or an error.\r\n\r\nIn the case of a result, magics should nicely print the result back.\r\nIn the case of an error, it should be clear to the user that an error just happened in the cluster. A stacktrace should be printed if available."},"comment":{"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/comments/163069491","html_url":"https://github.com/jupyter-incubator/sparkmagic/issues/60#issuecomment-163069491","issue_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/60","id":163069491,"user":{"login":"msftristew","id":13972471,"avatar_url":"https://avatars.githubusercontent.com/u/13972471?v=3","gravatar_id":"","url":"https://api.github.com/users/msftristew","html_url":"https://github.com/msftristew","followers_url":"https://api.github.com/users/msftristew/followers","following_url":"https://api.github.com/users/msftristew/following{/other_user}","gists_url":"https://api.github.com/users/msftristew/gists{/gist_id}","starred_url":"https://api.github.com/users/msftristew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msftristew/subscriptions","organizations_url":"https://api.github.com/users/msftristew/orgs","repos_url":"https://api.github.com/users/msftristew/repos","events_url":"https://api.github.com/users/msftristew/events{/privacy}","received_events_url":"https://api.github.com/users/msftristew/received_events","type":"User","site_admin":false},"created_at":"2015-12-09T00:44:44Z","updated_at":"2015-12-09T00:44:44Z","body":"I am currently investigating solutions to this issue.  Here is the problem we're facing:\r\n\r\n1. If we want to enable pretty-printing, the magics necessarily need to have a side effect.  So Livy would give us a dataframe, and the spark magic would return `None`. Returning `None` silences the kernel so nothing else would be printed to the screen.\r\n\r\n2. We can't return the result of the magic because if we did, then that would frequently cause results for an invocation to be printed twice: once (pretty-printed) by the magics, and once (improperly formatted) by the kernel.\r\n\r\n3. However, this prevents users from being able to do something like `x = %sql SELECT * FROM...`, or from doing `%sql SELECT * FROM ...` and using the `Out[]` array to get the dataframe from an invocation.  This limits the usefulness of the magics.\r\n\r\nIf we all agree this functionality is necessary, then we have a few options.\r\n\r\n1. Give different commands different names.  So we would have one command for running `sql` queries and pretty-printing the results, and one other command for running `sql` queries and getting the results of the query as a dataframe.  Alejandro and I think this is a pretty messy solution.\r\n\r\n2.  Add an option to silence and return the output of the magic.  So for example one could write `%sql SELECT * FROM ... -r`, which would have no output and simply return the dataframe.  What's unfortunate here is that you then can't pretty-print the results and also save the results as a dataframe, since silencing and returning the output are captured in the same option.\r\n\r\n3.  Similarly to option 2, add two options: `-r`, which returns the dataframe instead of `None`, and `-q`, which silences the pretty-printer.  This has all the power that one would need from the system, but now pretty-printing the output while also capturing the results as a dataframe (which might be a common pattern for some users) becomes pretty verbose: `%sql SELECT * FROM ... -r -q`.  \r\n\r\nWhat do you think?  Do we want the more verbose but more powerful options for the magics?\r\n\r\n@aggFTW @ellisonbg "}}