{"action":"created","issue":{"url":"https://api.github.com/repos/dejacode/about-code-tool/issues/204","labels_url":"https://api.github.com/repos/dejacode/about-code-tool/issues/204/labels{/name}","comments_url":"https://api.github.com/repos/dejacode/about-code-tool/issues/204/comments","events_url":"https://api.github.com/repos/dejacode/about-code-tool/issues/204/events","html_url":"https://github.com/dejacode/about-code-tool/issues/204","id":97816974,"number":204,"title":"Error log usability","user":{"login":"ottokruse-sd","id":12664344,"avatar_url":"https://avatars.githubusercontent.com/u/12664344?v=3","gravatar_id":"","url":"https://api.github.com/users/ottokruse-sd","html_url":"https://github.com/ottokruse-sd","followers_url":"https://api.github.com/users/ottokruse-sd/followers","following_url":"https://api.github.com/users/ottokruse-sd/following{/other_user}","gists_url":"https://api.github.com/users/ottokruse-sd/gists{/gist_id}","starred_url":"https://api.github.com/users/ottokruse-sd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ottokruse-sd/subscriptions","organizations_url":"https://api.github.com/users/ottokruse-sd/orgs","repos_url":"https://api.github.com/users/ottokruse-sd/repos","events_url":"https://api.github.com/users/ottokruse-sd/events{/privacy}","received_events_url":"https://api.github.com/users/ottokruse-sd/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dejacode/about-code-tool/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/dejacode/about-code-tool/milestones/2","html_url":"https://github.com/dejacode/about-code-tool/milestones/version%203","labels_url":"https://api.github.com/repos/dejacode/about-code-tool/milestones/2/labels","id":1229336,"number":2,"title":"version 3","description":"Major changes including using YAML, revamped command line interface and more","creator":{"login":"pombredanne","id":675997,"avatar_url":"https://avatars.githubusercontent.com/u/675997?v=3","gravatar_id":"","url":"https://api.github.com/users/pombredanne","html_url":"https://github.com/pombredanne","followers_url":"https://api.github.com/users/pombredanne/followers","following_url":"https://api.github.com/users/pombredanne/following{/other_user}","gists_url":"https://api.github.com/users/pombredanne/gists{/gist_id}","starred_url":"https://api.github.com/users/pombredanne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pombredanne/subscriptions","organizations_url":"https://api.github.com/users/pombredanne/orgs","repos_url":"https://api.github.com/users/pombredanne/repos","events_url":"https://api.github.com/users/pombredanne/events{/privacy}","received_events_url":"https://api.github.com/users/pombredanne/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":0,"state":"open","created_at":"2015-07-28T12:37:01Z","updated_at":"2015-07-29T12:58:37Z","due_on":null,"closed_at":null},"comments":1,"created_at":"2015-07-28T22:42:02Z","updated_at":"2015-07-29T13:29:50Z","closed_at":null,"body":"The error log is hard to use at the moment.  The following improvements would increase its usability:\r\n\r\n1.  Note whether particular line is WARNING or ERROR at start of line\r\n2.  Identifying information in each statement that links back to input or output of tool (BOM reference number or line number), as it stands you have to text search in the csv file to find things\r\n3. Identifying text at the start of the log file (when it was run, what was run perhaps by echoing command line, then a blank line before error log statements\r\n4. Print number of Warnings and Errors into log file"},"comment":{"url":"https://api.github.com/repos/dejacode/about-code-tool/issues/comments/125953218","html_url":"https://github.com/dejacode/about-code-tool/issues/204#issuecomment-125953218","issue_url":"https://api.github.com/repos/dejacode/about-code-tool/issues/204","id":125953218,"user":{"login":"pombredanne","id":675997,"avatar_url":"https://avatars.githubusercontent.com/u/675997?v=3","gravatar_id":"","url":"https://api.github.com/users/pombredanne","html_url":"https://github.com/pombredanne","followers_url":"https://api.github.com/users/pombredanne/followers","following_url":"https://api.github.com/users/pombredanne/following{/other_user}","gists_url":"https://api.github.com/users/pombredanne/gists{/gist_id}","starred_url":"https://api.github.com/users/pombredanne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pombredanne/subscriptions","organizations_url":"https://api.github.com/users/pombredanne/orgs","repos_url":"https://api.github.com/users/pombredanne/repos","events_url":"https://api.github.com/users/pombredanne/events{/privacy}","received_events_url":"https://api.github.com/users/pombredanne/received_events","type":"User","site_admin":false},"created_at":"2015-07-29T13:29:49Z","updated_at":"2015-07-29T13:29:49Z","body":"@ottokruse-sd agreed, the error logging is not great and we need to beef it up.\r\nWhat we have today are many confusing ways to handle this:\r\n\r\nIn the master branch and v2.x release stream.\r\n\r\n1. with `about.py`, we have  errors/warning mixed up on the same row as an about entry in two cells on the far right: `warnings` and `errors` columns. This is for the master branch and v2.x release stream.\r\n\r\n2. with `genabout.py`, we have  errors/warning logged in an `error.log` file\r\n\r\nIn develop, there is yet another an experimental approach which has not been firmed up\r\n\r\nI think these approaches could be improved this way:\r\nFor `about.py` proper:\r\n\r\n- always create a completely separate error log file, and not storing anything in the collected inventory CSV\r\n\r\nPros: this allows to have eventually more refined error info\r\n\r\nCons: this separates the data with errors from the corresponding error messages\r\n\r\n- keep the error/warning messages in-line with the collected CSV inventory but store it differently:\r\n - if there is an error or warning, create a new row in the CSV listing the error message in the same column that has an error/warning. Ensure that this row has a few essential fields such that the CSV sorts correctly and may be a \"row type\" column  to distinguish data rows from warning/errors message rows.\r\n - if there is no error/warning for a row, do not create such \"information\" additional row\r\n - we could use a also a top level/global information row to store housekeeping information such as time of the run, version in use, command line arguments\r\n\r\nPros: data and validation errors and warning are side-by-side. The validation message is located side-by-side with the invalid value, so no search is needed: errors can be fixed in place right away. The info rows can be easily filtered out of the output (and ignored when using such a  CSV file as an input). \r\n\r\nCons: none that I can think of.\r\n\r\nFor the the generation of ABOUT files and the attribution generation, we could also use the same approach: instead of providing a bare log file, we could instead provide a more structured output in a CSV, such that you have the validation messages side-by-side with the data that has an a validation problem. Since this CSV output could be having the same columns as an About inventory, it could become much easier to  fix the errors there and possibly regenerate your ABOUT files and rerun attribution from that basis.\r\n\r\nSo essentailly you would use a CSV and ABOUT files as an input and get something out which contains the original data and possible validation messages side by side on a different row.\r\n\r\nTo fix errors, you open the CSV, edit and save back, then rerun your command again  using thisfixed CSV as an input. You can iterate on this without having to  do back an forth betweeen the data and validation errors.\r\n\r\nThis could be an approach. What is your take? "}}