{"action":"created","issue":{"url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/34","labels_url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/34/labels{/name}","comments_url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/34/comments","events_url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/34/events","html_url":"https://github.com/hpcugent/hanythingondemand/issues/34","id":100251533,"number":34,"title":"clean up workdir to avoid issues","user":{"login":"boegel","id":620876,"avatar_url":"https://avatars.githubusercontent.com/u/620876?v=3","gravatar_id":"","url":"https://api.github.com/users/boegel","html_url":"https://github.com/boegel","followers_url":"https://api.github.com/users/boegel/followers","following_url":"https://api.github.com/users/boegel/following{/other_user}","gists_url":"https://api.github.com/users/boegel/gists{/gist_id}","starred_url":"https://api.github.com/users/boegel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/boegel/subscriptions","organizations_url":"https://api.github.com/users/boegel/orgs","repos_url":"https://api.github.com/users/boegel/repos","events_url":"https://api.github.com/users/boegel/events{/privacy}","received_events_url":"https://api.github.com/users/boegel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-11T08:05:27Z","updated_at":"2015-08-13T21:12:51Z","closed_at":null,"body":"Running multiple HOD jobs in parallel may lead to problems with the workdir:\r\n\r\n```\r\nerror:\r\nCaused by: org.apache.hadoop.fs.FSError: java.io.IOException: Disk quota\r\nexceeded\r\n         at\r\norg.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:221)\r\n         at\r\njava.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)\r\n         at\r\njava.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)\r\n         at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)\r\n         at java.io.DataOutputStream.flush(DataOutputStream.java:123)\r\n         at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)\r\n         at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)\r\n         at java.io.DataOutputStream.flush(DataOutputStream.java:123)\r\n         at org.apache.hadoop.mapred.IFile$Writer.close(IFile.java:163)\r\n         at\r\norg.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.finalMerge(MergeManagerImpl.java:739)\r\n         at\r\norg.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.close(MergeManagerImpl.java:363)\r\n         at\r\norg.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:158)\r\n         ... 6 more\r\nCaused by: java.io.IOException: Disk quota exceeded\r\n         at java.io.FileOutputStream.writeBytes(Native Method)\r\n         at java.io.FileOutputStream.write(FileOutputStream.java:345)\r\n         at\r\norg.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:219)\r\n         ... 17 more\r\n```"},"comment":{"url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/comments/130846318","html_url":"https://github.com/hpcugent/hanythingondemand/issues/34#issuecomment-130846318","issue_url":"https://api.github.com/repos/hpcugent/hanythingondemand/issues/34","id":130846318,"user":{"login":"ehiggs","id":28823,"avatar_url":"https://avatars.githubusercontent.com/u/28823?v=3","gravatar_id":"","url":"https://api.github.com/users/ehiggs","html_url":"https://github.com/ehiggs","followers_url":"https://api.github.com/users/ehiggs/followers","following_url":"https://api.github.com/users/ehiggs/following{/other_user}","gists_url":"https://api.github.com/users/ehiggs/gists{/gist_id}","starred_url":"https://api.github.com/users/ehiggs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehiggs/subscriptions","organizations_url":"https://api.github.com/users/ehiggs/orgs","repos_url":"https://api.github.com/users/ehiggs/repos","events_url":"https://api.github.com/users/ehiggs/events{/privacy}","received_events_url":"https://api.github.com/users/ehiggs/received_events","type":"User","site_admin":false},"created_at":"2015-08-13T21:12:50Z","updated_at":"2015-08-13T21:12:50Z","body":"Ok. I think this can be done when we have a `~/.config/hod/<jobid>/env dir we can figure out which directories belong to the job and blat them when we're done. That's actually really useful since it also trims where logs can hide."}}