{"action":"created","issue":{"url":"https://api.github.com/repos/lattice/quda/issues/218","labels_url":"https://api.github.com/repos/lattice/quda/issues/218/labels{/name}","comments_url":"https://api.github.com/repos/lattice/quda/issues/218/comments","events_url":"https://api.github.com/repos/lattice/quda/issues/218/events","html_url":"https://github.com/lattice/quda/issues/218","id":61714908,"number":218,"title":"Fused exterior twisted-clover fermions incompatible with tuning ","user":{"login":"mikeaclark","id":316771,"avatar_url":"https://avatars.githubusercontent.com/u/316771?v=3","gravatar_id":"","url":"https://api.github.com/users/mikeaclark","html_url":"https://github.com/mikeaclark","followers_url":"https://api.github.com/users/mikeaclark/followers","following_url":"https://api.github.com/users/mikeaclark/following{/other_user}","gists_url":"https://api.github.com/users/mikeaclark/gists{/gist_id}","starred_url":"https://api.github.com/users/mikeaclark/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikeaclark/subscriptions","organizations_url":"https://api.github.com/users/mikeaclark/orgs","repos_url":"https://api.github.com/users/mikeaclark/repos","events_url":"https://api.github.com/users/mikeaclark/events{/privacy}","received_events_url":"https://api.github.com/users/mikeaclark/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/lattice/quda/labels/bug","name":"bug","color":"e05555"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/lattice/quda/milestones/13","html_url":"https://github.com/lattice/quda/milestones/QUDA%200.7.1","labels_url":"https://api.github.com/repos/lattice/quda/milestones/13/labels","id":709742,"number":13,"title":"QUDA 0.7.1","description":"Relatively small changes and additions that did not make the cut for 0.7.","creator":{"login":"mikeaclark","id":316771,"avatar_url":"https://avatars.githubusercontent.com/u/316771?v=3","gravatar_id":"","url":"https://api.github.com/users/mikeaclark","html_url":"https://github.com/mikeaclark","followers_url":"https://api.github.com/users/mikeaclark/followers","following_url":"https://api.github.com/users/mikeaclark/following{/other_user}","gists_url":"https://api.github.com/users/mikeaclark/gists{/gist_id}","starred_url":"https://api.github.com/users/mikeaclark/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikeaclark/subscriptions","organizations_url":"https://api.github.com/users/mikeaclark/orgs","repos_url":"https://api.github.com/users/mikeaclark/repos","events_url":"https://api.github.com/users/mikeaclark/events{/privacy}","received_events_url":"https://api.github.com/users/mikeaclark/received_events","type":"User","site_admin":false},"open_issues":19,"closed_issues":9,"state":"open","created_at":"2014-07-03T18:42:12Z","updated_at":"2015-05-14T15:39:22Z","due_on":null,"closed_at":null},"comments":14,"created_at":"2015-03-14T23:48:47Z","updated_at":"2015-05-14T15:46:47Z","closed_at":null,"body":"(bug report from Alex)\r\n\r\nI'm still having problems with the fused kernels and tuning, I don't understand why. Disabling tuning yields the right result.\r\n\r\nYou can actually test it: if you use the same source (not point, fill the spinor with some random stuff), you can try to run invert_test with and without tuning on multiGPU with fused kernels, and you should see at the beginning:\r\n\r\n\"Prepared source = XXXXXXXXe+YY\"\r\n\r\nThe number should be the same, regardless of the tuning, but this is not the case.  I found some problems with the labeling during tuning: the application of the clover term was not labeled properly, and the dslash kernels could be labeled wrong. After correcting this in dslash_quda.cu:\r\n\r\n- Just add:\r\n\r\n```\r\n    if (twist == QUDA_TWIST_GAMMA5_DIRECT) {\r\n      writeAuxString(\"direct\");\r\n    } else {\r\n      writeAuxString(\"inverse\");\r\n    }\r\n```\r\nin the constructor, and change tuneKey to\r\n\r\n```\r\n    TuneKey tuneKey() const { return TuneKey(in->VolString(),\r\ntypeid(*this).name(), aux); }\r\n```\r\n\r\nAnd correcting dslash_twisted_clover.cu (removed tuneKey, so we use the standard one, and added fillAux with different endings depending on the kernel called) I tried again, but I'm getting the same wrong results.\r\n"},"comment":{"url":"https://api.github.com/repos/lattice/quda/issues/comments/102079205","html_url":"https://github.com/lattice/quda/issues/218#issuecomment-102079205","issue_url":"https://api.github.com/repos/lattice/quda/issues/218","id":102079205,"user":{"login":"AlexVaq","id":3840179,"avatar_url":"https://avatars.githubusercontent.com/u/3840179?v=3","gravatar_id":"","url":"https://api.github.com/users/AlexVaq","html_url":"https://github.com/AlexVaq","followers_url":"https://api.github.com/users/AlexVaq/followers","following_url":"https://api.github.com/users/AlexVaq/following{/other_user}","gists_url":"https://api.github.com/users/AlexVaq/gists{/gist_id}","starred_url":"https://api.github.com/users/AlexVaq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AlexVaq/subscriptions","organizations_url":"https://api.github.com/users/AlexVaq/orgs","repos_url":"https://api.github.com/users/AlexVaq/repos","events_url":"https://api.github.com/users/AlexVaq/events{/privacy}","received_events_url":"https://api.github.com/users/AlexVaq/received_events","type":"User","site_admin":false},"created_at":"2015-05-14T15:46:47Z","updated_at":"2015-05-14T15:46:47Z","body":"I tuned some exterior kernels and then substitute type=exterior_[xyzt] for type=exterior_all manually in the cache file (otherwise I could’t tune anything). It worked in most of the cases. Once I got an error about not enough resources, but then I suppose the tuned parameters of the non-fused kernel were not good for the fused one."}}