{"action":"created","issue":{"url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/620","labels_url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/620/labels{/name}","comments_url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/620/comments","events_url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/620/events","html_url":"https://github.com/xianyi/OpenBLAS/issues/620","id":102644343,"number":620,"title":"GEMM has probably a bug in RowMajor implementation","user":{"login":"Arech","id":778977,"avatar_url":"https://avatars.githubusercontent.com/u/778977?v=3","gravatar_id":"","url":"https://api.github.com/users/Arech","html_url":"https://github.com/Arech","followers_url":"https://api.github.com/users/Arech/followers","following_url":"https://api.github.com/users/Arech/following{/other_user}","gists_url":"https://api.github.com/users/Arech/gists{/gist_id}","starred_url":"https://api.github.com/users/Arech/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Arech/subscriptions","organizations_url":"https://api.github.com/users/Arech/orgs","repos_url":"https://api.github.com/users/Arech/repos","events_url":"https://api.github.com/users/Arech/events{/privacy}","received_events_url":"https://api.github.com/users/Arech/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-23T18:24:43Z","updated_at":"2015-08-25T07:46:05Z","closed_at":null,"body":"Hi\r\nI was playing with GEMM routine while stuck on something strange: when I fed the routine with ColMajor matrices, it works nice. But if I change ordering parameter from ColMajor to RowMajor (all other parameters remaining the same), routine fails with an error saying something about wrong parameter 10. From my point of view, this is strange, because all matrices dimensions remains the same and ordering should influence only how underlying data should be treated. With that in mind, I expected the GEMM to calculate matrix product (of course, the result should be different from the same product in ColMajor ordering), but it just failed.\r\nI was intrigued by the problem and dug into sources and here is what I've found:\r\n\r\nOpenBLAS uses a nice trick to allow RowMajor matrix multiplication, that is based on the fact that ColMajor'ed matrix unrolls into the same linear array as would unroll transposed RowMajor'ed matrix and vice-versa. Then using linear algebra properties we can get multiplication result of A * B->C in form of (B' * A')->C'\r\nThis means, that in order to get a matrix product of two RowMajor'ed matrices AA [m_rows * k_column] * BB [k * n] -> CC [m * n] we can use the GEMM routine for ColMajored matrices, provided that we call it like we want to get a product of BT[n * k] * AT[k * m]->CT[n * m]. Here ColMajor'ed matrix BT has exactly the same underlying linear representation as RowMajor'ed matrix BB, CM-matrix AT - like RM-matrix AA and CT - like CC.\r\nThat was the theory.\r\nIn practice the following call to ?gemm() to obtain RowMajor'ed product of AA[m * k] * BB[k * n]->CC[m * n]\r\n```\r\n?gemm(RowMajor, \"N\", \"N\",\r\n    m<-CC.rows()/*=m*/, n<-CC.cols()/*=n*/, k<-AA.cols()/*=k*/, alpha,\r\n    AA, lda<-m_param/*=CC.rows()=m*/,\r\n    BB, ldb<-k_param/*=AA.cols()=k*/,...)\r\n```\r\nshould be treated as ColMajor multiplication of matrices BT[n * k] * AT[k * m]->CT [n * m], which looks like this call:\r\n```\r\n?gemm(ColMajor, \"N\", \"N\",\r\n    m<-CT.rows()/*=CC.cols()=n*/, n<-CT.cols()/*=CC.rows()=m*/, k<-BT.cols()/*=BB.rows()=k*/, alpha,\r\n    BB, lda<-m_param/*=CT.rows()=CC.cols()=n*/,\r\n    AA, ldb<-k_param/*=BT.cols()=BB.rows()=k*/,...)\r\n```\r\nOr in short: treat A as B, m as n (and vice-versa). k- remains the same, lda changes from m to n, ldb remains the same.\r\nNow lets look at the code, file ./interface/gemm.c. Line 326 starts a segment that does this permutations. Lines 335-366 have this content:\r\n```c\r\n    args.lda = ldb;\r\n    args.ldb = lda;\r\n\r\n    /* skipped not relevant code here, it doesn't use neither ldb/lda, nor args.ldb / args.lda */\r\n\r\n    nrowb = args.k;\r\n    if (transb & 1) nrowb = args.n;\r\n\r\n/* not relevant */\r\n\r\n    if (args.ldb < nrowb)  info = 10;\r\n```\r\nSo what this basically does is just swap lda and ldb content, which is wrong idea provided that my previous thoughts are correct, and then compares invalid args.ldb with nrowb, and that in turn triggers an error that points to invalid parameter 10 (exactly the same error I got while playing with precompiled gemm).\r\n\r\nAny thoughts?\r\n\r\n[?gemm parameter reference](https://software.intel.com/ru-ru/node/520775#AE8380B9-CAC8-4C57-9AF3-2EAAC6ACFC1B)"},"comment":{"url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/comments/134512783","html_url":"https://github.com/xianyi/OpenBLAS/issues/620#issuecomment-134512783","issue_url":"https://api.github.com/repos/xianyi/OpenBLAS/issues/620","id":134512783,"user":{"login":"Arech","id":778977,"avatar_url":"https://avatars.githubusercontent.com/u/778977?v=3","gravatar_id":"","url":"https://api.github.com/users/Arech","html_url":"https://github.com/Arech","followers_url":"https://api.github.com/users/Arech/followers","following_url":"https://api.github.com/users/Arech/following{/other_user}","gists_url":"https://api.github.com/users/Arech/gists{/gist_id}","starred_url":"https://api.github.com/users/Arech/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Arech/subscriptions","organizations_url":"https://api.github.com/users/Arech/orgs","repos_url":"https://api.github.com/users/Arech/repos","events_url":"https://api.github.com/users/Arech/events{/privacy}","received_events_url":"https://api.github.com/users/Arech/received_events","type":"User","site_admin":false},"created_at":"2015-08-25T07:46:05Z","updated_at":"2015-08-25T07:46:05Z","body":"@xianyi , thanks for reply. Could you please elaborate, why do you think lda- and ldb-params should have the values you've set for them?\r\nAs far as I understand the documentation, lda=k and ldb=n are corresponds to transposed matrices, not the normal (as we're using)."}}