{"action":"created","issue":{"url":"https://api.github.com/repos/yixuan/rARPACK/issues/10","labels_url":"https://api.github.com/repos/yixuan/rARPACK/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/yixuan/rARPACK/issues/10/comments","events_url":"https://api.github.com/repos/yixuan/rARPACK/issues/10/events","html_url":"https://github.com/yixuan/rARPACK/issues/10","id":105130871,"number":10,"title":"Running out of memory in inverse and shift mode on sparse matrices","user":{"login":"swajnautcz","id":13739669,"avatar_url":"https://avatars.githubusercontent.com/u/13739669?v=3","gravatar_id":"","url":"https://api.github.com/users/swajnautcz","html_url":"https://github.com/swajnautcz","followers_url":"https://api.github.com/users/swajnautcz/followers","following_url":"https://api.github.com/users/swajnautcz/following{/other_user}","gists_url":"https://api.github.com/users/swajnautcz/gists{/gist_id}","starred_url":"https://api.github.com/users/swajnautcz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/swajnautcz/subscriptions","organizations_url":"https://api.github.com/users/swajnautcz/orgs","repos_url":"https://api.github.com/users/swajnautcz/repos","events_url":"https://api.github.com/users/swajnautcz/events{/privacy}","received_events_url":"https://api.github.com/users/swajnautcz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-09-07T01:47:16Z","updated_at":"2015-09-08T12:11:28Z","closed_at":null,"body":"Hello,\r\n\r\nI am trying to get the Eigenvalues of a large sparse matrix in R. For the large eigenvalues, I have no problems but for the small Eigenvalues, the \"SM\" does not converge even if I increase the number of iterations and tolerance. If I use sigma = 0, then I get a bad_alloc in a while. The matrix in question is real symmetric 174515 x 174515, with 9363966 nonzeroes and has just over 300 MB in the mtx (text format). I'm running 64-bit version of R and there is 16 GB of RAM in the box, I see little reason why this should happen. Obviously I'm doing something wrong. My R code is as follows:\r\n\r\n    library(Matrix)\r\n    f <- file(\"file://./lambda.mtx\", \"r\")\r\n    str(m <- readMM(f))\r\n    close(f)\r\n    # read sparse matrix\r\n\r\n    str(A <- sparseMatrix(c(m@i, m@j), c(m@j, m@i), index1 = FALSE, x = c(m@x, m@x), giveCsparse = TRUE, use.last.ij = TRUE))\r\n    # do not treat it as symmetric, this doubles the memory but avoids\r\n    # conversion to a full matrix in eigs() so it is a good deal ...\r\n\r\n    str(nnzM <- length(m@x))\r\n    str(nnzA <- A@p[A@Dim[2] + 1])\r\n    if(nnzM * 2 - m@Dim[2] != nnzA) {\r\n    stop(\"failed to form full matrix A\")\r\n    }\r\n    if(A@x[1] != m@x[1]) {\r\n        if(A@x[1] == 2 * m@x[1])\r\n            warning(\"failed to form full matrix A: diagonal elements added\")\r\n        else\r\n            warning(\"failed to form full matrix A: bad diagonal value\")\r\n    }\r\n    # make sure the repeated values of the diagonal were eliminated instead of added (use.last.ij = TRUE)\r\n\r\n    library(rARPACK)\r\n    str(le <- eigs(A, k = 5, which = \"LM\", opts = list(retvec = FALSE))) # or dsyMatrix # runs ok\r\n    str(se <- eigs(A, k = 5, sigma = 0, opts = list(retvec = FALSE))) # or dsyMatrix # bad_alloc\r\n    le$values[1]\r\n    se$values[se$nconv]\r\n\r\nI have never written anything in R, maybe that's the issue. Any insights?\r\n"},"comment":{"url":"https://api.github.com/repos/yixuan/rARPACK/issues/comments/138534767","html_url":"https://github.com/yixuan/rARPACK/issues/10#issuecomment-138534767","issue_url":"https://api.github.com/repos/yixuan/rARPACK/issues/10","id":138534767,"user":{"login":"yixuan","id":216398,"avatar_url":"https://avatars.githubusercontent.com/u/216398?v=3","gravatar_id":"","url":"https://api.github.com/users/yixuan","html_url":"https://github.com/yixuan","followers_url":"https://api.github.com/users/yixuan/followers","following_url":"https://api.github.com/users/yixuan/following{/other_user}","gists_url":"https://api.github.com/users/yixuan/gists{/gist_id}","starred_url":"https://api.github.com/users/yixuan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yixuan/subscriptions","organizations_url":"https://api.github.com/users/yixuan/orgs","repos_url":"https://api.github.com/users/yixuan/repos","events_url":"https://api.github.com/users/yixuan/events{/privacy}","received_events_url":"https://api.github.com/users/yixuan/received_events","type":"User","site_admin":false},"created_at":"2015-09-08T12:11:28Z","updated_at":"2015-09-08T12:11:28Z","body":"Thanks for the test.\r\n\r\nSince this package is using Eigen(http://eigen.tuxfamily.org/) for linear algebra computation, the factorization I'm using for sparse matrix is an implementation of SuperLU (http://eigen.tuxfamily.org/dox/classEigen_1_1SparseLU.html). This can be improved since we can use sparse LDL to optimize for symmetric matrices.\r\n\r\nEigen also provides other linear solvers including iterative methods (http://eigen.tuxfamily.org/dox/group__TopicSparseSystems.html). I have little experience on large sparse linear solver, so could you give me some hints on which solver I should use? Seeing that you could factorize the matrix at this scale in a second, I'm eager to learn what technique you are using. Thank you!"}}