{"action":"created","issue":{"url":"https://api.github.com/repos/NixOS/nixpkgs/issues/6772","labels_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/6772/labels{/name}","comments_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/6772/comments","events_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/6772/events","html_url":"https://github.com/NixOS/nixpkgs/issues/6772","id":60826942,"number":6772,"title":"numpy with openblas doesn't seem to actually use openblas","user":{"login":"aherrmann","id":732652,"avatar_url":"https://avatars.githubusercontent.com/u/732652?v=3","gravatar_id":"","url":"https://api.github.com/users/aherrmann","html_url":"https://github.com/aherrmann","followers_url":"https://api.github.com/users/aherrmann/followers","following_url":"https://api.github.com/users/aherrmann/following{/other_user}","gists_url":"https://api.github.com/users/aherrmann/gists{/gist_id}","starred_url":"https://api.github.com/users/aherrmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aherrmann/subscriptions","organizations_url":"https://api.github.com/users/aherrmann/orgs","repos_url":"https://api.github.com/users/aherrmann/repos","events_url":"https://api.github.com/users/aherrmann/events{/privacy}","received_events_url":"https://api.github.com/users/aherrmann/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/NixOS/nixpkgs/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-03-12T14:28:25Z","updated_at":"2015-04-16T13:58:12Z","closed_at":null,"body":"This issue is related to #6381.\r\n\r\ncc: @peti @olavks @ttuegel\r\n\r\nI just tried the openblas version of the numpy package. Either I'm doing something wrong or this doesn't actually activate openblas. I built numpy from the current master branch and get the following report on the numpy configuration:\r\n\r\n```python\r\n>>> import numpy as np\r\n>>> np.__config__.show()\r\nlapack_info:\r\n  NOT AVAILABLE\r\nlapack_opt_info:\r\n  NOT AVAILABLE\r\nblas_info:\r\n  NOT AVAILABLE\r\natlas_threads_info:\r\n  NOT AVAILABLE\r\nblas_src_info:\r\n  NOT AVAILABLE\r\natlas_blas_info:\r\n  NOT AVAILABLE\r\nlapack_src_info:\r\n  NOT AVAILABLE\r\nopenblas_info:\r\n  NOT AVAILABLE\r\natlas_blas_threads_info:\r\n  NOT AVAILABLE\r\nblas_mkl_info:\r\n  NOT AVAILABLE\r\nblas_opt_info:\r\n  NOT AVAILABLE\r\natlas_info:\r\n  NOT AVAILABLE\r\nlapack_mkl_info:\r\n  NOT AVAILABLE\r\nmkl_info:\r\n  NOT AVAILABLE\r\n>>> np.version.version\r\n'1.8.2'\r\n```\r\n\r\nand\r\n\r\n```\r\n$ ldd /nix/store/pvp2iqfs46msrvs4s56cv4n913bx92ms-python2.7-numpy-1.8.2/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so\r\n        linux-vdso.so.1 (0x00007ffff67fe000)\r\n        libgcc_s.so.1 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libgcc_s.so.1 (0x00007fd24aa76000)\r\n        libpython2.7.so.1.0 => /nix/store/5aq9sjd0g4frmpqni0a6zykzkcnzc3al-python-2.7.9/lib/libpython2.7.so.1.0 (0x00007fd24a66e000)\r\n        libpthread.so.0 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libpthread.so.0 (0x00007fd24a451000)\r\n        libc.so.6 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libc.so.6 (0x00007fd24a0b4000)\r\n        libdl.so.2 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libdl.so.2 (0x00007fd249eb0000)\r\n        libutil.so.1 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libutil.so.1 (0x00007fd249cac000)\r\n        libm.so.6 => /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib/libm.so.6 (0x00007fd2499a9000)\r\n        /nix/store/7yvf54nxwmaslcgyqfghqsqr1dwmr8ld-glibc-2.20/lib64/ld-linux-x86-64.so.2 (0x00007fd24b0a3000)\r\n```\r\n\r\nI.e., it's not actually using any blas/lapack library.\r\n\r\n---\r\n\r\nAccording to the [documentation](http://www.scipy.org/scipylib/building/linux.html), and [this blog post](https://leemendelowitz.github.io/blog/installing-numpy-with-openblas.html) numpy can be configured to use openblas by saving the following lines to `site.cfg`:\r\n\r\n```\r\n[openblas]\r\nlibraries = openblas\r\nlibrary_dirs = ${pkgs.openblas}/lib\r\ninclude_dirs = ${pkgs.openblas}/include\r\n```\r\n\r\n(The `BLAS`, and `LAPACK` environment variables are not necessary in that case.)\r\n\r\nWhich results in:\r\n\r\n```python\r\n>>> import numpy as np\r\n>>> np.__config__.show()\r\nlapack_opt_info:\r\n    libraries = ['openblas', 'openblas']\r\n    library_dirs = ['/nix/store/iihnm96k2d28wmf06rbhacpagdyb0hms-openblas-0.2.13/lib']\r\n    language = f77\r\nblas_opt_info:\r\n    libraries = ['openblas', 'openblas']\r\n    library_dirs = ['/nix/store/iihnm96k2d28wmf06rbhacpagdyb0hms-openblas-0.2.13/lib']\r\n    language = f77\r\nopenblas_info:\r\n    libraries = ['openblas', 'openblas']\r\n    library_dirs = ['/nix/store/iihnm96k2d28wmf06rbhacpagdyb0hms-openblas-0.2.13/lib']\r\n    language = f77\r\nblas_mkl_info:\r\n  NOT AVAILABLE\r\n```\r\n\r\nand\r\n\r\n```\r\n$ ldd /nix/store/v872lr2ds0ncw9r35rnia0nd1jm496d6-python2.7-numpy-1.8.2/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so\r\n        linux-vdso.so.1 (0x00007fff87fde000)\r\n        libopenblas.so.0 => /nix/store/iihnm96k2d28wmf06rbhacpagdyb0hms-openblas-0.2.13/lib/libopenblas.so.0 (0x00007fb427dee000)\r\n        libpython2.7.so.1.0 => /nix/store/mi3k70qyq1qd3cp913vnbwz7f5wbkg9a-python-2.7.9/lib/libpython2.7.so.1.0 (0x00007fb4279e5000)\r\n        libgfortran.so.3 => /nix/store/5pksr4hli52jvfznxw7g9fx8pmbz904p-gfortran-4.8.4/lib/libgfortran.so.3 (0x00007fb4276cd000)\r\n        libm.so.6 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libm.so.6 (0x00007fb4273ca000)\r\n        libgcc_s.so.1 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libgcc_s.so.1 (0x00007fb4271b3000)\r\n        libquadmath.so.0 => /nix/store/5pksr4hli52jvfznxw7g9fx8pmbz904p-gfortran-4.8.4/lib/libquadmath.so.0 (0x00007fb426f78000)\r\n        libc.so.6 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libc.so.6 (0x00007fb426bdb000)\r\n        libpthread.so.0 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libpthread.so.0 (0x00007fb4269be000)\r\n        libdl.so.2 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libdl.so.2 (0x00007fb4267ba000)\r\n        libutil.so.1 => /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib/libutil.so.1 (0x00007fb4265b6000)\r\n        /nix/store/93zfs0zzndi7pkjkjxawlafdj8m90kg5-glibc-2.20/lib64/ld-linux-x86-64.so.2 (0x00007fb428ed1000)\r\n```\r\n\r\nI.e. numpy is indeed using openblas.\r\n\r\nHowever, this introduces another issue. When I run the numpy test suite I get an illegal instruction in the dot-product test:\r\n\r\n```python\r\n>>> np.test('full', verbose=10)\r\nRunning unit tests for numpy\r\nNumPy version 1.8.2\r\nNumPy is installed in /nix/store/v872lr2ds0ncw9r35rnia0nd1jm496d6-python2.7-numpy-1.8.2/lib/python2.7/site-packages/numpy\r\nPython version 2.7.9 (default, Jan 01 1970, 00:00:01) [GCC 4.8.4]\r\nnose version 1.3.4\r\n# ...\r\ntest_blasdot.test_blasdot_used ... ok\r\ntest_blasdot.test_dot_2args ... Illegal instruction\r\n```\r\n\r\nIn fact just trying to evaluate a dot product causes an illegal instruction:\r\n\r\n```python\r\n>>> x = np.ones((2,3)); y = np.ones((3,4)); np.dot(x,y)\r\nIllegal instruction\r\n```\r\n\r\nI tested this on nixos with chroot builds enabled in a nixops virtualbox deployment on x64_86, and with nix on an opensuse 13.1 host running on a physical corei5 machine. Same symptoms in both cases.\r\n\r\nHave you experienced similar issues? Am I doing something wrong?\r\n\r\n---\r\n\r\nPatch for numpy openblas configuration:\r\n\r\n```\r\ndiff --git a/pkgs/top-level/python-packages.nix b/pkgs/top-level/python-packages.nix\r\nindex 0fbb4cc..85ce0c1 100644\r\n--- a/pkgs/top-level/python-packages.nix\r\n+++ b/pkgs/top-level/python-packages.nix\r\n@@ -6876,10 +6876,12 @@ let\r\n     preConfigure = ''\r\n       sed -i 's/-faltivec//' numpy/distutils/system_info.py\r\n       sed -i '0,/from numpy.distutils.core/s//import setuptools;from numpy.distutils.core/' setup.py\r\n-    '';\r\n-\r\n-    preBuild = ''\r\n-      export BLAS=${pkgs.openblas} LAPACK=${pkgs.openblas}\r\n+      cat >site.cfg <<EOF\r\n+      [openblas]\r\n+      libraries = openblas\r\n+      library_dirs = ${pkgs.openblas}/lib\r\n+      include_dirs = ${pkgs.openblas}/include\r\n+      EOF\r\n     '';\r\n\r\n     setupPyBuildFlags = [\"--fcompiler='gnu95'\"];\r\n```"},"comment":{"url":"https://api.github.com/repos/NixOS/nixpkgs/issues/comments/93740858","html_url":"https://github.com/NixOS/nixpkgs/issues/6772#issuecomment-93740858","issue_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/6772","id":93740858,"user":{"login":"teh","id":139251,"avatar_url":"https://avatars.githubusercontent.com/u/139251?v=3","gravatar_id":"","url":"https://api.github.com/users/teh","html_url":"https://github.com/teh","followers_url":"https://api.github.com/users/teh/followers","following_url":"https://api.github.com/users/teh/following{/other_user}","gists_url":"https://api.github.com/users/teh/gists{/gist_id}","starred_url":"https://api.github.com/users/teh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/teh/subscriptions","organizations_url":"https://api.github.com/users/teh/orgs","repos_url":"https://api.github.com/users/teh/repos","events_url":"https://api.github.com/users/teh/events{/privacy}","received_events_url":"https://api.github.com/users/teh/received_events","type":"User","site_admin":false},"created_at":"2015-04-16T13:58:11Z","updated_at":"2015-04-16T13:58:11Z","body":"Without patch:\r\n```\r\n>>> np.__config__.show()\r\natlas_blas_threads_info:\r\n  NOT AVAILABLE\r\nlapack_mkl_info:\r\n  NOT AVAILABLE\r\nblas_info:\r\n  NOT AVAILABLE\r\nblas_opt_info:\r\n  NOT AVAILABLE\r\nopenblas_info:\r\n  NOT AVAILABLE\r\nlapack_opt_info:\r\n  NOT AVAILABLE\r\nmkl_info:\r\n  NOT AVAILABLE\r\natlas_info:\r\n  NOT AVAILABLE\r\natlas_blas_info:\r\n  NOT AVAILABLE\r\natlas_threads_info:\r\n  NOT AVAILABLE\r\nlapack_src_info:\r\n  NOT AVAILABLE\r\nblas_mkl_info:\r\n  NOT AVAILABLE\r\nlapack_info:\r\n  NOT AVAILABLE\r\nblas_src_info:\r\n  NOT AVAILABLE\r\n```\r\n\r\nWith patch:\r\n```\r\n>>> np.__config__.show()\r\nblas_mkl_info:\r\n  NOT AVAILABLE\r\nlapack_opt_info:\r\n    library_dirs = ['/nix/store/q4p2s86969nxh3icpc4k178xfydhh3zx-openblas-0.2.13/lib']\r\n    libraries = ['openblas', 'openblas']\r\n    language = f77\r\nopenblas_info:\r\n    library_dirs = ['/nix/store/q4p2s86969nxh3icpc4k178xfydhh3zx-openblas-0.2.13/lib']\r\n    libraries = ['openblas', 'openblas']\r\n    language = f77\r\nblas_opt_info:\r\n    library_dirs = ['/nix/store/q4p2s86969nxh3icpc4k178xfydhh3zx-openblas-0.2.13/lib']\r\n    libraries = ['openblas', 'openblas']\r\n    language = f77\r\n```"}}