{"action":"created","issue":{"url":"https://api.github.com/repos/originell/jpype/issues/133","labels_url":"https://api.github.com/repos/originell/jpype/issues/133/labels{/name}","comments_url":"https://api.github.com/repos/originell/jpype/issues/133/comments","events_url":"https://api.github.com/repos/originell/jpype/issues/133/events","html_url":"https://github.com/originell/jpype/issues/133","id":67387604,"number":133,"title":"Reading from Java byte array with np.frombuffer","user":{"login":"caspervdw","id":8937919,"avatar_url":"https://avatars.githubusercontent.com/u/8937919?v=3","gravatar_id":"","url":"https://api.github.com/users/caspervdw","html_url":"https://github.com/caspervdw","followers_url":"https://api.github.com/users/caspervdw/followers","following_url":"https://api.github.com/users/caspervdw/following{/other_user}","gists_url":"https://api.github.com/users/caspervdw/gists{/gist_id}","starred_url":"https://api.github.com/users/caspervdw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/caspervdw/subscriptions","organizations_url":"https://api.github.com/users/caspervdw/orgs","repos_url":"https://api.github.com/users/caspervdw/repos","events_url":"https://api.github.com/users/caspervdw/events{/privacy}","received_events_url":"https://api.github.com/users/caspervdw/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":20,"created_at":"2015-04-09T15:32:35Z","updated_at":"2015-05-07T15:09:56Z","closed_at":null,"body":"I am trying to interface bioformats from python, as part of the PIMS project (https://github.com/soft-matter/pims/pull/144). My experience with JPype up to now is very good, but there is one issue that I hope to optimize.\r\n\r\nBioformats inputs images into a JByte array and separately I can readout the datatype which can be any number. Because we potentially have to readout a lot of them, I want this step to be as fast as possible.\r\n\r\nSo I read https://github.com/originell/jpype/issues/71 and https://github.com/originell/jpype/pull/73 and came up with a few options:\r\n\r\n    from __future__ import division\r\n    import numpy as np\r\n    import jpype\r\n    jpype.startJVM(jpype.getDefaultJVMPath())\r\n\r\n    nbytes = 2*1024**2\r\n    a = jpype.java.nio.ByteBuffer.allocate(2*1024**2).array()\r\n    _source_dtype = '>u4'\r\n    \r\n    def convert_JArray1(a):\r\n        arr = np.array(np.frombuffer(unicode(a), dtype=np.uint16), dtype=np.uint8)\r\n        return np.frombuffer(buffer(arr), dtype=_source_dtype)\r\n    \r\n    def convert_JArray2(a):    \r\n        java_str = jpype.java.lang.String(a, 'ISO-8859-1')\r\n        arr = np.array(np.frombuffer(java_str.toString(), dtype=np.uint16), dtype=np.byte)\r\n        return np.frombuffer(buffer(arr), dtype=_source_dtype)\r\n    \r\n    def convert_JArray3(a):\r\n        return np.frombuffer(np.array(a, dtype=np.byte), dtype=_source_dtype)\r\n    \r\n    %timeit convert_JArray1(a) # 100 loops, best of 3: 9.14 ms per loop\r\n    %timeit convert_JArray2(a) # 100 loops, best of 3: 7.9 ms per loop\r\n    %timeit convert_JArray3(a) # 1 loops, best of 3: 14 s per loop\r\n\r\nMy questions are:\r\n - am I doing something wrong in the 3rd option?\r\n - the second one works on my Win64 platform; on Travis it fails. Maybe something with endianness but I can't figure it out. Does anyone have a suggestion? (edit: with fails, I mean, the returned array has the wrong shape)\r\n\r\nI am using jpype 0.5.7 on py2.7. Thanks in advance for your response!\r\n"},"comment":{"url":"https://api.github.com/repos/originell/jpype/issues/comments/99902930","html_url":"https://github.com/originell/jpype/issues/133#issuecomment-99902930","issue_url":"https://api.github.com/repos/originell/jpype/issues/133","id":99902930,"user":{"login":"yetifrisstlama","id":7136811,"avatar_url":"https://avatars.githubusercontent.com/u/7136811?v=3","gravatar_id":"","url":"https://api.github.com/users/yetifrisstlama","html_url":"https://github.com/yetifrisstlama","followers_url":"https://api.github.com/users/yetifrisstlama/followers","following_url":"https://api.github.com/users/yetifrisstlama/following{/other_user}","gists_url":"https://api.github.com/users/yetifrisstlama/gists{/gist_id}","starred_url":"https://api.github.com/users/yetifrisstlama/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yetifrisstlama/subscriptions","organizations_url":"https://api.github.com/users/yetifrisstlama/orgs","repos_url":"https://api.github.com/users/yetifrisstlama/repos","events_url":"https://api.github.com/users/yetifrisstlama/events{/privacy}","received_events_url":"https://api.github.com/users/yetifrisstlama/received_events","type":"User","site_admin":false},"created_at":"2015-05-07T15:09:56Z","updated_at":"2015-05-07T15:09:56Z","body":"Hi,\r\n\r\nthanks, your explanation makes complete sense!\r\nI also agree that the [:] conversion should work for for any kind of n-d arrays.\r\n\r\nYour idea inspired me to do a really nasty hack for 2D arrays, which seems to improve performance quite a bit:\r\n\r\n```python\r\ndef convert2DJarrayToNumpy( jArr ):\r\n    arrShape = ( len(jArr), len(jArr[0]) )\r\n    arrType = type( jArr[0][0] )\r\n    resultArray = empty( arrShape, dtype=arrType )\r\n    for i,cols in enumerate( jArr[:] ):\r\n        resultArray[i,:] = cols[:]\r\n    return resultArray\r\n\r\n# 2D case, not so slow\r\nsourceDat = arange( 1000000, dtype=double ).reshape((1000,-1))\r\na = JArray( JDouble, sourceDat.ndim )( sourceDat.tolist() )\r\n%timeit convert2DJarrayToNumpy( a )\r\n# 10 loops, best of 3: 33.4 ms per loop\r\n\r\n```\r\n\r\n\r\n"}}