{"action":"created","issue":{"url":"https://api.github.com/repos/astropy/astropy/issues/2960","labels_url":"https://api.github.com/repos/astropy/astropy/issues/2960/labels{/name}","comments_url":"https://api.github.com/repos/astropy/astropy/issues/2960/comments","events_url":"https://api.github.com/repos/astropy/astropy/issues/2960/events","html_url":"https://github.com/astropy/astropy/issues/2960","id":43236061,"number":2960,"title":"\"underlying buffer is not C-contiguous failure\" error with Python 3 and numpy-dev","user":{"login":"astrofrog","id":314716,"avatar_url":"https://avatars.githubusercontent.com/u/314716?v=3","gravatar_id":"","url":"https://api.github.com/users/astrofrog","html_url":"https://github.com/astrofrog","followers_url":"https://api.github.com/users/astrofrog/followers","following_url":"https://api.github.com/users/astrofrog/following{/other_user}","gists_url":"https://api.github.com/users/astrofrog/gists{/gist_id}","starred_url":"https://api.github.com/users/astrofrog/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astrofrog/subscriptions","organizations_url":"https://api.github.com/users/astrofrog/orgs","repos_url":"https://api.github.com/users/astrofrog/repos","events_url":"https://api.github.com/users/astrofrog/events{/privacy}","received_events_url":"https://api.github.com/users/astrofrog/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/astropy/astropy/labels/Bug","name":"Bug","color":"eb6420"},{"url":"https://api.github.com/repos/astropy/astropy/labels/io.fits","name":"io.fits","color":"c7def8"},{"url":"https://api.github.com/repos/astropy/astropy/labels/Upstream%20Fix%20Required","name":"Upstream Fix Required","color":"444444"}],"state":"open","locked":false,"assignee":{"login":"mdboom","id":38294,"avatar_url":"https://avatars.githubusercontent.com/u/38294?v=3","gravatar_id":"","url":"https://api.github.com/users/mdboom","html_url":"https://github.com/mdboom","followers_url":"https://api.github.com/users/mdboom/followers","following_url":"https://api.github.com/users/mdboom/following{/other_user}","gists_url":"https://api.github.com/users/mdboom/gists{/gist_id}","starred_url":"https://api.github.com/users/mdboom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdboom/subscriptions","organizations_url":"https://api.github.com/users/mdboom/orgs","repos_url":"https://api.github.com/users/mdboom/repos","events_url":"https://api.github.com/users/mdboom/events{/privacy}","received_events_url":"https://api.github.com/users/mdboom/received_events","type":"User","site_admin":false},"milestone":null,"comments":18,"created_at":"2014-09-19T13:06:46Z","updated_at":"2015-02-17T21:27:40Z","closed_at":null,"body":"Now that the Jenkins instance is back, there is one failure with numpy-dev:\r\n\r\n```\r\ndef test_dist():\r\n        with get_pkg_data_fileobj(\r\n                os.path.join(\"data\", \"dist.fits\"), encoding='binary') as test_file:\r\n            hdulist = fits.open(test_file)\r\n            wcs1 = wcs.WCS(hdulist[0].header, hdulist)\r\n            assert wcs1.det2im2 is not None\r\n>           s = cPickle.dumps(wcs1)\r\n\r\nastropy/wcs/tests/test_pickle.py:29: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <astropy.wcs.wcs.WCS object at 0x114464ad0>\r\n\r\n    def __reduce__(self):\r\n        \"\"\"\r\n            Support pickling of WCS objects.  This is done by serializing\r\n            to an in-memory FITS file and dumping that as a string.\r\n            \"\"\"\r\n    \r\n        hdulist = self.to_fits(relax=True)\r\n    \r\n        buffer = io.BytesIO()\r\n>       hdulist.writeto(buffer)\r\n\r\nastropy/wcs/wcs.py:2583: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = [<astropy.io.fits.hdu.image.PrimaryHDU object at 0x114848890>, <astropy.io.fits.hdu.image.ImageHDU object at 0x114848f10>]\r\nfileobj = <astropy.io.fits.py3compat._File <_io.BytesIO object at 0x111349738>>\r\noutput_verify = 'exception', clobber = False, checksum = False\r\n\r\n    def writeto(self, fileobj, output_verify='exception', clobber=False,\r\n                checksum=False):\r\n        \"\"\"\r\n            Write the `HDUList` to a new file.\r\n    \r\n            Parameters\r\n            ----------\r\n            fileobj : file path, file object or file-like object\r\n                File to write to.  If a file object, must be opened in a\r\n                writeable mode.\r\n    \r\n            output_verify : str\r\n                Output verification option.  Must be one of ``\"fix\"``,\r\n                ``\"silentfix\"``, ``\"ignore\"``, ``\"warn\"``, or\r\n                ``\"exception\"``.  May also be any combination of ``\"fix\"`` or\r\n                ``\"silentfix\"`` with ``\"+ignore\"``, ``+warn``, or ``+exception\"\r\n                (e.g. ``\"fix+warn\"``).  See :ref:`verify` for more info.\r\n    \r\n            clobber : bool\r\n                When `True`, overwrite the output file if exists.\r\n    \r\n            checksum : bool\r\n                When `True` adds both ``DATASUM`` and ``CHECKSUM`` cards\r\n                to the headers of all HDU's written to the file.\r\n            \"\"\"\r\n    \r\n        if (len(self) == 0):\r\n            warnings.warn(\"There is nothing to write.\", AstropyUserWarning)\r\n            return\r\n    \r\n        self.verify(option=output_verify)\r\n    \r\n        # make sure the EXTEND keyword is there if there is extension\r\n        self.update_extend()\r\n    \r\n        # make note of whether the input file object is already open, in which\r\n        # case we should not close it after writing (that should be the job\r\n        # of the caller)\r\n        closed = isinstance(fileobj, string_types) or fileobj_closed(fileobj)\r\n    \r\n        # writeto is only for writing a new file from scratch, so the most\r\n        # sensible mode to require is 'ostream'.  This can accept an open\r\n        # file object that's open to write only, or in append/update modes\r\n        # but only if the file doesn't exist.\r\n        fileobj = _File(fileobj, mode='ostream', clobber=clobber)\r\n        hdulist = self.fromfile(fileobj)\r\n    \r\n        for hdu in self:\r\n            hdu._prewriteto(checksum=checksum)\r\n            try:\r\n>               hdu._writeto(hdulist.__file)\r\n\r\nastropy/io/fits/hdu/hdulist.py:662: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <astropy.io.fits.hdu.image.ImageHDU object at 0x114848f10>\r\nfileobj = <astropy.io.fits.py3compat._File <_io.BytesIO object at 0x111349738>>\r\ninplace = False, copy = False\r\n\r\n    def _writeto(self, fileobj, inplace=False, copy=False):\r\n        # For now fileobj is assumed to be a _File object\r\n        if not inplace or self._new:\r\n            header_offset, _ = self._writeheader(fileobj)\r\n>           data_offset, data_size = self._writedata(fileobj)\r\n\r\nastropy/io/fits/hdu/base.py:699: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <astropy.io.fits.hdu.image.ImageHDU object at 0x114848f10>\r\nfileobj = <astropy.io.fits.py3compat._File <_io.BytesIO object at 0x111349738>>\r\n\r\n    def _writedata(self, fileobj):\r\n        # TODO: A lot of the simulateonly stuff should be moved back into the\r\n        # _File class--basically it should turn write and flush into a noop\r\n        offset = 0\r\n        size = 0\r\n    \r\n        if not fileobj.simulateonly:\r\n            fileobj.flush()\r\n            try:\r\n                offset = fileobj.tell()\r\n            except IOError:\r\n                offset = 0\r\n    \r\n        if self._data_loaded or self._data_needs_rescale:\r\n            if self.data is not None:\r\n>               size += self._writedata_internal(fileobj)\r\n\r\nastropy/io/fits/hdu/base.py:636: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <astropy.io.fits.hdu.image.ImageHDU object at 0x114848f10>\r\nfileobj = <astropy.io.fits.py3compat._File <_io.BytesIO object at 0x111349738>>\r\n\r\n    def _writedata_internal(self, fileobj):\r\n        size = 0\r\n    \r\n        if self.data is not None:\r\n            # Based on the system type, determine the byteorders that\r\n            # would need to be swapped to get to big-endian output\r\n            if sys.byteorder == 'little':\r\n                swap_types = ('<', '=')\r\n            else:\r\n                swap_types = ('<',)\r\n            # deal with unsigned integer 16, 32 and 64 data\r\n            if _is_pseudo_unsigned(self.data.dtype):\r\n                # Convert the unsigned array to signed\r\n                output = np.array(\r\n                    self.data - _unsigned_zero(self.data.dtype),\r\n                    dtype='>i%d' % self.data.dtype.itemsize)\r\n                should_swap = False\r\n            else:\r\n                output = self.data\r\n                byteorder = output.dtype.str[0]\r\n                should_swap = (byteorder in swap_types)\r\n    \r\n            if not fileobj.simulateonly:\r\n                if should_swap:\r\n                    output.byteswap(True)\r\n                    try:\r\n>                       fileobj.writearray(output)\r\n\r\nastropy/io/fits/hdu/image.py:523: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <astropy.io.fits.py3compat._File <_io.BytesIO object at 0x111349738>>\r\narray = array([[ -1.27305422e-04,  -2.63464113e-04,  -5.10875077e-04, ...,\r\n          2.40933063e-04,   1.06801039e-04,  -3.66208042e-08]], dtype=float32)\r\n\r\n    def writearray(self, array):\r\n        \"\"\"\r\n            Similar to file.write(), but writes a numpy array instead of a string.\r\n    \r\n            Also like file.write(), a flush() or close() may be needed before\r\n            the file on disk reflects the data written.\r\n            \"\"\"\r\n    \r\n        if hasattr(self.__file, 'write'):\r\n>           _array_to_file(array, self.__file)\r\n\r\nastropy/io/fits/file.py:274: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\narr = array([[ -1.27305422e-04,  -2.63464113e-04,  -5.10875077e-04, ...,\r\n          2.40933063e-04,   1.06801039e-04,  -3.66208042e-08]], dtype=float32)\r\noutfile = <_io.BytesIO object at 0x111349738>\r\n\r\n    def _array_to_file(arr, outfile):\r\n        \"\"\"\r\n        Write a numpy array to a file or a file-like object.\r\n    \r\n        Parameters\r\n        ----------\r\n        arr : `~numpy.ndarray`\r\n            The Numpy array to write.\r\n        outfile : file-like\r\n            A file-like object such as a Python file object, an `io.BytesIO`, or\r\n            anything else with a ``write`` method.  The file object must support\r\n            the buffer interface in its ``write``.\r\n    \r\n        If writing directly to an on-disk file this delegates directly to\r\n        `ndarray.tofile`.  Otherwise a slower Python implementation is used.\r\n        \"\"\"\r\n    \r\n    \r\n        if isfile(outfile):\r\n            write = lambda a, f: a.tofile(f)\r\n        else:\r\n            write = _array_to_file_like\r\n    \r\n        # Implements a workaround for a bug deep in OSX's stdlib file writing\r\n        # functions; on 64-bit OSX it is not possible to correctly write a number\r\n        # of bytes greater than 2 ** 32 and divisble by 4096 (or possibly 8192--\r\n        # whatever the default blocksize for the filesystem is).\r\n        # This issue should have a workaround in Numpy too, but hasn't been\r\n        # implemented there yet: https://github.com/astropy/astropy/issues/839\r\n        #\r\n        # Apparently Windows has its own fwrite bug:\r\n        # https://github.com/numpy/numpy/issues/2256\r\n    \r\n        if (sys.platform == 'darwin' and arr.nbytes >= _OSX_WRITE_LIMIT + 1 and\r\n                arr.nbytes % 4096 == 0):\r\n            # chunksize is a count of elements in the array, not bytes\r\n            chunksize = _OSX_WRITE_LIMIT // arr.itemsize\r\n        elif sys.platform.startswith('win'):\r\n            chunksize = _WIN_WRITE_LIMIT // arr.itemsize\r\n        else:\r\n            # Just pass the whole array to the write routine\r\n>           return write(arr, outfile)\r\n\r\nastropy/io/fits/util.py:646: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\narr = array([[ -1.27305422e-04,  -2.63464113e-04,  -5.10875077e-04, ...,\r\n          2.40933063e-04,   1.06801039e-04,  -3.66208042e-08]], dtype=float32)\r\nfileobj = <_io.BytesIO object at 0x111349738>\r\n\r\n    def _array_to_file_like(arr, fileobj):\r\n        \"\"\"\r\n        Write a `~numpy.ndarray` to a file-like object (which is not supported by\r\n        `numpy.ndarray.tofile`).\r\n        \"\"\"\r\n    \r\n        if arr.flags.contiguous:\r\n            # It sufficies to just pass the underlying buffer directly to the\r\n            # fileobj's write (assuming it supports the buffer interface, which\r\n            # unforunately there's no simple way to check)\r\n>           fileobj.write(arr.data)\r\nE           BufferError: memoryview: underlying buffer is not C-contiguous\r\n\r\nastropy/io/fits/util.py:667: BufferError\r\n```\r\n\r\nThis only occurs on Python 3.x\r\n\r\ncc @embray\r\n"},"comment":{"url":"https://api.github.com/repos/astropy/astropy/issues/comments/74757617","html_url":"https://github.com/astropy/astropy/issues/2960#issuecomment-74757617","issue_url":"https://api.github.com/repos/astropy/astropy/issues/2960","id":74757617,"user":{"login":"astrofrog","id":314716,"avatar_url":"https://avatars.githubusercontent.com/u/314716?v=3","gravatar_id":"","url":"https://api.github.com/users/astrofrog","html_url":"https://github.com/astrofrog","followers_url":"https://api.github.com/users/astrofrog/followers","following_url":"https://api.github.com/users/astrofrog/following{/other_user}","gists_url":"https://api.github.com/users/astrofrog/gists{/gist_id}","starred_url":"https://api.github.com/users/astrofrog/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astrofrog/subscriptions","organizations_url":"https://api.github.com/users/astrofrog/orgs","repos_url":"https://api.github.com/users/astrofrog/repos","events_url":"https://api.github.com/users/astrofrog/events{/privacy}","received_events_url":"https://api.github.com/users/astrofrog/received_events","type":"User","site_admin":false},"created_at":"2015-02-17T21:27:40Z","updated_at":"2015-02-17T21:27:40Z","body":"@bsipocz - thanks for confirming! Have added a known issue in #3513"}}