Read uint32_t from binary file from numpy
<p>The tofile() method of numpy array was used to write a unsigned 32bit integer array as a binary file (test.bin). Then, c++ was used to read uint32_t from the file. The problem is the c++ program didn't read the correct value. How to fix that?</p>

<p>Hex editor shows the test.bin is correct because the value increases one by one, as expected.</p>

<p>fileio.py:</p>

<pre><code>import numpy as np
x = np.arange(20)+65536
y = x.astype(np.uint32)
print(y)
with open("test.bin", "wb+") as f:
    y.tofile(f)
</code></pre>

<p>output of fileio.py:</p>

<pre><code>[65536 65537 65538 65539 65540 65541 65542 65543 65544 65545 65546 65547
 65548 65549 65550 65551 65552 65553 65554 65555]
</code></pre>

<p>test.bin: (in hex editor. vim with %!xxd command)</p>

<pre><code>0000000: 0000 0100 0100 0100 0200 0100 0300 0100  ................
0000010: 0400 0100 0500 0100 0600 0100 0700 0100  ................
0000020: 0800 0100 0900 0100 0a00 0100 0b00 0100  ................
0000030: 0c00 0100 0d00 0100 0e00 0100 0f00 0100  ................
0000040: 1000 0100 1100 0100 1200 0100 1300 0100  ................
0000050: 0a                                       .   
</code></pre>

<p>fileio.cpp:</p>

<pre><code>#include &lt;iostream&gt;
#include &lt;vector&gt;
#include &lt;string&gt;
#include &lt;fstream&gt;
#include &lt;byteswap.h&gt;

using namespace std;

int main(void){
    vector&lt;char&gt; arr;
    char n;
    int i;
    int num_char=0;
    //ifstream f("test.bin");
    fstream f;
    f.open("test.bin", ios::in | ios::binary);

    while(f&gt;&gt;n){
        cout &lt;&lt; int(n) &lt;&lt; ' ';
        arr.push_back(n);
        num_char++;
    }
    cout &lt;&lt; endl;
    cout &lt;&lt; "number of characters: " &lt;&lt; num_char &lt;&lt; endl;
    size_t uint32t_size = arr.size() * sizeof(char) / sizeof(uint32_t);
    uint32_t* ptr_arr0 = (uint32_t*)arr.data();

    cout &lt;&lt; "uint32_t size: " &lt;&lt; uint32t_size &lt;&lt; endl;
    cout &lt;&lt; "uint32_t interpretatin:" &lt;&lt; endl;
    for (i = 0; i &lt; uint32t_size; i++){
        cout &lt;&lt; ptr_arr0[i] &lt;&lt; ' ';
    }
    cout &lt;&lt; endl;

    cout &lt;&lt; "32bit byteswap: " &lt;&lt; endl;
    for (i=0; i &lt; uint32t_size; i++){
        ptr_arr0[i] = bswap_32(ptr_arr0[i]);
        cout &lt;&lt; ptr_arr0[i] &lt;&lt; ' ';
    }
    cout &lt;&lt; endl;
    cout &lt;&lt; "characters after byte swap: " &lt;&lt; endl;
    for (i=0; i &lt; num_char; i++){
        cout &lt;&lt; int(arr[i]) &lt;&lt; ' ';
    }
    cout &lt;&lt; endl;

}   
</code></pre>

<p>output of fileio.cpp:</p>

<pre><code>0 0 1 0 1 0 1 0 2 0 1 0 3 0 1 0 4 0 1 0 5 0 1 0 6 0 1 0 7 0 1 0 8 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 14 0 1 0 15 0 1 0 16 0 1 0 17 0 1 0 18 0 1 0 19 0 1 0 
number of characters: 75
uint32_t size: 18
uint32_t interpretatin:
65536 65537 65538 65539 65540 65541 65542 65543 65544 256 16777217 65536 234881280 251658496 268435712 285212928 301990144 318767360 
32bit byteswap: 
256 16777472 33554688 50331904 67109120 83886336 100663552 117440768 134217984 65536 16777217 256 65550 65551 65552 65553 65554 65555 
characters after byte swap: 
0 1 0 0 0 1 0 1 0 1 0 2 0 1 0 3 0 1 0 4 0 1 0 5 0 1 0 6 0 1 0 7 0 1 0 8 0 0 1 0 1 0 0 1 0 1 0 0 14 0 1 0 15 0 1 0 16 0 1 0 17 0 1 0 18 0 1 0 19 0 1 0 0 1 0 
</code></pre>

<p>It seems that byte swapping is not the only problem. The c++ program didn't even read all the 8bit integers correctly, as in:</p>

<pre><code>... 1 0 8 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 14 ...
</code></pre>

<p>There is no number 9,10,11,12,13, despite these numbers exist in hex editor view of test.bin.</p>

<p>Possibly related questions:</p>

<ul>
<li><a href="https://stackoverflow.com/questions/19787186/reading-an-uint32-t-from-a-byte-stream">Reading an uint32_t from a byte stream</a> </li>
<li><a href="https://stackoverflow.com/questions/23418818/read-uint32-t-values-from-file">Read uint32_t values from file</a></li>
<li><a href="https://stackoverflow.com/questions/23831422/what-endianness-does-python-use-to-write-into-files">What endianness does Python use to write into files?</a></li>
</ul>

<p>Update:</p>

<p>readbinary.cpp
It is from <a href="https://stackoverflow.com/questions/15138353/reading-the-binary-file-into-the-vector-of-unsigned-chars">reading the binary file into the vector of unsigned chars</a></p>

<pre><code>#include &lt;iostream&gt;
#include &lt;vector&gt;
#include &lt;string&gt;
#include &lt;fstream&gt;
#include &lt;byteswap.h&gt;
#include &lt;iterator&gt;
using namespace std;

typedef unsigned char BYTE;
std::vector&lt;BYTE&gt; readFile(const char* filename)
{
    // open the file:
    std::ifstream file(filename, std::ios::binary);

    // Stop eating new lines in binary mode!!!
    file.unsetf(std::ios::skipws);

    // get its size:
    std::streampos fileSize;

    file.seekg(0, std::ios::end);
    fileSize = file.tellg();
    file.seekg(0, std::ios::beg);

    // reserve capacity
    std::vector&lt;BYTE&gt; vec;
    vec.reserve(fileSize);

    // read the data:
    vec.insert(vec.begin(),
               std::istream_iterator&lt;BYTE&gt;(file),
               std::istream_iterator&lt;BYTE&gt;());

    return vec;
}


int main(void){
    std::vector&lt;BYTE&gt;  arr = readFile("test.bin");
    int i;
    for (i=0;i&lt;arr.size();i++){
        cout &lt;&lt; int(arr[i]) &lt;&lt; ' ';
    }
} 
</code></pre>

<p>output of readbinary.cpp (it has 9,10,11,12).</p>

<pre><code>0 0 1 0 1 0 1 0 2 0 1 0 3 0 1 0 4 0 1 0 5 0 1 0 6 0 1 0 7 0 1 0 8 0 1 0 9 0 1 0 10 0 1 0 11 0 1 0 12 0 1 0 13 0 1 0 14 0 1 0 15 0 1 0 16 0 1 0 17 0 1 0 18 0 1 0 19 0 1 0
</code></pre>