Reading file from SFTP server using Node.js and SSH2
<p>I have a pretty strange problem working with read streams in Node.js. I'm using SSH2 to create a sftp connection between me and a sftp server. I then try to create a read stream from the sftp stream. From the read stream's emitted 'data' event I append the data to an array. When the read stream's 'close' event occurs I make a call to Buffer.concat to create concat all the chunks of data I have retrieved into one buffer. This is the same technique described in other questions asked here at stack overflow. For example <a href="https://stackoverflow.com/questions/14269233/node-js-how-to-read-a-stream-into-a-buffer">here</a>. However, I'm not able to use the data I retrieve. It seems like the buffer is 32 bytes less in size then the file that I'm trying to retrieve (from counting the length of the data retrieved). Could this have something to do with my SFTP connection? Or how I create my read stream?</p>

<p>If it matters, the file is of type zip. When I'm trying to unzip the file (in node.js and manually) after reading it to buffer it doesn't work. </p>

<p>After investigating I have found out that: </p>

<ol>
<li>When I use readdir on the file, the size of the file is correct. </li>
<li>Using FTP (JSFTP) against my development FTP server works just fine using the same technique above. </li>
</ol>

<p>Any advise is appreciated! </p>

<p>Here is my code: </p>

<pre><code>        var Client = require('ssh2').Client;
        var m_ssh2Credentials = {
           host: config.ftpHostName,
           port: config.ftpPort,
           username: config.ftpUser,
           password: config.ftpPassword,
           readyTimeout: 20000,
           algorithms: { cipher: ["3des-cbc", "aes256-cbc", "aes192-cbc","aes128-cbc"]}
        };
        ...
        var conn = new Client();
        var dataLength = 0;
        conn.on('ready', function() {
            conn.sftp(function(err, sftp) {
                if (err) {
                    writeToErrorLog("downloadFile(): Failed to open SFTP connection.");
                } else {
                    writeToLog("downloadFile(): Opened SFTP connection.");
                }

                var streamErr = "";
                var dataLength = 0;
                var stream = sftp.createReadStream(config.ftpPath + "/" + m_fileName)
                stream.on('data', function(d){
                    data.push(d);
                    dataLength += d.length;
                });
                .on('error', function(e){
                    streamErr = e;
                })
                .on('close', function(){
                    if(streamErr) {
                        writeToErrorLog("downloadFile(): Error retrieving the file: " + streamErr);
                    } else {
                        writeToLog("downloadFile(): No error using read stream.");
                        m_fileBuffer = Buffer.concat(data, dataLength);
                        writeToLog("Data length: " + dataLength);

                        writeToLog("downloadFile(): File saved to buffer.");
                    }
                    conn.end();
                });
            })
        })
        .on('error', function(err) {
            writeToErrorLog("downloadFile(): Error connecting: " + err);
        }).connect(m_ssh2Credentials);
</code></pre>