C++ / C# Communication using Named Pipes - reads block and don't return
<p>Working on a service that needs to do Named Pipe communication between a C# and a C++ application.  Eventually the C# will run as a service in session 0, and the C++ as a GUI in the user session.  I have created a Named Pipe server class for the C# application, and a Named Pipe client class for the C++ application, along with the necessary GUI to run tests.</p>

<p>From what I can see, I am able to connect correctly.  On connecting, the C++ client communication thread writes an initial "Connecting" message, and then sits waiting for the "Connected" response.  When debug into a run of the two applications, I get passed the named pipe connections in both.  The client sends its connection message and the return value is true.  Then the client sits waiting to read a response.  The server meanwhile went straight to waiting to read an initial message.  Both of the reads block and never return.</p>

<p>As far as I can see, the connection is good.  Both the C# and C++ applications have defined their pipe connections as READ/WRITE (inOut in C#), and both have the connection defined to use the Message mode.  The only real difference I can see is that I have defined the connection on the C# side as allowing 1 server instance, while on the C++ side I have not yet been able to find out how this definition is to be done.  I can't seem to find the answer, and I've looked online and don't see anyone stating this as an issue for them.</p>

<p>Running the C# client and server as separate apps connected and communicated perfectly, and running the C++ client and server as separate apps did too.  But the problem has come up when connecting the C# and C++ applications.</p>

<p>The code is as follows.  Sorry, but got it as small as I could without losing the intricacies that might actually be part of the problem.  I cut this down from a test project that has a server and client on both C# and C++ sides.  Any help here is appreciated.  If anyone wants a zip of the project, please let me know how I can get it to you:</p>

<p>C# code:</p>

<pre><code>using System;
using System.IO;
using System.IO.Pipes;
using System.Threading;

namespace NamedPipeCSharp
{
    class NamedPipeServer
    {
        /*******************************************
         * Member constants
         *******************************************/
        const int numConnects = 2;
        const int ConnSourceBE = 0;
        const int ConnSourceGUI = 1;

        /*******************************************
         * Public member variables
         *******************************************/
        private static bool m_shouldContinue = true;
        private Thread m_serverThread;

        public static void ServerThread(object data)
        {
            NamedPipeServerStream beServer = new NamedPipeServerStream("MyNamedPipe",
                PipeDirection.InOut, 1, PipeTransmissionMode.Message);
            beServer.WaitForConnection();
            StreamReader reader = new StreamReader(beServer);
            StreamWriter writer = new StreamWriter(beServer);
            String startProtocolMsg = "Connecting";
            String startProtocolRsp = "Connected";
            while (m_shouldContinue)
            {
                // block on read input
                String line = reader.ReadLine();
                if (line == startProtocolMsg)
                {
                    writer.WriteLine(startProtocolRsp);
                    writer.Flush();
                }
                Thread.Sleep(10);
            }
        }

        public void StartCommunication()
        {
            // Start a thread to send IPC communications as Backend and GUI
            m_serverThread = new Thread(NamedPipeServer.ServerThread);
            m_serverThread.IsBackground = true;
            m_serverThread.Start();
        }

        public void StopCommunication()
        {
            m_shouldContinue = false;

            if (m_serverThread != null)
            {
                m_serverThread.Join();
                m_serverThread = null;
            }
        }
    }
}
</code></pre>

<p>C++ NamedPipeClient class header:</p>

<pre><code>#ifndef NAMEDPIPESERVER_H
#define NAMEDPIPESERVER_H

#include &lt;QtWidgets/QMainWindow&gt;
#include &lt;Windows.h&gt;
#include &lt;queue&gt;

class NamedPipeClient
{
public:
    NamedPipeClient();
    ~NamedPipeClient();

    void startCommunication();
    void stopCommunication();
    bool shouldContinue();

    // message functions
    bool hasMessageIn();
    bool hasMessageOut();
    std::string nextMessageIn();
    std::string nextMessageOut();

private:
    // Thread function
    static DWORD WINAPI clientThread(LPVOID param);

    // Thread management
    HANDLE createThread(DWORD WINAPI threadFunc(LPVOID), void* objPtr);

private:
    HANDLE m_clientThreadHandle;
    std::queue&lt;std::string&gt; m_inMessageQueue;
    std::queue&lt;std::string&gt; m_outMessageQueue;
    unsigned long m_serverId;
    bool m_shouldContinue;
};

#endif // NAMEDPIPESERVER_H
</code></pre>

<p>C++ NamedPipeClient class definition:</p>

<pre><code>#include "NamedPipeClient.h"
#include "NamedPipeStream.h"

NamedPipeClient::NamedPipeClient() :
    m_clientThreadHandle(0),
    m_shouldContinue(true)
{
}

NamedPipeClient::~NamedPipeClient()
{
}

void NamedPipeClient::startCommunication()
{
    // Start a thread to send IPC communications from Backend
    m_clientThreadHandle = createThread(clientThread, this);
    if (m_clientThreadHandle &amp;&amp; m_clientThreadHandle != INVALID_HANDLE_VALUE)
    {
        ResumeThread(m_clientThreadHandle);
    }
}

void NamedPipeClient::stopCommunication()
{
    // set this particular named pipe server's should continue to false
    m_shouldContinue = false;

    if (m_clientThreadHandle != INVALID_HANDLE_VALUE)
    {
        WaitForSingleObject(m_clientThreadHandle, INFINITE);
        m_clientThreadHandle = 0;
    }
}

/**********************************
 *  Thread function definitions
 **********************************/

DWORD WINAPI NamedPipeClient::clientThread(LPVOID param)
{
    NamedPipeClientStream* beClient = new NamedPipeClientStream("MyNamedPipe", OPEN_EXISTING);
    if (beClient-&gt;validPipe())
    {
        NamedPipeClient* thisPtr = static_cast&lt;NamedPipeClient*&gt;((void*)param);
        // start by sending initial message to get things rolling
        std::string startProtocolMsg = "Connecting";
        std::string startProtocolRsp = "Connected";
        std::string message;
        std::string response;
        char buff[BUFSIZE];
        int iterNum = 0;
        int numTries = 0;

        // initial connection protocol
        message = startProtocolMsg;
        bool ok = beClient-&gt;writeToPipe(message);
        // wait for correct response
        response = beClient-&gt;readFromPipe();
        if (response != startProtocolRsp)
            return -1;

        while (thisPtr-&gt;shouldContinue())
        {
            if (thisPtr-&gt;hasMessageIn())
            {
                //send the next communication
                message = thisPtr-&gt;nextMessageIn();
                while (!beClient-&gt;writeToPipe(message))
                {
                    // try sending this message up to 10 times
                    numTries++;
                    if (numTries == 10)
                    {
                        break;
                    }
                    Sleep(2);
                    continue;
                }
                numTries = 0;

                // then sit and wait for an answer
                response = beClient-&gt;readFromPipe();
                if (response.length() == 0)
                {
                    Sleep(1);
                }
            }
            Sleep(10);
        }
        return 0;
    }
    else
        return -1;
}

HANDLE NamedPipeClient::createThread(DWORD WINAPI threadFunc(LPVOID), void* objPtr)
{
    // create a thread suspended and return the handle
    HANDLE hThread = CreateThread(
                NULL, // default security
                0, // default stack size
                (LPTHREAD_START_ROUTINE)threadFunc, // Thread
                (LPVOID) objPtr, // arguments
                CREATE_SUSPENDED, //0, // default creation flags
                NULL);     
    return hThread;
}

bool NamedPipeClient::shouldContinue()
{
    return m_shouldContinue;
}

bool NamedPipeClient::hasMessageIn()
{
    return (m_inMessageQueue.size() &gt; 0);
}

bool NamedPipeClient::hasMessageOut()
{
    return (m_outMessageQueue.size() &gt; 0);
}

std::string NamedPipeClient::nextMessageIn()
{
    std::string msg = m_inMessageQueue.front();
    m_inMessageQueue.pop();
    return msg;
}

std::string NamedPipeClient::nextMessageOut()
{
    std::string msg = m_outMessageQueue.front();
    m_outMessageQueue.pop();
    return msg;
}
</code></pre>

<p>C++ NamedPipeStream class header:</p>

<pre><code>#ifndef NAMEDPIPESTREAM_H
#define NAMEDPIPESTREAM_H

#include &lt;Windows.h&gt;
#include &lt;string&gt;

#define BUFSIZE 512

class NamedPipeStream
{
public:
    NamedPipeStream(std::string pipeName, int createMode);
    ~NamedPipeStream();

    virtual bool writeToPipe(std::string message);
    virtual std::string readFromPipe();
    virtual bool closePipe() = 0;
    bool validPipe();

protected:
    void createPipe(std::string pipeName);
    virtual HANDLE openPipe(wchar_t* pipeName) = 0;

protected:
    std::string m_pipeName;
    HANDLE m_pipeHandle; 
    int m_createMode;
    bool m_closePipe;
    bool m_isValid;
};

class NamedPipeClientStream : public NamedPipeStream
{
public:
    NamedPipeClientStream(std::string pipeName, int createMode = CREATE_NEW);
    ~NamedPipeClientStream();

    bool closePipe();

private:
    HANDLE openPipe(wchar_t* pipeName);
};
</code></pre>

<p>C++ NamedPipeStream class definition:</p>

<pre><code>#include "NamedPipeStream.h"
#include &lt;stdlib.h&gt;

/**********************************************
 *
 *  NamedPipeStream class
 *
 *  Definition: Abstract base class for named pipes
 *
 **********************************************/

NamedPipeStream::NamedPipeStream(std::string pipeName, int createMode) :
    m_pipeName(pipeName),
    m_createMode(createMode),
    m_closePipe(false),
    m_isValid(false)
{
}

NamedPipeStream::~NamedPipeStream()
{
}

bool NamedPipeStream::writeToPipe(std::string message)
{
    DWORD numToWrite, numWritten; 
    BOOL fSuccess = FALSE; 
    wchar_t* lpvMessage;

    // Send a message to the pipe server. 
    std::wstring wsMessage(message.begin(), message.end());
    lpvMessage = const_cast&lt;wchar_t*&gt;(wsMessage.c_str());

    numToWrite = (lstrlen(lpvMessage)+1)*sizeof(TCHAR);

    fSuccess = WriteFile( 
        m_pipeHandle,     // pipe handle 
        lpvMessage,       // message 
        numToWrite,       // message length 
        &amp;numWritten,      // bytes written 
        NULL);            // not overlapped 

    if (!fSuccess) 
    {
        return false;
    }

    return true;
}

std::string NamedPipeStream::readFromPipe()
{
    DWORD cbRead;
    TCHAR chBuf[BUFSIZE];
    BOOL fSuccess = FALSE;

    // Read from the pipe. 
    fSuccess = ReadFile( 
        m_pipeHandle,           // pipe handle 
        chBuf,                  // buffer to receive reply 
        BUFSIZE*sizeof(TCHAR),  // size of buffer 
        &amp;cbRead,                // number of bytes read 
        NULL);                  // not overlapped 

    if (!fSuccess &amp;&amp; GetLastError() != ERROR_MORE_DATA)
    {
        // return empty string on error
        return std::string(); 
    }

    std::wstring instr(chBuf);
    return std::string(instr.begin(), instr.end());
}

bool NamedPipeStream::validPipe()
{
    return m_isValid;
}

void NamedPipeStream::createPipe(std::string pipeName)
{
    BOOL   fSuccess = FALSE; 
    DWORD  cbRead, cbToWrite, cbWritten, dwMode; 
    char* szPipenameFormat = "\\\\.\\pipe\\";
    std::string pipeNameStr;
    size_t formatLen = 0;
    wchar_t* lpszPipenameFormat = NULL;
    char* szPipename = NULL;

    // convert std::string name to wide char pointer and append to format string
    pipeNameStr = szPipenameFormat;
    pipeNameStr.append(pipeName);
    lpszPipenameFormat = new wchar_t[pipeNameStr.length()+1];
    szPipename = const_cast&lt;char*&gt;(pipeNameStr.c_str());
    mbstowcs_s(&amp;formatLen, lpszPipenameFormat, pipeNameStr.length()+1, 
        szPipename, pipeNameStr.length());
    lpszPipenameFormat[pipeNameStr.length()+1] = NULL;

    // Try to open a named pipe; wait for it, if necessary. 
    // loop until get a valid named pipe
    while (1) 
    { 
        m_pipeHandle = openPipe(lpszPipenameFormat);

        // Break if the pipe handle is valid. 
        if (m_pipeHandle != INVALID_HANDLE_VALUE) 
        {
            DWORD err = GetLastError();
            break; 
        }

        // Exit if an error other than ERROR_PIPE_BUSY occurs. 
        if (GetLastError() != ERROR_PIPE_BUSY) 
        {
            DWORD err = GetLastError();
            m_isValid = false;
            return;
         }

        // All pipe instances are busy, so wait for 20 seconds. 
        if (!WaitNamedPipe(lpszPipenameFormat, 20000)) 
        { 
            m_isValid = false;
            return;
        } 
    } 

    // The pipe connected; change to message-read mode. 
    dwMode = PIPE_READMODE_MESSAGE; 
    fSuccess = SetNamedPipeHandleState( 
        m_pipeHandle,    // pipe handle 
        &amp;dwMode,  // new pipe mode 
        NULL,     // don't set maximum bytes 
        NULL);    // don't set maximum time 
    if (!fSuccess) 
    {
        m_isValid = false;
        return;
    }

    m_isValid = true;
}

/****************************************************
 *
 *  NamedPipeClientStream class
 *
 *  Definition: Class for creating server named pipe
 *              to open client connection
 *
 ****************************************************/

NamedPipeClientStream::NamedPipeClientStream(std::string pipeName, int createMode)
    : NamedPipeStream(pipeName, createMode)
{
    createPipe(pipeName);
}

NamedPipeClientStream::~NamedPipeClientStream()
{
}

HANDLE NamedPipeClientStream::openPipe(wchar_t* lpPipeName)
{
    HANDLE hPipe = CreateFile( 
        lpPipeName,     // pipe name 
        GENERIC_READ |  // read and write access 
        GENERIC_WRITE, 
        0,              // no sharing 
        NULL,           // default security attributes
        m_createMode,   // specifies whether create new or open existing
        0,              // default attributes 
        NULL);          // no template file 
    return hPipe;
}

bool NamedPipeClientStream::closePipe()
{
    m_closePipe = true;
    return (CloseHandle(m_pipeHandle) == TRUE) ? true : false;
}
</code></pre>