Synchronous RPCs and view changes


    When cluster-wide RPCs are invoke synchronously (ie., the caller is blocked until everyone has replied), there are issues with view changes.


    A simple example is cluster V2={A,B,C,D}.


    Let's say D crashes and - at the same time - A invokes a cluster RPC.


    If A received V3={A,B,C}, then it would wait for replies from itself, B and C. However, if A invoked the RPC first and only after returning received V3, then it would wait for D's reply forever.


    There are a few things that can be done to remedy this:

    • Bound the RPC with a timeout, say 5000. In this case, if the RPC is executed first and only then V3 is received, the RPC would return after 5000 with valid return values for A, B and C, and a null return value for D, which is marked as 'not-received'

    • Use asynchronous RPCs if you can. Sometimes, though, especially for data collection tasks, this won't do