Minecraft wiki entry for Calculator, associated text: In this machine, your inputs (in binary code) go into the bottom (yellow) full adders. Each adder needs the two inputs (A and B) with the same values. Also, the least significant bit has to be on the left, so they should all be connected by their carries. Basically, your inputs look like the wires in the #Input Wires (white and orange) section. Use basic bridges to pass wires over the others without connecting them.
Your A inputs (left) are the minuend (X in X-Y=Z), and go straight in the adders. The B inputs, your subtrahend (Y in X-Y=Z), have to pass through a multiplexer, made out of a modified version of a XOR gate which gives to the adder an inverted signal in case of a subtraction. The multiplexer is controlled by a switch (on the picture, that switch is on the left). The sums go into another multiplexer, which, again, gives an inverted input in case of a subtraction. This is controlled by an IMPLIES gate (in the top right) which gives a true output if the switch is on "Subtraction" AND if the last carry is true. This is required because on a subtraction, that last carry actually means the "-" (minus) sign.