I fancy he wants to give some of it back. It was possibly this remark which induced the driver to move away.
Perhaps the driver moved away because he wanted to give some of it back.
Perhaps the driver didn't move away because he wanted to give some of it back.