The meaning of the potential energy (stored work) is that movement of a test body from a larger radius to a smaller radius involves doing work against the centrifugal force and thus gaining potential energy. But this test body at the smaller radius where its elevation is lower has now lost equivalent gravitational potential energy.