You can get the [x y z] vector of each as follows:
r = 6371000 + alt
x = r*cos(lat)*cos(lon)
y = r*cos(lat)*sin(lon)
z = r*sin(lat)
Then subtract the vectors from each other. Make sure your lat and lon are expressed in radians.
Note that I have assumed a spherical model of the earth, which will yield very good answers in proportional terms — less than 0.3% worst case, based on a scratch-of-the-head "calculation", and much, much less for points that are within a few degrees of each other. If you need an answer that's good down to the meter regardless of the distance, you will need to use an ellipsoidal model, such as WGS 84 (thanks to @whuber for pointing this out). In that case, you can replace 6371000 with the equatorial radius, 6378137, and multiply z by (1 – 1/298.257223563).