Take the 2-minute tour ×  Is it possible to find the distance of
a pixel to a rendered sphere, in screen space? All my naive
solutions for just using the 2D screen distances are failing
because of the warping that occurs during projection.
share|improve this question See maybe Closest point on an
ellipsoid. But I'm not sure that's really what you are looking
for. Could you maybe explain what is the purpose of knowing that
distance? –  Sam Hocevar May 26 '12 at 14:24 add comment  2
Answers  up vote 2 down vote accepted  It's definitely possible,
though it doesn't have a clean explicit formula. The projection
of a sphere is an ellipse, and given the projection matrix you
should be able to find an explicit formula for the ellipse,
something of the form ax^2+bxy+cy^2+dx+ey+f=0 (the canonical
formula for a conic section); from this you can find a rational
parametrization of the ellipse (of the form x=f(t), y=g(t) with
f and g rational functions). The problem then comes down to
minimizing (f(t)-x_0)^2+(g(t)-y_0)^2