On the Potential of CLIP for Compositional Logical Reasoning
Justin Brody
In this paper we explore the possibility of using OpenAI's CLIP to perform
logically coherent grounded visual reasoning. To that end, we formalize our
terms and give a geometric analysis of how embeddings in CLIP's latent space
would need to be configured in order for the system to be logically coherent.
Our main conclusion is that, as usually configured, CLIP cannot perform such
reasoning.