Scaling Political Texts with ChatGPT
Gaël Le Mens Aina Gallego
We use GPT-4 to obtain position estimates of political texts in continuous
spaces. We develop and validate a new approach by positioning British party
manifestos on the economic, social, and immigration policy dimensions and
tweets by members of the US Congress on the left-right ideological spectrum.
For the party manifestos, the correlation between the positions produced by
GPT-4 and experts is 93% or higher, a performance similar to or better than
that obtained with crowdsourced position estimates. For individual tweets, the
positions obtained with GPT-4 achieve a correlation of 91% with crowdsourced
position estimates. For senators of the 117th US Congress, the positions
obtained with GPT-4 achieve a correlation of 97% with estimates based on roll
call votes and of 96% with those based on campaign funding. Correlations are
also substantial within party, indicating that position estimates produced with
GPT-4 capture within-party differences between senators. Overall, using GPT-4
for ideological scaling is fast, cost-efficient, and reliable. This approach
provides a viable alternative to scaling by both expert raters and
crowdsourcing.