Big City Bias: Evaluating the Impact of Metropolitan Size on
  Computational Job Market Abilities of Language Models
Charlie Campanella Rob van der Goot
Large language models (LLMs) have emerged as a useful technology for job
matching, for both candidates and employers. Job matching is often based on a
particular geographic location, such as a city or region. However, LLMs have
known biases, commonly derived from their training data. In this work, we aim
to quantify the metropolitan size bias encoded within large language models,
evaluating zero-shot salary, employer presence, and commute duration
predictions in 384 of the United States' metropolitan regions. Across all
benchmarks, we observe negative correlations between the metropolitan size and
the performance of the LLMS, indicating that smaller regions are indeed
underrepresented. More concretely, the smallest 10 metropolitan regions show
upwards of 300% worse benchmark performance than the largest 10.