It was the growth and military assertion of the newcomers that drove the  hairy people  (as they were labeled) back north to their present concentration in Hokkaido.
The hairy people had to be pushed back by the military power of the newcomers.