and they fled before the wrath of isildur , and did not dare to go forth to war on sauron 's part ; and they hid themselves in secret places in the mountains and had no dealings with other men , but slowly dwindled in the barren hills .