Minecraft wiki entry for Best_biomes_for_homes, associated text: There is little water which causes difficulty in many water-related activities, especially farming. However, villages and desert wells can be good sources of desert water.
The lack of rain also means you can't boost the time of fishing, and you can't hydrate crops if the desert is lacking any water sources.
No grass.
No reliable way to naturally get a lot of wood.
Sand can suffocate you if it falls on you while mining near the surface.
Like the plains, you can get shot down by skeletons due to the lack of foliage shielding its shots.
Husks, a variant of zombies, makes up 80%‌[Java Edition only]/70%‌[BE only] of zombies found there. They give the Hunger effect when hit. Also, husks will not burn in daylight, and can, therefore, be dangerous for players who have trouble during mob fights, or to villagers in desert villages.