besides , the main setback is the little fact the big guys are all worried that if we leave , there might not be enough fighters left behind to protect everyone , if any infected suddenly came this way .