Gauge invariance means that certain transformations of the gauge field do not change the energy at all. If an arbitrary gradient is added to A, the energy of the field is exactly the same. This makes it difficult to add a mass term, because a mass term tends to push the field toward the value zero. But the zero value of the vector potential is not a gauge invariant idea. What is zero in one gauge is nonzero in another.