Your Linux Data Center Experts

Earlier this week we had a client with a machine that was getting a distributed denial of service attack on their e-mail server. Over 4,000 hosts around the Internet were making connections and trying to send e-mail to a particularly formatted series of e-mail addresses, which were getting rejected at “RCPT TO” time. While mail wasn't flowing through, all their concurrent connections were getting blocked up by this attack. Not sure if it was a spam attack, dictionary attack, or “just” a DDoS… During that, however, Scott and I had a discussion about null-routing versus iptables.

Because there were over 4,000 host addresses involved, I decided to block them by null-routing. By this, I mean “ip route add $REMOTEADDRESS via”. This could also have been done with “iptables -I INPUT -s $REMOTEADDRESS -j DROP”. However, there is a big difference…

While iptables can accept tens of thousands of rules in a chain, the chains are walked sequentially until a match is found on every packet. So, lots of rules can lead to the system spending amazing amounts of CPU time walking through the rules. There is a project called “nf-HiPAC”, which means to resolve this, but as far as I know it is not available in any standard kernels.

If you are talking about TCP traffic, which requires a bi-directional hand-shake, you can also use null-routing. I typically do this by running “ip route add $REMOTEADDRESS via”. In other words, when trying to send any packets to that host, send them via the loopback interface to the local machine. This will fail, and responses will not reach the sender. Note that “ip route add” is the “IP Route 2” way of saying “route add”.

The routing rules are much simpler than iptables. With iptables, a match can be based on many different variables including protocols, source and destination packets, and even other packets that were sent before the current packet.

In routing, all that matters is the remote IP address, so it's very easy to optimize. Also, many systems have a lot of routing rules. A typical system may only have 5 or 10, but something that's acting as a BGP router can have tens of thousands. So, for a very long time there have been extensive optimizations in selecting the right route for a particular packet.

The route cache can be used to even more quickly find the correct route. Once the “slow path” has been used to select a route, that route information is cached for future lookups. So, when using the “ip route add” above, you then want to do an “ip route flush cache” to get the new routing rules to be used.

However, this won't protect against most UDP attacks and it's also important to realize that packets the remote end sends do reach the system. It's only responses that won't get back. So, a TCP connection cannot be established, blocking things like the e-mail distributed denial of service attack. Someone pounding on your DNS server with many UDP packets will not be blocked, however.

comments powered by Disqus

Join our other satisfied clients. Contact us today.