This is NAT for newbies and not so newbies. It's primarily a tutorial. If you're looking for something else, you're in the wrong room. Before we get started, a couple of things I want to do rather quickly here. Something I've always sort of wanted to do with a crowd like this, a special sort of DDOS, Distributed Denial of Service thing. So everyone who has a cell phone, please take it out. We're going to try something here. Take your cell phone out, hold it up in one hand.  Hold it up. Hold that cell phone up. OK, step one. Just for a moment, turn the power off. Turn the power off. OK, step two, put it in your pocket and leave it there for an hour. That's the world's first do-it-yourself distributed self-denial of service. Second issue, some of you have been and know that there's this spot the Fed contest.  It's really hot out there. Works so well. I know I got a monogram shirt on, but I got white socks and red Converse All-Star high tops on, so I couldn't be a Fed. So just thought I'd straighten that out. Hope this continues to work. Bad batteries? So how about this one? Let's see it. Who's got that phone? All right, we're going to talk about NAT, which is network address translation. I've been working with firewalls for a few years, primarily checkpoint firewall one. But network address translation is valuable with almost any firewall or almost any setup. I chose this topic because, in my opinion, it's sort of ignored. I think NAT is really useful. It's essentially free. It does all sorts of cool stuff. It gives you all sorts of protections. It's a really valuable thing, in my opinion. And I, looking at the firewall literature, I thought it wasn't being covered very well. So I'm sort of an amateur at this, but I think NAT speaks well for itself here. And so this is going to be two 50-minute sessions. We're going to go until 3.50 PM, take a 10-minute break, come back at 4. I'm eager for questions. This is a tutorial. The whole point here is that you learn and that you enjoy yourselves. So please, don't even do that raise your hand thing. I grew up in New Jersey. It's OK to just shout out, yo, at the top of your lungs. That's not an insult. I'll be glad to stop and answer questions. I'm going to run through a quick introduction. I don't know exactly when the break is going to occur here. We're going to talk about some basics of network address translation. I'm going to show you a typical NAT configuration.  We're going to discuss NAT in action. I'm going to go over some of the benefits. There are some special firewall configurations that I'm going to discuss. Those of you who have done network address translation know that there are some special DNS issues. If a single TCP IP host has multiple IP addresses, it's an interesting question, which IP address should your DNS server return? We'll talk about that. Quick summary, lots of Q&A. But please, don't save your questions till the end. Let's hear them as we go along. Some of you, this talk is on the newbie track. So some of you may be wondering, am I well enough qualified for this? Is this over my head? Is this basic stuff that's going to bore me? If you know some of these basic concepts, you're going to be plenty well qualified for it.  If you've ever done any of these, you're more than qualified to benefit from this tutorial. How many people here have configured a TCP IP host, router, or firewall? Knowingly configured one? Thank you. Connected some LAN users to the internet? Debugged an internet connectivity problem? OK, be careful now. Brought down Yahoo with a DDOS attack. OK, just a few of you. Here is one possible definition of NAT. I couldn't find anything official or formal on the internet anywhere, so I'm sort of making this up. The assigning of two different IP addresses to the same TCP IP host, a public one to be used when accessing the host from inside the security perimeter, and a private one to be used when accessing the host from outside the security perimeter. So as you all probably know, the basic concept of NAT is that you establish a security perimeter around your organization and use different IP addresses on the inside and on the outside. You need a network address translation box, a NAT box, sitting on the perimeter in order to do the intelligent translations back and forth. But this is the basic concept. If you have a better definition, I'd like to hear it. But I think this works pretty well. Some basics, we're going to talk about RFC 1918, the concept of private IP addresses and two general modes of operation for NAT. You've all probably heard of by now the Internet Engineering Task Force. They archive these things called RFCs. They are requests for comments. This is the, I guess, democratic unstructured way that the internet has developed. Somebody comes up with a bright idea and posts publicly a request for comment. Basically what they're saying is, I think this is a good idea. I think this is the way the world should do things. I assert this is the best we've got, and I'm posting this for comments. And only after it has been subjected to perhaps withering public scrutiny does it does do people sort of generally agree that, yeah, this works. And so most of the internet, the specifications, have been designated and cleared up and archived in these RFCs. The famous one for us in terms of network address translation is 1918. Anybody who knows NAT, anybody who can tell you the story about those funny IP addresses, 192.168. How many people here have seen some addresses, 192.168? And you know there's something funny about them, but you're not quite sure what it is. Hands, thank you. So in 1918, address allocation for private internet discusses some of this, but not all of it. You can find them all at ietf.org. There's all sorts of great stuff. On a boring rainy afternoon, it's a terrific way to learn some more things about the internet. OK, so let's talk about private IP addresses. On the bottom row there, you see the 192.168. Now, early in your careers when messing with TCP IP and routing issues, you quickly figured out that there was something different and strange and special about those IP addresses. If you were paying attention, you figured out that they couldn't route out to the internet. These are three different classes of IP address blocks that have been reserved and are private. One of them is a class A block, the 10s. So if you ever see any IP addresses starting with 10, you know that these are private. They're non-routable on the internet. We'll talk about some of these special characteristics here in a minute. This is particularly nice because you've got 16 million IP addresses. And you can use this no matter how big your organization is. You can use the 10 block to help get things structured internally. The class B blocks, which are the toughest trivia question here, ask somebody, what are the class B private address blocks? 172.16 through 172.31. There are 16 class B blocks available, each of which has 64,000 IP addresses. And there are 256 class C blocks, each of which has 256 IP addresses. So you should know, if you ever see any host with these IP addresses, that there's something special going on. These are private IP addresses. They are non-routable on the internet. And we'll talk about some of the special characteristics here. These private addresses are reserved in that they'll never be assigned to someone in the global internet routing scheme. There is no host out there at 192.168 on the internet somewhere. None. There are none. In fact, every router on the internet is required to drop all packets addressed to any IP address in these blocks. So you don't have to worry about it. So if you're on your corporate LAN and you know there's a web server that you're using at <IP_ADDRESS>, what does that say to you? Well, the immediate conclusion you can come to is that, well, that must be internal to my corporation somewhere, because you know for a fact it can't be over the internet, because you can't get to it. So it's nice that you never have to worry about a collision.  What if there is a 10.1.1 out there? I'm never going to be able to reach them now, I'm going to be routing back to myself.  Packets with a private address destination are dropped by internet routers. And we're going to talk about one of the benefits of this in the future. If one of your internal machines that you don't want people on the internet to have access to has one of these private IP addresses, it's a lot harder for someone on the internet to connect to it, because any packets that a hacker sends to 192.168, as soon as they get out on the internet, they're going to be dropped, right? This is even better than an access control list. This is, I don't have an address you can get to. Anyone can use them internally as they'll never collide with external addresses. OK, we've talked about RFC 1918 and some private addresses. If you've been playing with network address translation, you've probably seen that in most firewalls, in particular here, the checkpoints firewall one, if you notice here, I'm sort of using some other terminology. Well, I am. There are two modes, generally speaking, for network address translation. Hide mode, otherwise known as many to one, or static mode, which is one to one. And this is largely because when it comes to granting internet access in your organization, you can pretty much partition all of your TCP IP hosts into two groups, those of which should have some access by people on the internet, your mail server, your web server, your FTP server, and those that you never want anyone on the internet to have access to. Your payroll server, the accounting server, your workstation, say. I'm speaking only of connections coming from the internet in, not connections going from the inside out. So if you're like most organizations, you can partition all of your TCP IP hosts, all but five of them. You want to be internal and protected. For the purposes of this discussion, I'm going to call that the LAN, the local area network.  And you're going to want those to be allowed at least some access from people on the internet. And I'm going to call that the DMZ, demilitarized zone, which is sort of not very descriptive. But it's a special zone, perhaps it's even a special nick off of your firewall and a protected subnet in which you're allowing some access by people on the internet. Very filtered, very specific, very carefully monitored access. But you are allowing some access. But the machines on the inside, like the woman who does payroll, her laptop, her desktop machine, you don't want people on the outside to be able to connect to that under any circumstances. So those go on the LAN, and they'll be in something called hide mode. Static motor one to one is what you typically do for servers, machines that you wish to make available to the public. So the moral of the story is if you're going to set up NAT in your organization, one of the first things you need to do is basically think about all your TCP IP hosts and partition them into two groups, mutually exclusive, collectively exhaustive, and figure out which ones go on the DMZ and which ones go on the LAN. And that'll help you figure out which mode to use. OK, so we've talked about the first section here. Let's have some questions. I've talked about RFC 1918. I've talked about private IP addresses. I've talked about two modes of network address translation. Questions, things I've screwed up on. Yes? The question is, how would NAT work in a VPN situation? The short answer is with great difficulty. It's a real problem. I believe IPsec embeds the IP address of the host, and that goes out. It's some problems. I don't know a whole lot more than that that I can share with you at this point, but yes, there's some real difficulties if you're trying to use a VPN with NAT. Because NAT, the host and the server, or the two ends of the connection using NAT, basically don't need to, and perhaps cannot, be aware of the fact that network address translation was used. We'll show here. I'm going to go through some descriptions here and walk through with packets. And I'm going to show that both, for example, if you're browsing the web with a client using network address translation, neither side is even aware that NAT is going on. So for most protocols that operate with good manners and don't embed IP addresses in the payload, then NAT works just fine. But one of the security aspects of some VPN protocols is that in order to prevent various attacks, et cetera, they embed the IP address in the payload, and that can cause some problems. The best way to solve this problem is to have the VPN and your firewall working together. For example, Checkpoint Firewall 1, the example I know, I'm not trying to plug them. They do that well. You can mix the VPN with network address translation. Other questions? OK, this is a typical NAT configuration. This is a highly stylized, simple network. This is your new dot com company. Two years ago, they've been in business for two weeks. The Big Eye internet. We have a gateway router. It's your Cisco 1700 or your Cisco 3600, whatever it is. You're running a firewall there with NAT. And you'll see that there are two separate NICs coming out of that firewall. I'm really trying to separate those two LAN segments to the greatest degree possible. On the lower right, we have the DMZ. And you'll see we've got a worldwide web server. We have a mail server and an FTP server. And those machines are in the DMZ because we do want people on the internet to have some limited access to those machines. On the lower left-hand side is the LAN. And there's the server there and various users. And again, the purpose is we don't want anyone on the internet to have any access to the people in the LAN. So we've already done this partitioning. We've already got it split out here into two separate subnets with two separate policies. The one on the lower right is using static mode NAT. The one on the lower left is using hide mode NAT. Question. The question is, what if your router does your NAT for you? Can you still split it up into two different modes? OK, the question is particularly about low-end Cisco routers. I don't know. If there's a single NIC coming out on the bottom of the router, then it makes it a lot harder, obviously, to split it into two NICs. Put a switch underneath it and do the best you can. The point of security is not to be perfect. It's just to put up as many sort of walls to climb over as you can. One of the purposes of splitting it into two different NICs is it's like those lizards in the deserts of Arizona who have those big tails. And if a predator grabs the tail, well, the tail breaks off and the lizard gets away. And the DMZ is sort of the lizard's tail here, because you are allowing some access from the internet. There is still some chance of somebody taking over one of your servers. And so you set up your routing rules and the firewall rules in such a way that even if somebody takes over your web server, they still can't get into the LAN. This is the whole reason why you use two separate subnets and have two separate policies here.  But if it does, well, OK, you can flatten that hard drive and start over. But at no time did they have enhanced access to get back into the LAN and get into your payroll server, which is the real concern. So that's the motivation behind having these two different LAN segments. The question is that some routers will do specific port forwarding, limiting packets tested for certain ports only to certain IP addresses. That sounds like an access control list or basic firewall configuration issues. NAT in action. All right, we're going to, I wanted to have a little icon here of a guy named NAT.   So we're going to look here. There are four possibilities. We're going to look here at hide mode and static mode. There's another cell phone. And we're going to look at outbound connections and inbound connections and see what happens. OK. The first thing we're going to do here is talk about what's inside an IP packet. I could spend four days talking about what's inside an IP packet. I'm just going to mention a couple of basic fundamental things that you need to know from network address translation. You can get a PhD in IP packetology, I'm sure, from somewhere. Two of the most important things are the IP source address. Every TCP IP packet that goes anywhere contains the source address of the machine where it came from. Or if you spoof it, it can contain any IP address you want. But let's start with normal expected IP packets, which have the IP address of the TCP IP host that sent the packet. Another thing it also contains is the destination. You wouldn't be able to route the packet over your network or over the internet if it didn't have an address where it wants to go. So this is like a letter you're throwing in the mail or I guess in the example of the internet, the canonical example, it's a postcard you're throwing in the mail. And this postcard has a to address and it has a from address. You already know that. This is simple stuff. Something else that it has is a port number. Those of you who know the OSI model, this is layer four, this is the transport layer. For example, these are port numbers you've heard about. If you're talking to a web server, it's port 80 HTTP. If you're talking to an FTP server, it's port 21. Outbound email is SMTP, port 25, et cetera. You've heard these port numbers before. These get sent along with every packet. Again, this is a to address and a from address. The from address is actually sort of a little more mysterious. A lot of times when I'm teaching firewall classes, a lot of people are sort of uncertain or sort of surprised by this concept that every packet has a from port address. And it turns out it's actually fairly valuable. And it's one of the things that keeps firewalls, by which a firewall can keep its internal state tables straight. And we'll talk about that in a minute. Also inside an IP packet, there's lots of other stuff. There's the data payload. And this is your oversized ping of death packet or whatever it is you're sending today. Error correction information, et cetera. There's an awful lot of other stuff corresponding with other layers and flags, et cetera, et cetera. But for purposes of the discussion of network address translation, the only things that are really important for us today are the to and from address for the to and from IP address and the to and from port address. So we can talk about translating some of those addresses. IP in general. OK. So now we're going to go to hide mode outbound, the first one. Step number one. OK, this is hide mode. You remember we have the LAN users who are sitting inside the LAN. They're protected by hide mode. They have a non-routable address. They have one of these private addresses. It's like 192.168. So we know already that if they send a request out to some web server on the internet using their own address, where's the web server going to reply to? If this packet goes out just by itself, some web server somewhere on the internet is going to get a request, an HTTP request, from somebody who's identifying themselves as being <IP_ADDRESS>. That web server can't reply, because we know that you can't respond to an address like that. You can't send packets over the internet to one of those non-routable addresses. So we know already that some translation is going to have to occur here before that packet leaves your security perimeter. So the packet leaves the client. And you can see here, I've just enumerated the four sort of different embedded addresses in that packet. The first one is the IP source address is the private address. This is the private address of your web client there at work. The IP destination, I'm just using www.website.com. It's just a placeholder. Your web client is going to pick a random high TCP port number, and the destination is going to be port 80, and we know that because you're trying to talk to a web server. As it leaves your firewall or your NAT box, this is where the network address translation comes in, your NAT box overwrites the source address with the public IP address. So you see there the IP source. We've overwritten it with a virtual public address. This is an address that you own. You've been assigned it by your ISP or by IANA, and you own it, and it routes on the internet. And the other three addresses you leave unchanged. One of the important things here is that the firewall has to store information about this connection in the state table. Because this packet, this connection, well, the firewall is actually sort of maintaining two different connections now, one between the internal client and the firewall, and another one between the firewall and this web server out on the internet. And you have to keep all that straight. Because when the answer comes back, the firewall has to decide whether to let this in or not. So the firewall keeps a state table, just a little bit of RAM, just keeps a simple little table that says, OK, this inside user at this IP address, he has an open connection going out to this web server. His real address is this. The fake external address we gave him is that. And the connection is open. He went out on this port number, just some basic information. So the firewall keeps the state information in order to keep the channel open for the return traffic. The remote public server, this is the website you're going to, now he sees a request from a valid internet host, <IP_ADDRESS> or whatever. That's a valid, routable address. So the web server sees this and says, hey, I'm a web server. My job is to serve up web pages. I've got a valid HTTP request here from a valid internet address. So he just responds. So he sends back. Now, in a sense, it's easy to see he's just flipping the to and from on both the IP address and the port address. So he flips the to here. So this packet that leaves the server has a source address from him, which makes perfect sense. He's sending it back to the virtual public address. The port is on 80. The port source is on 80. And the destination is the original high TCP port number. Now, this is sort of one of the mysteries that everyone's sort of surprised to see, is that when answers, when you replies, come back from public web servers, or from any web server, the to port number is the same random high TCP port number that your machine selected at the beginning. Now, since the firewall has kept that information in the table, the firewall is able to disambiguate these replies when they come back and decide who gets this packet. Well, the question is, if you have multiple web browsers in hide mode in the same LAN, and they all want to go out and browse the web simultaneously, and they collide on high TCP port numbers, how do you resolve that? Well, the answer is we're much more willing to take the chance they're going to collide on port numbers than on the chance that they're going to collide on outbound IP addresses, which we know they're going to collide, because they're all hiding behind the same IP address. I don't know how the firewall resolves that. It probably rejects the connection and says, try again. But there's 64,000 or 63,000 random high TCP port addresses, so your chances of colliding are actually very small. I would imagine if you were a firewall and the way you would keep those straight would be either to reject the second connection or maybe do something smart, like keep track of, well, this guy's going to a different IP address, so when the answer comes back, you match both the IP address and the port number, and that's how you keep it straight. So I imagine if you did both of those, what are the chances of two LAN hosts simultaneously going to Yahoo both with the same random high port number? Pretty small. Excuse me? Who would pick a different high port number? The firewall. All right, the question is, is it possible the firewall would just pick a second high port number going to the same destination? Right. There's something called port address translation. The firewall does have the ability to play further games with this, pick different port numbers, et cetera. The chances of this happening are very small, so it's not something we worry about a whole lot. I don't know the details of how firewall one in particular resolves that. But given the fact that there are 63,000 random high TCP port numbers, and most LANs are hiding only a few hundred people behind a single NAT configuration, it doesn't seem to happen very often. I'm sure there's a solution. I'm sure they've worked on it, but I haven't considered it very much. Does the MAC address? Well, MAC addresses are local to your ethernet segment. They don't cross routers. They don't even cross switches. So whatever MAC address you have on your local LAN client could easily be long gone by the time you get to your gateway router or your NAT box. That's layer two ethernet only doesn't have any part of this type of NAT. Question? All right, the comment is, if you're using firewall one in hide mode, you're always going to be using port address translation. OK. The comment is, there's no reason to be maintaining force the same port number. OK, fine. This is not something I'm an expert on these details. I know that the firewall handles them just fine, and I haven't torn into it. The response finally comes back to the client. The client receives the reply from the internet host. So he's happy. He sent out a request to an internet server, and he got his HTTP request back. So he's perfectly happy. Number nine, he has no idea NAT was used. He has no idea that the web server thought he was really talking to 204, 205, 199, 1. He has no idea. And the server has no idea also. The server, as far as he knows, he received a valid request. So one of the important issues here, if NAT is working properly, is that neither the client nor the server have any idea that NAT was used, and it doesn't matter. They don't need to know, except for some unusual circumstances such as the VPN. OK, that was outbound. Now we're going to talk about hide mode and inbound connections. And if you're thinking ahead, if you're reading ahead, you might be thinking, something's not right here. Why should there be inbound connections in hide mode? And it's true. One of the purposes of hide mode is that there can't be inbound connections. So let's say that web server you just connected to has an evil process on it, and it tries to open another connection back to your web client. Those of you who have firewalls now and watch the logs, you see this all the time. I'm running firewall one in my lab right now, and I'm seeing it all the time. I connect to some web server somewhere on the internet. And three seconds later, there's some high TCP port attempted connection right back to my web client. My firewall drops them all and logs them, but this happens all the time. So in this scenario, the packet comes in from this website. Its destination is the public IP address, the routable one that your NAT box was using. Whatever the port source is in the port destination, if it's an evil process, it can fake it all at once. But it's coming in on something that, well, clearly you're not allowing any port numbers to come through, because you're in hide mode on the LAN. The firewall sees the incoming packet and is unable to match it with a known connection described in the state table. So remember before, we talked about this state table, where the firewall keeps information about connections that are outbound. So when packets come back in, it can look up in the table and say, oh yeah, I know about you. I was expecting you. You're the answer to this request from someone in my LAN. However, if a new connection tries to come through, the firewall looks at this and says, what is this? You're trying to connect to somebody who's in hide mode. And the firewall says, my job is to hide those hosts. So the firewall says, well, this is a virtual IP address only. It's not a real IP address. We just used it temporarily for hide mode NAT. Also, here's an interesting question. Even if the firewall could forward that packet to some host inside the LAN, it's an interesting question. Which host is it intended for? There could be 200 hosts in there. Lastly, hiding the internal hosts is the whole point of hide mode. So the next item here is, well, the next thing that happens is the connection fails. No connection can be made, hence hide mode. All right, we've talked about the first two in hide mode. I'll just more quickly go through static mode. OK, questions? Question here? Is there a cache that can be exploited from the hide mode local addresses? The only cache, so to speak, would be, my understanding, would be the state tables that are stored in the firewall. Well, sure. Once the connection leaves the firewall, it's subject to all sorts of other man in the middle attacks or whatever. At that point, it's just a standard connection going out over the internet. I suppose if you were able to hack into that, you could get back through. With the firewalls fairly clever, I don't know all the details of what it's got there to prevent that. But the firewall is only going to allow replies from that IP address with the correct port number and within a certain period of time. If you're hiding a couple 100 LAN hosts in hide mode, hiding behind a single IP address on the internet, and let's say you were sniffing that connection, you sniffed somebody's T1 line going out to the internet, what do you see? You see a single IP address that's incredibly chatty, right? You see a single IP address that's simultaneously talking to 53 websites. That's hard. That's harder. That's a real blizzard of information. And it's harder to keep all those data streams separate and tease them out and figure out who's what. It'd be hard work. But yeah, once the connection leaves the firewall and goes out on the internet insecurely, yes. There's some things that can be done. But the firewall is keeping a state table with all the relevant information and blocks everything that doesn't match exactly what it's expecting. The firewall is much smarter than a simple access control list, which just says, let this port number into this IP address. The firewall has a state table and says, only this connection and only for the next 30 seconds. And you have to be this IP address. You have to be this port number. Otherwise, you get dropped. Question. Well, the only connection that stays open is for return HTTP information. So it has to have a source port number of 80. So there is no way the firewall, in order to allow web traffic requests to come back to the client, has to allow web traffic requests to come back to the client. So as long as they look like that, yes, they're going to be let through. But if they come in as an attempted tellnet connection, no, the port numbers are wrong. They'll get blocked. So the firewall is only smart enough to let web traffic back. But if there's something dangerous in that, you've attached a Trojan horse.exe to it. Well, the firewall has to do other things to protect you against that. But the connections that is opened allows web traffic responses back from that IP address until the time is up. The question is that if the web server responds with the correct source IP address and the correct source port address that matches the existing connection, then the firewall will let that through. The answer is yes. I mean, that's what it does. The firewall has to do some other things in terms of further inspection of attached files, say, so much more high level inspection, layer seven in the packets. But for what I'm talking about here with network address translation, it's just looking at layers three and four. And so if something comes back through the firewall that looks like valid web response, it's something the web server sent, then yes, it's going to let it back in. This is just simply allowing web access from the land. Yes. Gentleman in the back, white shirt? All right, the question or the comment is that there are some known and perhaps patched issues involving firewalls with buffer overflows. Yeah, those are separate issues. I'm not really trying to talk about some of the more detailed security issues here. These are just, I'm going through the basics of NAT. Yes, there are other firewall issues. Right, if you can do a buffer overflow on a firewall, there's all sorts of things possible. Okay, we're in static mode here, outbound. This is going out. The firewall overwrites the IP source address with the public IP address. So this is just like we saw in hide mode. This is the packet going outbound. Since the from IP address on this packet is a local non-rattable address, the firewall has to change it on the way outbound. The question is, does the firewall accommodate a single or perhaps multiple hide addresses? My understanding with the checkpoint firewall, one product, it's only one IP address per, say subnet or per group or per rule. My experience with the Cisco PIX firewall is that you can designate a range of IP addresses and it will randomly select among them. That's even better. Present your opponents with 100 IP addresses and individual traffic is bouncing around between them. Sort of a spread spectrum sort of approach to it. So it depends which product you have. Firewall one, I think typically goes with one, but I know that Cisco PIX, you can put a range in. We're going to talk about it here in a minute, but of course any IP address that you're hiding behind for NAT has to be a public address that you own. You can't just pick some random IP address because then your traffic will go out, but it won't come back. The remote public server, it sees the request, it looks perfectly reasonable, it looks like a valid request, it just flips the two addresses, puts the payload in and sends it back. This is just a standard response from a web server. Coming back in, the firewall has to do the translation again since it was addressed to the virtual public IP address that now needs to be disambiguated and sent back to the correct internal address. So the moral of the story here is that your network address translation box changes one address on the way out and one address on the way in and that's it. You just have to get the right one. The DMZ server receives the reply from the internet host, the client has no idea that NAT was used and the server has no idea that NAT was used. So again, as long as you're using safe protocols that don't embed an IP address inside the payload, then this shouldn't be a problem. Let's do the last one here before we take the break and again, this is gonna be for people trying to connect to your web servers. These are people on the internet trying to connect into your web server, your FTP server, any of your public servers. And again, here's the four addresses. It's being addressed to your virtual public address. A leading question here. Where does the remote internal, the internet client get this static virtual public address? This is leading to the DNS discussion we're gonna have in the next hour. If you have more than one IP address assigned to a particular TCP IP host, you have to be careful to make sure that you load up your DNS servers with the correct address, depending on who's gonna be querying that DNS server. So if this is a public DNS server that you want people on the internet to be able to get to your server with, then you have to make sure that you're using one of your virtual IP addresses for it, right? You don't wanna load up a public DNS server with a 192.168, that won't do any good. They'll never get to you. So just, I want you to just keep in mind that one of the issues you have to resolve with network address translation is how to load up your DNS servers. On the way in, the firewall overwrites the destination address with your private address, your DMZ server sees a request from an internet host, it replies, the firewall, on the way out, this reply, the firewall has to overwrite the source address one more time, so the packet appears to be coming from a valid internet address. And lastly, the internet client out there receives the reply from the DMZ server. The client has no idea that NAT was used, the server has no idea that NAT was used. So I just wanted to enumerate here the four different types or the four different, you know, inbound and outbound with both hide mode and static mode and just sort of step through it. It's not that complicated once you sit down with paper and pencil and work it through, it's not that tough. You're only changing one address in each direction and all you're doing is just maintaining the consistency that the packets have correct addresses, whether they're on the inside or the outside. The question is, how do you protect your LAN from your compromised DMZ? Well, you do it in your firewall with rules in your firewall and that you don't allow any connections from hosts in the DMZ into hosts on the LAN. So you plan ahead and you say, well, what if somebody totally owned my web server? What's the first thing they're gonna do? They're gonna try to use that as a stepping stone, a springboard into the rest of your network. And at that point, the firewall is gonna stop them as they're trying to connect from the inside. So this is the main reason why you wanna have your DMZ and your LAN on separate NIC cards, separate LAN segments so that the firewall has the best chance possible of still providing a defense there. Your DMZ is sort of a land where things are partially trusted. You'll let people from the outside in to some degree to get on specific port numbers. But if something on the DMZ starts trying to connect back into your LAN, you don't allow it. And so even if it's totally compromised, that's okay because you still have a firewall protecting your inside LAN. Yes, the question is that could be improved with a cascading firewall? Sure, question. Are there, is there a reason why some services don't go through a firewall like voice? Some of the new multimedia protocols are very complex. They open up a control channel and then they send all sorts of port number information about it, about the connection and then they open up a whole bunch of random high TCP ports in order to send the data. Those of you who have worked with other access control lists, et cetera, you know that FTP is sort of a strange protocol, right? It's a strange, not well behaving protocol because once you do that port 21, port 20 thing, you get it set up and then all of a sudden there's this discussion between the server and the client about which random high TCP port number are we gonna use for the data channel? Well, if you have a static access control list, you can't prepare for that because if somebody's picking port 5,021, well you can't set up an access control list that you're constantly changing for that. So FTP is special, FTP is different and that's why you need a firewall that's smart enough to understand the protocol and actually sniff the packets.  Well, it temporarily makes a hole in the firewall just for those IP addresses and just that port number and only while the connection is valid. So if you're wanting some of these sort of the new Microsoft multi-media H323 or whatever they are, I don't know all the details. You need a firewall that understands them and can sniff the control channel and say, oh, okay, I need to open up these five high TCP ports. It has to be a dynamic intelligent thing. You can't just, you know, set something where we're gonna allow this or we're not gonna allow that. It requires some real dynamic thinking going on in order to make that work. It's just about time. So I'm gonna release you, we're starting again at four o'clock on the dot and ask you questions here in between. Of NAT for newbies and not so newbies. We're gonna run until 4.50 PM. We have several more issues to discuss. I'm gonna talk about the benefits of using NAT. I'm gonna talk about some special firewall configuration issues. I'm gonna talk about DNS. Questions at this point? Those of you who were here before, any lingering questions? Anything that we can talk about before we go forward? All right, the question is, even if you have NAT configured properly and you have a lot of people on your LAN who were using hide mode, the question is what else can you do to protect yourself from what?  Well, if you think the intrusion is only local to you, it's only between your gateway router and your ISP or something like that, go find another ISP somewhere who's willing to be the other end of a VPN and just tunnel to them. Find somebody, anybody on the internet who's willing to be the other end of a VPN and just tunnel all of your company's traffic through that box. In other words, a VPN, just to be really clear, is an authenticated, encrypted, private channel over the public internet or a public network. So you could have your entire company, VPN through your public ISP over to somebody, anybody. Pick somebody in Australia for 20 bucks a month. It doesn't matter. And then from there, you will emerge from the tunnel and be released out on the internet and your ISP or anybody who's sniffing your T1 line, what do they see? They see a ton of encrypted traffic and the value to them should be zero. So if it's local to you like that, then find another place to emerge on the internet or find another ISP or whatever. That would be one way to do it. Other questions? Okay, I'm gonna talk about the benefits of using NAT. The reason I developed this talk was because I think NAT is really cool. I think you get an awful lot of benefits for very little hassle. I find it really valuable. So there's basically four main issues here. You can hide your internal network structure from outsiders, structurally block all connections from the internet to your hosts using hide mode, preserve scarce IP addresses, and something called preventing leakage around your firewall, which we'll talk about. The first one here. Since network address translation provides the logical bridge between your internal LAN IP addresses which you created and your external IP addresses, which those are the ones you got from your ISP or IANA, since you can completely control what happens inside and you can completely control that logical bridging, you can do whatever you want on the inside and basically structure it so that you can control what people on the outside see. On the outside you could have five IP addresses in a numeric sequence, but they really represent servers that are on five different DMZ subnets. Whatever, what matters is you can completely delink what people see on the outside from what's actually happening on the inside. You know, we talk about tools like Nmap and port scanning, et cetera, right?  You're gonna try to scan their public IP addresses, right? You're gonna find out, well, what subnets do they have? What do they own? What domain names do they have registered? These IP addresses, okay, let's do a port scan. Okay, look, they've got a web server at these addresses. They respond to FTP at these addresses. Ooh, look, there's one of their mail servers, right? This is easy stuff. This is public information, basically. But the nice thing is, if you're using NAT, then getting all of that public external information gives you zero information about what's happening inside. So this is one of, this is the main benefit of NAT. Is you can make the outside look like whatever you want and it's completely orthogonal to what's happening on the inside. There's absolutely no relation at all. Question. Okay, the question is you've got a NAT box running which is also serving as an FTP server? Well, okay, well, I guess I'd start off by saying that's a bad idea. You should, you know, move your FTP server into the DMZ. I mean, FTP is notorious, you know. I mean, in my lab, my FTP server got ransacked two months ago through FTP. So I'm here to tell you, I don't like FTP. It's a bad thing to be serving.  Oh man, you just gave them the keys. So first of all, separate those out. Make your NAT box firewall just a NAT box firewall. In fact, strip that thing down so there's nothing on it, except your NAT and your firewall. So if I'm gonna, here, I'll rephrase your question. If you do have a FTP server inside and you're making it available to outsiders using, say, static mode NAT, yeah, people can tell that you've got, you know, people to a port scan, they will be able to tell, yeah, there's an FTP server there at that IP address. That's what you want, you're making it public. I mean, that's why you have an FTP server. So yes, if you want people to be able to connect to your mail server, web server, FTP server, then yes, if they do a port scan, they will see those ports open. That's the point. But you've limited it to exactly which IP addresses you want and exactly which port numbers you want. And you're not giving away any information about the inside structure. All you're giving away is that somewhere in your organization, there's an FTP server, but who knows what that address is.  You're giving away the fact that an FTP site exists at your company, but you've already told people that. So that's not giving away anything more. Other questions? Okay, so this is the main benefit of NAT, is you can completely decouple your internal structure from your external structure, say your internal IP structure. So let ScriptKitties do all the port scanning they want. All they're gonna find out is that, yeah, your company has a web server, duh. They're gonna find out, yeah, your company has an FTP server, you're a genius, and you've got a mail server. All right, you're still a genius. Okay, that doesn't do them any good. So your public IP address structure is designed independently of your internal structure. Another benefit to NAT, hosts are protected by Hyde mode don't have a public IP address. This is crucial. In the previous hour, we talked about in Hyde mode how someone can try to make a connection into one of the machines on your LAN. Well, the machines on your LAN don't have a public IP address. Okay, this is even better than an access control list.  It's even better because this IP address doesn't even exist. I mean, this is much better than access control list. This is, you've made an architectural change or it's being completely blocked by architecture. They can't even tell that those machines exist. Somebody on the outside could be pinging that machine all day long. Well, well, first of all, what address would they ping? Well, there is no address they could ping. So you can't even get started. So that's a real benefit. A third benefit to network address translation is to preserve scarce IPv4 IP addresses. How many people here have heard of something called IPv6 coming along real soon now? Okay, any year now, I know we're working on it.  Well, clearly we're running out. The same thing about 20 meg hard drives, right? I mean, if a guy like me can have 16 IP addresses in his house, you know, we're gonna run out soon. So there are only 128 class A IP address blocks. That's all there are. There are only 32,000 class Bs and only about 8 million class Cs. We're running out. I'll show you some examples here of people sort of grabbing more than they needed. There's some humorous examples here we'll get to. But the fact is, there just aren't enough.  Non-market allocation adds to scarcity. IP addresses, once they're assigned, they tend to have fairly strong squatters rights. So once your company gets a class B or a class A, you tend to hold on to it and resist people taking it from you. And we'll see here some examples. One of the reasons we tend to be running out of IP address blocks and also routing table restrictions. The central core routers on the internet have to have routing table entries for basically every single valid IP address in the world. And my understanding, I'm sure there's someone in this room who knows this better than I, but my understanding just a year or two ago, I read an article saying, well, it's bigger than 64 megs now and approaching 128 and it's really causing a problem because a lot of those routers only have 128 megs. Well, first of all, imagine how difficult it is for a router to dynamically search through a 128 megabyte routing table to figure out which Nick it should send this packet out on. So the more IP addresses you have out there and the more fragmented they become, the harder it is to keep straight where you route things. Because a router on the central part of the internet has to be able to handle pretty much every packet that comes its way. Okay, let's talk about squatters rights. This is interesting stuff. I was getting curious about who owns all the Class A address blocks. These are one through 127, I guess zero through 127. Here's some of them. 40, if you ever see a packet coming from 40, you can thank Eli Lilly and company. Now, what are Eli Lilly and company doing with the full Class A address block? I don't know. That makes them equivalent to Sprint, I think, in terms of how many IP addresses they have.  Prudential securities. Do I do Pond, Merck, Boeing? I mean, Boeing, give me a break. I mean, with a Class B, you could give an IP address to every seat on every plane Boeing has ever sold, I think. So, another one here, which I think recently got yanked was Mercedes-Benz had 53. I like Mercedes-Benz, but I'm pretty sure that if I were allowed to use NAT, I could hide any one of these companies behind a Class C address block, right? I mean, for any size company, I mean, how many public IP addresses do you need? Couple of web servers, couple of mail servers, couple of FTP servers, right? How many do you need? 10, 12? And then maybe one hide address for me to hide the rest of your people behind? Maybe a range, but still, I'm willing to bet. If you let me spend money on hardware like I want to, I could hide any corporation in America behind a single Class C IP address block. You don't need however 16 million or some ridiculous number of IP addresses. So I'm looking here at number 48, Prudential Securities. They got it in May of 1995. Well, everyone talks about 1995 as the year the internet really took off. It looks like they just got in under the wire. This was probably, from my looking through IANA's website, this is the last corporate Class A giveaway. Some lucky guys, they're over at Prudential, pulling down a whole Class A for the company. Now it would be a whole lot harder. So as you can see, if these guys like Eli Lilly can grab a whole Class A, obviously we're not allocating IPv4 addresses very efficiently. And this is one of the reasons we're running out. So here, back to that. So the reason you're able to use NAT to preserve scarce IP addresses is because all of your internal machines have private non-routables. And then the only ones that you need for the outside are the static addresses that you're using for people to come to your servers. And the hide mode addresses that you're using. So almost any company could turn in their Class A if they wanted to and pick up a Class C and probably do just fine. Another benefit of NAT, number four is preventing leakage around your firewall. How many people here are sort of network administrators or CIS admins or something like that or you're in charge of a network in some organization? How many people have had somebody in your organization they get either a dial-up account at their desk or they get a ricochet molotov or they get some other way to connect to the internet and before you know it, they're starting to route internal LAN packets through their private connection. Has that happened to anybody here? It's a real issue that somebody at work, well, as we all know, the most dangerous thing in the world is a helpful engineer with a screwdriver. But let's say you get somebody at your office who's sick and tired of not being able to get through the firewall to do whatever he wants. So he gets a dial-up account and he gets a modem line in his office and he starts dialing out. Well, let's say he's running Windows NT server, he accidentally checks the forward packets box and what do you know? He's got a server connected to the internet without protection and now he's routing between the internet and your LAN and it's not going through your firewall. Well, this is real trouble. So one of the things that network address translation does for you is it helps prevent this leakage. I call this leakage. If you're setting up a security perimeter and you've got your firewall, I mean the whole point is that all of the traffic has to go through that checkpoint, right? If some of your traffic is leaking around your firewall and you're in that box, well, that's not very secure because you can be sure hackers are gonna be leaking back, well, okay, leaking back in the other direction. So this is not good. So if you force all of your traffic, well, if you use network address translation, then you're forcing all of your traffic to go through the net box. You can't leak around it because your addresses aren't any good. So since the state tables on your net box are private to your net box, and since therefore nobody else can get in there and sort of figure out what's going on, if somebody does leak, the private address they have is worthless on the outside. So let's say somebody with a machine with a 192.168 dials up somewhere and connects to somebody and starts routing. Well, that 192.168 isn't going anywhere, right? It's a private non-routable address. That's the point. It's worthless on the internet. Public addresses are worthless on the inside because they have to go through the network address translation. And traffic that leaks around the firewall has senseless IP address translation. So one of the nice things about NAT is that you, not perfectly, but fairly effectively ruin the effectiveness of any traffic that tries to leak around your firewall. This is Windows 98. It's taking a little time here. Up here I have, I brought 50 copies, which clearly isn't enough, a printout of the presentation.  Oh yeah, by the end of the presentation I'll show you here, this presentation is already posted on the internet. Okay, the fifth issue here, let's talk about just some basics, how to configure a firewall for NAT because NAT is usually done through a firewall. If someone's gonna go through the trouble of writing, designing, coding a firewall, adding NAT is fairly easy. So you should get them as a bundle. I know that Checkpoint Firewall 1, you get NAT with it for free, as they say. And Cisco Pix has NAT also. The question is, what about the cheap, personal hardware firewalls like Linksys and D-Link? They're great. Hey, for 300 bucks, jeez, you can't go wrong. It's not perfect, you can't do all sorts of crazy configuration stuff. You can't get Microsoft Net Meeting to go through it or something. But man, I strongly recommend everybody have a personal firewall of some sort if you're gonna connect to the internet. And D-Link or Linksys, these guys out of Taiwan that make great stuff, really inexpensive, it works. I got a couple of them at home. I'm sorry, what about Black Ice? Black Ice, didn't, I think Steve Gibson's website, I think he definitively proves it's worthless. So go with something, in his opinion, go with something else. Zone alarm is very good. Yes, to what? To hardware firewalls and NAT boxes like the D-Link or the Linksys, do they add much network overhead? I think, are you really asking or do they add much latency? No, undetectable, milliseconds, not a problem. It's in hardware, it's an ASIC. The announcement is that many of those boxes have NAT only and do not have firewall. You're saying the Netgear in particular does have firewall? Okay, the Netgear box includes some rules-based firewall technology from Sonicwall. One of the ones I use is a D-Link box that also provides my wireless, my 802.11b Wi-Fi connection. It is a wireless node and NAT box and firewall with rules together for 230 bucks, something like that, whatever. The price is falling, you can't go wrong. You know, get one that has firewall and NAT. Other questions? The question is, am I going to address circumvention of NAT device? No, this is designed for newbies. This is a tutorial so that the goal is that at the end of this you have more confidence and information about setting up NAT rather than confidence and information about cracking it. That's the next hour. We're meeting at a bar downtown. So I hate to be so simplistic here to say that in order to configure NAT on your firewall, the first step should be document your network, but it's gosh, it's awfully important to do that. If you don't know what you have, you don't know where you're going and it's sometimes a lot of work to really figure out what you've got. I use what I call the pre-beta version of Vizio, Vizio 0.9, which is a piece of 8.5 by 11 paper in a pencil and I draw and I make the diagram and I draw in all my subnets and everything and it works just fine. I gotta tell you, it's actually more complex than you think to really sit down and diagram what you've got. You have to be really clear. You have to know, what are your IP subnets here? You know, what are the default gateways? What are the subnet masks? You gotta learn all that technology. You have to learn all that terminology. 255-255-255.248, what does that mean? Gotta figure that out. So that's the first part, document your network. Number two, make a plan. Figure out where you wanna be. If you're gonna be instituting NAT for the first time, there's actually some substantial changes you need to make to your network. You're changing all of your internal IP addresses. One of the technical things we gotta talk about here is ARP issues. Just briefly, ARP is a layer three, layer two protocol called address resolution protocol. This is how you convert layer three addresses into ethernet addresses. Think of it as a vending machine. You put in a layer three address and what you get out is a layer two address.  Whatever, we're gonna talk about ARP here in a minute but it's a technical detail you need to know. It's sort of one of the few things we need to talk about down at layer two. Also, we have to talk about routing issues because if you're changing all of your IP addresses, well clearly you have to change all your routing setups. Also, there's a special thing you have to worry about here. Your firewall is a router first and has a bad attitude second, right? That's what a firewall is. It's a router first with an added bad attitude. And when you're setting up your router, excuse me, when you're setting up your firewall, let's say if you're doing checkpoint firewall one, when it sits on top of a Solaris box or an NT box, you wanna get your NT box up and running first and make sure you can route through it first before you add the firewall and start dropping packets. So keep that in mind. A firewall is just a router with some advanced sophisticated access control lists and similar types of things. So it's an interesting question. If a packet is coming into your network, it needs to be knatted, say static mode. And while it's in that NAT box, it's to address for IP, it gets changed, right? It gets changed from the virtual public IP address into the private one. Well, that brings up an interesting question about routing because where you route a packet is based entirely upon what is the to address of that packet? So now you have a packet whose address, its mailing address is changing. So there's issues we have to go over here. We have to talk about exactly how do you handle that? How do you route a packet whose address is changing dynamically on you? Would you route it to the public one? Do you route it to the private one? Do you route first and NAT second? Do you NAT first? Whatever, we'll talk about that. Oh, I'm locked up. I should use Linux, thank you. Oh, here we go. I have this wireless LAN card here. So I'm probably being hacked. We should expect messages coming up on the screen here in a minute. Can somebody return one of the printouts, please? Thank you. Fortunately, this machine, oh, here we go. We just got some action. Okay, this is the next slide. So I'm gonna try to continue on this. If not, I can use the whiteboard. In terms of documenting your network, as I said, this is really just the basics. IP addresses, subnets, gateways, routers, firewalls, all connections through the security perimeter. Again, I'm just talking about documenting. What do you have just so you know what you've got currently going? I guess I'm gonna speak, I'm gonna continue to speak to the next slide while it slowly comes up here. Making a plan. The main issues are just what are your public and your private IP addresses? Whatever your documentation is, your hand drawing, et cetera, you need to have it at least as detailed for what you're doing going forward. And your NAT rules, you need to sort of, you need to figure out in advance what are you gonna do in terms of which machines are gonna be in the LAN and are gonna have hide mode network address translation and which machines are gonna be in your DMZ and have static mode. Man, this is annoying. I hate to reboot, because it just takes forever on this machine. I'm waiting for my mouse to come back. The question is, let's talk about DNS issues. DNS issues are three slides away. We've already been over a VPN. In case you missed the first hour, there was a question about VPN, about VPN compatibility with network address translation. It's the one sort of barely resolved issue VPN, okay, network address translation works best with, how do we describe them? Well-behaved protocols that do not embed the IP address inside the payload of the packet because VPNs, excuse me, because network address translation, your NAT box is changing your IP address as these packets go in and out of your security perimeter. So a well-behaved protocol, that doesn't matter, they don't care. They only look in the packet itself for the IP to and from destination and rely on that. VPNs are special because they've added some security issues in there. They've added some security protections and that they actually embed the IP address in the payload or encrypt it and embed it. And so you can get some problems because on the other end of your VPN, it sees a packet coming from one IP address and yet the inside embedded information gives a different IP address. So there are still issues with VPNs and network address translation. The best solution is to use an integrated firewall that has both network address translation and VPN and that issue will be resolved by itself. Yes? What about hardware VPN? Oh, I don't have a, I'm not able to really give a recommendation one way or the other. There's a lot of different solutions. IP SAC has emerged as the standard so if someone is using IP SAC, you should be fine. Yes. The question is, I've talked about web servers, FTP servers, mail servers as being servers that you want available to people on the outside, on the internet, and that you put them in the DMZ because of it. And the question is, does the same thing apply to SQL servers? You have SQL servers outside your firewall, ask the hackers who own that SQL server to do that for you. I mean, whatever, they need to talk, you have SQL servers outside the firewall that need to talk back to some, I guess I don't understand the configuration. Why do you have SQL servers outside the firewall? Well, SQL uses, as you mentioned, the standard port numbers where you can configure that with your firewall or with your access control and your router to allow or not allow those port numbers. And I think you can actually change those port numbers, but if you want to allow someone on the internet access to your, whatever server, put it in your DMZ and put the appropriate holes in the firewall, it's no different. Yeah, there's no difference between SQL server or web server. It's a little strange to make that accessible to some of them in the outside world. It sounds a little risky, but there shouldn't be any difference. Excuse me? Okay, question, yes. Okay, the question involves some complicated interactions between NAT and VPN, I guess in particular, using a cable modem at home and trying to use VPN to get back into work. Those interactions are a lot more, oh, I love this, are a lot more complicated than, well, it looks like we've got an hour now, don't we? Yeah, here comes safe mode, I bet. It's a complicated interaction between, I'm so embarrassed. It's a complicated interaction between NAT and VPN. I'm not prepared to get into the details on it. That's a lot tougher problem to solve. What kind of NAT box we're using? OpenBSD, okay. Okay. Okay, other questions? Yes? Do ISPs use Natting? It's a public, okay, the question is, the question is, you went to a Steve Gibson's website, you tested the Shields Up thing, and it reported back that you were on a private network, congratulations, and this was news to you. Are you on a public IP address? It looks like a regular, is it possible that your ISP is doing some firewalling for you in protecting some of those port numbers? But it might be the, well, you've got a public, routable IP address, and I've never heard of an ISP issuing non-routables to anybody. So, I don't know. You've got an ISP for dial-up who gives you non-routables? Really? Oh, sorry to hear that. Other questions? Yes? The question is, should I assume, this sounds like a hacker question. All right, the question is, you've always assumed most companies use Nat, and now, I guess I'm talking here, sort of implying that not all companies use Nat, and maybe they should get with the program. I don't really know. I think there's a lot of companies, I mean, all the companies I've worked for, I mean, everybody is desperately short of time in resources, and if a company's been around for a long time and they've got public addresses and it seems to work and, you know, they haven't been hacked severely recently, there's no incentive for people to change that.  You're going to make as much use of those addresses as you can, right?  You're going to make as much use of that as you can. The coffee pot at Eli Lilly has its own IP address. So, I don't know. I don't really know how to really answer that question. I suspect there's an awful lot of organizations out there who many years ago got the network working once and then dropped it and went on to other emergencies and haven't been back, you know?  You get it working once and then it's not your most important five alarm fire and so you don't get back to it and, well, you know, everyone's getting on the internet and no one's complaining and, you know, I, you know, I got to do something else. I don't have time. So I suspect there's lots of companies out there who have firewalls who aren't using that and yeah, I imagine so, but I don't have data on it. Yes, question. This is a good observation that converting IP addresses or switching IP addresses at your organization is a real pain and you're exactly right. You sort of have to do it in one swell foop. It's a lot of work. It's a long, bad weekend and there's a lot of ways to go wrong on it. I'll talk about that a little bit. Yeah, if you're a company who's already got a few thousand users set up with public IP addresses, who wants the headache of converting them over to private? I completely agree. Yes. Okay, so here's a war story up here about it took three days to change IP addresses when you changed ISPs. If you were using private IP addresses and you change ISPs, you only have to reconfigure your NAT box because none of your internal addresses change. Just your external ones. You get a new Class C from Bob's ISP. You're up and running in 10 minutes, right? It's not that tough. Another real benefit of going with NAT. Yes. This leads to my question. What's trouble is that to maintain that side of it? The question is, what if you have a NAT box and on the outside, the public IP address side of it, you're using a DHCP address, an address that's dynamically given to you by your internet service provider? Traditionally, typically, from what I've seen in NAT box is you have to hard code in the external IP address, but this is trouble if you're getting it dynamically. The new boxes now allow for that. I think my D-Link Wi-Fi hub, but it's a combination of a switch, a firewall, a NAT box, and a coffee maker, I think. And it's got, I think you can do DHCP on the outside. So new hardware will allow that. Checkpoint firewall one, I don't think so at this point. There's a need for that and it's easy to code it, so expect it soon. Question, the suggestion is a way that you can incrementally walk your company through a major IP address change. And that is, if you've broken your network up into subnets, you can do them one at a time and use NAT boxes and have the outside of the NAT box still be in the old regime while the inside or the downside, excuse me, the inside of the NAT box with the LAN that they're all gonna be on the new addresses. So if you're thinking cleverly through it and you're willing to go through some of that, yeah, you can sort of walk your way through one LAN at a time and then maybe at the end there's a bad weekend where you have to sort of yank all those out or do something else, but yeah, you can incrementally work your way through.  Yes, in the back. Okay, another tip here, DHCP is particularly helpful. Those of you who haven't discovered it yet, Dynamic Host Configuration Protocol, it's a great way to propagate network changes through your network and your users don't even know. I need to move on. We're only got about seven or eight minutes left, so I'm gonna try to make some more tracks here. Windows 98 seems to be, I spoke to soon. There we go. Okay, we talked about making a plan. Back to address resolution protocol. If you're going to be using on the outside of your NAT box, FACO IP addresses. You know, these are virtual public IP addresses, but for example, this is really an issue, say, on a checkpoint firewall when you're on Solaris or on Windows NT. If you're going to be using external public IP addresses that aren't native to that box, you have to make sure that that box will respond properly to ARP broadcasts from your router. If you have a gateway router who's trying to find this virtual IP address, you have to ensure that the outside of your NAT box is gonna respond to the ARP broadcasts and announce itself properly as being the Ethernet host for that IP address. A lot more complicated details. In firewall one, you handle it in two different ways, depending if you're on Solaris or if you're on Windows NT. Just keep in mind you have to get that straight. On routing, as I mentioned, a firewall is a router first and it's a firewall second. Important question, how do you route a packet whose IP destination is changing since routing is based upon what is your IP destination? So the question sort of falls down to this. Which comes first, the routing or the natting? And it depends on your firewall. You have to figure this out. With Checkpoint, the routing comes first and the natting is the very last thing that happens as the packet flies out the door. So you have to route to the correct NIC first, whether it's the LAN or the DMZ, and then just as it's leaving, the very last step is the destination address gets changed. So it can go either way. You can route first or you can nat first. You just have to find out what it is in your firewall and make sure you understand that. But keep in mind, you're trying to route a packet that has a dynamically changing destination address and as you can imagine, that can get tricky if you get it wrong. DNS issues, here's the problem. If you have two different IP addresses for a given host, one on the inside and one on the outside, which IP address should be returned with the DNS query? Users on the inside, they need to get DNS resolved to the inside address. Users on the outside need to get DNS resolved to the outside address. So the question is, which answer should DNS resolve?  Well, which address do you want to resolve to? The outside one or the inside one? Here's the answer. You have to have something called split DNS. You need to maintain both an inside and an outside DNS server. Insiders resolve to the inside, outsiders resolve to the outside, and you limit the number of entries in the outside DNS server to just the minimum required. So this is additional enhanced security. You don't want people on the outside to use NS Lookup and just run through all the tables on your DNS server and figure out all your internal machines, right? That wouldn't be good. So you list the five boxes that are in your DMZ, you list their public IP addresses and outsiders can see that. But in terms of all the inside machines, you use inside DNS and so that enhances your security by keeping the, again, you're hiding the internal structure of your network from outsiders. Okay, a summary here. NAT provides security at the network architecture level. That's one of the reasons I think it's so good. It's below and before you get issues like access control lists. How many people here have screwed up an ACL and accidentally let something through they shouldn't or accidentally block something they shouldn't have? I better see every hand go up if you've ever touched a router. People do it all the time. It's much harder to screw up NAT. Well, once you get NAT working the first time, then it's harder to get it wrong later. It provides security at a structural level. The nice thing is all the machines in your hide mode, all the machines on your LAN, they don't even have public IP addresses. So the question of whether somebody from the outside can connect to it is sort of moot. I've talked here about the many benefits of NAT. Sort of a warning, a caveat. It may require some significant restructuring of your network. It's some real work to do IP address changes in your company, but I believe the benefits are pretty darn nice if you can get a chance to pull it off. Where to get more information? There's the Internet Engineering Task Force. This is my company, Information Engine. This presentation is already posted on the site. If you go to InformationEngine.com, you see a section called DEFCON 9, and you can find my PowerPoint presentation here. I wanna do some more Q&A, and this is about us. It's sort of a one-man corporation. I'm definitely looking for more interesting opportunities. I've read some haze up here. You know how to get to my website. Talk to me if you wish. And in the few minutes we have left, actually about two minutes, more questions, and I'll handle more questions when we're done here until we lose the room. Thank you.


Hello everyone, welcome to the weekly infrastructure team meeting. Today we are the 21 of December. We have today, Stefan and Harvey with me. It's start to be Christmas holidays. Okay. Mark won't be able to join us today. His half team won't be able to join the Azos on duties. So we can get started. First announcement that the proposer, I propose to cancel next week. Team meeting between Christmas and New Year's Eve. Because almost no one will be there except me as far as I know. So let's see each other in two weeks in 2022. Unless there is someone that disagree and want to join during their early days here. No, thank you. It's good. See each other in 2020. Second announcement, the Jenkins Weekly. Jenkins Weekly release. 2.326 has been released successfully. Thanks for Mark, even though he was expected to be in day off. So Mark, when you hear this recording, that's a blame for you. But thanks anyway. I triggered the release. And team was able to fix the last part that failed. In validation under the script. So we'll have to diagnose that further. In a few days. And the container images failed to build the window spot because not in this space. That's something that happens sometime. You might want to increase the disk space for these missions. I don't know. Do you have. Do you have other announcement or elements works? Nope. Okay, so let's go ahead. We have a pretty pile agenda. First point. Just a note. Thanks for taking that job. We renamed. Two of our main repositories used for the Kubernetes management. So now the name of the repositories and their documentation have been updated. To have more understandable names. So now charts has been renamed to Kubernetes management, which is the repository in charge of managing our Kubernetes applications. And public charts is now Elm charts. So it's clear that it hosts a set of Elm charts that we use and that and people could use if they want. Thanks for this part that will help a lot for contribution and making it clearer for newcomers. And also for us. One point with since two days we have TCP connection errors on the build agent running on the Azure Kubernetes cluster. We don't know if it's related to an external outage on networks or to some network error that we discovered in the Diagnose panel of the Azure UI. But at least we have that error since a few month is now that say we have an overlap of some network ranges. So the overlap is not with the pod subnet. It's with the service subnet. So these are the virtual IP that are that are used as external IP for each Kubernetes service in the cluster. And the documentation of Azure clearly states that these say there must not overlap with the virtual network use for the not pool or the pod IPs. The thing is that we are we overlap because it's a subnet. So we'll have to change that. However, the documentation that I've linked there clearly says, oh, no, I need to add the correct link. So it has been confirmed on one of the GitHub issue of a case that we cannot change the service on a running cluster. We have to create a brand new cluster. So I need to update the links here. So the proposal here will be for the upcoming weeks. The target will be before end of January, at least to create a new cluster named private cavities that will sit on the private network as per the GP that Olivier and Tyler wrote a few years ago. That describe the kind of a network IOP sorry. The IOP infrastructure enhancement proposal where a way to write RFC life for the infrastructure. And so there was one related to the network. I think it's the first one. No, I'm not sure not the IOP one is. So I don't remember exactly which one. Search. Code. IOP two. Yes. It was about network segregation. Right now, our public case cluster sits on the public production. But that cluster is also hosting services that should sit on the private production network behind the VPN gateway. So here will be to create a private cavities cluster inside that private production with the correct with a correct set of network settings and maybe other changes that we will discover to have a proper cluster. Then we can start the proposal is to start migrating private services such as infrasci really see I the graphana or anything behind the VPN out from public to the private. Then we will see if it's changed the TCP issues. And if we have other issue after one or two week on that one we will be able to then create a new instance of a new public cluster and migrate the public the public element on that one. That's the idea. We might have a dependency on terraform. But it's a nice to have. We can start right now using the GUI and managing the new cluster manually. However, it could be easier to change, especially the DNS port if we manage all of these on terraform. My proposal given it's almost Christmas is that we delay these changes to the next week. If it sounds good for you. Yes. Brand new year. A word about the public gates migration. Most of the public application that we are hosting are stateless or depend from on an external manage database, which is not hosted inside the cluster, which means that should be easy. Except maybe for the LDAP that's the only exception for all there's a service that should be easy to have a copy of the services running on the second cluster, and we can do a B with the DNS. So that should be that that should be good enough. Let's see. Worst case that will be an outage when we try the migration. We'll have to do that with a full day with services down and we'll see. I prefer having a bigger outage but planned ahead with people being aware. And so we don't rush the migration. I prefer taking a full day if it's required to be to feel at ease. Is there any question about that? So for people not available right now because of Christmas or not available to reach, don't hesitate to ask questions and I see or on the mailing list. I will send a specific email on that topic to discuss that as synchronously by email. Sounds good for you. Yes, perfect. No question. The next topic is Netlify. So thanks, Gavin. Gavin, add the needs for a spawning preview of the pull request on some of our projects. Most of the time these are static websites such as plug-in Jenkins IO for instance. And the goal is on each pull request from a contributor or from an external maintainer to have a preview of the changes in real time. So we don't restrict the pull request only to code review but also to validate how it behave before merging. Netlify is a SaaS service that allows that feature only on static sites but that could be really interesting because it takes care of the cleaning of cleaning up. Alternatively or complementary we could also bootstrap a Jenkins cluster that provide the same feature. We will have more things to manage but we will be able to have more features and we will manage the cloud resource will pay for this but we'll manage it. Gavin started a support request to Netlify asking for an open source sponsoring an open source account and there is a discussion where Netlify ask us which makes sense to add the Netlify logo and eventually some other things on our website publicly to say we are sponsored by them. That's the condition for getting the account started. There is a discussion about that I'm not sure about all the details but this has to go through the Jenkins board to validate the sponsorship conditions. And Gavin also opened the door to maybe thinking about hosting Jenkins IO on Netlify. Are there is technical question asked to us that we will have to answer even though I'm not completely sure about the exact scope of the question but it's about the bandwidth that we have on Jenkins IO today. I'm not really sure if we have to target the bandwidth with Fastly on the cluster or without Fastly. I have no idea. So I'm going to ask Gavin to clarify before giving an answer. Because the source of truth for this bandwidth could be totally different. Is there any question about that topic? No. So let's go ahead. Next topic award about rating the Jenkins.io. So this is a web service that helps to give user rating about the releases of Jenkins IO. So on the Jenkins.io website. When you go to the latest changelog release. Then changelog. And as you can see after one or two seconds you have some weather icons. These are user feedbacks about these releases. So we have a virtual machine that has to service rating the Jenkins.io, which is responsible for the weather part of the changelog. When the service is down or not acting or not behaving as expected, the weather icon disappear from that page. The rest of the page is static and hosted on Jenkins IO. So that web service was down. First of all, we weren't able to catch any alert about that because in fact we had a bunch of alert due to the TCP issue we mentioned earlier. So the alert was hidden in the pile. That's a shame but thanks for the user who report that on the community forum that helped a lot. So we were able to react promptly after that. Thanks Hervé for jumping on that. So we had to restart the virtual machine. The virtual machine was in bad state. The AWS console wasn't able to reach the machine and was triggering alert. So we took the opportunity to restart it, change its size so it was migrated to a more modern supervisor on EC2 and we upgraded the packages on the machine. Hervé also worked on the Docker image that ran the service inside that VM. So thanks for that. We tried to put that Docker image under the same recipe as all of our Docker images. There are still a bit of work to do though. We are missing the automatic release ports. That one is an easy one. Stefan, if you're interested, that could be an easy one to do. It's one YAML file to put on the correct, we can point this to you. Okay. Let's try. And the Apache version is shown. There are two Apaches server. There is one on the virtual machine, which is not showing its version because Hervé already fixed that a few weeks ago on all the platform on the pooped side. But there is an Apache server for running the PHP request inside the container. And that one should be configured to not show it first. So that's a task in progress. Is there any question about this topic or things unclear? No. Cool. Let's move on. Ongoing and huge work on the upgrading all of our update CLI manifest. New version tracking all dependencies of all Docker images, all charts. So there is a heavy lifting here. We discover every day a new repository with a new set of dependency to add. So it's like the unfinished stories, but almost there. We might want to follow Olivier's advice though on updating the Docker image tag. Of the default value of our own private, our own charts three public. We might have to roll back what we started to do, where we were overriding the default tag on the Elm files on Kubernetes management. So that means a bit more time because we will sequentialize all the updates. But that might that also means easier to understand for newcomers or maintenance. That's always a balance to find. We have a few incoming requests to take care of one from Daniel about conditionally remove some Apache directive for spam protection on the third CI Jenkins instance, which is a private instance. A request from Basil, which is minor but might need help from Jenkins core maintainer. Because he is so that the Jenkins core pull request or builds are not able to restart after a full restart of CI Jenkins. They should be able to start again. But the thing here is that Jenkins see that it has to restart the pipeline so it works as behaved. However, the agent with the full file context on the working directory that are Kubernetes pods. These agent are gone. So it's hard to restart. So either we'll have to work with Jenkins developer to improve the pipeline so it can restart without state. Or we can work on trying finding a way to persist the Kubernetes container use for these patterns. I'm not really sure about the correct way. So we need to ask for help on that part. I'm not good enough on that element. Minor request about Jira setup. Someone want to change your field name, which I don't disagree with. Just don't see the value, but I mean, it doesn't cost nothing to do it. And it's very well documented. So why not. I've been set owner of the Jenkins if her team private mailing list, which is used to contact privately the Jenkins in front team and use as a external communication email. So now I have to add their V and Stefan as members of that mailing. So you should have all the information we mentioned earlier, Netlify stuff, discussion with G frog, etc. I still have to contact Tyler and Olivia to ask them. What's the purpose of the service and since Jenkins IO. Because we don't know. Thanks, Tim and Mark for enabling the Azure centralized logging on Oracle Cloud, which mean you have you only have to authenticate using your Azure credential with centralized authentication and it should be able to access the Oracle account. So I've, I think RV that was good for you. It was good for me. So we'll have to check with you, Stefan, but you should be able to do it. And on the hold we have the puppet cleanup and the IO domain renewal. This element will have to wait for next year. IO domain renewal is, I think we'll miss that part. It's automatically renewed and we'll have to pay but the difference in cost is not so much. It's like 30 bucks for one year. So that's not so much. Okay, that's all we had. Don't know if you have other topics to bring. Good. Good for you, folks. Wishing a Merry Christmas for everyone. Thank you. Wishing a Merry Christmas to you. Happy New Year and see you in two weeks. Bye bye.