OS X: How to Setup NAT on Lion and Mountain Lion

It appears Apple changed the way NAT works, starting in Lion and this upset a lot of people. Apparently, nat would just stop working for some folks after they upgraded to Lion from Snow Leopard.

Finding really in-depth information about Apple's NAT implementation is surprisingly hard. While attempting to figure out how to setup a virtual network for two MacBooks connected to one another over Ethernet, I found a few helpful resources, but none helped me understand exactly how OS X does natd within InternetSharing and no matter whose blog or tutorial I followed, I just couldn't get /usr/sbin/natd to forward my packets. This is how my setup first looked...

I experimented with VLANs, bridges, routing tables, Parallels NAT, Apple NAT, InternetSharing, ipfw, config files and other stuff I've forgotten. I typed in netstat -nr|more and ifconfig more times than I can count. One thing I didn't get into was pf, though. It looked a little too complex.

First, the reason(s) why I didn't want to use Apple's InternetSharing app even though while using it, the VMs on both MacBooks could ping one another and the Internet. I'm a control freak. I want to manage my own everything. My own DNS, my own DHCP server and I want to pick my own damn subnet. Apple says that you can change the subnet by modifying SharingNumberStart in the nat defaults file but this won't work if you already have any device assigned to an IP in that subnet. To avoid conflicts, InternetSharing takes it upon itself to modify the subnet by one octet (172.20.0.1 -> 172.20.1.1) which is the opposite of what I want. I was also unable to find any documentation on how to modify the subnet mask, so I don't even think that's possible with InternetSharing.

My network requirements aren't that common, admittedly. It's not often that someone will want to hook up two MacBook Pros and run a lil virtual network between them using only an Ethernet cable but, eh, I do. Btw, did you know that you don't need a crossover cable for MacBooks to talk to one another directly over gigabit Ethernet? Me either. Anyway, I didn't want to buy a gigabit router because this is a very short term project. And I thought I'd enjoy the challenge of figuring this out.

I knew my nat configs were mostly right, I was just missing something with the routes. I finally found the last piece using route monitor and watching what InternetSharing changed when it was turned on and all client VMs could ping each other and the Internet(s).

Check out the routing table InternetSharing created:

192.168.1.1 is my Wireless connection, 192.168.2.1 is Apple's NAT subnet and the other nets were created by Parallels. Most network admins I talked to immediately thought the multiple gateways were causing an issue but this is the setup that actually worked, it just wasn't customizable enough. By the way, BSD has one of the few tcp/ip stacks that allow you to have multiple gateways (so I read.)

So after about a week of prodding, I figured out that this is what Apple basically does within InternetSharing:

1gwdev=en1
2ifconfig bridge0 create
3ifconfig bridge0 up
4ifconfig bridge0 addm en0
5ifconfig bridge0 172.20.0.1
6route add default -interface bridge0 -ifscope bridge0 -cloning
7sysctl -w net.inet.ip.forwarding=1
8/sbin/ipfw add 100 divert natd ip from any to any via $gwdev
9/usr/sbin/natd -interface $gwdev -use_sockets -same_ports -unregistered_only -dynamic -clamp_mss -enable_natportmap -natportmap_interface en0

That and a bunch of error handling and things related to DHCP and DNS. It also checks to see if the Internet connection exists and deletes the nat/bridge if it doesn't and recreates it when the connection is back up.

Here's a visual of my network after running that script:

Below is the code I've started using on my machine. My guest OS network runs a Windows domain and has its own DNS and DHCP servers so those are not addressed in this script. They are really important services, though so if you don't have them setup, you'll want to do that.

Oh, it also doesn't make any changes to the firewall except to add a forwarding rule. You'll want to setup your firewall, too. Annnd this script doesn't use a specific network interface. I make it detect the current gateway then use that interface. If it can't find a gateway, it uses en1. I should probably make it just die but whatever.

 1#!/bin/bash
 2
 3# Get the interface name for the gateway
 4gwdev=`netstat -nr | grep default | awk '{ print $6 }' | head -1`
 5
 6# If none are found, set the gateway to en1 (generally Wifi on OS X)
 7if [ -z "$gwdev" ]; then
 8gwdev=en1
 9fi
10
11# Create a bridge, add the Ethernet device
12ifconfig bridge0 create
13ifconfig bridge0 up
14ifconfig bridge0 addm en0
15# Give it an IP, route bridge0's traffic to bridge0
16ifconfig bridge0 172.20.0.1
17route add default -interface bridge0 -ifscope bridge0 -cloning
18
19# Enable IP forwarding, add a firewall rule to send all natd traffic to the real gateway
20# Start natd with a whole bunch of options
21sysctl -w net.inet.ip.forwarding=1
22/sbin/ipfw add 100 divert natd ip from any to any via $gwdev
23/usr/sbin/natd -interface $gwdev -use_sockets -same_ports -unregistered_only -dynamic -clamp_mss -enable_natportmap -natportmap_interface en0

And to stop...

 1#!/bin/bash
 2# All these steps look excessive but address
 3# network instability issues created by not doing them
 4
 5gwdev=`netstat -nr | grep default | awk '{ print $6 }' | head -1`
 6
 7if [ -z "$gwdev" ]; then
 8gwdev=en1
 9fi
10
11killall natd
12/sbin/ipfw delete 100 divert natd ip from any to any via $gwdev
13sysctl -w net.inet.ip.forwarding=0
14ifconfig $gwdev down
15ifconfig en0 down
16route delete default -interface bridge0 -ifscope bridge0
17ifconfig bridge0 172.20.0.1 delete
18ifconfig bridge0 deletem en0
19ifconfig bridge0 destroy
20ifconfig en0 up
21ifconfig $gwdev up

One big important thing for laptop users is that natd becomes unstable after the laptop wakes from sleep. To address that, I created the scripts above, installed sleepwatcher, and set sleepwatcher to run /sbin/stopnat on sleep and /sbin/startnat it on wake. This makes natd run 24/7 so make sure that's what you want. I also need to put it in the startup but haven't yet.

Here's that setup, real quick: sudo port install sleepwatcher /usr/local/sbin/sleepwatcher -d --wakeup /sbin/startnat --sleep /sbin/stopnat

It's likely I'll keep develop this more over time, but for now, my clusters need my attention. I know it's a bit of a pain just copy/pasting so give me about a month or eight and I'll package up the files nice and put em up here.

Update: If you're looking to do this without NAT, check out Thomas' post about bridging ethernet interfaces in OS X.

References: OS X Advanced Server Guide, Mac OS X for Unix Geeks, a blog post by akutz and every support forum for OSX, FreeBSD and Parallels ever.