What to do when eth0 isn't eth0
I tripped on this issue awhile back (when a lightning strike killed one of my nics), but in all the recovery chaos I neglected to document the issue. And so I tripped on it again just yesterday when I juggled nics between two machines.
The issue is that udev makes a note to itself of which mac addresses correspond to which ethernet device indicators (eth0, eth1, etc..), when they're initially discovered. And then it 'reserves' that indicator, so if, say, you pull out the nic that was eth0 and put in a new one, it won't get eth0, which is non-intuitive (at least if you've been installing nics for years).
The fix is easy, though. Just edit the following file:
/etc/udev/rules.d/70-persistent-net.rules
You can either delete the line that contains the indicator that you want to reassign, or you can just edit the mac address itself. A reboot (or possibly even a restart for udev - I don't know I didn't try) will get the new nic exposed as expected.
It can also be helpful to just run: 'ifconfig -a' if you're not sure which nic got assigned which ethernet device indicator.