Ubuntu: Network driver reset to eth0?
Ubuntu Server has some custom udev rules in place that allow it to "cache" the ethernet devices that have been plugged in to the machine. The delay you're experiencing is caused, as you suspect, in the /etc/network/interfaces file. While it may be viable to constantly add new devices to the file, in your case, you just want to remove Ubuntu's ethernet cache:
rm -f /etc/udev/rules.d/70-persistent-net.rules
Which will restart your network interfaces back to eth0 on the next boot.