the issue

A few times I’ve faced this issue:

ISP/Network Guys - We’ll provide you with the 204.154.207.0/28 and 188.174.36.128/29 networks. Your gateway is 154.205.0.1.

Me - Oh! Great! So the gateway for these networks is… 154.205.0.1?

ISP/Network Guys - Yes, that’s correct. The transit VLAN is already enabled and available to you.

Me - Ah… Thank you! I’ll… Let you know if I find any issues…

ISP/Network Guys - You’re welcome!

I’m not a network engineer. Actually, most times this issue was already solved before it reached me, and I never really tried to understand how. Until it punched me in the face. Ok, ok… I didn’t get punched as I was lucky enough to be punched before by other “advanced?” networking stuff to figure this one out quite quickly, but this answer still doesn’t show up easily in my internet searches, so I’ll let you have it, for free ;)

The trick here is to “forget” a little bit about the strict theory on networking, so you don’t get into “how to get out of the building, if the door is on the other room?”. If you think of it, the answer is actually quite simple! “Use the door from your room to the other room, then you can exit the building from the other room.”. I know, I’m genius!

the concept

You can set a host from a different subnet as the default gateway by routing to it first. Remember that while there are “safety locks” preventing you to mess around freely with the network, the packet is nothing more than a binary stream, which can have any source address and any destination address. The network devices will receive that and decide what to do with the information they have. A router can take any packet and forward it, with translations or whatnot, to wherever it decides too.

Also consider that routing happens as you send an IP packet, with an external destination on the layer 3 header, to the mac address of your router. A layer 2 device (a simple switch) won’t check the IP header, just the mac to which it should forward the packet to. As long as it lets it exit through the right port it will eventually reach the router somehow. The router then will be able to forward it further.

That being said, assuming that the network is set up accordingly, all you need is to make sure that the packet leaves your computer from the right interface, as it will get to the switch and find its way. You can do it by creating a route to the gateway network, through one of the interfaces first, then routing whatever you want to that gateway. Your operational system should be able to understand that it needs to resolve one route before the other.

the practice

Assume the following setup: The IP of your host: 192.168.100.33/24 The IP of your gateway: 10.0.32.1 Your network interface name: eno4

The following ip commands should do that for you (sorry unix guys, I’m still mostly on linux only and even forgetting stuff about ifconfig):

# ip route add 10.0.32.1/32 dev eno4 src 192.168.100.33
# ip route add default via 10.0.32.1

If you just tried the default route first you’d probably get a Nexthop has invalid gateway error. This can even be used as post-up scripts.

complications

If you’re working with aliases on the interface, multiple subnets, actual vlans, asymetrical routes or other stuff I don’t understand as much as I should, this can fail on some way or another. I can’t give a one size fits all answer, so I’ll just give a tip to look into routing tables, as it solves most stuff I can think of, and to thank the person that may end up helping you learn it!

bonus

If you acquired multiple IP ranges (not a a single IP, like you get as residential or small setups), even if they are contiguous, without a configuration like this you may be forced to “lose” some of the IPs as broadcast, network and gateway IPs. Sometimes even 2 or more other gateway redundancy IPs. We love to use what we pay as much as we can, and we hate to configure stuff that may fail (I’m sure someone out there may have used a 255.0.0.0 mask somewhere). This can give you the ability to use some of these very expensive addresses as actual hosts! I hope I don’t discover later on that it shouldn’t ever be done because of X, Y or Z. Until now, it looks fine for me.

the disclaimer

Just let me enforce that I’m strictly staying out of a best practices discussion here. I’m not a network engineer and far from comfortable in discussing that seriously, it just seemed to be useful to others to share the way I know this problem can be “solved” and break a few concepts that took me a while to sink in.

Until the next one!