Three ISPs, four routers, one MikroTik: rebuilding a homelab network

The homelab accumulated the way most do. A router came with each internet connection, a second one extended wifi to the far room, a third arrived with a device that wanted its own network. By the time I looked at it properly there were three internet connections, four routers each running its own DHCP server, and at least three subnets in play, some of them the same subnet in two places.

Symptoms: devices that could reach the internet but not each other, a camera recorder visible from one room and not another, hypervisor nodes whose addresses changed after a power cut, and wifi that connected fine and passed no traffic. Each of those looked like a separate problem. They were one problem.

The actual problem

Four devices were each acting as the router for their own idea of the network. Nothing was wrong with any individual configuration. What was wrong was that there were four of them and no agreement about who owned what.

  BEFORE

  ISP 1 ─ router A (DHCP, 192.168.0.0/24) ─┐
  ISP 2 ─ router B (DHCP, 192.168.1.0/24) ─┼─ switch ─ everything
  ISP 3 ─ router C (DHCP, 192.168.0.0/24) ─┘
              │
              └─ router D (repeater, its own DHCP)

  a device's address, gateway, and DNS depended on
  which DHCP server answered first

Two of those routers handed out addresses in the same range. A device could receive an address from one router and a gateway pointing at a network segment it was not on, and this changed on every lease renewal.

The overlapping-subnet part deserves specific attention, because it produces the most confusing failure I know of at this layer. When the same subnet exists on two sides of a router, the router has no way to decide whether a destination is local or needs forwarding. It answers for addresses it should route and routes addresses it should answer for. The symptom is “connected but no internet”, or a host that pings from one machine and not another, and the instinct is to go look at DNS. DNS is fine. The routing table is ambiguous.

The design

One device is the router. Everything else is a switch, an access point, or a modem.

  AFTER

  ISP 1 ─ WAN1 ┐
  ISP 2 ─ WAN2 ├─► MikroTik ─► main switch ─┬─ hypervisor nodes  (static lease)
  ISP 3 ─ WAN3 ┘   (sole DHCP,   │          ├─ camera recorder   (static lease)
                    sole NAT,    │          ├─ access points     (static lease)
                    sole router) │          └─ everything else   (pool)
                                 │
                                 └─ tunnel + mesh VPN for remote access

The rules that fall out of it:

One DHCP authority. Every other router has its DHCP server disabled and is demoted to an access point or a plain switch. This single change fixed more symptoms than everything else combined.

Each WAN gets its own subnet, and none of them is the LAN subnet. Three upstream routers all defaulting to 192.168.0.1 is a routing conflict waiting to happen. Give each upstream link a distinct range.

Infrastructure gets static leases, not static IPs. Reservations by MAC on the router, so the address lives in one place I can read rather than in a config file on each machine that I will forget about. The hypervisor nodes and the camera recorder needed this most, since they are the things I need to reach when something is wrong.

Failover uses monitored routes, not just a metric. A default route with a worse metric does not fail over if the upstream link stays technically up while its internet connection is dead, which is the common consumer-ISP failure. The route needs to depend on something actually reachable through it.

Policy routing only where a service requires a specific link. Everything else takes the healthy default. Per-service routing rules are a maintenance cost, so they should be exceptional.

Hardware decision

The existing MikroTik was a hAP ac lite, adequate for wifi and inadequate as the routing authority for three WANs. Choosing between the RB2011 and the RB4011:

The RB4011 was the right call. The deciding factor was not today’s traffic, it was that the router is the one device where being underpowered degrades everything simultaneously and gives you no signal about why.

Remote access

Public services reach the lab through an outbound tunnel connector, so no inbound port is published. Administrative access uses a mesh VPN, deliberately separate from the public path. Two independent paths means a problem with the public one does not lock me out of the thing I need to fix it with.

That separation has paid off more than once. A tunnel misconfiguration takes a service offline. It does not take away my ability to log in and fix the tunnel.

Lessons

Consumer routers are appliances that assume they are the only router. Every one ships as DHCP server, NAT gateway, DNS forwarder, and wifi access point in one box, and that bundle is exactly right when it is the only device and exactly wrong when it is the fourth. Consolidation is mostly the work of turning off features, not adding them.

Overlapping subnets are worth ruling out first. They produce symptoms that look like DNS, like firewalling, like a bad cable. Two minutes checking that no two interfaces share a range saves hours.

Static leases beat static IPs. The address is authoritative in one place, which means an address change is one edit, not a hunt across machines.

Plan the management address before you change the addressing. Switching a hypervisor host to DHCP without a reservation in place means finding it again by scanning, from a laptop whose own network settings just changed. Ask me how I know.

Technologies: MikroTik RouterOS, multi-WAN with monitored routes, NAT, DHCP reservations, VLAN separation, Proxmox, Cloudflare Tunnel, Tailscale.