For a rather long while now, I’ve been running with a virtual machine as a router, first for a decent while running pfSense and then at some point switching to a simple Debian installation due to some irritation with how poorly pfSense handled IPv6. It all ran rather well, and setting up a router using systemd was actually a rather decent experience since there is actually a decent amount of flexibility built into it assuming one is using a recent enough version.
However, having recently started tinkering with other virtual machines on the same host more, and specifically things like PCIe passthrough, it became apparent that there was a decent amount of value in having the router actually be a separate physical box. Since I had been curious about trying VyOS anyway, the EdgeRouter X seemed like a good fit for the task, while still being a rather small, cheap and silent device.
Configuration of the device has been rather easy for the most part, though I have had to do a lot of it through the CLI—which to be fair is my preferred method anyway—since the WebUI has a woefully terrible support for IPv6; it’s strange that manufacturers think that’s acceptable in this day and age, but I guess I paid for the device too so maybe they’re correct? I have also ended up simplifying my network somewhat from the time I was using pfSense which is fortunate, since looking at how the router seems to set up things like router advertisements seems to be handled the same way in VyOS which means I would be encountering the same problems as I had with pfSense again.
Another change I ended up making from my previous setup due to besser hardware support was changing my VPN tunnels from OpenVPN to IPsec, since the EdgeRouter only has hardware offloading for IPsec meaning the performance of those tunnels is going to be better. This was a nice opportunity for me, since I had actually been curious about playing around with IPsec anyway but due to the increased complexity in comparison to OpenVPN had never really had the motivation to actually properly get started with it, which was compounded by the previous other endpoint of the tunnel being on an OpenVZ VPS, which introduced some additional difficulties in configuring IPsec since it’s a kernel module and OpenVZ is paravirtualization. The changeover was probably unecessary since I believe the true bottleneck will end up being our Internet upink anyway, but it did prove to be a fun learning experience—even the extra effort of switching from PSK to public key authentication for the tunnel, even if I was rather frustrated at times at the somewhat unclear documentation when one wanted public key without a certificate authority. Another thing that required some figuring out was allowing IPv6 traffic over the IPsec tunnel as well, since while the EdgeRouter does support configuring an IPsec tunnel with a DHCP interface instead of a fixed address which is necessary for my setup, this option is unexplicably disabled when using IPv6 addresses which means there was no straightforward way to configure it. What I ended up doing was running a GRE tunnel over the IPsec tunnel and using that to pass the IPv6 traffic, meaning a bit of extra overhead but it is at least a functional solution and the overhead should be negligible for my use-case anyway.
The final piece of the changeover puzzle was figuring out DNS-level adblocking, to protect devices that don’t allow installing an adblocker on the device or software itself. This was luckily enough solved easily by a package by britannic that even ended up having the usual filter list that I use included meaning it was zero configuration needed for me, I only needed to actually install the package itself. I has worked nicely so far, and I’m happy to see such simple solutions be available and tailored for these devices, especially since any custom hacks that I might build myself while certainly functional would mean an increased maintenance burden.
Overall, I’ve been happy with the switch so far, and everything seems to be functioning rather nicely, and getting to finally properly use VyOS was a nice bonus on top. It does feel a bit unfortunate to lose the “magic box” cool factor of having the router run in a virtual machine, but the separation of concerns makes up for that. Also, it’s still a tiny highly configurable “magic box” that makes the Internet work, so there is still a certain cool factor in that as well.