A while back I wrote some basic information on how to get started implementing multi-homed internet using BGP. The details and configurations listed in that post are enough to get the connection up and running – but not quite in an ideal state. So today I want to share some quick tips that will help you maintain a better and more secure BGP connection.
Securing your BGP peering (Know who you’re connecting to)
BGP is a little different from most other routing protocols, since it uses a single unicast TCP connection between peers to exchange routing updates. Lucky for us, that means that we can easily filter traffic from only known peers. Once you have direct connectivity up between your edge router/firewall and your direct peer, lock down that connection with an ACL. Permit TCP port 179 traffic ONLY from your directly connected peer IP – no one else.
While you’re at it, let’s take it another step further: Request that your ISP set up BGP authentication. Sure, a majority of BGP implementations today still require use of MD5 for auth (which is terrible) – but some authentication is still better than none. This can usually be arranged at the time of turning up peering. Both sides configure the same authentication password and with any luck the peering still establishes.
BGP by nature is unfortunately not the most secure protocol – but a few simple steps like this will help ensure you’re only connecting out to authorized peers.
Route filtering (Don’t trust anyone)
Usually when you’re filling out the BGP peering paperwork for your service provider, they will ask you what kinds of routes you want. In most cases, you should be able to request one of the following:
Default only – Exactly what it sounds like. Your provider will only advertise a route for 0.0.0.0/0. In many cases, this is probably what you’re going to want. With this type of advertisement, each upstream provider will just give us the same default route to the internet. From there we can weight which one we want to use, and traffic will automatically fail-over to the secondary connection should the primary fail.
Partial – If for any reason you want to weight routes to certain destinations differently, then we might request this. In this case, you’re probably going to still receive 0.0.0.0/0 plus any specific routes you ask for. A good example of this is if we wanted to specifically manipulate routes for a remote office we have. Maybe we want to weight Internet traffic for one uplink, and VPN traffic to a remote office on the other uplink.
Full – In 99% of typical business cases, this won’t be required. This option means the upstream providers will be dumping the entire Internet routing table on you. While this offers you a ton of control over path manipulation, it also requires significant memory resources on your routers in order to maintain that routing table.
After we figure this out, the next step is to make sure we are filtering the routes we accept from the upstream provider. Wait – didn’t we just tell them exactly what routes to send us? Why do we need to filter them? Well you can never be too safe here – and we would rather perform an unnecessary filtering than have an ISP accidentally misconfigure route advertisements. So if you’re only expecting a route for 0.0.0.0/0, then filter your inbound route advertisements so you only accept that route.
Same thing goes for outbound route advertisement – if we own a /24 of public IP space, then we only want that range to be advertised out. Some providers may already filter this on their end, but again it doesn’t hurt here to be extra cautious. If we are accepting anything other than a default route from our provider, then we run the risk of leaking those additional routes between the two providers – which would lead to inadvertently becoming a transit AS. Chances are pretty good that you don’t want that, so make sure you configure filtering for all outbound route advertisements.
Minimum Advertisement (Oh no, we have to re-address everything)
I mentioned this in the original post – but typically when you are peering with two separate upstream providers, you need to advertise no less than a /24. We ran into this at my last job, where we had been provided a /25 by AT&T but we needed to bring in a second carrier via BGP. The reasoning behind this is to keep global routing tables as small as possible, by not allowing them to end up flooded with a ton of routes for smaller subnets. It makes sense, but on the other hand I feel like requiring a /24 in all cases can be a bit wasteful. My last job only required maybe 30 publicly addressable hosts – which meant that the remaining addresses went unused.
At any rate – should you find yourself in this scenario then you’re going to have to face the inevitable: Renumbering into a new IP space. Any time you have to do this, it’s going to be a bit of a pain – but for external addressing like this it might be easier. So in our case, the entire /25 space was hosted on our external firewall then NAT’ed into DMZ servers.
Here is the quick steps that I used to do a side-by-side migration without taking any significant downtime:
- Get the new subnet up and running – assign the interface addresses on your firewall and BGP up and running
- Assign new IP addresses to all of your existing services
- Configure NAT rules for the new external IP addresses to the DMZ hosts – while leaving the existing NAT rules for the old subnet (Also make sure your firewall rules permit the same traffic to either IP)
- Migrate DNS entries externally to point to the new IP space
- Once traffic stops flowing to the old IP, remove the old NAT
As a side note – if you procure redundant internet connections through the same upstream provider, then you might be able to work out something else. They may be able to provide you a private ASN to use, and they will likely accept any minimum advertisement – since they will be summarizing upstream within their network anyways.
I had a few more things I originally intended to cover here – but it seems that these topics are filling way more space than I thought they would. Specifically, I’m thinking about a dedicated post to BGP path manipulation – which is probably something you’re going to want to implement after peering is established.
Hopefully these tips help! If you have any questions, throw them in the comments below.