So in my last post, I picked up a Qotom Mini-PC to run OPNsense on. After a few months, the device has been running well & I’m very happy with it.

One of the new things I got to try out with OPNsense was Wireguard VPN. I had previously been using something else for VPN connectivity back to my home network - but I heard good things about Wireguard & wanted to give it a try.

So far my experience has been good! I’ve been pleasantly suprised with how easy it is to configure & get running. In addition, the performance & overall experience has been very positive. The VPN connects quicker than anything I’ve used in the past, and just simply works without issue.

All that being said - I wanted to put together a quick guide on how to configure Wireguard on OPNsense. Specifically, this configuraion will be for remote-access VPN - where clients will connect to a VPN headend. We’ll walk through the OPNsense configuration & a few clients as well. So let’s dig in!

Topology

For the purpose of this blog post, we’ll be using the lab topology below:

I will be using the reserved IP range 203.0.113.0/24 for the WAN-side addressing. I’ll have one Windows & one Android client that we’ll walk through & connect to the VPN.

Installing the Wireguard Plugin

To get started, first thing we will want to do is install the Wireguard plugin for OPNsense. By default, OPNsense will have standard IPSec & OpenVPN already available - but other VPN options can be enabled easily.

So in OPNsense, we’ll navigate down to System > Firmware > Plugins, then search for wireguard and click the plus icon.

This should pull down the package & install pretty quickly. No reboot required here!

Once installed, you may have to refresh the page or navigate to a new page so that the menu bar has a chance to reload. Then we’ll have a new option under VPN:

Wireguard Tunnel Configuration

Next we’ll begin configuring Wireguard on the OPNsense side.

There is a little bit of a chicken & egg scenario here since everything is based on cryptographic keys. We’ll need to generate keys on the firewall, which we need to enter on the client - but we also need the client keys to enter on the firewall. A bit of bouncing between the two - but for now we’ll try to complete as much as we can on the firewall side.

We’ll enable Wireguard by dropping down to VPN > WireGuard then clicking Enable and Apply

Next we’ll set up the Wireguard tunnel interface on OPNsense. This will be a virtual tunnel interface that will be created as interface wg<instance number>.

To do this, we’ll navigate to the Local tab, and click the plus icon to add a new tunnel.

In the above screenshot, I’ve filled in just a few details.

For Name, I’ve entered our virtual interface name wg1. Since OPNsense shows the Instance as 1 - it will create a wg interface with that instance number.

We’ll leave Public Key & Private Key blank for now. OPNsense will auto-generate these keys once we save this config.

For Listen Port, I’ve set this to 51820 which is the default for Wireguard. It’s not stated here, but this is a UDP tunnel.

For Tunnel Address - this is where we add the IP address of the virtual tunnel interface. This will be the gateway for our remote clients. In my lab I’ll be using 10.50.50.1/24 here.

We have no peers configured yet - so we can’t select any here. We’ll leave this blank for now, but come back later.

We will leave Disable Routes unchecked. By default, OPNsense will add static/connected routes for any client via the tunnel interface. You might not want this behavior if you wanted to do custom routing - for example, in a site-to-site VPN connection - but we’ll leave this enabled.

Once we’re done, we’ll click Save.

Like I mentioned before, OPNsense will now auto-generate our crypto keys for the tunnels. So if we edit our tunnel, we’ll now see those fields populated:

We’ll want to copy the Public Key & save it for later. This will need to be imported onto our clients, so that they can communicate securely with our firewall.

Wireguard Interface Assignment

Now that we have our headend tunnel interface defined, we can map our wg1 interface to an OPNsense interface. The OPNsense documentation suggests this is optional, but I would recommend it since it will allow us to create firewall rules to permit/deny access to clients.

We’ll navigate to Interfaces > Assignments, and we should see a New interface available: our wg1 tunnel.

We can assign this a name, then click the plus icon & Save.

Next we’ll enable the interface by navigating to Interfaces > WG1. Here we’ll only need to click Enable & save the change - nothing else is necessary.

Of course, we’ll be prompted to apply the changes - which we will do:

By default, all traffic through our WG1 firewall interface will be blocked - so please make sure to configure a firewall rule to permit traffic from the Wireguard clients.

Firewall Rules: Allow Inbound Wireguard Traffic

Next we need to permit the Wireguard traffic into our firewall. By default the WAN interface will block all traffic that isn’t explicitly allowed - including our Wireguard traffic.

For this, we’ll navigate to Firewall > Rules > WAN. Then click the plus icon to add a new rule.

The screenshot above shows what our firewall rule will look like.

Here’s the summary:

• Action: Pass
• Interface: WAN
• Direction: In
• TCP/IP Version: IPv4
  • You can enable IPv6 as well, if you have IPv6 connectivity (this is a lab box, which does not)
• Protocol: UDP
• Source: Any
  • This will allow anyone on the internet to reach our VPN. We could restrict source IP addresses, if our clients had permanent, static IPs.
• Destination: WAN Address
  • The firewall itself is the destination for this traffic
• Destination Port Range: (other) / 51820

I also enabled logging & added a quick description. After we have all this configured, we can click Save - then Apply Changes.

Client Setup - Windows

So first we’ll start with an easy configuration on a Windows client. Wireguard client software can be found on the Wireguard site here.

For the sake of the walkthrough, we will manually configure each client. However, this can be a difficult task if there is a large number of clients. Wireguard does support importing configurations, and there are a number of free tools available to help automate generating config files for clients - including some which will generate QR codes for easy import on mobile clients.

So on my lab Windows machine, we’ll open up the Wireguard client & click Add Empty Tunnel:

Then we’ll be given a blank config file, with only the devices public & private key pair generated for us.

We’re going to fill in details similar to the below screenshot:

The configuration under [Interface] is the local, client-side configuration. I’ve added the client’s tunnel address - which will be 10.50.50.15/32. I’ve also configured DNS servers which the client can reach via the VPN.

Then we’ll add the [Peer] section, which contains info about our VPN headend. Here’s where we’ll need the public key from our OPNsense firewall. We’ll also specify the Endpoint address, which is the IP or hostname of our VPN headend & the port (which by default is 51820).

We’ve also configured AllowedIPs as 0.0.0.0/0. This will force all client traffic over the VPN tunnel - including general internet traffic. However, we could limit this to specific subnets. For example, let’s say your network only used 172.16.90.0/24 & 10.1.1.0/16 subnets & we only wanted the user to be able to access those. We would configure the following: AllowedIPs = 172.16.90.0/24, 10.1.1.0/16. In this case, only traffic for those subnets would be routed over the VPN - any other traffic would use the devices default internet connection.

Now, we’ll save this - and again need to copy the device’s Public Key, which we’ll need to enter on the OPNsense firewall.

Client Setup - Adding Clients to OPNsense

In order for the Windows machine to connect to OPNsense, we’ll also need to configure a client profile on the firewall.

In OPNsense, we’ll navigate back to VPN > WireGuard, then click on the Endpoints tab.

Here we’ll configure a name for our client & paste in the client’s Public Key.

We’ll also set AllowedIPs to the client’s IP address, which we have configured as 10.50.50.15/32. This controls what IP addresses are reachable via this endpoint.

We do have some additional fields available, which we will leave blank. For example - Endpoint Address & Endpoint Port would be used to define a public IP that we expect our client to connect from. Since a remote-access client could connect from any IP, we leave those fields blank to allow this.

Once we’re done, we’ll click Save then Apply.

Next we’ll jump back to the Local tab, and edit our headend tunnel configuration.

We should now see our windows client in the Peers dropdown. We’ll select that client, so that Wireguard will permit that client to connect via this tunnel interface.

Then, as always, click Save & Apply

Last but not least - we should restart our Wireguard server on OPNsense. This can be done by either disabling & re-enabling Wireguard - or by navigating back to the OPNsense dashboard & clicking the restart icon next to the wireguard-go service.

Testing The Connection

Okay - now that we have all that completed, it’s finally time to test connectivity from our client.

On my Windows client, I’ll just click the Activate button for the tunnel:

And we’ll see that the Status shows Active, and we’ll start to see updates to the Last Handshake & Transfer fields - indicating we are connected & sending data.

To further validate, we can check the Log tab:

The key things to look for here, are the following messages:

• Receiving handshake response which indicates our firewall responded to our request to connect
• Keypair 1 created which indicates that our connection is healthy to our peer
• Receiving keepalive packet from peer - we should see these periodically to maintain our connection. If keepalives stop flowing, then we may have a break in connectivity between client & peer.

We should also be able to reach out wg1 tunnel address from the Windows client:

On the OPNsense side of things, we can check what client is connected via the List Configuration tab, under VPN > Wireguard:

On this screen, we can see we have 1 peer connected - which matches the IP & public key of our Windows client. We’ll see similar output like the Windows client, where we can see the latest handshake & data transfer.

Additionally, for easier access we can add the Wireguard widget to the OPNsense dashboard.

If we navigate to our dashboard, then click Add widget - we can add the Wireguard widget.

Here’s what that looks like:

Additional Client Setup - Android

Let’s also take a quick look at a mobile client. I’ll get through this pretty quickly, since the configuration will be very similar to our other client - just a different UI.

On my Android device - if I open up the Wireguard app, I have a few options for creating or importing a tunnel:

In this case, we’ll again walk through creating a tunnel manually. Again, it’s worth mentioning that there are 3rd party apps available to auto-generate config imports or QR codes to make this easier.

First we’ll have a handful of fields to fill in about the Android-side configuration. This includes a name for the tunnel, an address, and DNS servers.

We can click on the refresh icon in the Private Key field to auto-generate our key pairs:

One of the interesting parts of the mobile app is the ability to permit/exclude individual apps from traversing the VPN tunnel.

Here’s a quick screenshot, where we can pick either to allow only certain apps to use the VPN - or permit all except a few we choose to exclude:

In my case, I’ll keep this set to allow all applications.

Next we’ll work on the peer config:

After that, all we need to do is save our VPN configuration - then we can toggle the tunnel on or off:

And similar to the Windows client, we can click on the tunnel itself to see current status / data transfer:

So it looks like we should be connected & can try accessing VPN resources!

We can also use the same methods as earlier to check connectivity from the OPNsense side.

Okay - that’s all I wanted to share today. I’ve been quite pleased with how easy to use WireGuard has been - and how well it performs! I hope this blog post was helpful if you’re interested in trying it out.

Thanks!!

In a recent post, I walked through setting up a Netgear LTE cellular modem with Google Fi. Might seem random - but I had a plan for this!

In trying to ensure I keep a stable internet connection for work, I eventually led myself down the path of buying a cellular modem. That was part one. The next step was being able to somewhat-intelligently monitor my home internet connection, and then fail over to the cell modem when connectivity was poor (think lazy SDWAN).

At first I figured this would be easy…. but of course nothing is :)

I currently have a Cisco FirePower 1010 appliance that I’m using for a home firewall. Unfortunately, while this thing does support IP SLA - It wouldn’t quite accomplish what I wanted to do. And to be fair, this box is intended to be a security appliance - not SDWAN.

Anyways - I talked myself into just writing some Python automation to monitor my home internet, and dynamically inject/remove static routing entries. I needed a new project anyways

For those of you wanting to jump straight to the code, the GitHub repo is here

Part 1 - Monitoring Packet Loss & Latency

So first thing - I regularly experience both complete internet outages (always at the worst times), as well as very high latency. Packet loss seems to be less common with my ISP - but hey, it happens sometimes too.

I opted to use the pythonping module as an easy way to collect some of the info I needed.

After importing the module, it’s as easy as specifying a destination, packet size, and number of ICMP messages to send:

from pythonping import ping
result = ping('8.8.8.8', size=2, count=10, verbose=True)

To simplify at least one part of the operation, pythonping has a built in function to return the average round trip:

>>> print(result.rtt_avg_ms)
74.23

The actual contents of our ping results are going to be in the following format: Reply from 8.8.8.8, 10 bytes in 43.82ms

So a quick way for me to figure out packet loss was to simply search for the “Reply from” text in each response:

lost = 0
for packet in result:
    if "Reply from" in str(packet):
        pass
    else:
        lost += 1

Easy enough - and we can use that to calculate the amount of packet loss against the 10 total packets sent:

lossperc = 100 * lost / 10

Once all that is figured out, we can use a simple expression to determine whether or not the loss or latency thresholds have been exceeded:

if response >= MAX_LATENCY or loss >= MAX_LOSS:
    print("Loss/Latency violate thresholds.")
    return True
else:
    print("Loss/Latency within thresholds.")
    return False

The code I wrote then takes one of two paths.

• If the loss and latency is ABOVE our thresholds, call to the FirePower module to inject a static default route over the LTE modem
• If the loss and latency is BELOW our thresholds, call to the FirePower module to delete the static default route - which returns traffic to the primary internet

These functions within the FirePower module were written so that each change will only be made once. For example, if the script checks and finds that the loss/latency is bad, but the static route towards the LTE modem already exists - then no additional changes are made.

So next - Let’s take a look at the FirePower module.

Part 2 - Authenticating to FDM & Initial Checks

This was my first time getting into the FirePower FDM APIs - but they ended up being fairly straightforward to use.

Turns out that FDM has built-in API documentation, which is extremely helpful. This can be found at https://<FDM IP>/#/api-explorer

Okay - So first thing’s first. Authenticating to the firewall:

oauth_data = {
    "grant_type": "password",
    "username": "admin",
    "password": "Cisco1234!"
}
headers = {
          "Content-Type": "application/json",
          "Accept": "application/json"
}
baseurl = "https://" + FDM + "/api/fdm/latest"

authurl = baseurl + "/fdm/token"
print("Posting AUTH request to FDM")
# POST request to FDM with headers / oauth data
resp = self.s.post(authurl, headers=headers,
                   data=json.dumps(oauth_data),
                   verify=False)
# If success - only pull out & return the auth token
if resp.status_code == 200:
    print("Auth success - got token.")
    return json.loads(resp.text)['access_token']
else:
    print("Authentication Failed.")
    print(resp.text)

In the code above - we take the headers and some basic authentication info (username, password, grant type) and send it in an HTTP POST request to https://<FDM IP>/api/fdm/latest/fdm/token

This returns an authentication token, which we’ll need to include in any further HTTP request to the firewall. This can be done by sending a new set of headers in the future:

headers = {
          "Content-Type": "application/json",
          "Accept": "application/json",
          "Authorization": "Bearer " + self.token
}

Next we need to figure out which routing table to insert the route into. Since I am only using the default routing table, the script will just grab the UID for the default global routing table:

vr_url = baseurl + "/devices/default/routing/virtualrouters"
routing_table = requests.get(vr_url, headers=headers)

I mentioned earlier that the script will not make any changes if the firewall is already in the desired state. So we will take time now to check what state the firewall is in, before attempting to make any changes.

This is accomplished by making our first primary action scanning the routing table to see if our route already exists:

route_url = baseurl + "/devices/default/routing/virtualrouters/" + \
            self.globalVR + "/staticrouteentries"
route_data = requests.get(route_url, headers=headers)

# Convert returned JSON object
current_routes = json.loads(route_data)['items']
# Run through each route returned and see if it matches the one we're looking for
for route in current_routes:
    # For each route, we need to manually go look up the uid of our gateway & network objects
    gateway = self.getNetworkObject(route['gateway']['id'])
    dest_network = self.getNetworkObject(route['networks'][0]['id']).split('/')[0]
    # Match based on route prefix & upstream next hop gateway
    if gateway == GATEWAY and dest_network == ROUTE.split('/')[0]:
        print("Found route to %s via %s" % (dest_network, gateway))
        return route

Depending on whether we were looking to add or remove a route, the script may proceed with doing so - or just quit if the action has already been taken previously.

Part 3 - Failover (Add Static Route)

Assuming that we need to Add a route, we need to sent a POST request to https://<FDM IP>/api/fdm/latest/devices/default/routing/virtualrouters/<Virtual Router ID>/staticrouteentries with some required information (like subnet info, next-hop, etc)

Unfortunately, we can’t just send a POST with the target subnet & gateway. Those need to be created as network objects on the FirePower box beforehand. For each network object, we need to build a quick JSON object with the required info:

host_data = {}
host_data['name'] = name
host_data['description'] = "Created by ISP Failover automation"
host_data['subType'] = subtype
host_data['value'] = address
host_data['type'] = "networkobject"

Then we can send a POST to the FDM API to create the object with our supplied parameters:

posturl = baseurl + "/object/networks"
requests.post(posturl, headers=headers, json=host_data)

Once we have those objects created, we can compile all of that info into a JSON object with all the components needed to create a static route entry:

routeobject = {}
routeobject['name'] = "route_BACKUP"
routeobject['description'] = "Created by ISP Failover automation"
routeobject['iface'] = {}
routeobject['iface']['id'] = iface_ID
routeobject['iface']['type'] = "physicalinterface"
routeobject['iface']['name'] = iface_name
routeobject['networks'] = [{}]
routeobject['networks'][0] = {}
routeobject['networks'][0]['id'] = network_ID
routeobject['networks'][0]['type'] = "networkobject"
routeobject['networks'][0]['name'] = network_name
routeobject['gateway'] = {}
routeobject['gateway']['id'] = gateway_ID
routeobject['gateway']['type'] = "networkobject"
routeobject['gateway']['name'] = gateway_name
routeobject['metricValue'] = 1
routeobject['ipType'] = "IPv4"
routeobject['type'] = "staticrouteentry"

Then pass all that into a POST request to create the route object:

add_url = baseurl + "/devices/default/routing/virtualrouters/" + self.globalVR + "/staticrouteentries
requests.post(add_url, headers=headers, json=routeobject)

Success? Well not quite yet.

FirePower requires all changes to be deployed (or applied) to the system before they take effect.

So next we need to initiate a deployment task - and periodically check on it. Deployments can take a while to complete, depending on number of changes, processing power of the appliance, etc.

# First send a POST to the deployment endpoint:
deploy_url = baseurl + "/operational/deploy"
deploy_response = requests.post(deploy_url, headers=headers)
# Grab the deployment task UID:
deploymentID = json.loads(deploy_response.text)['id']

# Loop while deployment is running:
while deployed is False:
    # Sleep for a few seconds:
    sleep(5)
    # Get list of all deployment tasks:
    tasklist = json.loads(requests.get(deploy_url).text)
    for task in taskList['items']:
        if task['id'] == deploymentID and task['state'] == 'DEPLOYED':
            print("Deployment status is: " + task['state'])
            deployed = True
            return True
        elif task['id'] == deploymentID and task['state'] != 'DEPLOYED':
            # If changes not yet deployed, check again momentarily
            print("Deployment status is: " + task['state'])
            deployed = False

So in the above block, we POST a request to deploy changes. The response of that POST request contains our deployment ID, which we keep to check the status later.

Then the script waits a few seconds, and pulls a list of all deployment tasks. Search through that list for our deployment ID - and check to see if it has been completed yet. If not, wait and run the loop again.

And that’s it! Once we get confirmation that our changes have been deployed - we’ve now routed traffic over the secondary connection (an LTE modem, in my case).

Part 4 - Fail-Back (Delete Static Route)

Okay - this section will be fairly quick. We’ve already looked at authenticating, checking if our route already exists, and how to add a new route. So the only thing remaining is removing our route if we wanted to migrate traffic back to the primary connection.

Same logic applies as earlier. If our path monitoring script runs and finds that loss & latency are within the thresholds we set, then we make a call to the firepower module. First we check to ensure there is a route for us to delete, and if so - proceed with deleting it.

Adding a route is a little more work, since we may need to create network objects. However, when we delete the route - we’ll just leave those objects on the FirePower box. They’ll be there for the next time we need them (which also speeds up deployment times).

To get started, we just need the UID for the route we want to delete.

Remember that code I showed above, where we check to see if the static route exists? And match on our intended subnet / gateway pair? Well, I just made that into a function that will return the UID of the route if it finds it. Easy enough!

Next, we just send an HTTP DELETE to the static routing endpoint - and reference that UID:

del_url = baseurl + "/devices/default/routing/virtualrouters/" + \
          self.globalVR + "/staticrouteentries/" + route['id']
requests.delete(del_url, headers=headers)

Once that succeeds - we use the same logic as earlier to deploy the changes.

After that, our route is removed & traffic should be flowing over our primary connection again.

If it helps anyone - I also threw together a quick diagram to explain the flow of operations here:

Well - that’s it. This was a fun side project to work on over the past week or two. I hadn’t spent much time with the FirePower/FDM APIs just yet. While there was a bit of work in creating all the necessary network objects, the overall process was fairly simple.

It helped tremendously that the FDM API explorer exists, and is available on-box. This utility allows you to see all the available API calls, what parameters they require, and even run test calls from the web UI. This greatly reduced the time needed to figure this out.

Hope this was interesting. If you would like to see the whole project - check out the repo on my GitHub page.