With a lot of the recent nonsense going on after VMware was acquired last year, I’m seeing people start to look for ESX alternatives - especially for home labs. In fact, I also just migrated my home lab from ESX to Proxmox.

One of the things I needed to keep running was Cisco Modeling Labs (CML). I often use CML to build quick virtual topologies for testing things, so it was important for me that it worked on Proxmox.

It’s worth noting that Proxmox isn’t an officially supported platform. The docs state that only bare metal & VMware platforms are supported.

That being said, I’m happy to say that Proxmox has worked just fine for me - it just needed a few tweaks. Though just be warned that you may still run into issues and it is technically an unsupported configuration.

In the post below, I’ll share the steps that I used to get CML up & running on Proxmox.

Prerequisites

To start with, we’ll need a few things:

• A server running Proxmox
  • I’m currently using version 8.1.3
• A CML install ISO, found here
  • We’ll be using version 2.6.1 for this guide
• CML’s reference platform ISO, also found at the link above

Our Proxmox machine will still need enough resources to build a VM with the minimum requirements listed here.

The CML & refplat ISOs should be placed into a Proxmox ISO storage location.

Creating the VM

Next we can get to the fun part: Setting up the VM in Proxmox.

We’ll first locate a node in Proxmox to place our VM. Then right-click that node & select Create VM:

Then, we’ll give our VM a name & ID. I’ve named mine CML:

On the next page, we can select our CML ISO from the correct storage device. We can leave the Guest OS as Linux and 6.x - 2.6 Kernel:

Next, we’ll need to make some minor adjustments on the System tab.

The default BIOS will likely be set to Default (SeaBIOS). We’ll change that to OVMF (UEFI), which will also give us a few additional options.

We’ll keep Add EFI Disk checked, and select a storage location for that EFI disk.

On the Disks tab, feel free to increase the size of the VM disk. While 32GB is the minimum required, you’ll likely want more than that. I’ve increased mine to 50G for now. Keep the image format as QEMU.

We’ll also want to update the Async IO setting to native. This is hidden under the Advanced settings:

On the CPU tab, we’ll update the core count to a minimum of 4 per the requirements. Of course, you’re welcome to increase this as needed.

More importantly however, we’ll need to set the CPU type to host. This allows the VM to use the underlying nested virtualization features:

Next we can assign memory to our VM.

The requirements state a minimum of 8GB, however this will likely only accommodate simpler labs. Some individual CML nodes require more than that to start.

In my case, I’ll start with 32GB - and we can always increase this later:

Lastly, on the Network tab, no changes are necessary. Of course, you can assign a VLAN if needed.

After that, we can head over to the Confirm tab & finish up the wizard.

Add Refplat ISO

Next, we’ll need to make sure our reference platform ISO is connected to the VM.

We’ll head over to the VM’s hardware tab, then click Add and select CD/DVD Drive:

Then we’ll select the appropriate ISO storage, and pick our refplat ISO file:

Install CML

Once that’s done, we can go ahead and power on our VM & pop open the console!

If you’ve installed CML previously on another platform, this process is just the same.

At boot, we’ll select Install CML.

After a moment & a few screens, we’ll start the system configuration.

First, we’ll input a system hostname:

Then, we’ll input credentials for the system management user.

Note: This user isn’t used to log into the CML software. Rather, it’s used for system administration tasks via the Cockpit UI, such as: Installing updates, checking logs, joining a domain, or powering off / rebooting the system.

After that, we can configure our CML admin user:

Next we have our network config. By default, CML will prompt to use DHCP, but we’ll likely want to change this to a static IP assignment:

After that, CML will prompt us to confirm our settings - then it will begin the installation. Since a lot of the VM images come from the reference platform ISO, this process may take a while as each of those images are copied over to the system.

When CML is ready, we’ll see something similar on the console telling us how to reach the web UI:

Again for reference, the main UI is reachable at https://<CML IP> and the Cockpit management UI is at https://<CML IP>:9090.

At this point, we can log in & start building labs!

The process for setting up CML on Proxmox isn’t super crazy, but there are just a few tweaks that need to be made during setup. I wrote this hoping that if anyone else out there is trying to set this up, maybe it would help.

As always, Thanks for reading!

In this blog post, I wanted to spend just a few minutes going over how to solve an issue one of my peers ran into recently.

The Issue

Let’s say we were trying to use Ansible to update the interface admin state on a Cisco Nexus switch. In this case, I’m using a virtual Nexus 9000 which is running NX-OS version 10.2(6).

This switch has a loopback interface (lo100), which we want to disable.

Reading the Ansible documentation for the NETCONF module, we might assume that the following would be enough to get the job done:

- name : NX-OS Interface Updates
  hosts: all
  connection: netconf
  gather_facts: no

  tasks:
    - name: Update Interface State
      ansible.netcommon.netconf_config:
        content: |
            <config>
              <System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
                <intf-items>
                  <lb-items>
                    <LbRtdIf-list>
                      <id>lo100</id>
                      <adminSt>down</adminSt>
                    </LbRtdIf-list>
                  </lb-items>
                </intf-items>
              </System>
            </config>

However, once we run that playbook, we’ll get an error back:

(ansible) matt@ansible-dev:~/ansible/nxos$ ansible-playbook nx-update-interface.yaml -i inventory.yml

PLAY [NX-OS Interface Updates] **********************************************************************************************************

TASK [Update Interface State] ***********************************************************************************************************
fatal: [10.122.2.229]: FAILED! => {"changed": false, "msg": "Request without namespace and filter is an unsupported operation"}

PLAY RECAP ******************************************************************************************************************************
10.122.2.229               : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

The Fix

Luckily, this has an easy solution.

According to the Cisco NX-OS Documentation, there was a change in version 9.3(1) that requires a filter to be sent along with any GET or GET-CONFIG request.

But wait - aren’t we updating the config via an EDIT-CONFIG request? We shouldn’t need to send a filter, since it’s not a GET or GET-CONFIG, right?

Well because of the way Ansible works, we actually send two GET-CONFIG requests along with our EDIT-CONFIG. One before the change, and another one after making the change.

These are used to validate that we actually updated the config. Ansible uses these two GET-CONFIG requests to compare & see if there is actually any difference between the two.

So to address our error above - when we send our config change in Ansible, we also need to include the get_filter parameter. This will give Ansible a NETCONF filter use when it issues the GET-CONFIG requests to validate our changes.

In our case, that GET-CONFIG filter could look something like this:

<System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
  <intf-items>
    <lb-items>
      <LbRtdIf-list>
        <id>lo100</id>
        <adminSt/>
      </LbRtdIf-list>
    </lb-items>
  </intf-items>
</System>

That filter would pull back just the admin state for our lo100 interface, which then Ansible could use to compare after we issue our request to disable the interface.

For what it’s worth, we don’t necessarily have to be that specific with our filter. We just need to ensure that whatever config returned by the filter would also include the section we intend to change.

So for example, we could actually just filter by System and this would still work:

<System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
</System>

The main difference here is that the switch would return a much larger amount of data, which then Ansible would also need to process to detect the change. So for efficiency & accuracy, it’s probably still helpful to keep our get_filter scoped to a reasonably narrow amount.

With all that covered, here’s the example of our final playbook with the new filter:

- name : NX-OS Interface Updates
  hosts: all
  connection: netconf
  gather_facts: no

  tasks:
    - name: Update Interface State
      ansible.netcommon.netconf_config:
        get_filter: |
          <System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
                <intf-items>
                  <lb-items>
                    <LbRtdIf-list>
                      <id>lo100</id>
                      <adminSt/>
                    </LbRtdIf-list>
                  </lb-items>
                </intf-items>
              </System>
        content: |
            <config>
              <System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
                <intf-items>
                  <lb-items>
                    <LbRtdIf-list>
                      <id>lo100</id>
                      <adminSt>down</adminSt>
                    </LbRtdIf-list>
                  </lb-items>
                </intf-items>
              </System>
            </config>

And, if we try to run the playbook again:

(ansible) matt@ansible-dev:~/ansible/nxos$ ansible-playbook nx-update-interface.yaml -i inventory.yml

PLAY [NX-OS Interface Updates] **********************************************************************************************************

TASK [Update Interface State] ***********************************************************************************************************
changed: [10.122.2.229]

PLAY RECAP ******************************************************************************************************************************
10.122.2.229               : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

That looks much better!

Thanks so much for reading this blog post. I really hope it was helpful to you!

If you found value in this, please let me know! Feel free to leave me a comment below, or follow me on YouTube!

Oh, and for those of you on Mastodon - You can find me at @matt@0x2142.com.

Thanks again!

Back when I started diving into studying for the Cisco DevNet certifications, I thought a lot of the REST API stuff was fairly easy. But I always struggled a bit with wrapping my head around YANG models & how exactly NETCONF/RESTCONF worked. A lot of it seemed more abstract, and browsing through YANG models isn’t quite as intuitive as most REST API documentation.

These days I end up working quite a bit with NETCONF - and if there’s one tool that’s helped me more than anything, it’s certainly YANG suite. So in this post, I wanted to spend a little bit of time walking through how to get set up with this tool - and walk through a quick example of how it can be used.

What’s YANG Suite?

To put it simply: YANG Suite is a web-based tool that allows you to easily browse through YANG models & find what you need. It also has a bunch of tools to run test queries against devices, which makes developing NETCONF automation easier. It can also auto-generate code snippets - and who doesn’t like that? 😄

YANG suite is an open source tool maintained by Cisco & the GitHub repo here.

Initial Setup

YANG suite is a containerized application, so we’ll need a container host to run it on. You could use something like Docker Desktop to run it on your local PC, but I’ll be using a dedicated Ubuntu machine in my lab with Docker installed.

Note: YANG suite now supports a local Python pip-based install. Check out the GitHub repo for more info on that.

Deploy with Docker

First thing we’ll do is pull down the code from GitHub:

matt@yangsuite:~/yangsuite/docker$ git clone https://github.com/CiscoDevNet/yangsuite.git

Next, we can execute the setup script to help provision the YANG Suite containers, which is located at yangsuite/docker/start_yang_suite.sh. The script will prompt you to set up an admin user and generate self-signed certificates:

matt@yangsuite:~/yangsuite/docker$ cd yangsuite/docker
matt@yangsuite:~/yangsuite/docker$ ./start_yang_suite.sh
Hello, please setup YANG Suite admin user.
username: matt
password:
confirm password:
email: matt@0x2142.local

Setup test certificates? (y/n): y
  -- output omitted -- 
  
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: US
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:
Certificates generated...
Building docker containers...

Once all of the above info is provided, the script will build the YANG Suite contianer images & launch them. However, by default it will launch in the foreground - so we can kill the script with Ctrl-C & relaunch the containers with docker-compose up -d.

Allow Access from Remote IPs

We may need to make a quick change depending on how we’ll be accessing YANG Suite.

By default, YANG Suite will only permit access to the web UI from localhost - so if you’re running Docker on your local PC, then no change is necessary.

However, if you’re running Docker on a dedicated VM like I am, then we’ll need to allow external IPs to access the YANG Suite UI. This is controlled by the DJANGO_ALLOWED_HOSTS environment variable that gets supplied to the container.

As long as we’ve already run the setup script once - our environment variables will be saved to yangsuite/docker/yangsuite/setup.env:

DJANGO_SETTINGS_MODULE=yangsuite.settings.production
MEDIA_ROOT=/ys-data/
STATIC_ROOT=/ys-static/
DJANGO_STATIC_ROOT=/ys-static/
DJANGO_ALLOWED_HOSTS=localhost
YS_ADMIN_USER=<removed>
YS_ADMIN_PASS=<removed>
YS_ADMIN_EMAIL=<removed>

We’ll need to change DJANGO_ALLOWED_HOSTS=localhost to the IP address of the Docker host. So if your Docker host is reachable at 192.168.1.1, then we’ll set: DJANGO_ALLOWED_HOSTS=192.168.1.1.

After which, we can bring up the containers again with docker-compose up -d:

matt@yangsuite:~/yangsuite/docker$ docker-compose up -d
Recreating docker-yangsuite-1 ... done
Recreating docker-backup-1    ... done
Recreating docker-nginx-1     ... done

Once everything is running, the web UI can be reached at https://<ip address>:8443.

Loading YANG Models

After we get logged in, we’ll need to load the appropriate YANG models for whatever devices we’re using.

For example, I’ll be using Cisco IOS-XE devices running version 17.6.x code - so I’ll need to download the YANG models from: https://github.com/YangModels/yang/tree/main/vendor/cisco/xe/1761.

Within YANG Suite, we’ll navigate to Setup > YANG files and repositories. Then click the button to add a New repository:

You can name the repo whatever you like. In my case, I named it ios-xe-1761 which describes the models it will hold.

Next we can tell YANG Suite where we want to pull our models from. There are a couple of options, including a direct upload from your local PC or querying models directly from a device - but I’ll choose Git.

After which, we’ll need to fill in the following info:

The Repository URL will be the base Git repo that we’re pulling from. In this case, that’s https://github.com/YangModels/yang.git.

We’ll also need to specify which Git branch to pull from, which for the YangModels repo will be main.

Then we’ll specify which directory to import. The YangModels repository holds models for a large number of vendors & devices - so we only want to pull in what is necessary. In my case, I’ve set this to vendor/cisco/xe/1761 - which will only pull in the YANG models for Cisco IOS-XE 17.6.x devices. I’ve also checked the box for Include subdirectories.

Once that’s all set, we can hit Import YANG files. Depending on how many models are being downloaded & how quick your Docker host is, this may take a few minutes.

After it finishes, mine displayed a message showing that 827 models had been added:

Selecting YANG Module Sets

Once we have all of our models downloaded into YANG Suite, it’s time to set up some module sets. You can think of these as filters for the YANG models.

For example, later on we’ll show how to change a wireless PSK with NETCONF. So since I already know what type of information I’ll be working with, I can create a module set to only contain those models.

YANG Suite will require at least one module set to be configured, even if you don’t want to filter anything out.

To do this, we’ll navigate to Setup > YANG model sets. Then click New YANG set:

Again, since we’ll be working with wireless data in a bit, I’ll name my model set wireless. We’ll also have to select which YANG repository this will pull from, which will be ios-xe-1761.

Next, we’ll get to select which YANG models we want to include in our module set. To make things easy, I’ll filter the models by wireless and then click Add selected.

Once those are added, you’ll likely see an error about missing dependencies. Some of the models we selected may have sub-models or data types that are located in other models.

In order to quickly add these, YANG Suite provides a one-click-button to Locate and add missing dependencies:

That’s it - our model set is ready to use.

Adding Devices

Okay, so now that we have all of our necessary YANG models set up, let’s take a add a device that we can use for testing. I’ll be adding a Catalyst 9800 Wireless controller.

To do this, we’ll head over to Setup > Device Profiles. Then click on Create new device.

Here we’ll fill in a name for our device profile, along with the device’s IP/FQDN and login credentials.

Scrolling down a bit, we’ll also need to make sure we enable the appropriate management protocols. So I’ll be checking the box for both NETCONF and RESTCONF, since I have both enabled. I’ll also add SSH, which isn’t shown in the screenshot below.

Please note, you’ll likely want to check the box for Skip SSH key validation for this device under the NETCONF settings.

After that’s all done, we can click Create Profile at the bottom.

Then, to validate everything works - we’ll select our device & click the button for Check selected device’s reachability.

With any luck, you should see a similar result as shown below:

Example: Changing Wireless PSK

Alright, now let’s take a look at how we might use YANG Suite to test NETCONF / RESTCONF calls to a wireless controller.

In this example, perhaps we’re developing automation to rotate the guest wireless network password on a regular basis. We need to figure out how specifically to accomplish that by finding the right YANG models & understanding what data is sent and received with those calls.

We’ll explore how to do this with both NETCONF and RESTCONF.

Via NETCONF

To start off, we’ll navigate to Protocols > NETCONF. On this page, we’ll have full access to explore models, build XML payloads, and test them against our configured device.

Get-config

First thing we’ll need is to select our YANG set & Modules. In my case, the YANG set will be wireless, and I’ll select the module: Cisco-IOS-XE-wireless-wlan-cfg.

Then click the button to Load Modules:

By default, our NETCONF Operation will be set to get-config, which will work for now since we only want to retrieve configuration and not change anything yet.

Under the Nodes pane, we can expand the tree a bit & dig in to find which call we need.

For our example, we’ll need to examine the current configured WLAN profiles. So we’ll check the box next to wlan-cfg-entries:

This will retrieve all configured WLAN profiles on the controller. For now, let’s do that so we can figure out what data we’ll get back.

We can now click the button to Build RPC, and YANG Suite will automatically build the appropriate XML payload for us:

You’ll also notice that I selected the catalyst 9800 as the target device. Let’s go ahead and run this with the Run RPC(s) button, which should pop open a new window & establish a connection to our device.

We can see in the response above, that we have two WLAN profiles configured: test-lab-network and test-guest-network. For the purposes of this demo, I have opted to have the guest network PSK stored in plain text, so we can validate our changes more easily.

Okay, so what if we wanted to refine our query XML to only retrieve data on the test-guest-network? Maybe we also want to filter our the additional information returned, and only see the current PSK?

Back on the payload editor, we can expand wlan-cfg-entries and input our desired profile-name:

We’ll also find the psk item further down in the list. Now we want to pull back this field, regardless of what the PSK is currently set to. So we’ll click the value field, but leave it blank:

Once we click Build RPC, we can see our payload now shows the additional XML filters asking only for a single profile & it’s associated PSK:

If we run that RPC call, we’ll now see that we only get back a limited set of data:

Now we know exactly what payload to send if we want to get back the current configured PSK for any specific WLAN profile.

Let’s try changing it!

Edit-config

Lucky for us, we’re already nearly set up for that. We’ll need to make two quick changes. First, set the NETCONF Operation to edit-config:

Then, under the YANG tree, we’ll set a new PSK:

Again, we’ll click Build RPC - and take a quick look at the proposed change:

Look good to you? Okay, let’s send it:

Sure enough, the new configuration was sent to the wireless controller and we got a response of: ok.

But of course, we can check quickly by switching our operation back to get-config and re-running our previous XML payload:

Auto-generated snippets

Well if you’re new to NETCONF, you might be asking yourself “Okay great, but how do I get this into Python?!?!”

YANG Suite has some built in tools to auto-generate Python code or Ansible playbooks, which can help us get started on our code more quickly.

To generate these, click the Replays button:

For example, I selected to auto-generate a Python script. Here’s what that looks like:

#! /usr/bin/env python
import traceback
import lxml.etree as et
from argparse import ArgumentParser
from ncclient import manager
from ncclient.operations import RPCError

payload = [
'''
<get-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <source>
      <running/>
    </source>
    <filter>
      <wlan-cfg-data xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-wireless-wlan-cfg">
        <wlan-cfg-entries>
          <wlan-cfg-entry>
            <profile-name>test-guest-network</profile-name>
            <psk/>
          </wlan-cfg-entry>
        </wlan-cfg-entries>
      </wlan-cfg-data>
    </filter>
  </get-config>

''',
]

if __name__ == '__main__':

    parser = ArgumentParser(description='Usage:')

    # script arguments
    parser.add_argument('-a', '--host', type=str, required=True,
                        help="Device IP address or Hostname")
    parser.add_argument('-u', '--username', type=str, required=True,
                        help="Device Username (netconf agent username)")
    parser.add_argument('-p', '--password', type=str, required=True,
                        help="Device Password (netconf agent password)")
    parser.add_argument('--port', type=int, default=830,
                        help="Netconf agent port")
    args = parser.parse_args()

    # connect to netconf agent
    with manager.connect(host=args.host,
                         port=args.port,
                         username=args.username,
                         password=args.password,
                         timeout=90,
                         hostkey_verify=False,
                         device_params={'name': 'csr'}) as m:

        # execute netconf operation
        for rpc in payload:
            try:
                response = m.dispatch(et.fromstring(rpc))
                data = response.xml
            except RPCError as e:
                data = e.xml
                pass
            except Exception as e:
                traceback.print_exc()
                exit(1)

            # beautify output
            if et.iselement(data):
                data = et.tostring(data, pretty_print=True).decode()

            try:
                out = et.tostring(
                    et.fromstring(data.encode('utf-8')),
                    pretty_print=True
                ).decode()
            except Exception as e:
                traceback.print_exc()
                exit(1)

            print(out)

Via RESTCONF

What if we prefer working with REST & JSON instead of XML? This time we’ll navigate to Protocols > RESTCONF.

Here, we’ll select our YANG set & modules again. In case you skipped the NETCONF section, we’re working with the wireless YANG set & the Cisco-IOS-XE-wireless-wlan-cfg module.

We’ll also select a device to use, then click Load Module(s):

Again, we’ll see a similar structure to the NETCONF side. But this time, we’ll utilize it quite a bit differently.

In the screenshot above, I’ve already selected the wlan-cfg-entries entry. Next, we’ll click the button to Generate API(s):

We’ll see auto-generated API documentation for all of the available calls. This list will contain every variation possible, so it can be quite long. Since we already went through the NETCONF method, we know what data we need to find / use which should make this easier.

To start with, I’ll expand the GET for /data/Cisco-IOS-XE-wireless-wlan-cfg:wlan-cfg-data/wlan-cfg-entries. We’ll have the ability to test the RESTCONF calls directly from this interface as well by clicking on Try it out:

In the screenshot above, I’ve already submitted the test request & gotten a response from the controller. While the screenshot is cut off a bit, we can see both of our wireless profile names.

If we wanted to see only the PSK, like we did with our previous example - we could scroll down quite a bit to find the GET request for /data/Cisco-IOS-XE-wireless-wlan-cfg:wlan-cfg-data/wlan-cfg-entries/wlan-cfg-entry={wlan-cfg-entry-profile-name}/psk

In the screenshot below, I’ve already input our test-guest-network profile name & executed the API call:

Sure enough, we see the PSK that we had just configured via NETCONF a few minutes ago.

Now let’s change it. We’ll expand the PATCH API call for the same endpoint, and fill in the request body with our new PSK:

Executing that call, the wireless controller returned a status code of 204.

So let’s check again to see if that updated properly:

Now that we have all the appropriate REST calls, it should be much easier to start developing our automation.

Okay, I think that’s about all I wanted to cover in this post. I’ve seen a lot of people struggle with NETCONF much like I used to, and I think the real key to getting familiar is just having the right tools & spending enough time with it. YANG Suite has really helped me get more familiar with the various model structures and how to use them.

Hopefully this information helps others to get started. NETCONF & YANG are certainly big hills to climb, but the right tools can make all the difference.

A few months ago, I had written a blog post on building a simple chatbot with Webex. You can find that here.

At the time I was already thinking about some follow-up content, where I could walk through how to use AdaptiveCards to make the bot a little fancier - but I decided to wait a bit & see how the first post performed first.

Well, now it’s a few months later & I suppose it’s about time I write this 😄.

So in this post - We’ll build on the code I used in the last post. We’ll show how to add cards to both display information, as well as take user input.

What are AdaptiveCards?

AdaptiveCards are actually not a Webex/Cisco-specific thing! They were actually created by Microsoft with the intention of being a platform-agnostic way of displaying information. So in theory, a card schema written for a Webex bot could be taken & re-used on another messaging platform that implements AdaptiveCards - without any/much rework.

Hopefully this should mean a more consistent user experience for a bot that is supported across multiple platforms. The cards themselves can also enhance the experience by making your bot more dynamic & interactive. Maybe a developer or network engineer is okay with issuing specific syntax-based commands to a bit - but what about someone who is less computer savvy? Using cards makes it easy to provide guided input with textboxes, buttons, and toggles.

AdaptiveCards are built on a standard JSON structure with a pre-defined schema of card properties. Let’s take a quick look at a simple example:

{
    "type": "AdaptiveCard",
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.2",
    "body": [
        {
            "type": "TextBlock",
            "text": "Hello there! What's your name?",
            "wrap": true
        },
        {
            "type": "Input.Text",
            "placeholder": "Your name here",
            "id": "user_name"
        },
        {
            "type": "ActionSet",
            "actions": [
                {
                    "type": "Action.Submit",
                    "title": "Submit"
                }
            ]
        }
    ]
}

So in the JSON payload above, we’re creating a card that asks for user input. We have a block of text that asks the user “What’s your name?”, then we provide a text input field, and finally a submit button. Looks pretty straightforward, yeah?

We’ll get into the specifics around how to create that card JSON in a bit, but you can always use Microsoft’s AdaptiveCard Designer to play around with card elements/structure.

So what would that card look like when rendered in Webex? Let’s take a look:

That card seems a lot more user-friendly than trying to explain to your users that they need to remember some sytax like /botcommand username set <firstname> <lastname>. Instead, they can just enter the info on the card & click submit.

Sending a Static, Default Card from a JSON File

So first we’ll start off with the easier scenario. We’ll have a pre-defined JSON card built, and use the bot to load & send this card response.

One note before we get started - I’ll be building on my simple weather chatbot code that I mentioned earlier. If you’re following along, you can pull down that code from here.

Okay, to start off - we’ll be creating a card that allows a user to enter their desired zip code. Instead of having to remember the specific command syntax, a user will instead just issue the weather command & our bot will prompt for information via a card.

I’ve used the Microsoft Adaptive Card Designer to build out the basic card structure, which I’ve kept simple for now. Here’s what that looks like:

{
    "type": "AdaptiveCard",
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.2",
    "body": [
        {
            "type": "TextBlock",
            "text": "Get Current Weather",
            "wrap": true,
            "size": "Medium",
            "weight": "Bolder"
        },
        {
            "type": "TextBlock",
            "text": "Enter a Zip code:",
            "wrap": true
        },
        {
            "type": "Input.Number",
            "placeholder": "00000",
            "id": "zip_code",
            "min": 1,
            "max": 99950
        },
        {
            "type": "ActionSet",
            "actions": [
                {
                    "type": "Action.Submit",
                    "title": "Get Weather",
                    "data": {
                        "callback_keyword": "weather"
                    }
                }
            ]
        }
    ]
}

This card looks pretty similar to the example I used earlier. We have an initial text block that displays the title “Get Current Weather”, and we’ve added attributes for bold text & a medium size.

Next, we have a normal text block prompting the user to enter their zip code. This is followed by a number input box for that data entry.

The input box has a “min” & “max” value for the zip code from 1 to 99950 - which is the full range of US zip codes. We also need to assign any input a unique “id” value - which we’ll see later is used to pull out the user input from the response.

Last, we have our actions - which similar to earlier. This will present as a simple submit button with the text “Get Weather”. The important note here is adding a data dictionary with a callback_keyword sub-element. We set the callback_keyword to the command we want this submission to be sent to. In our case, we want the input to be sent back to the weather command.

Now to implement. Back in our bot, we’ll edit the weather.py file to load the card & assign as the default command response. Let’s take a look:

### Script snippet

with open("./input-card.json", "r") as card:
    INPUT_CARD = json.load(card)

class WeatherByZIP(Command):
    def __init__(self):
        super().__init__(
            command_keyword="weather",
            help_message="Get current weather conditions by ZIP code.",
            card=INPUT_CARD,
        )
        
### Script snippet

In the above snippet, I’ve extracted just the pieces we’ll be editing. For full examples, please see the final example repo here.

I’ve saved that JSON payload from earlier as input-card.json. Then we’ll open that card file & assign that JSON payload to a variable called INPUT_CARD.

Since we’ll expect this card to be the primary method of collecting a zip code - we’ll make this card the default response from our bot when the command is triggered. In my original code, we initialized the WeatherByZIP class with card=None - which instead passed our input directly to the execute() function. In this case, we’ll set that to card=INPUT_CARD to send our form.

With all that done, let’s test it out:

Collecting & Processing Card Submissions

Now that we have our input card, let’s move on to processing the response.

In the original example, we expected the user to enter a command like weather 12345 - and it was easy enough to just strip the 5-digit zip code from the response.

Since a card could have many input boxes, we instead receive that data back in a structured JSON format.

If we check out the logging in the webex_bot console, we can see the payload that gets returned to us when the card is submitted:

2022-03-03 14:59:26  [INFO]  [webex_websocket_client.root._process_incoming_websocket_message]:62 attachment_actions from message_base_64_id: Webex Teams AttachmentAction:
{
  "id": "--removed--",
  "type": "submit",
  "messageId": "--removed--",
  "inputs": {
    "callback_keyword": "weather",
    "zip_code": "12345"
  },
  "personId": "--removed--",
  "roomId": "--removed--",
  "created": "2022-03-03T19:59:24.820Z"
}

Using this output, we can see exactly what data is passed back to our bot. Under imports, we do see the correct callback_keyword that we defined in our card JSON. But we also see the user input - which is 12345. This is listed under the key zip_code which is the id we assigned to our input field earlier.

Now we can easily dig through this response to pull out the data we need. This JSON payload is delivered to the execute function under the attachment_actions parameter.

So we’ll make some minor modifications to our original code to pull that value out.

def execute(self, message, attachment_actions, activity):
    zip_code = attachment_actions.inputs['zip_code']

As a reminder, zip_code is the variable we’re using inside the execute() function to store the requested zip code - which is then passed to the OpenWeather API.

Previously, we had just assigned the value of message to zip_code, which was the incoming text message from the user.

Instead, this time we’ll fetch the zip code from the incoming card response, using attachment_actions.inputs & pulling out the zip_code key.

With that simple change completed, we should be able to ask the bot for the weather, input our zip code, then receive the same text-based output as before.

Here’s what that looks like now:

As we would expect, upon hitting Get Weather - the bot returns the text-based weather report.

Next, we’ll take a look at dynamically building a card to display weather info.

Dynamically Building Cards

Now we get to the real fun part. We’ll need to dynamically build an AdaptiveCard response, depending on what values we want to present to the user.

I will be using a Python module called adaptivecardbuilder. There are a handful of modules available that can help dynamically build cards, but this was the one I found to be most intuitive for me. While I will walk through an example here, I would also recommend reviewing their docs for additional context/examples.

You could also generate a JSON payload manually, or build one using the card designer & try to edit the JSON dynamically to change card values - but I won’t be doing that here.

So we’ll start off by installing the adaptive card module:

pip install adaptivecardbuilder

Before we build out the full card response, let’s take a quick look at how we can use this module.

In our execute() function, we’ll create a simple card that shows what zip code was entered.

First, we’ll need to add two new imports:

from webex_bot.models.response import Response
from adaptivecardbuilder import *

We’ll import the Response module from webex_bot, which will be required to send any attachments / custom response to the client. We’ll also import our adaptivecardbuilder module.

Next, we’ll modify the execute() function to look like the following:

    def execute(self, message, attachment_actions, activity):
        zip_code = attachment_actions.inputs['zip_code']
        
        card = AdaptiveCard()
        card.add(TextBlock(text=f"Weather for {zip_code}", size="Medium", weight="Bolder"))
        card_data = json.loads(asyncio.run(card.to_json()))

        card_payload = {
            "contentType": "application/vnd.microsoft.card.adaptive",
            "content": card_data,
        }

        response = Response()
        response.text = "Test Card"
        response.attachments = card_payload

        return response

First we create a new AdaptiveCard() instance. With this new object we can add different card elements.

In this case, we add a single card element - a TextBlock. This has multiple attributes, including the actual text content, and we’re specifying that we want a medium font size & bold text.

Next we serialize the card payload to JSON using asyncio.run.

In order to prepare the card object to be attached to our response, we will need to set the appropriate headers. In this case we need to wrap the JSON structure with the contentType header. We could use the response.attachments object to send any number of different file types to the user, but in this case we’ll set that value to application/vnd.microsoft.card.adaptive.

Then we can assemble the response to the client. We’ll create a new instance of the Response() object, where we’ll define some necessary attributes.

We’ll set our response.text first. This will be the text message sent to the client. If the client supports adaptive cards, then this message won’t be shown to the user - however, it may still appear in pop notifications. If the client does not support adaptive cards, only this message will be shown to the user.

Next we attach our complete card payload using response.attachments.

Last but not least, we can return that response object - which is then sent back to the user.

In this case, we can see the bot responds back to our request with a simple card that displays the zip code.

Let’s move onto building out the remainder of the card. For the purposes of this example, I’ll also be pulling out a few additional items from the OpenWeather API - including sunrise, sunset, pressure, and high/low temperatures.

I’ll go ahead and show the full card build here:

card = AdaptiveCard()
card.add(TextBlock(text=f"Weather for {city}", size="Medium", weight="Bolder"))
card.add(ColumnSet())
card.add(Column(width="stretch"))
card.add(FactSet())
card.add(Fact(title="Temp", value=f"{temperature}"))
card.add(Fact(title="Conditions", value=f"{conditions}"))
card.add(Fact(title="Wind", value=f"{wind}"))
card.up_one_level()
card.up_one_level()
card.add(Column(width="stretch"))
card.add(FactSet())
card.add(Fact(title="High", value=f"{high_temp}"))
card.add(Fact(title="Low", value=f"{low_temp}"))
card.add(Fact(title="Humidity", value=f"{humidity}"))
card_data = json.loads(asyncio.run(card.to_json()))

If this looks a little messy, it is. There is a cleaner way to do this, which I’ll show in a moment.

The first thing to know is that the card JSON structure is hierarchical. So individual card items need to be added under the correct hierarchy - for example, you create a ColumnSet first, then add a Column underneath, then the items within that Column last.

See the below visualization of this card:

Top Level
 ↳ Text Block
 ↳ ColumnSet
   ↳ Column
     ↳ Fact Set
       ↳ Fact
       ↳ Fact
       ↳ Fact
    ↳ Column
     ↳ Fact Set
       ↳ Fact
       ↳ Fact
       ↳ Fact

So on our card, we first placed the title TextBlock. Underneath that we created a ColumnSet, which will contain two Columns.

Each one of those Columns contains a FactSet, which is a key/value pair listing of information. And of course, we add each Fact pair underneath the FactSet.

You’ll notice after building one Column and FactSet, we use card.up_one_level() twice - to return back to the ColumnSet level, where we then add a second Column.

This can certainly get a little confusing at first, but I’ll show a more intuitive method in a moment.

Let’s take a peak at what this looks like now:

Sub-cards & Better Card Building

In this section we’ll do two quick things. First we’ll take a look at a simpler, more visual way of building the cards. Then we’ll also expand our example with sub-cards.

The adaptivecardsbuilder module supports a second way of building cards, which I personally find to be much easier to use.

In this method, we just create a single card.add() object - and feed it a list of all the items on the card.

For me, this makes it easier because you can add indentation to each list item - which helps keep track of where you are at in the hierarchical structure.

To navigate within the structure, we use "<" to go up one level, or "^" to return to the top level.

I’ve re-written the earlier example with this new structure below:

card = AdaptiveCard()
card.add(
    [
        TextBlock(text=f"Weather for {city}", size="Medium", weight="Bolder"),
        ColumnSet(),
            Column(width="stretch"),
                FactSet(),
                    Fact(title="Temp", value=f"{temperature}F"),
                    Fact(title="Conditions", value=f"{conditions}"),
                    Fact(title="Wind", value=f"{wind} mph"),
                    "<",
                "<",
            Column(width="stretch"),
                FactSet(),
                Fact(title="High", value=f"{high_temp}F"),
                Fact(title="Low", value=f"{low_temp}F"),
                "^",
        ActionSet(),
            ActionShowCard(title="More Details"),
                ColumnSet(),
                    Column(width="stretch"),
                        FactSet(),
                            Fact(title="Humidity", value=f"{humidity}%"),
                            Fact(title="Pressure", value=f"{pressure}"),
                            "<",
                        "<",
                    Column(width="stretch"),
                        FactSet(),
                            Fact(title="Sunrise", value=f"{sunrise}"),
                            Fact(title="Sunset", value=f"{sunset}"),
    ]
)
card_data = json.loads(asyncio.run(card.to_json()))

Again, you’re welcome to use whichever method suits you - but I find this method to be a bit cleaner & easier to read/troubleshoot.

You can also see that I’ve added a new ActionSet, with a single action: ActionShowCard. This adds a nice little button to our card where we can optionally display additional information.

Here’s what that looks like now:

And if we expand that sub-card:

Sub-cards can be an easy way to hide excess information to keep the response clean & focus on the important information.

Well that’s it for this blog post. There’s a lot more you can do with cards, including adding images, video, tables, dropdown menus, etc. But I just wanted to show a few examples of building cards. They can be a simple way to make chatbots a little less boring!

Enjoy!

I’ve been spending some of my time lately working on a new project that involves an interactive chatbot - where you can send commands to the bot & ask for it to take certain actions or return information.

I didn’t previously have a ton of experience building something like this. Mostly what I’ve done before is just using Webex or Discord APIs to send alerts or messages to a room.

As with anything new - I’ve stumbled my way through a couple of things, but have been enjoying the challenge of trying something different. Building out some of the more interactive parts of the bot hasn’t been quite as difficult as I originally imagined, and some of the newer modules & SDKs have made the process much simpler.

With all that being said - I wanted to throw together a quick guide on how to get started building a basic chatbot, in case anyone else is interested but worried that it may be too complicated.

Repo here for all the sample code used below

Step 1: Getting API Keys

So the first thing we’ll worry about is getting signed up for a Webex developer account & generating API keys.

So head on over to the Webex Developer site & log in (or sign up for a new account).

Once logged in, we’ll go to the upper-right corner of the screen and click on our user icon - and select “My Webex Apps” from the drop down:

Then we’ll be asked what type of app we’re creating. In this case, we’ll select bot:

Next we’ll need to provide some details about our bot. This includes a display name, username, image, and some details. Once the bot is completed, we have the ability to publish it via the Webex App Hub where anyone could interact with it. If you’re looking to do this, make sure you give a good description of your bot!

After we finish all that, we’ll finally get our API key!

Webhook vs Websocket: Which to use?

Next, I wanted to cover the two primary methods of sending & receiving messages from the Webex APIs: Webhooks and websockets.

Webhooks

Likely webhooks are the method most people are familiar with. Using this method, we need to run out bot code on a publicly reachable URL. When we run our bot, one of our first actions will be sending a request to the Webex APIs with our bot URL. This way, every time Webex receives a message from a user that wants to interact with our bot, the Webex cloud knows where to send that chat message.

Now, the potential downside of using webhooks is that we need to 1) manage a web server and 2) expose the bot publicly to the internet. Both of these could be overcome with easy workarounds, if needed. For example, we could use something like FastAPI to quickly build our web listener & handle incoming POST requests. We could also use a tunneling method (like ngrok) to avoid having to directly open ports to our webserver.

Websockets

On the other hand, we could use websockets to accomplish the same function. With a websocket, we’re instead opening a direct, persistent TCP connection to the Webex APIs. Through this persistent connection, we can send & receive messages very quickly and easily.

This method offers a few advantages over webhooks. Primarily not having to expose our app publicly to the internet. Websockets act almost similar to something like ngrok, where we create an outbound tunnel - except in this case, only Webex knows about it and has access to it (where ngrok is still providing you a public URL that anyone could hit, if they knew about it). Additionally, this method removes the need for maintaining a web server & having to handle all of the incoming HTTP requests in your bot’s code. There may also be a handful of API calls which are only supported via websockets today as well - like having your bot show that they’ve read a message, for example.

I’ve also found that the websocket method seems to be much more responsive, likely because there is a persistent TCP connection open - and you no longer have to rely on a full HTTP session establishment & teardown for every message to the bot.

Historically, websockets were only officially supported via the Webex JavaScript SDK. However, it seems recently that someone has built a Python equivalent earlier this year - which we’ll be using throughout this blog post. You can find that package here

Getting started: Registering the Bot with Webex

So we’ll start with some basic functionality. Let’s grab the modules we need, apply our API key, and see if we can get a response from our bot.

Luckily, the module we’re using makes this super easy. So let’s install via pip:

pip install webex_bot

Once that’s installed, we’ll create a new Python file - I’ll call mine bot.py

In that file, we’ll start our script like this:

from webex_bot.webex_bot import WebexBot

api_token = "<TOKEN_HERE>"

bot = WebexBot(api_token, approved_domains=["0x2142.com"])

bot.run()

Looks easy enough, yeah? We’ll import our webex_bot module first, as always. Then we’ll define our API key. Obviously this can be stored as an environmental variable or another more secure location, but we’ll define it here for demonstration.

Then we create a new instance of WebexBot, by providing our API key and an optional approved_domains value. In my case, I’ve provided the 0x2142.com domain - which means the bot will reject messages from anyone who isn’t from that organization.

Last, we start our bot with bot.run().

Let’s go ahead and run our bot with python bot.py and see what happens!

You should see something similar to this:

matt@0x2142:~/demo-webex-bot$ python bot.py
2021-10-25 15:42:52  [INFO]  [webex_bot.webex_bot.webex_bot.__init__]:39 Registering bot with Webex cloud
2021-10-25 15:42:52  [INFO]  [restsession.webexteamssdk.restsession.user_agent]:167 User-Agent: webexteamssdk/0+unknown {"implementation": {"name": "CPython", "version": "3.8.10"}, "system": {"name": "Linux", "release": "5.10.16.3-microsoft-standard-WSL2"}, "cpu": "x86_64"}
2021-10-25 15:42:52  [WARNING]  [command.webex_bot.models.command.__init__]:33 no card actions data so no entry for 'callback_keyword' for echo
2021-10-25 15:42:52  [INFO]  [command.webex_bot.models.command.set_default_card_callback_keyword]:47 Added default action for 'echo' callback_keyword=callback___echo
2021-10-25 15:42:53  [INFO]  [webex_bot.webex_bot.webex_bot.get_me_info]:74 Running as bot '0x2142 Bot' with email ['0x2142@webex.bot']
2021-10-25 15:42:54  [INFO]  [webex_websocket_client.root._connect_and_listen]:151 Opening websocket connection to wss://mercury-connection-partition2-a.wbx2.com/v1/apps/wx2/registrations/06b5c858-174a-4977-a268-04530d777563/messages
2021-10-25 15:42:54  [INFO]  [webex_websocket_client.root._connect_and_listen]:154 WebSocket Opened.

So long as we continue to run this script in the foreground - we’ll be able to see the live log of anything that happens, including messages to our bot. So for now, we’ll leave this up & running.

Now we should be able to try sending our bot a message in Webex.

In the Webex app, if we do a quick search for our bot (by email or name), we should see it pop up pretty quickly:

With our new space, we can interact with our bot. I’ll send a “Hello” message:

And the webex_bot module will automatically respond back with a built-in help card.

Currently the bot only supports one command: echo. We’ll add our own commands shortly, but for now let’s give that a try:

So in the example above, I used the echo command. The bot returns a card with an input field. Once you fill out the textbox & hit submit, the bot will echo back whatever text was provided!

So now we have a running bot. But what about implementing our own commands? Let’s take a look!

Creating custom bot commands

Now we get to the fun part!! To demonstrate, let’s add a command to ask our bot for the current weather conditions. To accomplish this, we’ll use the OpenWeather APIs.

So to begin - We’ll probably want a separate Python file for each command we want to add. First we’ll add weather.py to our current directory.

Next, we’ll throw in a bit of boilerplate code that will be used for each command:

from webex_bot.models.command import Command
import logging

log = logging.getLogger(__name__)

class WeatherByZIP(Command):
    def __init__(self):
        super().__init__(
            command_keyword="weather",
            help_message="Get current weather conditions by ZIP code.",
            card=None,
        )

    def execute(self, message, attachment_actions, activity):
        return f"Got the message: {message}"

So first we’ll import Command from our webex_bot module, which will be used to create our custom command class.

Next we create a new class, in this case WeatherByZIP, and inherit the Command class from webex_bot. Within this new class, we’ll create our __init__ function & define some information about our command.

So using super().__init__(), we’ll provide what keyword should trigger this command, any help message to be provided to the user, and an optional card. We saw how the card could work earlier with the echo command - but to keep this module simple, we’ll go without a card for now.

After that - we just need to create an execute function. This is what gets called by webex_bot when our command is triggered. By default, webex_bot will pass us the user’s message - and, in the case of an action (like a card submission), it will provide those details.

Two things to keep in mind with our execute function:

1. the message variable will contain the user’s message with the command keyword removed. So in our case, if the user’s message was “weather 12345” - then the message variable would just contain “12345”.
2. Any text we return from our function will be sent via the bot as a chat response. If we wanted to provide extras, like cards or attachments, we would need to use the webex_bot Response object. I might cover this in a separate blog post later

Okay! So right now our new command is configured to just echo back any test that is sent to the bot - similar to the echo command earlier, except without the card.

We’ll add our weather functionality shortly - but let’s first add our command to the bot & give it a quick test.

In our bot.py file, we’ll just need to import our command module and call bot.add_command():

from webex_bot.webex_bot import WebexBot
from weather import WeatherByZIP

api_token = "<TOKEN_HERE>"

bot = WebexBot(api_token, approved_domains=["0x2142.com"])

bot.add_command(WeatherByZIP())

bot.run()

That’s it! Now we should be able to try it out. We’ll restart our bot, and send the command weather 12345:

Sure enough, our bot gets the message, passes it to our custom command, and returns the stripped message - all as expected.

So let’s add our weather query & actually make this thing work.

Using the OpenWeather APIs, I’ll create a new API key - then create a variable in the command module called OPENWEATHER_KEY. Then, I’ve updated my execute function to look like this:

def execute(self, message, attachment_actions, activity):
    # Message may include spaces. Strip whitespace:
    zip_code = message.strip()

    # Define our URL, with requested ZIP code & API Key
    url = "https://api.openweathermap.org/data/2.5/weather?"
    url += f"zip={zip_code}&units=imperial&appid={OPENWEATHER_KEY}"

    # Query weather
    response = requests.get(url)
    weather = response.json()

    # Pull out desired info
    city = weather['name']
    conditions = weather['weather'][0]['description']
    temperature = weather['main']['temp']
    humidity = weather['main']['humidity']
    wind = weather['wind']['speed']

    response_message = f"In {city}, it's currently {temperature}F with {conditions}. "
    response_message += f"Wind speed is {wind}mph. Humidity is {humidity}%"

    return response_message

So because of the way that message is passed to us, we’ll first need to strip whitespace. This is because when the user enters a command like “weather 12345”, webex_bot will only remove the “weather” portion - leaving us with " 12345". So by stripping that value, we’re left with just the numeric ZIP Code: “12345”.

Next we assemble our URL, supplying our parameters: zip, units, and appid. Zip is the numeric ZIP code provided by our user, units will be either imperial or metric, and appid is our API key.

After that, it’s as simple as sending the GET request & parsing the data. In the code above, I assemble the final text response into response_message which is then returned to the user.

All that being done - Let’s try it out!!

In this example, I used the ZIP code for Richfield, OH - and our bot quickly returned that information to us.

And that’s it! In just a few steps we created a new Webex bot & added custom commands. Once we have that process down, it’s easy enough to continue extending our bot by adding additional commands.

There are obviously more advanced use cases - and a whole lot of fun that comes with using Adaptive Cards… But I hope this post was helpful in getting you started!

Update 2022/03/12 - If you’re interested in seeing additional content around Cards, please check out the new blog post & video here

I’ve been getting a handful of questions lately on the process of bringing a Cisco Catalyst 8000v or CSR 1000v into an SD-WAN environment. So I figured maybe I should put something together to share.

On a related note - I’ve been debating for a while doing a blog and/or video on building out a Cisco Viptela SD-WAN lab in EVE-NG. This would (tentatively) include everything from building controllers, bringing up remote sites, and template/policy configs. If this is something you might get value from, please let me know!! I’m looking for some motivation :)

Okay - all that being said, let’s go ahead and get started.

Note: This guide should work with CSR 1000v devices as well. But it will NOT be 100% accurate for physical ISR/IOS-XE routers, as there are some additional steps with certificates & the Plug and Play portal to get those running.

Topology

So to start with, figured I would share the topology that I’m working from. If you read my last post you might recognize this, but with an added location. This new location (site id 400) contains our Catalyst 8000v VM, running IOS-XE version 17.04.01a.

Controller or Autonomous Mode?

Back in the earlier days of IOS-XE SD-WAN, there used to be two separate software images to load on your network appliance - one for traditional IOS-XE, and one for SD-WAN code.

With the newer releases of IOS-XE, we’re now getting a unified image that contains both software sets. So our options now are two modes: autonomous (traditional IOS-XE) or controller (SD-WAN).

One way we can check this, is by running a show version and looking for Router operating mode:

Router# show version
<-- Output omitted -->
cisco C8000V (VXE) processor (revision VXE) with 2035355K/3075K bytes of memory.
Processor board ID XXXXXXXXXXX
Router operating mode: Autonomous
4 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
3965744K bytes of physical memory.
5234688K bytes of virtual hard disk at bootflash:.

Configuration register is 0x2102

Okay, and looks like the Catalyst 8000v image I’m using booted up in autonomous mode. No big deal, we can change modes pretty easily!

So in normal exec-mode, we’ll use the command controller-mode enable:

Router# controller-mode enable
Enabling controller mode will erase the nvram filesystem, remove all configuration files, and reload the box!
Ensure the BOOT variable points to a valid image
Continue? [confirm]

As noted in the snippet above - this command will erase the config! So you will want to take a backup snapshot of your existing configuration, if this is already being used. In my case, it’s a brand new VM - so we’ll continue on.

Once the device is back online, we’ll log in with the default login of admin/admin. Note that you will be forced to change this upon first login.

User Access Verification

Username: admin
Password:

Default admin password needs to be changed.

Enter new password:
Confirm password:
Router#

And just for fun, we’ll run a show version again to ensure we’re in controller mode:

Router# show version 
<-- Output omitted -->
cisco C8000V (VXE) processor (revision VXE) with 2035355K/3075K bytes of memory.
Processor board ID XXXXXXXXXXX
Router operating mode: Controller-Managed
4 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
3965756K bytes of physical memory.
5234688K bytes of virtual hard disk at bootflash:.

Configuration register is 0x2102

Catalyst 8000v Initial Config

So since this is a lab environment, and I’m not trying to provide any Day0 provisioning files - I’ll have to complete some manual configuration to get started.

First, we’ll start with some SD-WAN specific config (site id, org name, etc). I’ll be using the values that apply to my lab, so be sure to change these in yours!

Note: If you haven’t used IOS-XE SD-WAN previously, be aware that “conf t” doesn’t work! In SD-WAN/controller mode, you’ll use “config-transaction”. In this mode, all changes will need to be committed before they’re applied to the device.

Router# config-transaction
 ! Set hostname
Router(config)# hostname Cat8k-Site400
 ! Required SD-WAN system configs
Router(config)# system
Router(config-system)# organization-name "SDWAN-LAB"
Router(config-system)# site-id 400
Router(config-system)# vbond 192.168.99.241
Router(config-system)# system-ip 10.10.10.237
Router(config-system)# commit
Commit complete.

Cat8k-Site400# 

Once we have those basics configured, at a minimum we’ll also need to configure our internet-facing tunnel interface. This allows our Catalyst 8000v to communicate with our control plane for bring-up.

Cat8k-Site400#config-transaction
 ! Config physical interface (This is a lab, so I'm using a static IP)
Cat8k-Site400(config)# interface GigabitEthernet1
Cat8k-Site400(config-if)# ip address 192.168.99.237 255.255.255.0
Cat8k-Site400(config-if)# no shut
 ! Config tunnel interface
Cat8k-Site400(config-if)# interface Tunnel1
Cat8k-Site400(config-if)# no shut
Cat8k-Site400(config-if)# ip unnumbered GigabitEthernet1
Cat8k-Site400(config-if)# tunnel source GigabitEthernet1
Cat8k-Site400(config-if)# tunnel mode sdwan
Cat8k-Site400(config-if)# exit
 ! SD-WAN tunnel config
Cat8k-Site400(config)# sdwan
Cat8k-Site400(config-sdwan)# interface GigabitEthernet1
Cat8k-Site400(config-interface-GigabitEthernet1)# tunnel-interface
Cat8k-Site400(config-tunnel-interface)# encapsulation ipsec
Cat8k-Site400(config-tunnel-interface)# color biz-internet
Cat8k-Site400(config-tunnel-interface)# exit
 ! Default route to our internet gateway
Cat8k-Site400(config)# ip route 0.0.0.0 0.0.0.0 192.168.99.1
Cat8k-Site400(config)# commit
Commit complete.

Cat8k-Site400#

A Note on Certificates

Since this is a lab environment, I’m using self-signed local certificate authority to provision all of my certificate infrastructure. Because of this, I’ll need to install my local CA certificate on the Catalyst 8000v. If you’re using the default Cisco-provisioned certificate setup, you won’t need to do this.

Since my SD-WAN lab doesn’t have direct access to my local TFTP server - I do have an out-of-band management interface connected to my Cat 8000v. We’ll start by configuring that interface:

Cat8k-Site400# config-transaction
Cat8k-Site400(config)# vrf definition 512
Cat8k-Site400(config-vrf)# address-family ipv4
Cat8k-Site400(config-vrf)# exit
Cat8k-Site400(config)# interface GigabitEthernet 3
Cat8k-Site400(config-if)# vrf forwarding 512
Cat8k-Site400(config-if)# ip address dhcp
Cat8k-Site400(config-if)# exit
Cat8k-Site400(config)# ip tftp source-interface GigabitEthernet 3
Cat8k-Site400(config)# commit

The standard management VRF/VPN for SD-WAN is 512, so I kept that config to match when I configured this management interface. This will all be over-written anyways once we get connected to vManage & configure/push our template configs.

Once that’s done, we can go ahead and copy our CA certificate to bootflash.

Cat8k-Site400# copy tftp://10.0.0.2/cacert.pem bootflash:
Destination filename [cacert.pem]?
Accessing tftp://10.0.0.2/cacert.pem...
Loading cacert.pem from 10.0.0.2 (via GigabitEthernet3): !
[OK - 1406 bytes]

1406 bytes copied in 0.112 secs (12554 bytes/sec)

Cat8k-Site400# dir bootflash: | inc cacert.pem
16      -rw-             1406  May 14 2021 14:41:27 +00:00  cacert.pem
```text

Then we'll use the command below to install the CA certificate:

```text
Cat8k-Site400# request platform software sdwan root-cert-chain install bootflash:cacert.pem
Uploading root-ca-cert-chain via VPN 0
Copying ... /bootflash/cacert.pem via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain

And we can validate by using the show sdwan certificate root-ca-cert command:

Cat8k-Site400 show sdwan certificate root-ca-cert
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            11:3b:0a:12:17:b0:e0:b5:4b:fa:c2:e9:2c:9c:12:84
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: DC = local, DC = 0x2142, O = SDWAN-LAB, CN = 0x2142-0XWIN1-CA-2
        Validity
            Not Before: Nov 29 20:00:11 2018 GMT
            Not After : Nov 29 20:10:11 2043 GMT
        Subject: DC = local, DC = 0x2142, O = SDWAN-LAB, CN = 0x2142-0XWIN1-CA-2
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
        <-- Output omitted -->

Activating the Catalyst 8000v

Almost there! Now that we’ve finished our pre-config & added our root CA certificate - we’re ready to join the Catalyst 8000v to the SD-WAN fabric.

We’ll start over in vManage - by going to Configuration > Devices.

Then we’ll find our target, unused Catalyst 8000v device. Click the ellipsis on the right side, then select Generate Bootstrap Configuration

This will give us a prompt to select which configuration style to generate. We’ll leave this on “Cloud-init”:

Once we hit okay - we’ll be presented with the info we need. We won’t necessarily need all of this information, but we’ll want to take note of our uuid and otp:

We’ll drag this info back over to our Catalyst 8000v, and we can now use it to activate the device & join to our SD-WAN fabric.

For the command below, chassis-number will be our uuid value - and token will be our otp.

Cat8k-Site400# request platform software sdwan vedge_cloud activate chassis-number C8K-178BXXXX-XXXX-XXXX-XXXX-XXXXXXXXBC24 token 421ecxxxxxxxxxxxxxxxxxxxxxbd53b5

Validation

After a few moments, we’ll see some log messages start to appear showing our control connections coming up. You might see these on the terminal if you’re using the console port, or you can use show log:

*May 14 15:39:06.910: %Cisco-SDWAN-Cat8k-Site400-OMPD-3-ERRO-400002: R0/0: OMPD: vSmart peer 10.10.10.242 state changed to Init
*May 14 15:39:07.980: %DMI-5-AUTH_PASSED: R0/0: dmiauthd: User 'vmanage-admin' authenticated successfully from 10.10.10.240:48140 and was authorized for netconf over ssh. External groups:
*May 14 15:39:08.798: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400002: R0/0: OMPD: vSmart peer 10.10.10.242 state changed to Handshake
*May 14 15:39:08.804: %Cisco-SDWAN-Cat8k-Site400-OMPD-5-NTCE-400002: R0/0: OMPD: vSmart peer 10.10.10.242 state changed to Up
*May 14 15:39:08.808: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400005: R0/0: OMPD: Number of vSmarts connected : 1
*May 14 15:39:09.827: %Cisco-SDWAN-Cat8k-Site400-OMPD-3-ERRO-400002: R0/0: OMPD: vSmart peer 10.10.10.243 state changed to Init
*May 14 15:39:10.584: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
*May 14 15:39:11.756: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400002: R0/0: OMPD: vSmart peer 10.10.10.243 state changed to Handshake
*May 14 15:39:11.762: %Cisco-SDWAN-Cat8k-Site400-OMPD-5-NTCE-400002: R0/0: OMPD: vSmart peer 10.10.10.243 state changed to Up
*May 14 15:39:11.762: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400005: R0/0: OMPD: Number of vSmarts connected : 2
*May 14 15:39:12.738: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400007: R0/0: OMPD: Using policy from peer 10.10.10.242
*May 14 15:39:13.570: %Cisco-SDWAN-Cat8k-Site400-FTMD-6-INFO-1000020: R0/0: FTMD: SLA class added : class 'Default' at index '1' loss = 25%, latency = 300ms, jitter = 100ms, app-probe-class = None
*May 14 15:39:14.738: %Cisco-SDWAN-Cat8k-Site400-OMPD-6-INFO-400007: R0/0: OMPD: Using policy from peer 10.10.10.242

We can also see our control connections using the command show sdwan control connections:

Cat8k-Site400# show sdwan control connections
                                                                                       PEER                                          PEER                                          CONTROLLER
PEER    PEER PEER            SITE       DOMAIN PEER                                    PRIV  PEER                                    PUB                                           GROUP
TYPE    PROT SYSTEM IP       ID         ID     PRIVATE IP                              PORT  PUBLIC IP                               PORT  ORGANIZATION            LOCAL COLOR     PROXY STATE UPTIME      ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart  dtls 10.10.10.242    100        1      192.168.99.242                          12446 192.168.99.242                          12446 SDWAN-LAB  biz-internet    No    up     0:00:00:51  0
vsmart  dtls 10.10.10.243    100        1      192.168.99.243                          12446 192.168.99.243                          12446 SDWAN-LAB  biz-internet    No    up     0:00:00:49  0
vmanage dtls 10.10.10.240    100        0      192.168.99.240                          12646 192.168.99.240                          12646 SDWAN-LAB  biz-internet    No    up     0:00:00:52  0

Of course, we can also check to see our device status in the vManage dashboard as well.

Over on the Monitor > Network page, we can see that our new Catalyst 8000v is now online:

Extra: How do I check the routing table on an IOS-XE SD-WAN device?

So - if you’ve only used the vEdge software devices, you may be used to using the show ip route or show ip route vpn 10 commands.

In the IOS-XE world, most SD-WAN commands are prefixed with the sdwan keyword. For example: show sdwan bfd sessions (where on vEdges, it would just be show bfd sessions)

This might lead you to believe that you can use show sdwan ip route or show sdwan ip route vrf 10 - but these won’t work! In fact, you’ll get the following message:

Cat8k-Site400# show sdwan ip route
% Error: This command is not supported

For the IOS-XE based devices, they actually just use the standard IOS-XE routing table and VRF constructs.

So on a vEdge, you would have VPN 0 as your transport VPN. On IOS-XE, this is just the default global routing table, shown with show ip route.

But what about our LAN-side service VPNs? In this case, our routes are being dumped into a VRF on the IOS-XE device.

So for example, I have VPN 10 in my lab which is used for LAN-side clients. We can use the show vrf command to see that this exists, and then show ip route vrf 10 to see the routes from our other SD-WAN locations:

Cat8k-Site400# show vrf
  Name                             Default RD            Protocols   Interfaces
  10                               <not set>             ipv4        Gi2
  512                              <not set>             ipv4        Gi3
  65528                            <not set>             ipv4        Lo65528

Cat8k-Site400# show ip route vrf 10

Routing Table: 10
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
       n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       H - NHRP, G - NHRP registered, g - NHRP registration summary
       o - ODR, P - periodic downloaded static route, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR
       & - replicated local route overrides by connected

Gateway of last resort is not set

      10.0.0.0/24 is subnetted, 2 subnets
m        10.2.2.0 [251/0] via 10.10.10.235, 00:09:02, Sdwan-system-intf
m        10.3.3.0 [251/0] via 10.10.10.236, 00:09:02, Sdwan-system-intf

Okay, that’s it! Pretty quick process overall - and now we can get into applying our device/feature templates.

Hope this was helpful!!

In this post, we’ll walk through configuring a Cisco Viptela SD-WAN network to integrate with Cisco Umbrella’s Secure Internet Gateway (SIG) & Secure Web Gateway (SWG).

If you’re interested in reading more about what SIG & SWG are, please check out my last post where I walked through this integration with a Meraki MX firewall.

For additional reading, there’s also a good Umbrella blog post on all the components of Cisco’s SASE architecture, which you can find here.

Okay, with that being said - let’s get started!

Step 1: Umbrella API Keys

Okay, so we’ll start on the Umbrella side. We’ll need to generate a set of API keys, which we’ll push out to our WAN edge devices. These API keys will allow our remote devices to reach out to Umbrella & auto-configure their SIG tunnels.

We’ll log into our Umbrella dashboard at https://login.umbrella.com/

Once we log in, we’ll hop over to Admin > API Keys. Then up in the upper-right corner, click Create.

For this integration, we’ll need to select Umbrella Management to make sure our API keys have the access they need. Then click Create. Easy enough, right?

Once we do that, the Umbrella dashboard will display our API keys - which we’ll need to copy over to our Viptela SD-WAN environment.

We’ll also need to collect our Umbrella Organization ID. This is actually just embedded within the Umbrella Dashboard URL, and should be a seven-digit number. For example, if we look at our current URL on the API keys page:

https://dashboard.umbrella.com/o/<ORG ID>/#/admin/apikeys

And that’s all we need for now from the Umbrella side. So let’s hop over to Viptela.

Step 2: Viptela Templates & Config

For this post, I’ll be using my SD-WAN lab in EVE-NG. Currently I just upgraded the lab to run Viptela version 20.4.1, though the SIG integration has been supported since 20.1.

Just for reference, here is the lap topology that I’m working with:

This lab includes a bridged connection to allow direct internet access from the lab network.

To configure our SIG automatic tunnels, we’ll need to create / update a few templates:

• Create a SIG Credentials feature template
• Create a SIG feature template
• Assign SIG templates to device templates
• Edit Service-side VPN Template to inject a service route

2a: Creating a SIG Credentials Template

First we’ll create a template which will store our Umbrella API credentials.

In the vManage dashboard, we’ll go to Configuration > Templates, then drop into the Feature tab, and click on Add Template.

Once we get to the next screen, we’ll select our device type. In my case, I’m using a vEdge Cloud.

A list of templates will pop up - we’ll drop down to SIG Credentials, which will be near the bottom under the Other Templates header.

We’ll then be taken to the page where we create our template.

We’ll fill in our template name & description, then paste in our Umbrella details (Org ID, API Key, API Secret).

Once we’re done - We’ll hit Save.

Note: At time of writing, the “Get Keys” button only functions if you’ve purchased your SD-WAN subscription with DNA Premier licensing. This license level includes Umbrella SIG, and allows this 1-click integration that pulls your Umbrella API keys automatically.

2b: Creating a SIG Tunnel Template

Okay, next we’ll need to create another feature template to specify our IPSec tunnel configuration.

Just like before, we’ll head back over to Configuration > Templates > Feature > Add Template.

This time, after selecting our device type, we’ll choose Secure Internet Gateway (SIG) / WAN - which is located under the VPN header.

In the template configuration, we’ll give the template a name & description as always. Then we’ll jump down to the tunnel config.

We’ll select Umbrella for SIG Provider, then click Add Tunnel.

Within the tunnel config, we’ll specify an interface name - I’ll name mine ipsec1 (and I’ll create an ipsec2 shortly). We’ll also specify a Tunnel Source, which in my lab is ge0/0 for the biz-internet VPN 0 interface.

We’ll keep Data-Center as Primary, then click Add. Then - we’ll add a second tunnel configuration, but using Data-Center as Secondary and the interface name as ipsec2.

When all that is done, we should have the following:

If we scroll down to the bottom of the template config, we’ll have some settings for High Availability. Here, we’ll specify what our active & backup IPSec tunnels are, and their weights. I’ll specify ipsec1 as active & ipsec2 as backup. Then click Save to finish our template.

Note: Weight settings are only available starting with 20.4.1 & allow for ECMP routing to SIG. Not shown above, you can create up to four HA tunnel pairs. Depending on weighting, you can equal-cost or unequal-cost load balance between those pairs.

Why would you want to load balance across multiple tunnels? At time of writing, each IPSec tunnel is limited to a maximum 250Mb/s throughput. By creating multiple tunnels & load balancing, we can overcome this limitation if we need higher bandwidth.

2c: Attaching SIG to Device Templates

After we’ve built out our two SIG templates, we can now attach them to our device templates.

For me, I currently only have a single device template which is applied to all of my remote WAN edge devices.

So we’ll go back to Configuration > Templates and find whichever device template we want to use - then click the ellipsis on the far right, end select Edit.

In the device template, we’ll scroll down and look for our VPN 0 configuration under Transport & Management VPN. We’ll attach our SIG tunnel template to VPN 0, since that’s where those IPSec tunnels are being sourced from.

On the right side, under Additional VPN 0 Templates, we’ll click Secure Internet Gateway to add our template. Then from the drop-down, we can select the feature template we just created:

We’ll also include our SIG credentials template, which we can find at the bottom of the page, under Additional Templates:

Once we’re done, we can click Save at the bottom. This will take us through the process of pushing out these changes to our remote WAN edge devices. When this configuration is deployed, each vEdge will now reach out to Umbrella & auto-configure an IPSec tunnel.

Note: While the IPSec tunnels will be established when this configuration is pushed - no traffic will flow over them yet. We’ll need to add a service route for that, which we’ll do next!

2d: Injecting a Service Route

Now we have our IPSec tunnels deployed - all we need to do is start routing traffic out to Umbrella. We’ll accomplish this using a service route on our LAN-side VPN.

So we’ll go over to Configuration > Templates > Feature - and we’ll scroll down & find whichever template we’re currently using on our LAN side. For me, I want VPN 10 to route over the tunnels, so I’ll be editing my VPN 10 template.

Within the template, we’ll scroll down to the Service Route section, click New Service Route, and we’ll enter our desired prefix. This will be what traffic will be routed over the IPSec tunnel to Umbrella. Since this is intended to be an internet gateway, I’ll enter 0.0.0.0/0 to inject a default route to Umbrella.

Service should be pre-selected as SIG, so we’ll click Add, then Update & deploy our changes to the remote edge devices.

Step 3: Validation & Troubleshooting

Awesome! Now we should have everything configured & working. So let’s jump through some initial testing and validation we can do.

Within vManage, we can check the status of our IPSec tunnels. We’ll go over to Monitor > Network then select one of our WAN edge devices.

We’ll click on the Interfaces tab on the left side - and we should be able to see a list of all interfaces on our device. This should include an ipsec1 & ipsec2 interface.

I’ve also selected the Real Time monitor on mine, and filtered to just show traffic on the ipsec1 interface:

You might recall from my topology above, that I have a linux VM running behind each of the two vEdge Cloud appliances. Using these, I can also test web access & see what my external IP is:

And sure enough, I am getting a 146.112.x.x address - which belongs to the Umbrella datacenter.

If we log into one of our vEdge devices, we can check the routing table with show ip route vpn 10 (or whichever VPN you’re using for the LAN-side). We should see our default 0.0.0.0/0 route via ipsec1, with a next-hop-VPN of VPN0:

vEdge-01# show ip route vpn 10

     ADDRESS               PATH             PROTOCOL          NEXTHOP  NEXTHOP                         NEXTHOP
VPN  FAMILY   PREFIX       ID    PROTOCOL   SUB TYPE  METRIC  IFNAME   ADDR     TLOC IP  COLOR  ENCAP  VPN      STATUS
------------------------------------------------------------------------------------------------------------------------
10   ipv4     0.0.0.0/0    0     std-ipsec  -         0       ipsec1   -        -        -      -      0        F,S
10   ipv4     10.2.2.0/24  0     connected  -         0       ge0/2    -        -        -      -      -        F,S

We can also check specifically our SIG tunnel status using the command show secure-internet-gateway tunnels:

vEdge-01# show secure-internet-gateway tunnels

TUNNEL                                                        API   LAST
IF                                                            HTTP  SUCCESSFUL     TUNNEL
NAME    TUNNEL ID  TUNNEL NAME                     FSM STATE  CODE  REQ            STATE
-------------------------------------------------------------------------------------------
ipsec1  530233543  SITE200SYS10x10x10x235IFipsec1  st-tun-up  200   create-tunnel  -
ipsec2  530233542  SITE200SYS10x10x10x235IFipsec2  st-tun-up  200   create-tunnel  -

In that output above, we’ll see that we have both ipsec1 & ipsec2 tunnels shown. The last API queries to Umbrella have a HTTP 200, which is good. We’ll also see our tunnel names, which we can use to find our device tunnel configuration in the Umbrella dashboard.

The tunnel name might look like a mess at first, but it’s a unique identifier to represent each tunnel that was created. So if we break it down for this vEdge:

• This vEdge is at Site ID 200
  • Shown as “SITE200”
• This vEdge has a system IP of 10.10.10.235
  • Shown as “SYS10x10x10x235”
• This vEdge has two IPSec interfaces, ipsec1 & ipsec2
  • Shown as “IFipec1” and “IFipsec2”

If we jump over to the Umbrella dashboard, we’ll see the same. Within the Umbrella dashboard, we can jump to Deployments > Network Tunnels.

On this page, we should see a total of four tunnels listed (two from each vEdge appliance):

And with everything configured & validated - now we can move onto configuring firewall and web filtering policies within Umbrella!

If you’re interested in seeing how to configure Umbrella’s cloud firewall & web filtering policies - please check out my YouTube Video above!

In today’s blog post - we’ll be checking out how to integrate a Meraki MX firewall to Cisco’s Umbrella Secure Internet Gateway (SIG) & Secure Web Gateway (SWG) services.

Interested in seeing how to do this with Cisco Viptela SD-WAN? Check out this post

Note: This integration is still rather new - so I anticipate some of the screenshots & steps below may differ as this post ages.

What are SIG & SWG?

Good question! A lot of us are probably familiar with Cisco Umbrella (formerly OpenDNS) for the DNS-layer security products. Using Cisco Umbrella, we can configure our end PCs or DNS forwarders to use the Cisco DNS servers. Then, we apply security policies to allow/deny DNS queries based on our policy.

Now applying security policy at the DNS level is great! Typical web filtering only inspects HTTP/HTTPS traffic, but inspecting DNS traffic allows us to stop threats that might not use those typical ports. While there may be some applications or malware that use hard-coded IP addresses, a good majority use a domain name that requires a DNS lookup.

But what if we wanted to still do web filtering too? Or maybe we want a centralized, cloud-hosted firewall service? That’s where Umbrella SIG & SWG come in.

Secure Web Gateway (SWG) is pretty much exactly what it sounds like. It’s a cloud-hosted web filter or web proxy, where we can set URL-based policys to permit or deny access. It supports the usual allow-lists, deny-lists, HTTPS inspection, file inspection, and policies based on user/group identities.

Secure Internet Gateway (SIG) is a large, cloud-hosted firewall service. What we can do with SIG is tunnel all of our traffic to one centrallized location to apply firewall policies (permit/deny/etc).

Why would we want either of these functions to be cloud-hosted? Well we might not have the resources at every remote site to perform firewalling & web filtering - or we don’t want to invest in beefy hardware in our corporate datacenter, and backhaul all of our remote traffic back for inspection. The cloud-hosted piece also means a central management point, so only one place to make change for all of our remote sites - rather than having to touch dozens or hundreds of remote devices…..Did someone say SASE? 🙃

With all that said - Let’s get into making this work!

Note: There are many ways to use Umbrella SIG/SWG (IPSec tunnel, PAC file, Anyconnect, etc). This particular post will only cover IPSec via Meraki MX.

Step 1: Getting our Keys

First thing we’ll need to do is on the Umbrella side. We’ll need to generate tunnel keys for our Meraki MX to use for IPSec negotiation.

We’ll log into our Umbrella dashboard at https://login.umbrella.com/

Once we’re in, we’ll navigate to Deployments > Core Identities > Network Tunnels. Shown below, we currently have no tunnels configured:

In the upper right corner, let’s click the Add Tunnel button.

We’ll now see the following screen to begin creating our IPSec tunnel:

Here, we’ll need to specify a name for our tunnel & which device type. The name can be anything we choose that helps us identify what locations are using this tunnel. I’ve specified MX64 as my tunnel name, since I only have one MX that I’ll be testing this with. We also have Meraki MX as an option in the Device Type drop down - so we’ll select that as well. Then, we’ll click Next.

Then we’ll need to configure our Tunnel ID and Passphrase:

The Tunnel ID we’ll be sending later, as part of our IPSec local ID. The passphrase will be used as our pre-shared key for the tunnel config. Once we’re done with that, click Save.

As long as our ID & passphrase are all good, we’ll be presented with a box to easily copy these parameters into the Meraki config:

Finally, we’ll be dropped back to our Network Tunnels page - where we can see that our tunnel is configured, but not yet established. Let’s fix that & hop over to the Meraki dashboard!

Step 2: Meraki MX IPSec Configuration

Okay, now onto the fun stuff.

We’ll log into our Meraki Dashboard at https://account.meraki.com/secure/login/dashboard_login

Once we’re in, you’ll need to select the network with the MX you want to connect. For me, I currently only have a single network, which contains my MX firewall.

Then we’ll navigate to Security & SD-WAN > Configure > Site-to-site VPN.

Assuming we have no other VPNs configured, you’ll have to change the VPN Type to Hub (Mesh) or Spoke. In my case, Umbrella SIG will be the only VPN I’ll have configured - so I’ll go ahead and use Hub (Mesh). Don’t mind the error regarding exit hubs, as we’ll be configuring a non-Meraki peer below.

Okay, scrolling down the page just a bit - we’ll need to specify which VLANs/internal subnets will be routed across the VPN & pushed through Umbrella for SIG/SWG:

For testing purposes, I have a VM in a subnet labeled DMZ that I’ll be using - So that will be the only VLAN that I’ll set to VPN On.

Next we’ll take a look at configuring Non-Meraki VPN Peers.

As shown in the screenshot below, we’ll need to configure the following settings:

• Name - This is locally significant, I set mine to Umbrella_SIG
• IKE Version - Set this to IKEv2
• IPSec Policies - Custom - See below
• Public IP - This is the peer Umbrella Datacenter
  • Choose the DC closest to you here
  • For this post, I’m using the New York DC
• Local ID - Set this to the Tunnel ID we got from Umbrella
• Remote ID - Leave blank
• Private Subnets - This is what destination IPs will be routed over the tunnel
  • Since this is intended as an Internet gateway, we’ll use 0.0.0.0/0
• Preshared Secret - Set this to the passphrase we configured in Umbrella
• Availability - Set this to which networks this tunnel should apply
  • Since I only have 1 network & 1 MX, I set this to All Networks

Okay - I mentioned above we’ll need to set some custom IPSec parameters. Here’s what we’ll configure as a custom policy:

Note: Turns out in the drop down menu, there is also an option for “Umbrella”, so you don’t have to create a custom policy. That being said, the Umbrella preset uses DH group 5, but Umbrella’s docs ask for DH group 14.

Lastly, we’ll be able to configure whether or not we want firewall logging (I enabled this) and also whether we want to add access-lists to permit/deny any traffic over the VPN. For now I’ll be leaving this as permit any.

Finally - we can click Save to push our configuration to the MX appliance!

Step 3: Validation & Testing

Hopefully once we get all that configuration done, we’ll be able to see our tunnel come up.

We can validate from both sides, but let’s start with Meraki. In the Dashboard menu, we’ll navigate to Security & SD-WAN > Monitor > VPN Status.

Then we’ll need to click the tab for Non-Meraki peer:

With any luck, we should see the little green circle showing that our tunnel is online.

If your tunnel doesn’t come up immediately, you may have to generate some traffic from your VPN subnet to the internet. This will force the MX to try and connect to Umbrella.

It’s also worth noting - sometimes it may take the Meraki dashboard to show a successful connection. A better indicator is by checking the Umbrella dashboard, or checking manually with a device you are expecting to route over the VPN.

If for any reason we have trouble, we can also check our VPN negotiation logs by going to Network-Wide > Monitor > Event Log.

On this screen, make sure you’ve selected for security appliances at the top. If needed, we can also filter events by All Non-Meraki VPN / Client VPN.

Since my tunnel came up with no issues, the output above shows a successful tunnel negotiation.

Let’s jump over to the Umbrella side, and see what the Umbrella dashboard shows.

Back on the Network Tunnels page - we should see that our tunnel now shows as Active.

This screen will also so the public IP that our MX is using to connect, as well as which Umbrella datacenter we’ve connected to.

Okay! That’s it for now. To try and keep this blog post from getting too long, I’ve decided to split this into two parts.

If you’re interested in seeing how to configure Umbrella’s cloud firewall & web filtering policies - please check out my YouTube Video above!

This post has been on my backlog for a while… I’ve been intermittently trying out the IOS-XE guestshell feature for a few different projects, and never feeling like any of them were good enough to write about.

So why not just write about guestshell itself? It’s such a nifty tool.

What is Guestshell?

Now that Cisco’s Catalyst platform has standardized on IOS-XE, we can take advantage of some of the features of the Linux-based platform. One of which being Linux Containers (LXC).

Most of the latest Catalyst routers, switches, and access points support what Cisco calls Application Hosting - which is a fancy way of just saying it can run containers on-box. Another acronym to be aware of is IOx - which is what Cisco calls their IOS-XE/Linux appliction framework.

Guestshell is a specific feature within IOS-XE, where a dedicated pre-built container is enabled for on-box access to a Linux shell.

What Hardware Supports Guestshell?

Okay - so for this example, I’ll be using the new Catalyst 8000V platform (a virtual Catalyst router). I do also have a Catalyst 9200L on hand, but unfortunately it’s one of the few Catalyst platforms that does not support Guestshell (due to lower-end HW specs).

You can also use the Catalyst 8000V to test this in a lab, or you could use most of the Catalyst 8000 / 9000 series platforms if you had them available (or the CSRv & some ISR routers!).

You can find a list of supported platforms & other hardware requirements here.

All that being said - let’s get started!

Enabling IOx

So before we can actually use the guestshell feature, there is some pre-configuration to be done.

First we’ll need to ensure that the IOx application framework is enabled & running. We can check this by using the show iox command:

0x8KV01# show iox

IOx Infrastructure Summary:
---------------------------
IOx service (CAF)              : Not Running
IOx service (HA)               : Not Supported
IOx service (IOxman)           : Not Running
IOx service (Sec storage)      : Not Supported
Libvirtd 5.5.0                 : Running

And as we see above - most components are listed as “Not Running” or “Not Supported”.

So we’ll enter config mode, and use the iox command to enable these services:

0x8KV01# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
0x8KV01(config)# iox 
0x8KV01(config)#
0x8KV01(config)# exit
0x8KV01# show iox

IOx Infrastructure Summary:
---------------------------
IOx service (CAF)              : Running
IOx service (HA)               : Not Supported
IOx service (IOxman)           : Running
IOx service (Sec storage)      : Not Supported
Libvirtd 5.5.0                 : Running

Configuring Guestshell Networking

There’s a good chance we’ll want our container to have access to the outside world, so let’s take a quick look at the network configuration.

Note: The networking config syntax will differ if you are using a Catalyst router or a Catalyst switch. Since I am using a Catalyst 8000V - we will look at the config for a Catalyst router first.

In order to allow external access to our container, we’ll need to configure a gateway interface and NAT translations. Then we can provide a full network configuration to our container:

## Enable NAT on our external interface - in my case, gigabitEthernet1
0x8KV01(config)# interface gigabitEthernet 1
0x8KV01(config-if)# ip nat outside

## Create a virtual interface for the guestshell container
0x8KV01(config-if)# int virtualportgroup0
0x8KV01(config-if)# ip address 192.168.100.1 255.255.255.0
0x8KV01(config-if)# ip nat inside

## Create an ACL to match traffic from guestshell & Enable NAT
0x8KV01(config)# ip access-list standard IOX_NAT
0x8KV01(config-std-nacl)# permit 192.168.100.0 0.0.0.255
0x8KV01(config)# ip nat inside source list IOX_NAT interface gigabitEthernet 1 overload

## Assign network configuration to our guestshell container
0x8KV01(config)#app-hosting appid guestshell
0x8KV01(config-app-hosting)# app-vnic gateway0 virtualportgroup 0 guest-interface 0
0x8KV01(config-app-hosting-gateway0)# guest-ipaddress 192.168.100.5 netmask 255.255.255.0
0x8KV01(config-app-hosting-gateway0)# app-default-gateway 192.168.100.1 guest-interface 0
0x8KV01(config-app-hosting)# name-server0 8.8.8.8

Now, as I mentioned before - this config will look different if you’re using a switch vs a router.

So let’s take a quick look at what this would look like, if we were on a Catalyst Switch instead:

Cat9k(config)# interface AppGigabitEthernet 1/0/1
Cat9k(config-if)# switchport mode trunk

Cat9k(config)# app-hosting appid name
Cat9k(config-app-hosting)# app-vnic AppGigabitEthernet trunk
Cat9k(config-app-hosting-trunk)# vlan 10 guest-interface 0
Cat9k(config-app-hosting-vlan-access-ip))# guest-ipaddress 192.168.100.1 netmask 255.255.255.0

Cat9k(config-app-hosting)# app-default-gateway 192.168.100.1 guest-interface 0
Cat9k(config)# name-server 8.8.8.8 

Once all that config is done - Let’s move on to getting our container started!!

Managing the Guestshell Process

In order to start a container on our Catalyst device, we’ll need to go through a process of deploying the app, activating it, then starting it.

For guestshell, there is a shortcut command that will perform all of these steps for you - and make managing the guestshell process much easier.

So let’s go ahead and spin up our guestshell container with the command guestshell enable:

0x8KV01# guestshell enable
Interface will be selected if configured in app-hosting
Please wait for completion
guestshell installed successfully
Current state is: DEPLOYED
guestshell activated successfully
Current state is: ACTIVATED
guestshell started successfully
Current state is: RUNNING
Guestshell enabled successfully

0x8KV01# show app-hosting list
App id                                   State
---------------------------------------------------------
guestshell                               RUNNING

And as you can see above, everything gets automatically deployed & started. We can verify using the show app-hosting list command to see that our container has been started.

If we need to, we can also stop & deactivate our container using the command guestshell disable.

We could also use the commands below if we wanted to perform these steps manually:

0x8KV01# app-hosting activate appid guestshell
0x8KV01# app-hosting start appid guestshell

0x8KV01# app-hosting stop appid guestshell
0x8KV01# app-hosting deactivate appid guestshell

Before we move onto accessing the guestshell interface, I also wanted to show where you can get more detail about the guestshell container.

Using the show app-hosting detail appid guestshell command, we can see details around the container version, current MAC/IP config, and resource consumption:

0x8KV01# show app-hosting detail appid guestshell
App id                 : guestshell
Owner                  : iox
State                  : RUNNING
Application
  Type                 : lxc
  Name                 : GuestShell
  Version              : 3.2.0
  Description          : Cisco Systems Guest Shell XE for x86_64
  Path                 : /guestshell/:guestshell.tar
  URL Path             :
Activated profile name : custom

Resource reservation
  Memory               : 256 MB
  Disk                 : 1 MB
  CPU                  : 800 units
  CPU-percent          : 3 %
  VCPU                 : 1

Attached devices
  Type              Name               Alias
  ---------------------------------------------
  serial/shell     iox_console_shell   serial0
  serial/aux       iox_console_aux     serial1
  serial/syslog    iox_syslog          serial2
  serial/trace     iox_trace           serial3

Network interfaces
   ---------------------------------------
eth0:
   MAC address         : 52:54:dd:d:bf:da
   IPv4 address        : 192.168.100.5
   IPv6 address        : ::
   Network name        : VPG0

Port forwarding
  Table-entry  Service  Source-port  Destination-port
  ---------------------------------------------------

Accessing the Guestshell CLI

Now for the fun part! Let’s get into our guestshell container.

We’ll just need to use the guestshell command, and we’ll be dropped into the Linux shell:

0x8KV01# guestshell
[guestshell@guestshell ~]$
[guestshell@guestshell ~]$ uname -a
Linux guestshell 5.4.40 #1 SMP Tue Oct 6 17:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Let’s also make sure our network & NAT configuration worked:

[guestshell@guestshell ~]$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=64 time=3.57 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=64 time=2.64 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=64 time=2.54 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 4ms
rtt min/avg/max/mdev = 2.543/2.916/3.567/0.465 ms

If needed, we can also run commands on our Catalyst device CLI without leaving the guestshell interface by using the dohost utility:

[guestshell@guestshell ~]$ dohost "show app-hosting list"
App id                                   State
---------------------------------------------------------
guestshell                               RUNNING

[guestshell@guestshell ~]$

Just a note, the reverse is also possible. Using the guestshell run command from our Catalyst CLI, we can run commands in our container:

0x8KV01# guestshell run uname -a
Linux guestshell 5.4.40 #1 SMP Tue Oct 6 17:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

0x8KV01#

It’s also worth reiterating, that the guestshell container is a full Linux container (It runs CentOS). So if we need to install additional Linux packages, we can use our Linux package manager (YUM) to do so.

Anyways - Let’s wrap this up by looking at the built-in Python modules & capabilities

Using Python in Guestshell

One of the more appealing use cases for guestshell is the ability to run native Python scripts & code directly on your Catalyst device.

A couple of possible use cases could be:

• Troubleshoot intermittent issues - Automatically collect device state (routes, MAC changes, etc) on a regular basis to log locally or send via email
• Automated resolution - If a certain type of state change is observed, we could run a pre-set list of commands to try and automatically fix the issue
• Local monitoring / troubleshooting - We could run a series of ping, web page load tests, etc from a python script to help monitor performance from the switch’s perspective
• Change management / notification - A script could be written to monitor the device configuration for any changes made, then send an automated email when something was detected
• Have a risky change that might sever your connection to a remote device? A script could execute the change for you, and if unsuccessful, revert to a working configuration (without doing a reload after)

These are just a handful of ideas that came to mind while writing this blog post… But there are many more possibilities!

Within the guestshell container, there is a special cli python module that comes pre-loaded. This module allows us to run CLI commands on our catalyst device directly from a python script & receive the command output for parsing.

Let’s look at a quick example using the Python interactive interpreter to save the device configuration:

[guestshell@guestshell ~]$ python3
Python 3.6.8 (default, Aug 24 2020, 17:57:11)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import cli
>>> status = cli.execute("write memory")
>>> if "OK" in status: print("Device configuration was saved!")
...
Device configuration was saved!

The example above uses the cli.execute Python function, which will run commands in Cisco’s exec mode.

What if we wanted to make configuration changes? In that case we would use the cli.configure function:

>>> config_data = """interface loopback200
... ip address 172.16.99.10 255.255.255.0
... """
>>>
>>> cli.configure(config_data)
'Line 1 SUCCESS:  interface loopback200\nLine 2 SUCCESS:  ip address 172.16.99.10 255.255.255.0\n'
>>>
>>> cli.execute("show run interface loopback200")
'Building configuration...\nCurrent configuration : 68 bytes\n!\ninterface Loopback200\n ip address 172.16.99.10 255.255.255.0\nend\n'

In the above example, we used a multi-line comment to provide the list of commands to enter. The cli.configure function also supports a Python list - so we could provide the commands in that format as well.

You’ll notice that the Python interpreter returns a status for each command that we attempted to run. This should allow us to error-check & ensure that all of our intended configuration was accepted.

Lastly, we used the cli.execute command again to validate that our running configuration looks correct.

There’s a bit more functionality built into the cli Python module, but I just wanted to cover some of the common use cases. More details can be found here

A Few Common Questions

Does this give me access to the underlying IOS-XE Linux shell?

Nope, anything running in guestshell is running in a container - which runs on-top of the IOS-XE image.

What about resource conflicts?

The linux containers running on the IOS-XE platform are given their own dedicated resources. Any resource constraints that affect the containers will not impact normal network functions.

I have a local Cisco SD-WAN lab environment running at home, which was built in EVE-NG. It’s what I use whenever I need to test something for a customer, or just play around with the templates or APIs.

I’m planning on spending some hands-on time with new features soon, along with working on some automation projects - so it’s well past time to upgrade my lab.

Currently I’m running on version 18.4.302, but my intent is to upgrade to 20.3.2 - which is the latest version available today.

In this blog post, we’ll walk through how to upgrade a Cisco SD-WAN / Viptela network, including:

• Control Plane:
  • vManage
  • vBond
  • vSmart(s)
• Data Plane:
  • vEdge / vEdge Cloud

While my lab is using the on-prem controllers, these steps will work just the same if you’re using Cisco’s cloud-hosted controllers.

Downloading the Software Images

First thing’s first - in order to upgrade our environment, we need the correct software images!

Head over to Cisco Software Downloads, and search for SD-WAN (or click here!).

Once on this page, I just want to call out that we will need to click on the SD-WAN Software Update link. This may seem simple enough, but the other links for vManage Software or vSmart Software only contain images for a new install.

After that - just select the image version that you would like to upgrade to, and download both images.

In my case, since I’m downloading version 20.3.2, I’ll be using the following images:

• vSmart, vEdge Cloud, vEdge 5000, ISR1100 series and vBond upgrade image
  • File name: viptela-20.3.2-x86_64.tar.gz
• vManage upgrade image
  • File name: vmanage-20.3.2-x86_64.tar.gz

Adding Images to the Software Repository

The process for upgrading a Cisco SD-WAN environment is pretty straightforward.

The control plane can be upgraded independently of the edge devices, so long as everything stays within the bounds of the compatability matrix. In my case, I’ll be upgrading to 20.3.2 - and the controllers could still support edge devices as far back as 17.2. So no issues here!

Alright - let’s get started!

First, we’ll need to upload our images into our local vManage software repository. This is the file storage for all upgrade images, and it’s where the controllers & edge devices will go to pull their images from.

In the vManage dashboard, we’ll go to the Maintenance tab and select Software Repository.

Then, click on Add New Software.

In here we’ll see a few options: vManage and Remote Server / Remote Server - vManage.

We’ll use vManage if we want to upload & distribute images from the local vManage server we’re logged into currently.

Alternatively, we could use a remote file storage server by using the Remote Server option. If you choose to go this route, don’t forget to ensure that ALL controllers & WAN edge devices have access to this storage location.

After you select an option, it’s an easy drag & drop to upload the software images.

Planning the Upgrade

So before we get into actually applying our image upgrades - let’s address the questions of “What order do I upgrade things in?” and “What’s the impact?”.

Since this is a lab environment for me, I’ll be upgrading everything all at once - since uptime / outages aren’t a factor here

If you’re doing this in a production environment, I highly recommend performing these upgrades in an outage / maintenance window - or at least an off-peak time.

Yes, you can upgrade the controllers at any time without causing any issue. Yes, you can upgrade a redundant pair of vEdge devices and keep a branch online. However, I would advise you to try these out off-hours first - and get your own understanding of how this works & what to expect before doing it in production.

As for the upgrade order, we’re going to start at the top of the food chain and work our way down:

• Upgrade vManage first
• Then vBond
• Upgrade ONE vSmart controller & wait for it to come online / re-establish control connections
• Then upgrade the second / redundant vSmart controller
• After the control plane is upgraded & stable - move onto the edge devices

These steps can also be found under the Best Practices section of the Cisco SD-WAN Getting Started Guide. Cisco’s official recommendation is to wait 24 hours in between a few of those steps, to ensure platform stability - but for my lab that won’t be necessary

Upgrading vManage

Once we have uploaded our images & we have our upgrade plan - we can move forward with actually performing the image upgrades.

Starting with vManage - we’ll go to Maintenance > Software Upgrades > vManage.

Then we’ll click on Upgrade and select our version - in my case 20.3.2. After that, just click Upgrade

Now, what this does in the background is just pre-stage the vManage image for an upgrade. The actual software upgrade is not occurring just yet.

Think of this step like you might prepare an IOS/IOS-XE router: Copying the image to the device flash. The image is there and ready - but we haven’t booted to it yet.

Once that’s all done, we’ll go back to the vManage upgrade page and click Activate, select our image again, then click Activate.

Now this step is where vManage will reboot, apply the upgrade, and come back online with the new image.

Again, back to the IOS/IOS-XE analogy: this is the equivalent of setting out boot system flash:<image name> to the new image, then rebooting our router.

vManage may take a short while to complete & reinitialize. In my lab, about 10-15 minutes.

Upgrading vBond & vSmart

After vManage is done, it’s time to work on the real heart of our control plane: vBond & vSmart.

Similar to vManage, we’ll start by going to Maintenance > Software Upgrades > Controller.

Here we will need to select the devices we want to upgrade. As I mentioned earlier, you may want to do these one at a time & in a phased approach. However, in my lab I’ll select all of them to upgrade at once.

Now in this case, vManage will still follow the proper order (vBond, then vSmart), and even perform a rolling upgrade one device at a time. This may be suitable for you in production, but again I would urge you to test it for yourself first!

The other big difference here, as you can see in the screenshot above, is the presence of the Activate & Reboot checkbox.

This does exactly as you would anticipate. Instead of doing the two-step process with vManage where we staged the image, then performed the activation/reboot - this checkbox will do all of that in one step.

In my lab environment, I did check this box & allowed everything to reboot automatically.

Why is there a separation between uploading the image & rebooting / activating it? To allow better granularity over the process.

For example, maybe you have a poor internet connection at a branch site & the image upload may take a long time. This separation of tasks allows you to stage all of the images independently of applying them. If you have a short outage window, this could help you save time by pre-staging the images ahead of time.

Back to the upgrade - just like vManage we’ll select the version we’re applying then click Upgrade

Again - Depending on the resources available to your controllers, the image upload / activation process may take a short while…

Upgrading the WAN Edge Appliances

In my lab, I’m currently using a handful of vEdge Cloud VMs as branch office routers. The upgrade process here should apply to other edge devices as well.

After we’re confident our controller upgrades have been successful & all control connections have been re-established - we can move to upgrading our edge devices.

It’s also worth mentioning that my lab currently has no redundant deployments of WAN edge appliances. All of my test ‘branch offices’ are single-homed to one vEdge Cloud - so an outage will be required to apply the images.

If you’re using a redundant configuration at a remote site, ideally you would upgrade ONE edge device first. Then only upgrade the second after control connections & routing adjacencies had been re-established on the first device. This should allow for an upgrade with minimal downtime.

The process for upgrading the edge devices mirrors what we saw for vBond / vSmart.

We’ll go to Maintenance > Software Upgrade > WAN Edge, then select the edge devices we want to upgrade.

Click Upgrade, select the target version from the drop-down, then click Upgrade. Again, in my case, I also selected the Activate & Reboot checkbox

NOTE: It’s worth mentioning that for the WAN Edge upgrade, vManage will push the upgrade to all devices simultaneously. If you would like to perform a rolling upgrade here, you’ll have to manage it yourself. In the case of a site where a redundant vEdge is deployed & they share the same site ID, vManage automatically will handle upgrading only one at a time to maintain uptime (Thanks Tim McConnaughy for clarifying this!).

Once again, we’ll give the edge devices a few minutes to download & apply their software upgrades. Then we’ll check to ensure all of the control connections re-established & traffic is flowing.

Wrap up

Once your edge devices are back online, it’s all done! The network has been upgraded to the new version.

There are a handful of ways to check this, but one easy way is via the device monitor page: Monitor > Network

This page will list a summary of everything in the network, including the current software version & number of established control connections. For me, it’s an easy way to get a one-page summary of the network.

From the screenshot above - we can see that all of my lab devices are back online & running software version 20.3.2.

That’s it! Hope this post was helpful to you.

Thanks for reading!

In the last post, I covered a simple project that I’m working on while studying for the Cisco DevNet certifications. This is part two of a series, which will continue on while I work through this project.

As a brief summary - I started using a combination of Scrapli & Cisco Genie for a short network automation script. This script connects to a list of network switches & outputs a spreadsheet of interface statistics. This provides a quick snapshot view into the current port capacity within your network.

In this post, I’ll be showing off a bit of how to build a simple web frontend - which I’ll put together using Flask and Bootstrap.

Getting Started with Flask

Okay so I have a bad habit of making a lot of Python scripts which are essentially just command-line utilities.

In theory, this isn’t necessarily problem. But what if I want to share some of the tools I build? Maybe not everyone wants to compile config files & figure out what command-line arguments are needed. For some, they may prefer to have an easy-to-use web interface instead.

So for this project I opted to venture a little outside my typical comfort zone & build a web dashboard. I’ll admit I’ve done this once or twice before, but certainly not something I would claim proficiency in!

The good thing is - building a web interface doesn’t have to be complex. My intent with this post is to try & break it down a bit - and hopefully encourage you to try it yourself!

Installing Flask & Hello World

Installing Flask is much like any other python module - we’ll use pip:

pip install flask

Once installed, we can import the module into our python script:

from flask import Flask

Simple enough!

Now let’s look at how easy it is to set up the web server & return a simple web page:

app = Flask(__name__)

@app.route('/')
def main():
    return "Hello there!"

if __name__ == '__main__':
    app.run()

First we create a new instance of Flask, using the syntax app = Flask(name).

Next, we just create a quick python function to return the text Hello there! You’ll notice that there is a decorator above the function - this is used by Flask to route incoming HTTP requests.

In this example, we use @app.route(’/’). This tells Flask to register this python function for any HTTP calls to http://<url>/. If we wanted a function for any HTTP requests to http://<url>/networking, we would modify the decorator to @app.route(’/networking’).

Lastly, we need to start the Flask web server when the script is run. This is accomplished using app.run()

Let’s go ahead and try to run the script - which should start the Flask server:

In the above screenshot, you’ll see that by default Flask will start on http://127.0.0.1:5000. You may also notice that Flask will print out real-time logging of incoming HTTP requests.

If we visit our page, we’ll see pretty much what we might expect:

Before we move onto the next section, I did want to show one more example.

Let’s say that we have a python variable (or dictionary, etc). Could we pass that as part of our web page function?

Let’s look at this modified example:

web_page_text = {"Hello": "There!"}

@app.route('/')
def main():
    return web_page_text

If we reload our local web page, we’ll now see the following:

Keep this in mind - We’ll use this later!

Making life easier with HTML templates

Now then - you could build all of your HTML in python, then return the entire HTML page as a response to the function call. That being said, just because you can doesn’t mean you should 🙃

Let’s take a quick look at how we can use HTML templates to help build our web page.

First we’ll create a new directory called templates, and create an HTML file. In this case, I named my template main.html:

To start with, we’ll just add some simple text into the HTML file:

<body>
    Hello There - From the template!
</body>

On the Flask side, we’ll have to modify our imports to include the render_template module. We’ll also update what we’re returning when an HTTP request is received:

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def main():
    return render_template('main.html')

if __name__ == '__main__':
    app.run()

So now instead of returning plain text, we’ll return render_template(‘main.html’).

Flask uses Jinja2 for templating, which we’ll get into shortly. The above function call tells our app to use main.html as a base template for the content that is returned to the end user.

Let’s check out the page:

Awesome - we receive our HTML file as a response this time.

Extending functionality with Jinja2

Now we can start getting into the fun stuff.

Manually writing out a bunch of HTML is fun and all - but what if we could auto-generate everything programmatically?

Jinja2 allows for exactly that. We’ll be able to insert variables, if/then statements, and looping logic directly into our HTML template.

As an example, let’s say we had the following python dictionary - which we wanted to display on a web page:

switchList = [
        {
        "name": "C9200",
        "serial": "ABCD00010",
        "ip": "192.168.1.1",
        "check": True,
        "total": 28,
        "up": 10,
        "down": 5,
        "disabled": 13,
        "capacity": 35       
    },
    {
        "name": "C9300",
        "serial": "ABCD00011",
        "ip": "172.168.44.2",
        "check": False,
        "total": 48,
        "up": 32,
        "down": 6,
        "disabled": 10,  
        "capacity": 67       
    },
        {
        "name": "C9400",
        "serial": "ABCD00012",
        "ip": "172.33.44.2",
        "check": True,
        "total": 48,
        "up": 40,
        "down": 6,
        "disabled": 2,  
        "capacity": 82       
    }
]

This dictionary is an example from the scrapli script I’ve been working on (more detail in my last post).

One way for us to present this using an HTML template would be the following:

<body>
    {% block content %}
    
    {% for switch in switches %}
    Switch Name: {{ switch.name }} 
      - Serial: {{ switch.serial }} 
      - Reachable at: {{ switch.ip }} 
      
    {% endfor %}
    
    {% endblock %}
</body>

As you can see - that’s a bit of a change from what we were using in our template before.

The Jinja2 syntax above uses a combination of curly braces & percentage signs to indicate which parts of our template are logic that should be processed before returning content to the user. Variables are represented with double curly braces.

Once our template receives the switchList dictionary, we can iterate through it similar to how we might in python. Using the {% for switch in switches %} syntax, we can iterate through each item - and pull out relevant data.

Within our for loop, we can reference the individual values of each item in the dictionary. Using the double braces, we can insert a placeholder for where a piece of content should be inserted. For example, in the above template we have a spot where we want to insert the name of the switch. We use {{ switch.name }} as a placeholder, where the value from the dictionary will be inserted once our template is processed.

Okay - so before we test this, we have one minor modification to our python function:

@app.route('/')
def main():
    return render_template('main.html', switches=switchList)

We’ve changed our function render_template, but also include the python object that we’ll be passing to our template.

Let’s test this out:

Now there’s some magic! Using a combination of python & our Jinja2 templates - we can drastically reduce the amount of HTML code that we need to write.

Okay, but that’s not pretty

Yes I know. But we needed to get the basics done first!

Here’s where Bootstrap comes in.

I went out to Google and searched for good CSS templates. I’m not a web developer, nor am I good at design - so I’ll leave that work to someone else.

In my searching, I found quite a bunch of free templates… but I fell in love with one called Lux by Bootswatch.

They provide a great sample page for the template, which shows off different variations. But the best part is that it also includes code samples!

After we pick & download our CSS template, we’ll place it in a new directory named static:

To get started, we’ll need to install one additional python module:

pip install flask_bootstrap

Then we’ll also need a quick modification to our base Flask app:

from flask_bootstrap import Bootstrap

... code omitted ...

if __name__ == '__main__':
    Bootstrap(app)
    app.run()

I removed some code to focus on just the two parts that change.

First, we need to import our new module. Second, we need to inject our bootstrap plugin into the Flask app. We do this with Bootstrap(app) prior to app.run().

Let’s move back over to our HTML template, since this is where the most of our changes will be.

First, we’ll need to include a few things to make sure our CSS file is loaded:

{%- extends "bootstrap/base.html" %}

<head>
    {% block styles %}
    {{super()}}
    <link rel="stylesheet"
            href="{{url_for('static', filename='bootstrap.css')}}">
    {% endblock %}

</head>

I won’t dive in too deep on this piece, as it’s fairly straightforward. First, we’ll need to include the base/default bootstrap template. Next, we’ll include a reference to where our CSS file is located & it’s name - so that it gets loaded when our page is rendered.

Okay, now for the body of the page:

<body>
    {% block content %}
    <table class="table table-hover">
        <thead>
          <tr>
            <th scope="col">Name</th>
            <th scope="col">Current Capacity</th>
          </tr>
        </thead>
        <tbody>
          {% for switch in switches %}
            {% if switch.capacity > 75 %}
                <tr class="table-danger">
            {% else %}
                <tr class="table-success">
            {% endif %}
            <th scope="row">{{ switch.name }}</th>
            <td>{{ switch.capacity }}%</td>
          </tr>
          {% endfor %}
        </tbody>
      </table> 
    {% endblock %}
</body>

Now instead of just returning a plain-text list of all three switches, we’ll use our CSS template to generate a colorful table. For the purpose of this example, I kept the data to just showing the switch name & it’s current capacity.

Most of the template is fairly simple. First we build our table header using the <thead> tags. Next we’ll build the rows of our table using <tbody> and <tr>.

For the Jinja2 logic - We’ll iterate through our list of switches, and add a new table row (<tr>) for each device. I also added logic to check the current capacity of the switch. If the switch capacity is greater than 75%, we create the table row using <tr class=“table-danger”> - which is a reference in our CSS stylesheet that will color the row red. If capacity is less than 75%, we use table-success for a green color.

Let’s go ahead and check out what this looks like:

And as we expect - we get our nicely formatted table & the correct colors on each row.

Bringing it all together

Now that we’ve gotten some background on how to use Flask & Bootstrap, let me show you a bit of what I’ve been working on.

I won’t cover much code in this section, but you can check out the GitHub repo if you’re interested in seeing the details.

The scrapli script that I wrote previously has been extended a little. The biggest change is that instead of writing its output to a spreadsheet, it will insert the data into a sqlite database.

The web frontend has been assembled using the same basic ideas covered in this blog (plus a lot of banging my head against the desk, trying to figure out why things don’t format or align properly 🙂). Whenever a request is made to the web dashboard, it will query the sqlite database for the requested information.

So, without further delay - here it is!

I currently have one physical device running, plus a handful of virtual nexus 9k & CSRv devices running in Cisco Modeling Labs. So all of the data shown in this dashboard is being pulled from real(ish) equipment - no mock data.

This was one of those projects where I started off thinking my dashboard was going to be really simple… but then I kept having new ideas and getting carried away with the design & functionality.

The table above shows each switch, it’s serial number, management IP, current software version, and the port inventory. Similar to the example earlier in this post, I have a small progress bar that shows switch capacity - and will change color depending on percentage of in-use ports.

If you click on any of the switch names, you’ll be taken to a page with additional detail on that particular device:

On this first tab, you’ll see some quick info - mostly the same that is on the main page of the dashboard. However, I added two additional tabs here - one for a detailed port breakdown & one for a dump of the raw CLI output.

If we click on the port info:

We have a breakdown of how many ports are currently in use vs down, as well as our breakdown of port media & operational speed. Since I account for switches that are anywhere from 10M up to 100G, the data for operational speeds will only show the speeds that are actually in-use on the switch.

Just in case it’s needed, I added a tab to show the raw CLI output from the switch:

This could be a quick way to check port counters & errors, or any other information that isn’t already presented via the dashboard.

Lastly, I built a separate page to provide statistics on ALL devices in the network:

This page will show port count & availability network-wide. However, I also added some spaces to show the top 5 hardware models & software versions that are in-use across the network.

I went into this project a bit worried because frontend development isn’t my strength. That being said, I really enjoyed working on this project. It was a different challenge than I’m normally used to - and it gave me a creative way to try and format & display the data I am collecting.

I hope this post was useful to you. Please leave a comment below - and check out my Github repo for the full code.

About ARIN

If you haven’t already worked with them - ARIN is the American Registry for Internet Numbers. They are a non-profit organization and their purpose is to assign/manage Internet number resources for all of North America. This includes IPv4/IPv6 addresses and BGP Autonomous System Numbers (ASNs). ARIN is one of five Regional Internet Registries (RIRs) - each managing Internet resources for it’s own individual region. All of these report back to a top-level organization, the Internet Assigned Numbers Authority (IANA).

What I didn’t know: ARIN actually used to manage resources for all of South America and Africa as well. LACNIC formed and took ownership of South America in 2001, and AFRINIC took Africa in late 2004. ARIN itself has only been around since 1997, and will be celebrating it’s 20th anniversary this December.

Outside of assigning/managing number resources - ARIN manages a huge manual of numbering policies and standards (The Number Resource Policy Manual). A good note here is that these policies are heavily influenced by the community - so if any individual or group of network operators want to change/modify or add new policies, then they can submit proposals to do so.

IPv4 Depletion

I was very interested to hear about what’s going on with IPv4/IPv6 - mostly because I’ve been trying to push for IPv6 in many of the places I have worked. The ARIN group spent a little bit of time talking about how the depletion of IPv4 addresses has affected their workload. Overall, it seems like their work has remained about the same - but it has transitioned from mostly IPv4 allocations to more IPv4 transfer requests.

An interesting note from this discussion was that ARIN only performs the backend registration changes for IPv4 block transfers. They play no part in the actual negotiations between two organizations. However, they do perform their own investigations during transfers to ensure that the source organization legitimately owns the IP block, and the destination organization can justify the use of the space.

I had heard previously that ARIN kept a block of IPv4 addresses for transition to IPv6 - but I never researched it further. This was a topic ARIN touched on during the event. Essentially, they have kept ownership of a /10 block of addresses, which is split up into individual /24 blocks for assignment. Any organization can request one of the /24s when they request a block of IPv6 addresses. The organization must fill out a justification form, in which they demonstrate how the IPv4 blocks will be used to help transition to IPv6. Organizations can request one of these blocks every 6 months, provided they can still justify the need for them. This is all documented in NRPM section 4.10.

The somewhat surprising thing here is that ARIN was actively encouraging people to take advantage of this. Probably because they need to push IPv6 adoption in any way they can. As of the date of the event, ARIN stated that only ~60 /24 blocks had been assigned so far.

IPv6 Adoption

This part of the event wasn’t quite everything I wanted it to be. Overall ARIN touched on statistics from Google and other organizations that show the trending uptake in IPv6 network access. They also spoke briefly about how the structure of IPv6 addresses makes life easier - because the last 64 bits can always be used for host-based MAC autoconfig, then network operators only worry about subnetting above that.

Interestingly enough, ARIN was advocating for the method of ‘assign way more addresses than you’ll ever need’ mentality for IPv6. Another attendee asked the question ‘Won’t we run into the same thing as IPv4, if we just throw out v6 blocks like candy’? This actually led to hearing something I wasn’t aware of - IANA has currently only made 1/8th of IPv6 blocks public available for use. The current numbering scheme/standard will be used for this first block of addresses. If we run through them too quickly, then we can step back and re-evaluate best practices before handing out the next 1/8th block of addresses.

DNSSec

Initially I was a bit confused that DNSSec was on the topic list - but I figured maybe ARIN was just trying to push this for the betterment of the Internet. While they spoke a bit about DNSSec for forward DNS, their primary topic was how DNSSec for reverse DNS isn’t something people are normally thinking about. As it turns out, ARIN offers reverse-lookup DNSSec for any IP blocks that they assign out. This is good to know, since reverse DNS can be important for things like email security - and its certainly something I’ve never really considered in the past.

If you have purchased IPv4/v6 blocks directly from ARIN - I would recommend that you check this out.

RPKI

Resource Public Key Infrastructure (RPKI) is a way of cryptographically validating ownership of IP address space or routing objects. Since BGP is primarily a trust-based protocol between organizations, RPKI allows network operators to implement additional security by providing a certificate-based system of trust. The majority of this discussion was around how bad BGP security is, and that overall North America is far behind on implementing RPKI.

ARIN has a service available where they will act as your Certificate Authority (CA) for RPKI - so it only requires network operators to sign records then implement a few device changes.

My Thoughts

Overall the event was fairly informative! It wasn’t quite everything I wanted it to be, but I did walk away with additional knowledge that I didn’t have before. I was really hoping to learn more about how other organizations are implementing IPv6, or even how other people are convincing their employers to take IPv6 adoption seriously. When I spoke with some other attendees, it seemed like not many people had IPv6 running in a production environment yet - only a few of them had even started testing. Surprisingly, even the ARIN reps were repeatedly asking people to contact them if they had an IPv6 success story to share.

One thing I found really interesting was surrounding DNSSec/RPKI. A few attendees asked about how many people are actually validating signed resources. It’s one thing to implement signing, but it won’t matter if no one validates the resources, right? Surprisingly, ARIN had no statistics about this - and stated the point that they cannot enforce adoption of these standards. It certainly makes sense, but it’s not something I gave much thought to previously. Since they’re just a registry, they can only make these services available - not enforce their usage. This is why they put on events such as this to raise awareness and provide education.

ARIN pushed the fact that all of their policies are community driven. There were quite a few examples throughout the event of how individual members of the community could impact changes to their policies. My primary concern is that it seemed like a majority of the individuals in attendance represented government or educational organizations - and not a lot who worked in similar network environments to what I manage. They raised their own concerns and questions, which were certainly valid for the types of infrastructure and designs that they maintain. However, a number of these things don’t really apply to my infrastructure in quite the same ways.

If I have to make one point here: If you’re a network operator, go subscribe to ARINs mailing lists and get involved. Maybe you don’t have any ideas for policy changes, but you never know what might come up that you could provide meaningful input on. The ARIN reps provided an example or two of when a smaller group of people suggested policy changes which drastically affected bigger companies - and almost no one opposed it until it took effect. Only you have the ability to voice your opinion and concerns about how a proposed policy could affect your network. If not, the next time you try to request a block of IP addresses or a BGP ASN, you could potentially run into roadblocks because of a policy change proposed by someone with very different needs.

The staff at ARIN don’t live and work in the networks that we do. They try to work with network operators to understand use cases and the possible ramifications of policy changes - but ultimately they are a small non-profit. They can’t think of everything, nor can they force network operators to contribute their opinions. Get involved and make a difference.

As a final note, ARIN has a Fellowship Program where you can apply to attend one of their Public Policy meetings for free. Fill out an application and if you’re chosen they’ll provide a ticket, hotel room, and travel expenses. It’s a great opportunity to experience one of these meetings, especially if you might not have the financial means to otherwise.

The slide deck from the event is publicly available on ARIN’s website: here.

So I just scheduled an upcoming attempt at the Developing Applications using Cisco Core Platforms and APIs (DEVCOR 350-901) exam.

It’s coming up quick in two weeks here & I’m starting to do some review / final prep before taking the exam.

Why does this matter? Well - I plan on doing a few write-ups & videos on some of the content I’m studying. There are a lot of people diving into the DevNet exams, and I want to do what I can to help other people succeed.

All that being said, check back here over the next few weeks - or subscribe on YouTube - to see the additional content that will follow. While this specific blog doesn’t cover much from the exam blueprint, it’s the beginning a simple project I’ll be using in the further content.

Getting Started with Scrapli

For this project I opted to start with scrapli, and possibly migrate to using RESTCONF later on in the process. Mostly this was an excuse for me to try out scrapli, as I’ve heard a few people using it recently & was interested.

Scrapli is a screen scraping module for Python. If you’re not familiar with screen scraping, it’s the process of connecting to something via telnet/ssh/etc, then literally scraping or dumping the contents of the screen. There are other modules that do this as well, like paramiko, netmiko, or expect scripting.

In networking, too many of our devices still rely on CLI and don’t have proper API endpoints. We’re working on it, but yet still a ways from having it everywhere. So in the meantime, we still need screen scraping for automation.

On the whole, this isn’t necessarily a bad thing. For example, most network engineers are very familiar and competent with the CLI of any network operating system. It’s easier to jump into the world of automation if you can start with something you know, the CLI. It’s harder to force someone to start their automation journey by giving up everything they know and shoving REST APIs at them.

Okay - enough rambling. Let’s get scrapli installed and see what it can do.

Installing the module

Easiest part of the whole project. Install with pip:

pip install scrapli

Next we’ll go ahead & import into our script.

Scrapli supports quite a handful of operating systems (including a few non-Cisco platforms as well!). In the case of my project though, I’m only using Cisco IOS-XE switches - so we’ll only import the scrapli driver for that.

from scrapli.driver.core import IOSXEDriver

Connecting to a Device

Okay - now that we’re all setup with the module, it’s time to get working!

First thing, we need to authenticate to our device. Building off of the example code from the scrapli page - we’ll create a short dictionary of authentication parameters first:

switch = {
    "host": "10.1.1.1",
    "auth_username": "net_api",
    "auth_password": "net_api_pass",
    "auth_strict_key": False
}

cli = IOSXEDriver(**switch)
cli.open()

Once we build out dictionary, we’ll use **switch to unpack our key/value pairs into the IOSXEDriver object & assign it to a variable called cli. Then, all we need to do is call cli.open() and scrapli will open a connection to our target device.

Now we can run any command we want by calling cli.send_command():

sh_int = cli.send_command("show interface")
print(sh_int.output)

So in the above example, I want to issue a show interface - then print the output.

What we get is shown below:

GigabitEthernet1/0/1 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 78bc.1a81.e101 (bia 78bc.1a81.e101)
  Description: Test Port
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:11, output 00:00:03, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 198
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 9000 bits/sec, 11 packets/sec
  5 minute output rate 2066000 bits/sec, 221 packets/sec
     2647548 packets input, 371883264 bytes, 0 no buffer
     Received 6985 broadcasts (6757 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 6757 multicast, 0 pause input
     0 input packets with dribble condition detected
     50905390 packets output, 60134059101 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

[**Output Truncated**]

What’s that look like? Exactly the same output we would get if we entered show interface on the command line ourselves! (Only Gig1/0/1 shown here to keep this short & clean)

Using Cisco Genie to Parse CLI Output

So what if we wanted to pull a list of every interface on the switch? Maybe tally how many ports are connected vs down vs admin disabled? Or even check operational speeds & media types?

Well that’s what I’m looking to do for this project.

Now at first it may seem like we have to use regular expressions to try & match the content we need from the output. However, there is an easier way!

Cisco Genie is an open source project, and part of the larger PyATS automated network testing suite. If you haven’t looked at any of that yet, I would highly recommend you check it out.

So what’s Genie do? It’s a utility that handles all the output parsing for us. There are a ton of pre-existing parsers written, which handle all the hard work so we don’t have to.

Best yet - scrapli has native integration into Genie. All we have to do is install one additional component:

pip install scrapli[genie]

And to make use of the power of Genie, we just need to pass our raw output to it.

So our new code will look like this:

sh_int = cli.send_command("show interface")
sh_int_parsed = sh_int.genie_parse_output()
print(sh_int_parsed)

That’s it. And now, instead of that raw output - we get a native Python dictionary that’s ready to consume by our script:

{
   "GigabitEthernet1/0/1":{
      "port_channel":{
         "port_channel_member":False
      },
      "enabled":True,
      "line_protocol":"up",
      "oper_status":"up",
      "connected":True,
      "type":"Gigabit Ethernet",
      "mac_address":"78bc.1a81.e101",
      "phys_address":"78bc.1a81.e101",
      "description":"Test Port",
      "delay":10,
      "mtu":1500,
      "bandwidth":1000000,
      "reliability":"255/255",
      "txload":"1/255",
      "rxload":"1/255",
      "encapsulations":{
         "encapsulation":"arpa"
      },
      "keepalive":10,
      "duplex_mode":"full",
      "port_speed":"1000mb/s",
      "media_type":"10/100/1000BaseTX",
      "flow_control":{
         "receive":False,
         "send":False
      },
      "arp_type":"arpa",
      "arp_timeout":"04:00:00",
      "last_input":"00:00:05",
      "last_output":"00:00:08",
      "output_hang":"never",
      "queues":{
         "input_queue_size":0,
         "input_queue_max":2000,
         "input_queue_drops":0,
         "input_queue_flushes":0,
         "total_output_drop":198,
         "queue_strategy":"fifo",
         "output_queue_size":0,
         "output_queue_max":40
      },
      "counters":{
         "rate":{
            "load_interval":300,
            "in_rate":8000,
            "in_rate_pkts":11,
            "out_rate":1868000,
            "out_rate_pkts":205
         },
         "last_clear":"never",
         "in_pkts":2658450,
         "in_octets":372811780,
         "in_no_buffer":0,
         "in_multicast_pkts":6783,
         "in_broadcast_pkts":6783,
         "in_runts":0,
         "in_giants":0,
         "in_throttles":0,
         "in_errors":0,
         "in_crc_errors":0,
         "in_frame":0,
         "in_overrun":0,
         "in_ignored":0,
         "in_watchdog":0,
         "in_mac_pause_frames":0,
         "in_with_dribble":0,
         "out_pkts":51057453,
         "out_octets":60309072263,
         "out_underruns":0,
         "out_errors":0,
         "out_interface_resets":2,
         "out_collision":0,
         "out_unknown_protocl_drops":0,
         "out_babble":0,
         "out_late_collision":0,
         "out_deferred":0,
         "out_lost_carrier":0,
         "out_no_carrier":0,
         "out_mac_pause_frames":0,
         "out_buffer_failure":0,
         "out_buffers_swapped":0
      }
   }
}

This format makes our lives a lot easier. For example, let’s say we wanted to determine the operational state of a port. Without the Genie integration, we would need to write some regex to comb through the raw output & find what we needed.

However, with Genie involved - its as easy as this:

print(sh_int_parsed['GigabitEthernet1/0/1']['oper_status'])

Collecting the Useful Bits

Now that we have everything in an easy-to-use format, all we need to do is write a few loops to run through our interface list & collect data.

The purpose of my script is to automate the collection of port utilization. For example, think about performing a switch refresh or some form of capacity planning scenario. You might need to inventory how many switches you have, how many ports are utilized vs how many are available, or count the number of copper vs fiber ports.

For this first iteration, we’ll just collect the data then dump it out to a CSV file. As I mentioned earlier, this is just the start of a small project I’ll be using to study for the DEVCOR exam - so this will be evolving into a web service later on.

The full script can be found here. But I’ve posted just a snippet of how we count the different interface characteristics:

# Count all Ethernet interfaces
interfaceStats['total'] += 1
# Count admin-down interfaces
if not sh_int_parsed[iface]['enabled']:
    interfaceStats['intdisabled'] += 1
# Count not connected interfaces
elif sh_int_parsed[iface]['enabled'] and sh_int_parsed[iface]['oper_status'] == 'down':
    interfaceStats['intdown'] += 1
# Count up / connected interfaces - Then collect current speeds
elif sh_int_parsed[iface]['enabled'] and sh_int_parsed[iface]['connected']:
    interfaceStats['intup'] += 1
    speed = sh_int_parsed[iface]['bandwidth']
    if speed == 10_000:
        interfaceStats['intop10m'] += 1
    if speed == 100_000:
        interfaceStats['intop100m'] += 1
    if speed == 1_000_000:
        interfaceStats['intop1g'] += 1
    if speed == 10_000_000:
        interfaceStats['intop10g'] += 1
# Count number of interfaces by media type
try:
    media = sh_int_parsed[iface]['media_type']
    if '1000BaseTX' in media:
        interfaceStats['intmedcop'] += 1
    else:
        interfaceStats['intmedsfp'] += 1
except KeyError:
    interfaceStats['intmedsfp'] += 1

Once all that runs, we create a CSV file to dump all of the data to.

For example, running the complete script against my lab Catalyst 9200 switch would output the following:

And that’s it! Using the combination of scrapli & genie made this script very quick and easy to write - which means now I can focus more time on the next steps.

As I’ve mentioned a few times, this is hopefully the start of a short series of blog posts & videos as I wrap up studying for the DEVCOR exam.

If you found this content helpful - or you’re interested in the future content - please check back soon! Or consider subscribing to my YouTube channel, where new content will be coming shortly.

So if you’ve read some of my recent posts - you may have seen that I purchased a NetGear LB 1121 LTE Cellular modem to use for home internet backup.

Well - I decided to upgrade!

After using the NetGear modem for a while, I started having some issues where it would disconnect from the cellular connection intermittently. Since it’s not necessarily intended for the purpose I’m using it for, there wasn’t any good way to set up monitoring for it either.

So I opted to upgrade to a Meraki MG21. This is one of the latest additions to the Meraki family of network devices & is available with internal (MG21) and external (MG21E) antenna.

I was pretty excited to get one, since Meraki tends to have decent analytics & configuration - and they make everything so easy!

We’ll walk through the setup of the MG below - but if you’re interested in seeing what the device looks like as well, definitely check out the video above!

MG Setup - Changing the APN

Okay - so after I got the SIM card inserted into the MG, the first configuration step is making sure we have the correct APN configured. As a reminder, I’m using Google Fi as my cellular provider.

This change will need to be done on the local web management interface - not from Meraki Dashboard.

When the MG powers on, by default it will hand out DHCP addresses to any device connected to port 1. At the time of writing, these addresses were in the 192.168.5.x range.

Connecting a PC directly to the MG port, we should be able to reach the local management web page - either by typing in the IP address into our web browser, or using mg.meraki.com:

In my case, I could see that the MG had auto-detected my Carrier as Google Fi. However, it still had the incorrect APN.

Using the Configure tab, we can change that setting. You’ll be prompted for a username & password. By default, the username will be the serial number of the device (including dashes) with a blank password.

As shown in the screenshot above, we have a handful of options - though we’ll only care about APN.

First - change the Cellular Override option to Override SIM Settings. Then type in your APN. In my case, it’s h2g2 for Google Fi. Lastly, hit save at the bottom (just outside the view in the screenshot above).

With any luck, the modem will connect and you’ll see something like this:

The modem connects, gets an IP from the provider, and is able to validate connectivity to both the internet & Meraki Cloud.

When I originally set up my MG, I did run into some issues with this. My MG connected to the internet successfully, but said it couldn’t reach the Meraki Cloud. Not sure what caused it, but it shortly resolved itself within a few minutes. Just gotta be patient sometimes, I suppose!

Configuring the MG in Meraki Dashboard

Note: I won’t get into how to claim your device in dashboard or how to attach it to a network. If you need help, please check out the video above where I did show how to accomplish these steps

Okay! Now that our MG is configured for the correct cell network, we can log into the Meraki Dashboard and begin configuring it.

After we’ve added the MG to our network, we’ll see a new Cellular Gateway menu:

We’ll start first by going over to Configure > Settings

First section we’ll see is for Addressing & NAT:

As of today, the MG doesn’t support any form of direct internet pass-through. Instead, our only option is routed mode - where the MG will hold the IP provided by our Carrier & NAT any requests from the devices behind it.

We can change the DHCP subnet configuration here, which will affect what IP addresses are handed to clients behind the MG. In my case, I’m connecting this directly to a firewall as a secondary internet uplink - so the addressing & subnet doesn’t matter as much. By default, the MG will always consume the first available address as it’s own.

Next, we have a section for DHCP & subnets:

Here we can change our DHCP lease time, and what DNS servers are provided to our clients. The DNS setting does have pre-defined options for Umbrella DNS, Google DNS, or using whatever the upstream carrier provides. You’re also welcome to manually specify which DNS servers to use.

We can also configure reserved & fixed IP addresses here.

Reserved IP ranges are IP addresses that we don’t want the MG to provide via DHCP. So if we had any statically configured IP addresses, we could reserve them here.

Fixed IP addresses are for any client that needs a DHCP address, but we want that IP assignment to be permanent. We’ll enter the client name & MAC Address here, as well as the IP we want assigned to that device. In my case, I went ahead and inserted my firewall MAC address - and I’ll just allow the firewall to get its IP via DHCP from the MG.

By default, the MG will block all inbound traffic from the cellular network. If we need to allow any traffic inbound, we can change the Port Forwarding settings:

This allows for a light configuration of an inbound NAT. Right now, I probably won’t be using this. However, I may permit VPN access into my network via the MG at a later date.

If I needed to add anything here, the MG allows us to translate an external / public IP & port to any internal IP / port combination. It appears we can even add a IP filter to permit only trusted source addresses.

Lastly - We can configure settings for Traffic Shaping:

In this section, we can throttle our cellular throughput & configure uplink monitoring.

By default, the cell bandwidth is set to unlimited - but we can drop this down if needed. In my case, I am not using an unlimited cell data plan - so I will throttle cellular speeds to preserve data & reduce charges.

In addition, we can configure one or more IP addresses to check uplink connectivity. These addresses will be used to collect loss & latency data via the cellular connection. The MG monitoring dashboard will collect & graph this data for easy insight into the performance metrics.

Note: As a word of warning, these uplink monitors are constantly sending ICMP/ping requests. If you have a limited amount of cellular data, this may consume more data than you would like. In my testing, using only one IP for uplink monitoring consumed about 70-100M per day. More on this below…

Monitoring the MG

Now we get to the good stuff! The primary reason I opted to buy an MG was for monitoring & analytics.

Back on the dashboard, if we use the lefthand menu - we’ll go over to Cellular Gateway > Monitor > Cellular Gatways. Then select our MG out of the list.

The primary summary page isn’t too exciting:

The MG does have two gigabit ethernet ports - and we’ll see the status here.

We’ll also see the connectivity history to the Meraki cloud - which in my case is nearly 100%. Seems like one very minor blip just after 4am.

We can also see the current network utilization on the MG. This is great to have - though my current utilization is pretty low… (I am using this as a backup modem, after all).

On the left side of the page, we’ll see some of the usual info we expect from a Meraki device. Current IP, location, Serial number, and IMEI. Just below the view of the screenshot, there is also an indicator for firmware version.

Onto the Uplink tab! Let’s see what we have:

First we’ll see the Configuration section. This just gives us a quick view into what settings the MG currently has.

We’ll see the current IP info provided by our carrier, and also some statistics on our cellular connection.

Just below that info, we’ll see our cellular graphs:

This is what I wanted! It’s great to see a quick view into what our active uplink traffic is - as well as look back historically at what our LTE signal quality has been.

Not pictured here - but there is also a section of graphs on this page for our uplink monitor. This is where we can see our current & historical loss & latency stats for the cellular connection. After a few days of use - I disabled the uplink monitor due to the amount of data the feature consumes.

Finally, we also have the DHCP tab:

This will show our current DHCP subnet & any clients that have been provided an address. In my case, there isn’t any current leases here - because my firewall has a fixed IP.

Performance && Considerations

I’ve had the MG running for about a week now, and wanted to provide some things to think about.

First - How does the modem perform? Well, the first day I had it set up - I was able to get ~150M download speeds using the MG’s built-in speed test utility:

That being said - I’m lucky if I get 30-50M on an average day. I might have just gotten lucky that day with some light cellular utilization in my area. Overall though, I’m pleased with the speeds I get - they’ll certainly fit my needs.

For the few days that I had the uplink monitoring running, I saw good results. Usually 0% packet loss, with a rare spike of 5-10%. Latency was a little less reliable, but usually bounced between 50-150ms. This was also much less than the NetGear modem I had been using, which averaged 200-250ms.

Speaking of uplink tests! Let’s talk about data usage….

By default, the MG communicates intermittently with the Meraki cloud - which consumes some data. By my measurement, this is usually less than 10Mb/day. No problem here.

The uplink tests, on the other hand, do consume a bit of data. I’m not sure what frequency these run on, but it’s fairly often. Even with one uplink monitor to 8.8.8.8 configured, I was seeing data usage of 70-100Mb a day.

While seeing those metrics is valuable to me, it’s also not worth the data charges. If I was using a SIM card with an unlimited data plan - no doubt I would keep this feature enabled. However, since I am paying for the cell data used - I opted to disable this feature.

The MG does still perform it’s check-ins to the Meraki Cloud - so you’ll have availability statistics & monitoring… But disabling the uplink monitor means you’ll lose the granular data on loss & latency.

Lastly, and another word of warning, when you’re actively viewing the MG monitoring page - this also consumes additional data. To demonstrate - I’ll post a snippet of the screenshot from earlier:

If you notice, all the way on the far left there was barely any activity. However, once I loaded the MG monitoring page - you begin to see minor spikes in data usage as the Meraki Cloud starts actively polling the MG for data.

In my experience so far, this isn’t a ton of data. I’ve checked in to see how the MG has been performing a few times this week, and each time has totaled around 10-15Mb of data usage.

To sum up - I’m cheap and want to avoid excess data usage. Just wanted to provide some of that info as something to be aware of.

Final Thoughts

I’m only a week in, but pretty pleased with the MG’s performance. It’s maintained a very solid & stable connection compared to the NetGear modem it replaced. The device is intended to provide LTE connectivity or backup service for business networks, so I would certainly hope it would meet my home needs :)

Outside of that, I do wish there was a little better documentation & clarity from the Meraki team on data usage. Right now their documentation only mentions the 6-8Mb of usage due to backend data to/from the Meraki Cloud:

I would be happy to see additional settings on the uplink monitor to allow me to choose the polling frequency. I feel like throttling down the amount of requests could reduce data to a point where I would be comfortable re-enabling that feature.

Oh - and currently there are no native email alerts for the MG. So if the MG goes offline, etc… there is no alerting from the Meraki Dashboard. This kinda sucks. I’m sure this is coming soon, but for the time being I’m inclined to write my own monitor using the Dashboard APIs. (see note below!)

Overall, I’m happy with the device. Definitely looking forward to future feature & firmware updates to see where the Meraki team takes this platform!

Update 08/28/2020 - Looks like in the week since I posted this, Meraki added alerting for the MG! Now you can be notified if the cell gateway goes offline:

In a recent post, I walked through setting up a Netgear LTE cellular modem with Google Fi. Might seem random - but I had a plan for this!

In trying to ensure I keep a stable internet connection for work, I eventually led myself down the path of buying a cellular modem. That was part one. The next step was being able to somewhat-intelligently monitor my home internet connection, and then fail over to the cell modem when connectivity was poor (think lazy SDWAN).

At first I figured this would be easy…. but of course nothing is :)

I currently have a Cisco FirePower 1010 appliance that I’m using for a home firewall. Unfortunately, while this thing does support IP SLA - It wouldn’t quite accomplish what I wanted to do. And to be fair, this box is intended to be a security appliance - not SDWAN.

Anyways - I talked myself into just writing some Python automation to monitor my home internet, and dynamically inject/remove static routing entries. I needed a new project anyways

For those of you wanting to jump straight to the code, the GitHub repo is here

Part 1 - Monitoring Packet Loss & Latency

So first thing - I regularly experience both complete internet outages (always at the worst times), as well as very high latency. Packet loss seems to be less common with my ISP - but hey, it happens sometimes too.

I opted to use the pythonping module as an easy way to collect some of the info I needed.

After importing the module, it’s as easy as specifying a destination, packet size, and number of ICMP messages to send:

from pythonping import ping
result = ping('8.8.8.8', size=2, count=10, verbose=True)

To simplify at least one part of the operation, pythonping has a built in function to return the average round trip:

>>> print(result.rtt_avg_ms)
74.23

The actual contents of our ping results are going to be in the following format: Reply from 8.8.8.8, 10 bytes in 43.82ms

So a quick way for me to figure out packet loss was to simply search for the “Reply from” text in each response:

lost = 0
for packet in result:
    if "Reply from" in str(packet):
        pass
    else:
        lost += 1

Easy enough - and we can use that to calculate the amount of packet loss against the 10 total packets sent:

lossperc = 100 * lost / 10

Once all that is figured out, we can use a simple expression to determine whether or not the loss or latency thresholds have been exceeded:

if response >= MAX_LATENCY or loss >= MAX_LOSS:
    print("Loss/Latency violate thresholds.")
    return True
else:
    print("Loss/Latency within thresholds.")
    return False

The code I wrote then takes one of two paths.

• If the loss and latency is ABOVE our thresholds, call to the FirePower module to inject a static default route over the LTE modem
• If the loss and latency is BELOW our thresholds, call to the FirePower module to delete the static default route - which returns traffic to the primary internet

These functions within the FirePower module were written so that each change will only be made once. For example, if the script checks and finds that the loss/latency is bad, but the static route towards the LTE modem already exists - then no additional changes are made.

So next - Let’s take a look at the FirePower module.

Part 2 - Authenticating to FDM & Initial Checks

This was my first time getting into the FirePower FDM APIs - but they ended up being fairly straightforward to use.

Turns out that FDM has built-in API documentation, which is extremely helpful. This can be found at https://<FDM IP>/#/api-explorer

Okay - So first thing’s first. Authenticating to the firewall:

oauth_data = {
    "grant_type": "password",
    "username": "admin",
    "password": "Cisco1234!"
}
headers = {
          "Content-Type": "application/json",
          "Accept": "application/json"
}
baseurl = "https://" + FDM + "/api/fdm/latest"

authurl = baseurl + "/fdm/token"
print("Posting AUTH request to FDM")
# POST request to FDM with headers / oauth data
resp = self.s.post(authurl, headers=headers,
                   data=json.dumps(oauth_data),
                   verify=False)
# If success - only pull out & return the auth token
if resp.status_code == 200:
    print("Auth success - got token.")
    return json.loads(resp.text)['access_token']
else:
    print("Authentication Failed.")
    print(resp.text)

In the code above - we take the headers and some basic authentication info (username, password, grant type) and send it in an HTTP POST request to https://<FDM IP>/api/fdm/latest/fdm/token

This returns an authentication token, which we’ll need to include in any further HTTP request to the firewall. This can be done by sending a new set of headers in the future:

headers = {
          "Content-Type": "application/json",
          "Accept": "application/json",
          "Authorization": "Bearer " + self.token
}

Next we need to figure out which routing table to insert the route into. Since I am only using the default routing table, the script will just grab the UID for the default global routing table:

vr_url = baseurl + "/devices/default/routing/virtualrouters"
routing_table = requests.get(vr_url, headers=headers)

I mentioned earlier that the script will not make any changes if the firewall is already in the desired state. So we will take time now to check what state the firewall is in, before attempting to make any changes.

This is accomplished by making our first primary action scanning the routing table to see if our route already exists:

route_url = baseurl + "/devices/default/routing/virtualrouters/" + \
            self.globalVR + "/staticrouteentries"
route_data = requests.get(route_url, headers=headers)

# Convert returned JSON object
current_routes = json.loads(route_data)['items']
# Run through each route returned and see if it matches the one we're looking for
for route in current_routes:
    # For each route, we need to manually go look up the uid of our gateway & network objects
    gateway = self.getNetworkObject(route['gateway']['id'])
    dest_network = self.getNetworkObject(route['networks'][0]['id']).split('/')[0]
    # Match based on route prefix & upstream next hop gateway
    if gateway == GATEWAY and dest_network == ROUTE.split('/')[0]:
        print("Found route to %s via %s" % (dest_network, gateway))
        return route

Depending on whether we were looking to add or remove a route, the script may proceed with doing so - or just quit if the action has already been taken previously.

Part 3 - Failover (Add Static Route)

Assuming that we need to Add a route, we need to sent a POST request to https://<FDM IP>/api/fdm/latest/devices/default/routing/virtualrouters/<Virtual Router ID>/staticrouteentries with some required information (like subnet info, next-hop, etc)

Unfortunately, we can’t just send a POST with the target subnet & gateway. Those need to be created as network objects on the FirePower box beforehand. For each network object, we need to build a quick JSON object with the required info:

host_data = {}
host_data['name'] = name
host_data['description'] = "Created by ISP Failover automation"
host_data['subType'] = subtype
host_data['value'] = address
host_data['type'] = "networkobject"

Then we can send a POST to the FDM API to create the object with our supplied parameters:

posturl = baseurl + "/object/networks"
requests.post(posturl, headers=headers, json=host_data)

Once we have those objects created, we can compile all of that info into a JSON object with all the components needed to create a static route entry:

routeobject = {}
routeobject['name'] = "route_BACKUP"
routeobject['description'] = "Created by ISP Failover automation"
routeobject['iface'] = {}
routeobject['iface']['id'] = iface_ID
routeobject['iface']['type'] = "physicalinterface"
routeobject['iface']['name'] = iface_name
routeobject['networks'] = [{}]
routeobject['networks'][0] = {}
routeobject['networks'][0]['id'] = network_ID
routeobject['networks'][0]['type'] = "networkobject"
routeobject['networks'][0]['name'] = network_name
routeobject['gateway'] = {}
routeobject['gateway']['id'] = gateway_ID
routeobject['gateway']['type'] = "networkobject"
routeobject['gateway']['name'] = gateway_name
routeobject['metricValue'] = 1
routeobject['ipType'] = "IPv4"
routeobject['type'] = "staticrouteentry"

Then pass all that into a POST request to create the route object:

add_url = baseurl + "/devices/default/routing/virtualrouters/" + self.globalVR + "/staticrouteentries
requests.post(add_url, headers=headers, json=routeobject)

Success? Well not quite yet.

FirePower requires all changes to be deployed (or applied) to the system before they take effect.

So next we need to initiate a deployment task - and periodically check on it. Deployments can take a while to complete, depending on number of changes, processing power of the appliance, etc.

# First send a POST to the deployment endpoint:
deploy_url = baseurl + "/operational/deploy"
deploy_response = requests.post(deploy_url, headers=headers)
# Grab the deployment task UID:
deploymentID = json.loads(deploy_response.text)['id']

# Loop while deployment is running:
while deployed is False:
    # Sleep for a few seconds:
    sleep(5)
    # Get list of all deployment tasks:
    tasklist = json.loads(requests.get(deploy_url).text)
    for task in taskList['items']:
        if task['id'] == deploymentID and task['state'] == 'DEPLOYED':
            print("Deployment status is: " + task['state'])
            deployed = True
            return True
        elif task['id'] == deploymentID and task['state'] != 'DEPLOYED':
            # If changes not yet deployed, check again momentarily
            print("Deployment status is: " + task['state'])
            deployed = False

So in the above block, we POST a request to deploy changes. The response of that POST request contains our deployment ID, which we keep to check the status later.

Then the script waits a few seconds, and pulls a list of all deployment tasks. Search through that list for our deployment ID - and check to see if it has been completed yet. If not, wait and run the loop again.

And that’s it! Once we get confirmation that our changes have been deployed - we’ve now routed traffic over the secondary connection (an LTE modem, in my case).

Part 4 - Fail-Back (Delete Static Route)

Okay - this section will be fairly quick. We’ve already looked at authenticating, checking if our route already exists, and how to add a new route. So the only thing remaining is removing our route if we wanted to migrate traffic back to the primary connection.

Same logic applies as earlier. If our path monitoring script runs and finds that loss & latency are within the thresholds we set, then we make a call to the firepower module. First we check to ensure there is a route for us to delete, and if so - proceed with deleting it.

Adding a route is a little more work, since we may need to create network objects. However, when we delete the route - we’ll just leave those objects on the FirePower box. They’ll be there for the next time we need them (which also speeds up deployment times).

To get started, we just need the UID for the route we want to delete.

Remember that code I showed above, where we check to see if the static route exists? And match on our intended subnet / gateway pair? Well, I just made that into a function that will return the UID of the route if it finds it. Easy enough!

Next, we just send an HTTP DELETE to the static routing endpoint - and reference that UID:

del_url = baseurl + "/devices/default/routing/virtualrouters/" + \
          self.globalVR + "/staticrouteentries/" + route['id']
requests.delete(del_url, headers=headers)

Once that succeeds - we use the same logic as earlier to deploy the changes.

After that, our route is removed & traffic should be flowing over our primary connection again.

If it helps anyone - I also threw together a quick diagram to explain the flow of operations here:

Well - that’s it. This was a fun side project to work on over the past week or two. I hadn’t spent much time with the FirePower/FDM APIs just yet. While there was a bit of work in creating all the necessary network objects, the overall process was fairly simple.

It helped tremendously that the FDM API explorer exists, and is available on-box. This utility allows you to see all the available API calls, what parameters they require, and even run test calls from the web UI. This greatly reduced the time needed to figure this out.

Hope this was interesting. If you would like to see the whole project - check out the repo on my GitHub page.

Hey there! First thing’s first - Hope all is well with everyone. The past few months have seen all sorts of craziness going on in the world. I’ve been lucky in the sense that my current job role was already well set up to make an easy transition to working from home all the time. That being said, I know it hasn’t been easy for everyone - and I know quite a few people who have lost their jobs, etc.

Second thing! As I’m sure you may have already seen above - I opted to spend some of my new-found free time trying out a new format. I’ve had a couple of ideas in the past for making short videos, but never forced myself to sit down and give it a shot. So here we are - after about a month of on-and-off work - I finally have something to share. Please give it a look if you’re interested, and I would appreciate any comments & feedback. I still have a few other ideas - so if this one goes well I may pursue producing a few more of these.

Okay - Now onto the real content!

A few months ago I had the chance to pick up a pair of Cisco Catalyst 9100 series access points from work. My home network has been running on Ubiquiti APs for years - but they’re getting old and I desperately needed to replace them. So along comes the Catalyst 9100 APs, which are capable of supporting 802.11ax clients - and why not take the time to upgrade & future proof? I needed some practice anyways, as I have a few customers at work that I think will be interested in these in the near future. I ended up with two 9120AXI APs.

So in the process of trying to dive on the opportunity to play with something new - I completely missed the fact that the 9100 APs have two different product SKUs. One which ships the AP pre-configured with the standard lightweight AP code that you might use if you had an external wireless LAN controller (WLC). And a second SKU that ships the AP with the Catalyst 9800 Embedded WLC (eWLC) software loaded. Of course, I grabbed the lightweight APs without checking.

Product SKUs (Example, using 9120AXI):
 - Standard Lightweight AP: C9120AXI
 - Pre-load with Embedded WLC: C9120AXI-EWC

I wrote in a previous blog about getting MAC address filtering set up on the Catalyst 9800 WLC. When I wrote that blog I was actually using the C9800-CL, which is a virtual machine version of the Catalyst 9800 controller software. I was originally excited to run the controller as a VM and not need a hardware appliance - but then got to thinking that maybe I should try and save VM resources as well. Which led me to looking at the 9800 eWLC.

Finding the Software Image

In order to convert an existing lightweight access point to one running the embedded WLC software - we first need to grab a copy of the software images from Cisco.com. Search for the model of your 9100 access point (in my case, the 9120AXI).

You’ll see two options for Software Type - and it may not immediately be obvious - but we’ll need to look under IOS XE Software to find the eWLC images:

Then we’ll grab the EWC AP image bundle. In my case, I was waiting on a specific feature set that wasn’t available until 17.x - so I downloaded the 17.2.1 software:

Okay - after we’ve done that - let’s drop everything onto a server/laptop/etc which has console connectivity to our AP and a running TFTP server.

Once you un-zip the contents of that AP image bundle, you’ll notice there are quite a number of files. We’ll only need two of them - one image to load onto the AP itself, and one that will get loaded for the WLC software container. Within the image bundle, there will be a readme.txt file that will tell you which image to use with your AP:

ap1g5 : AP1815, AP154x
ap1g4 : AP180x, AP183x, AP185x
ap1g7 : C9115, C9120
ap1g6 : C9117
ap1g6a : C9130
ap3g3 : AP380x, AP280x, AP156x

So in my case, I would use the file named ap1g7 since I had the 9120 APs. In addition, you should also see a file named C9800-AP-iosxe-wlc.binwhich we’ll need to load the controller.

What’s with that second image? Well - the Catalyst 9100 APs include a feature called Application Hosting (also found on some of the Catalyst 9000 series switches). This is equivalent to running a Linux container or Docker container directly on the AP hardware. At time of writing, this is only available for the embedded WLC software. However, there will be a future software update that will allow you to provision other software containers as well (according to the data sheet).

Converting the Access Point

Now we can get started on the actual conversion process. If it’s a new AP, we can just go ahead and boot it up with a console cable connected. If it’s already running something, you’ll likely want to factory reset the AP first. You can find detailed instructions here, but a quick summary - power off the AP, hold the mode button while plugging the AP back in, and keep the mode button held for at least 20-30 seconds. If you’re logged into the console during this time - you’ll actually see the AP counting how long you’ve held the button for. Once that counter shows at least 20 seconds, release the modebutton and allow the AP to reboot.

Once the AP is booted. The default login is Cisco / Cisco. The enable password is also Cisco. You may also want to quickly issue a show version, and take note of the current software version running on the AP (hint: we’ll need that later).

Next - it’s important to note that after we convert the AP - our default CLI will actually be the WLC, not the AP (even when connected via console). So we should configure a hostname and IP address for the AP now:

0xAP# capwap ap hostname <hostname>
0xAP# capwap ap ip <ip-addr> <netmask> <gateway>

Then we can load the new AP image and WLC image. Depending on which software version you’re running today, there is a different command to load the image:

0xAP# ! If you're running software 8.9 or lower, use the following:
0xAP# ap-type mobility-express <TFTP path to AP image> <TFTP path to WLC image>
0xAP# ! If you're running anything above 8.9, use the following:
0xAP# ap-type ewc-ap <TFTP path to AP image> <TFTP path to WLC image>

Once those commands are submitted, the AP will begin copying the software from the TFTP server. The AP will automatically reboot after the image load is completed.

Quick note: For one of my APs, the WLC software didn’t load correctly - and when my AP rebooted I was left with an error that said “EWC-AP in Recovery Mode”. Re-copying the WLC image fixed it without much trouble. If this happens, the AP will print out the command to re-image the WLC software, but I’ll also put it here for reference:

0xAP# ! If you end up in "EWC-AP Recovery Mode", run the following:
0xAP# archive download-sw ewc-ap <TFTP path to WLC image>

Okay - Once our AP has come back from loading up all the new software, we can get on with a bit of minimal config required to access the WLC web UI.

First, we’ll configure the WLC hostname & our local user account. This can be switched over later to your choice of directory service:

0xC9800eWLC# config t
0xC9800eWLC(config)# hostname <hostname>
0xC9800eWLC(config)# user-name <admin username>
0xC9800eWLC(config-user-name)# password <admin password>
0xC9800eWLC(config-user-name)# privilege 15

Next we’ll also provide the WLC with the administrative credentials used to manage the connected APs:

0xC9800eWLC(config)# ap profile <profile-name>
0xC9800eWLC(config-ap-profile)# mgmtuser username <AP admin user> password 0 <AP admin password> secret 0 <AP admin secret>

After that, we’ll go ahead and configure our basic network settings. This will be the IP address & gateway info that will be used to connect to the WLC web UI and SSH:

0xC9800eWLC(config)# interface gigabit 0 
0xC9800eWLC(config-if)# ip address <managemnt IP> <Network mask>
0xC9800eWLC(config-if)# no shut
0xC9800eWLC(config)# ip default-gateway <Gateway IP address>

Lastly - we’ll need to actually enable the web server. In my case, I opted to not enable the standard HTTP server. I only enabled the encrypted SSL-enabled web server:

0xC9800eWLC(config)# ! I only enabled the HTTPS server:
0xC9800eWLC(config)# ip http secure-server
0xC9800eWLC(config)# ! If you wanted to enable the plain-text / unencrypted web server:
0xC9800eWLC(config)# ip http server
0xC9800eWLC(config)# 
0xC9800eWLC(config)# ! Finally - Save the config
0xC9800eWLC(config)# exit
0xC9800eWLC# wr mem