Category: Networking

Categories
MikroTik Networking Windows Server

MikroTik VPN with Windows NPS RADIUS

With the advance of cheap MikroTik routers and ready to use CHR instances, setting up a VPN concentrator for remote access has become an easy task. Moving even further, a single router could provide VPN access and dynamic routing to integrate remote networks to the backbone.

I have started a gig as a consultant and sysadmin for a logistics insurance company, and one of my first proposals was to improve the network acess for road warriors and remote workers.

The past

There was a Proxmox hypervisor, with some Windows 2012 R2 servers, providing Terminal Services, to execute a locally installed client for an ERP system. Proxmox was also using iptables on its the Debian backend of the to masquerade the VM networks with a public IP address, for Internet connectivity, dstnat rules for a NGINX reverse proxy, and RDP for the Windows servers..

I guess we all know having internet-exposed RDP is not a good idea, even if it is running in a non default port, so the former sysadmin transitioned to a SSH tunnel system, where the users connected to the hypervisor via SSH to establish tunnel to the desired server.

This solution, which I considered not elegant, was the only available at the moment due to networking constraints of the VPS provider, so really it was the best they were able to do, and it worked fine for them.

Over the Proxmox hypervisor, they also had a MikroTik CHR instance, with a P1 license, which was used to make a L2TP tunnel to a RB2011UiAS-rm located on their HQ.

Networks behind the tunnel endpoints were routed with static routers, so I configured a quick multi-area OSPF routing system, with the directly connected networks on area 0, along with the /30 network of the tunnel. I added an additional area on both ends, for the future VPN networks. Once OSPF was working as expected, I remove the static routes.

Securing the tunnels

This interconnection via the L2TP tunnel was just plain ol’ L2TP, without IPsec. This is no bueno, and could be improved. Fortunately, IPsec configuration on MikroTik is trivial. Just select “Use IPsec” on both ends, and use the same IPsec pre-shared key.

Configuring via Winbox

This can of course be configured via CLI. Would you like some RouterOS configuration Ansible on next posts? Let me know in the comments. 

/interface l2tp-server server
set authentication=mschap1,mschap2 default-profile=VPN enabled=yes ipsec-secret="PUT_A_SECRET_HERE" use-ipsec=yes

VPN profiles

It’s always a good idea to copy the default-encryption profile, and create a new one based on that template. I set up a local address which was of course, part of the networks announced in a separate area by the OSPF process. I also added a IP pool to be able to provide dynamic addresses for the VPN users.

PPP profile

Maybe you are aware that in the Cisco world, you have to use tcp adjust-mss to adjust the maximum TCP segment size, to advoid fragmentation of packets over the tunnel. Fortunately, this is configured by default on RouterOS.

We don’t want any fragmentation

Finally, to be able to redirect the dial-in to a RADIUS server, we need to instruct the PPP AAA system to use RADIUS, as shown next.

Setting up RADIUS authentication

RADIUS servers are very simple to set up on RouterOS.

Under the RADIUS submenu, add a new server for PPP service, and configure the following parameters.

  • IP address of the radius server
  • RADIUS secret
  • Authentication and accounting ports, usually 1812 and 1813. Some servers use 1645 for accounting. Those are all UDP.
  • REALM if your server supports that extension
  • Which source address should the router use for its NAS-IP-Address
Configuring RADIUS

Using Windows NPS as a RADIUS server

NPS can work without a Certificate Authority but if you are working in an Active Directory environment, you’ll save a lot of headaches by installing the CA role.

Installing NPS and CA
Selecting Roles

In my particular scenario, the server was not part of a domain so the certificate generation and association was skipped.

Once the roles have been configured, I headed to the NPS service configuration, and add new RADIUS client.

Make sure to match the RADIUS secret and the source IP address as you configured on the MikroTik side.

New RADIUS Client

Next, the network access policies. I wanted to match the NAS IPv4 address, and the authentication types. If you are not familiar with the RADIUS lang, NAS stands for Network Access Server, which in this case, is the MikroTik router which provides the VPN service.

MikroTik source address

I had to use a CHAP fallback due to some legacy devices withuout MSCHAP support.

Authentication methods

Next, I added a new Access condition, matched the NAS address once again, and selected the local server as point of authentication.

Authenticate on this server

Once everything was properly configured, I set up the VPN client on my side, which looks like as follows. The idea of using NPS as RADIUS was to be able to use my Windows account credentials for the VPN.

VPN client on Windows 10

I verified the successful authentication on the router logs, and the VPN was sucessfully connected.

Categories
Ansible Networking Projects

Creating passwordless logins with Ansible

What kind of users? Well, a special user called Ansible, which will use SSH keys to login into remote devices, allowing for full automation on playbooks.

Creating a new key

If you have been following the series, maybe you remember that we already created keys on the Juniper Junos SSH Keys post.

To create a new key, let’s issue the ssh-keygen command as follows. The -f flag tells the output path, and the -C flags specifies a comment.

$ ssh-keygen -f ansible.key -C ansible-login-passwordless

This should output two files, ansible.key and ansible.key.pub.

The public key should look something like this.

$ cat ansible.key.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtJUPfzJY6vKqLUssPPQe+LD7qRmIPbVhb/1i4Qab7T0Vf3x+ItfJyV4Ej4FsnRSU8iMU8J5eIdcetGQfsmwIZAm8glB0T6En5F9lvq2Yd+3RKIvxM3UlrIH6EaRedhsRUyV96CHfIO2nVqS9dmFfgrOJMIOwfTWIiRDNczUPw7aqw0FExslw9ZC0FO/1A6hYgofkGLrdIu9gK/WkNg5BE1EUCYPqbDBEHnnhv3C33LqiSJZnXJyqu53qz+jlv+1LZxerNHuovMGZMkjQsBo2f3r9Gk/9HqBmT0rcLr5prm4CqqryJ3S9VyVVlF599BlqYMuMjj+fCj277R8kSnLxl ansible-login-passwordless

Of course we need an inventory to use, which has the following content.

$ cat inventory.yml 
---
all:
  hosts:
    vars:
      ansible_ssh_user: ansible
      ansible_ssh_private_key_file: ansible.key
      ansible_python_interpreter: auto_silent
    hosts:
      localhost:

This inventory only has one host, localhost, and uses three main variables:

  • ansible_ssh_user, which tell Ansible to use the user ansible
  • ansible_ssh_private_key_file, which indicates the key for this user
  • ansible_python_interpreter, just to avoid non needed logs

The playbook will looks like this. Notice we don’t need to gather_facts here, and we will instruct ansible to use become to gain privileges on the destination host.

---
- hosts: all
  become: yes

  tasks:

    - name: Make sure we have a "wheel" group
      group:
        name: wheel
        state: present

    - name: Allow 'wheel' group to have passwordless sudo
      lineinfile:
        dest: /etc/sudoers
        state: present
        regexp: '^%wheel'
        line: '%wheel ALL=(ALL) NOPASSWD: ALL'
        validate: 'visudo -cf %s'
        
    - name: Create "ansible" user
      user:
        name: ansible
        comment: Ansible Automation User
        groups: wheel

    - name: Add ssh key
      authorized_key:
        user: ansible
        state: present
        key: "{{ lookup('file', './ansible.key.pub') }}"

First, we want to make sure there is a group called wheel which will group users with administrative privileges.

Then, the /etc/sudoers file will be edited by allowing the wheel group to gain privileges, with a failsafe using a visudo validation.

Once the group has been created, the new user will be created, and a SSH key will be added to it.

It seems allright, but, how should we run the playbook, if the default user is ansible and this user does not exists yet? Let’s give it a try.

$ ansible-playbook create-user.yml -i inventory.yml 

PLAY [all] ************************************************************************

TASK [Gathering Facts] ************************************************************
fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ansible@localhost: Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************
localhost                  : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

It fails, as expected, because the ansible user does not exists in the host.

Well, there is a way to provide a one-time password by connecting a as different user. You will need to install sshpass with your favourite package manager, like apt install sshpass.

One installed, run the playbook once again with the following arguments:

  • -e “ansible_ssh_user=xxxxx”, where xxxxx is a valid user on the remote host
  • -kK, which tell Ansible to ask for a login and a sudo password
$ ansible-playbook create-user.yml -i inventory.yml -e "ansible_ssh_user=arturo" -kK
SSH password: 
BECOME password[defaults to SSH password]: 

PLAY [all] ************************************************************************

TASK [Gathering Facts] ************************************************************
ok: [localhost]

TASK [Make sure we have a "wheel" group] ******************************************
changed: [localhost]

TASK [Allow 'wheel' group to have passwordless sudo] ******************************
changed: [localhost]

TASK [Create "ansible" user] ******************************************************
changed: [localhost]

TASK [Add ssh key] ****************************************************************
changed: [localhost]

PLAY RECAP ************************************************************************
localhost                  : ok=5    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Awesome, we have sucessfully created a new user!

Let’s try to connect using the ansible user with its key, as defined in the playbook.

$ ansible -m ping -i inventory.yml all
localhost | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "ping": "pong"
}

Stay tuned for more automation using Ansible.

Categories
Ansible Juniper Networking

Ansible and Juniper – SSH Keys and Prompts

On previous posts we’ve seen how to connect with Ansible using credentials stored in a inventory file, and using SSH keys for authentication.

However, it isn’t a good idea to store credentials in plain text files, neither have to rebuild your inventory when you want to switch over to key authentication.

A possible solution is to first ask for credentials, run a playbook to install the SSH key, and then use this key for authentication on later playbooks.

You can find all the files for this post on the following repo.

https://github.com/baldoarturo/ansible-ssh-keys

Variable prompts

  vars_prompt:
    - name: "ansible_user"
      prompt: "Username"
      private: no

The vars_prompt section is used to prompt the user for information, which is stored in variables. System variables can be populated, for example the ansible_user and ansible_password variables, allowing us to provide credentials to connect.

Take a look to the new version of the uptime playbook.

---
- hosts: all
  gather_facts: no

  vars_prompt:
    - name: "ansible_user"
      prompt: "Username"
      private: no
      unsafe: yes

    - name: "ansible_password"
      prompt: "Password"
      private: yes
      unsafe: yes

  tasks:
    - name: Get uptime
      junos_command:
        commands:
            - show system uptime
      register: uptime
    
    - name: Show uptime
      debug: var=uptime

We’re prompting for the username and password on the vars_prompt section. The private settings indicates if the user input should appear on the screen. The unsafe option allows to enter special chars.

The task to execute are:

  • Get system uptime via the junos_command module, with “show system uptime”
  • Print the result using debug

And the new (and definitive) inventory looks like this now.

all:
    hosts:
      "192.168.227.101":
    vars:
      ansible_connection: netconf
      ansible_network_os: junos
      ansible_ssh_private_key_file: juniper-hosts.key
      ansible_python_interpreter: auto_silent

The ansible_python_interpreter variable is set to auto_silent just to avoid the warning about no Python interpreters on the remote end.

Let’s give the playbook a run, trying to login with user and password. If you have not been following the Ansible series, let me tell you that there is an user admin with a password of Password$1 on the router. Note that the password won’t be seen on the screen.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ansible-playbook junos-auth-with-key.yaml -i junos-hosts.yaml 
Username: admin
Password: 

PLAY [all] ******************************************************************************************************************

TASK [Get uptime] ***********************************************************************************************************
ok: [192.168.227.101]

TASK [Show uptime] **********************************************************************************************************
ok: [192.168.227.101] => {
    "uptime": {
        "ansible_facts": {
            "discovered_interpreter_python": "/usr/bin/python"
        }, 
        "changed": false, 
        "failed": false, 
        "stdout": [
            "Current time: 2020-01-13 17:12:32 UTC\nSystem booted: 2020-01-13 14:55:46 UTC (02:16:46 ago)\nProtocols started: 2020-01-13 14:56:03 UTC (02:16:29 ago)\nLast configured: 2020-01-12 16:09:02 UTC (1d 01:03 ago) by admin\n 5:12PM  up 2:17, 2 users, load averages: 0.00, 0.00, 0.00"
        ], 
        "stdout_lines": [
            [
                "Current time: 2020-01-13 17:12:32 UTC", 
                "System booted: 2020-01-13 14:55:46 UTC (02:16:46 ago)", 
                "Protocols started: 2020-01-13 14:56:03 UTC (02:16:29 ago)", 
                "Last configured: 2020-01-12 16:09:02 UTC (1d 01:03 ago) by admin", 
                " 5:12PM  up 2:17, 2 users, load averages: 0.00, 0.00, 0.00"
            ]
        ]
    }
}

PLAY RECAP ******************************************************************************************************************
192.168.227.101            : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Great, the prompts work.

What if we try to login with the user ansible we configured on the previous post? This user has an SSH key installed on the router, and the local private key is on juniper-hosts.key.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ansible-playbook junos-auth-with-key.yaml -i junos-hosts.yaml 
Username: ansible
Password: 

PLAY [all] ******************************************************************************************************************

TASK [Get uptime] ***********************************************************************************************************
ok: [192.168.227.101]

TASK [Show uptime] **********************************************************************************************************
ok: [192.168.227.101] => {
    "uptime": {
        "ansible_facts": {
            "discovered_interpreter_python": "/usr/bin/python"
        }, 
        "changed": false, 
        "failed": false, 
        "stdout": [
            "Current time: 2020-01-13 17:32:05 UTC\nSystem booted: 2020-01-13 14:55:46 UTC (02:36:19 ago)\nProtocols started:
 2020-01-13 14:56:03 UTC (02:36:02 ago)\nLast configured: 2020-01-12 16:09:02 UTC (1d 01:23 ago) by admin\n 5:32PM  up 2:36, 
1 user, load averages: 0.00, 0.01, 0.00"
        ], 
        "stdout_lines": [
            [
                "Current time: 2020-01-13 17:32:05 UTC", 
                "System booted: 2020-01-13 14:55:46 UTC (02:36:19 ago)", 
                "Protocols started: 2020-01-13 14:56:03 UTC (02:36:02 ago)", 
                "Last configured: 2020-01-12 16:09:02 UTC (1d 01:23 ago) by admin", 
                " 5:32PM  up 2:36, 1 user, load averages: 0.00, 0.01, 0.00"
            ]
        ]
    }
}

PLAY RECAP ******************************************************************************************************************
192.168.227.101            : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Excellent, by using the user ansible without password, it will fallback to the key authentication.

Categories
Ansible Juniper Networking

Ansible and Juniper Junos – Using SSH Keys

Previous posts introduced basics connection methods to manage Juniper devices using Ansible playbooks. The inventory files had sensitive information and credentials which should not be accessible to anyone.

SSH and NETCONF over SSH requires client authentication, for example with and username and password, which could looks like this:

admin> show configuration system login 
user admin {
    uid 2000;
    class super-user;
    authentication {
        encrypted-password "$1$./TeE4CZ$uAMigDedlRuuJgcZx4hYk0"; ## SECRET-DATA
    }
}

If you are a frequent SSH user, maybe you are aware that there are other login methods besides using usernames and passwords. By using a key-pair, with public and private keys, a password is no longer needed. The public key is installed on the remote host, and the private key is kept on the control node.

Although by using keys a password is no longer needed, a passphrase can be used with a key, adding an additional security factor to the connection. In fact, using SSH keys with passphrases is considered best practice. However, a private key with a passphrase is less useful for scheduled automation tasks because an operator may not be available to enter the passphrase at the scheduled time.

Creating a Key Pair

A key pair is a set of two cryptographic keys, a public one and a private one. The public key, as its name says, is the one we expose to the public. The private key, must be kept in a secure location.

To create a key pair, lauch ssh-keygen on a new console and follow the prompts. This utility will create two files, which are the public and private keys. Use the -f flag to specify the destination of the output files.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ssh-keygen -f ./juniper-hosts.key
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in ./juniper-hosts.key.
Your public key has been saved in ./juniper-hosts.key.pub.
The key fingerprint is:
SHA256:Kc/MZ11dLlXpcrK9PKO4L6XzaTrdczaek1ydzaFTFXw arturo@arturo-ThinkPad-L440
The key's randomart image is:
+---[RSA 2048]----+
|              ..+|
|               oE|
|              . =|
|         .   o O.|
|      . S     @.B|
|       *   . * +=|
|        = o = = +|
|         o =.o.%+|
|           +X=o+B|
+----[SHA256]-----+
arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ls 
juniper-hosts.key  juniper-hosts.key.pub

Now that we have created the key pair, let’s examine them to find out how a key looks like.

This is the private key, which in fact is a plain text file with a RSA key inside it.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ cat juniper-hosts.key
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAn0XtdTPJxWHQHeWi8IwSOGtsgWwW3z86Z91edH0dBS6SWDzm
seWshSTD2PdD8EU0mGac1V9+rWIJYIw0VZlpTeEKiNnS+1/feHxln+S0LLwn91E6
fgVRt6kE/9uTv217Pa5eP5HLbPuSwPsVMwRgdCJL1pYIFagJLyrsakDWE0qDtBXZ
0fendmUr6NrrCLofjHBkRASviJ9E4Vvx4YqrG9ak+2jX21g0LRNxZ/nFv2uHwVeI
gibC+1JiZ/IGaBSNCovF131wb1AUyLm5Z5DNvM6hu2C6+cMTNNQ5fiBfgMpryboW
4+MVCGJQ31EthyFEx0XPgjos/EcS9Pp436OppwIDAQABAoIBAQCVczEws4qV2oVF
OG/fFSAXrr0e6ATCMHsmcLKrzaZIcX3CrEqwDNoICQp4cPRf5SBIDKkHElc0a/Ru
ksCcvZnxCMQwy2vMkhaH4PoewaRLAbbiu2aOT4FxO3jEeA44JovowdAQCEcAmUMI
L9GhkG7NKk1NKnSllYogpz81KGd3qw21sRqb1NTLAlYnE4KOhJz+GKmJV8NdAaRj
zjkVeaLf3t/FCxPRhdAtoADkRQSS1KSCjU0hx0lDCmsdJzM8gFJykltzzBQJ9tZu
voPZ0TkaIjrR7o8+Oez/pkvUSa1AJPhmH7l6P3RqHdSzMJGQraM1yuvBOTixbnkt
lsQ26tZpAoGBAM/CbeNhJvGfl18kLLLLSsNb4femXXlBimo6TW4tOGdOunD6+fyt
LiA0FMLpWfvAh++/yvWX/jee+E3uXkDLLfQVWqWBbfFqIhU4VOKLpMvE5sMQzkOc
OyooZNR5hui9e2+eU5P2qND/MUVy4YrUBdHtK58Or/cqYT6sHA78e9OtAoGBAMRB
Y8F79BaqYoH7x0lJf51A45U5rLzKom8eJ+aJujDx9RgDvCEWCZmZ53q6MjvNgNBp
v5AZptCDn0nIfAVOn2hCmTIPs1IvaLgVfMtmufxzdM1aGdYF7wEu0u7DV+Dzspf2
h9Q/9C4Or4YZ5oe25Qf5mkwb+xnnmTZCWZETC70jAoGALvskplp91/3i2RzxDq1y
BqNsgfgZAyaTClqMz/Fh49qlxo66oSz4VUfxufHS618qXkjcuJTaY/GK7PSOU9Ce
X6fEi9Cs7/60HmBSsbgqV/n6xPmz6w4VQv9HbdTdcRwIIcGH3NnWawyKM846uo4f
ks0zJBDKMfZfbzC0V5840TECgYEAraoTZR6Tsw7ZFp6/DZoNZBEMknsz4OgK7vsn
YbiUW0VwleyQKFMA8bwf+xkS5JqIF2TMT+5zD+a5KKhRHr0hEDiGqab9DofHScYx
5SelAsEEJcdKP3qGsWxG2WNguz3K1vAf5/Ej2THDno4C0itE5la4dAr6m0S27i2u
ZlMNOzMCgYEAr3/7pkN2LdUCVZYEjVMAW4YxSzi4R5JeixyidZwihVF4GqqqJ2Nl
VaOQzleNYUg23QWBs/n6yz2iDQaQdKXNCcOwrsQYzkX0aMuEz+4iWXkDKcRIvGWt
5H/7ShsKoXFsvwbjXkaTnirBMqAJh4jyv4R3oAqIy966zJ5K2vd0nT4=
-----END RSA PRIVATE KEY-----

And the public keys looks like this.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ cat juniper-hosts.key.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfRe11M8nFYdAd5aLwjBI4a2yBbBbfPzpn3V50fR0FLpJYPOax5ayFJMPY90PwRTSYZpzVX36tYglgjDRVmWlN4QqI2dL7X994fGWf5LQsvCf3UTp+BVG3qQT/25O/bXs9rl4/kcts+5LA+xUzBGB0IkvWlggVqAkvKuxqQNYTSoO0FdnR96d2ZSvo2usIuh+McGREBK+In0ThW/Hhiqsb1qT7aNfbWDQtE3Fn+cW/a4fBV4iCJsL7UmJn8gZoFI0Ki8XXfXBvUBTIublnkM28zqG7YLr5wxM01Dl+IF+AymvJuhbj4xUIYlDfUS2HIUTHRc+COiz8RxL0+njfo6mn arturo@arturo-ThinkPad-L440

This keypair can authenticate an user which connects via SSH.

Installing the public key on Junos

We already know that a basic authentication schema on Junos looks like this.

admin> show configuration system login 
user admin {
    uid 2000;
    class super-user;
    authentication {
        encrypted-password "$1$kYNQ.bg0$4T3W7GAPuXwsX3nbbsRCb/"; ## SECRET-DATA
    }
}

The main idea of using SSH keys, is to avoid user interaction, by trusting the keys instead of a credentials combination.

As seen above, the keys are plain text files. We need to install the public key on the Junos configuration, either configuring it manually, or using Ansible to configure it.

Manually configuring the key

I added a new user called ansible, set its class as super-user and configured its authentication as ssh-rsa.

[edit]
admin# show system login | display set                                                                                                                                 
set system login user admin uid 2000
set system login user admin class super-user
set system login user admin authentication encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."
set system login user ansible uid 2001
set system login user ansible class super-user
set system login user ansible authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfRe11M8nFYdAd5aLwjBI4a2yBbBbfPzpn3V50fR0FLpJYPOax5ayFJMPY90PwRTSYZpzVX36tYglgjDRVmWlN4QqI2dL7X994fGWf5LQsvCf3UTp+BVG3qQT/25O/bXs9rl4/kcts+5LA+xUzBGB0IkvWlggVqAkvKuxqQNYTSoO0FdnR96d2ZSvo2usIuh+McGREBK+In0ThW/Hhiqsb1qT7aNfbWDQtE3Fn+cW/a4fBV4iCJsL7UmJn8gZoFI0Ki8XXfXBvUBTIublnkM28zqG7YLr5wxM01Dl+IF+AymvJuhbj4xUIYlDfUS2HIUTHRc+COiz8RxL0+njfo6mn arturo@arturo-ThinkPad-L440"

The SSH public key is copied and pasted between double quotes.

Using Ansible to configure the key

Altough the key can be configured manually on the remote hosts, what if we have hundreds, or thousands of hosts to configure?

The idea behind this series of posts is to use Ansible whenever possible, so, let’s write a quick playbook to automate the key configuration.

rturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ cat junos-install-ssh-key.yaml 
---
- hosts: all
  gather_facts: no

  vars:
    - auth_key: "{{lookup('file', '{{ key_file }}')}}"    

  tasks:
    - name: Install SSH key on remote host
      junos_config:
        lines:
          - set system login user ansible authentication ssh-rsa "{{ auth_key }}"
          - set system login user ansible class super-user

The playbook starts as usual, matching all hosts in the inventory, and without gathering facts, just for the sake of speed.

On vars, we are using the lookup plugin to read from a file and store its contents on a variable. Lookup can retrieve data from multiple sources, for example, take a secret from Hashicorp’s Vault. In this scenario, it will read a file which name is take from the key_file variable from the inventory.

It is possible to set a fixed file name on the playbook, but by taking the filename as a variable from the inventory, it gives us more flexibility. We could have multiple keys and rotate them by just changing the file name on the inventory, or use different keys per host group, and still apply the playbook to the full inventory, while using the proper key for each group.

The inventory for this playbook looks like the following. Notice the key_file variable which tells the playbook where to look for the key.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ cat junos-hosts.yaml 
all:
    hosts:
      "192.168.227.101":
    vars:
      ansible_connection: netconf
      ansible_network_os: junos
      ansible_user: admin
      ansible_password: Password$1
      key_file: juniper-hosts.key.pub

Running the playbook to install the key

The current configuration of router logins is:

admin> show configuration system login 
user admin {
    uid 2000;
    class super-user;
    authentication {
        encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."; ## SECRET-DATA
    }
}

Let’s run the playbook to apply the new configuration which will create the ansible user, set ssh-rsa authentication for it, and set its class as super-user.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ansible-playbook junos-install-ssh-key.yaml -i junos-hosts.yaml 

PLAY [all] *****************************************************************************

TASK [Install SSH key on remote host] **************************************************
changed: [192.168.227.101]

PLAY RECAP *****************************************************************************
192.168.227.101            : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Ok, the playbook executed with no errors, and Ansible says there is 1 changed host, which is what we expected.

Let’s check the router configuration again.

admin> show configuration system login    
user admin {
    uid 2000;
    class super-user;
    authentication {
        encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."; ## SECRET-DATA
    }
}
user ansible {
    uid 2001;
    class super-user;
    authentication {
        ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfRe11M8nFYdAd5aLwjBI4a2yBbBbfPzpn3V50fR0FLpJYPOax5ayFJMPY90PwRTSYZpzVX36tYglgjDRVmWlN4QqI2dL7X994fGWf5LQsvCf3UTp+BVG3qQT/25O/bXs9rl4/kcts+5LA+xUzBGB0IkvWlggVqAkvKuxqQNYTSoO0FdnR96d2ZSvo2usIuh+McGREBK+In0ThW/Hhiqsb1qT7aNfbWDQtE3Fn+cW/a4fBV4iCJsL7UmJn8gZoFI0Ki8XXfXBvUBTIublnkM28zqG7YLr5wxM01Dl+IF+AymvJuhbj4xUIYlDfUS2HIUTHRc+COiz8RxL0+njfo6mn arturo@arturo-ThinkPad-L440"; ## SECRET-DATA
    }
}

There is a new user called ansible, with all the parameters we specified. That’s great!

Authenticating using keys

I wrote another playbook to show the system uptime

---
- hosts: all
  gather_facts: no

  tasks:
    - name: Get uptime
      junos_command:
        commands:
            - show system uptime
      register: uptime
    
    - name: Show uptime
      debug: var=uptime

And our inventory now looks like this.

all:
    hosts:
      "192.168.227.101":
    vars:
      ansible_connection: netconf
      ansible_network_os: junos
      ansible_user: ansible
      ansible_ssh_private_key_file: juniper-hosts.key
      ansible_python_interpreter: auto_silent

There is no plain text password, but instead, by setting up the ansible_ssh_private_key_file variable, we are instructing Ansible to authenticate using the private key.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ansible-playbook junos-auth-with-key.yaml -i junos-hosts-w-key.yaml 

PLAY [all] *****************************************************************************

TASK [Get uptime] **********************************************************************
ok: [192.168.227.101]

TASK [Show uptime] *********************************************************************
ok: [192.168.227.101] => {
    "uptime": {
        "ansible_facts": {
            "discovered_interpreter_python": "/usr/bin/python"
        }, 
        "changed": false, 
        "failed": false, 
        "stdout": [
            "Current time: 2020-01-12 16:25:10 UTC\nSystem booted: 2020-01-12 13:42:06 UTC (02:43:04 ago)\nProtocols started: 2020-01-12 13:42:27 UTC (02:42:43 ago)\nLast configured: 2020-01-12 16:09:02 UTC (00:16:08 ago) by admin\n 4:25PM  up 2:43, 3 users, load averages: 0.08, 0.02, 0.01"
        ], 
        "stdout_lines": [
            [
                "Current time: 2020-01-12 16:25:10 UTC", 
                "System booted: 2020-01-12 13:42:06 UTC (02:43:04 ago)", 
                "Protocols started: 2020-01-12 13:42:27 UTC (02:42:43 ago)", 
                "Last configured: 2020-01-12 16:09:02 UTC (00:16:08 ago) by admin", 
                " 4:25PM  up 2:43, 3 users, load averages: 0.08, 0.02, 0.01"
            ]
        ]
    }
}

PLAY RECAP *****************************************************************************
192.168.227.101            : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

This is great, now Ansible authenticates using the SSH key. You maybe are thinking:

“Do i have to edit my inventory every time i want to use keys?”

The answer is, no, and in the next post we will set a interactive prompt to connect using user and password to run the first playbook, which will configure the key, and then we will run all the other playbooks connecting with this key.

Stay tuned for more!

Categories
MikroTik Netbox Networking

Contributing to Netbox devices library

The Netbox community has launched a repository for standard devices.

https://github.com/netbox-community/devicetype-library

This comes handy for new and existing Netbox installations, because now you can populate your database with predefined device models for the most common networking gear manufacturers.

I have contributed to the repository and created the entire set of MikroTik routers and switches, updated to December 2019.

https://github.com/netbox-community/devicetype-library/tree/master/device-types/MikroTik

Feel free to use it and contribute!
And as always, you can find me on Github at https://github.com/baldoarturo

Categories
Ansible Juniper Networking

Ansible and Juniper Junos – Playbook Variables and Raw Commands

As seen on previous posts, Ansible has a decent catalog of modules for Junos, allowing various operations like:

  • Manage layer 2 and layer 3 intefaces
  • Manage LAG and LACP
  • Configure VLANs and VRFs
  • And many others

But, what happens if you want to automate an operation which doesn’t has its own dedicated module. For example, routing protocols.

Enter junos_config

As you might guess, the junos_config module allows us to execute arbitrary commands on the Junos CLI, using Ansible. This maybe doesn’t makes sense for a single host (or does it?), but when you have a dozen, a hundred, or thousands hosts with similar configuration, it is a life-saver.

Using Ansible, you could update the NTP servers for your entire fleet, or just a group of hosts. Maybe you need to bulk update passwords, change routing protocols keys, deploy a new service, or any other repetitive task.

Even if Ansible can help you with routine and repetitive tasks, another of its main features is idempotency. For Ansible it means after 1 run of a playbook to set things to a desired state, further runs of the same playbook should result in 0 changes. In simplest terms, idempotency means you can be sure of a consistent state in your environment. This can ensure consistent configuration across all the devices.

Configuring OSPF in Junos using Ansible

For this example scenario we’ll use two Juniper vSRX instances linked to each other via ge-0/0/1.

The only commands to be entered from the router CLI are:

  • IP addresses for the management interface ge-0/0/0.
    • vSRX01 ge-0/0/0 192.168.100.101/24
    • vSRX02 ge-0/0/0 192.168.100.102/24
  • Enabling SSH, and NETCONF over SSH

The objective is to use Ansible to configure the rest.

  • Create a new non-root user
    • As this is a lab environment, we won’t be using AAA, and the user will be authenticated locally with admin for the username, and Password$1 for the password.
  • Proper IP address for the interfaces
    • vSRX01 ge-0/0/1 will use 10.10.10.1/30
    • vSRX02 ge-0/0/1 will use 10.10.10.2/30
  • Establish a OSPF relation between them to exchange routes
    • Interfaces in area 0, no authentication for now.

Cloud tap1 is connected to a host-only network of the GNS3 VM, in fact a tap inteface, in the 192.168.100.0/24 subnet. This keeps the management interfaces in a common subnet, which is also accessible to the physical machine.

This is a reference configuration for vSRX01:

admin@vSRX01# show | display set             
set version 12.1X47-D20.7
set system host-name vSRX01
set system root-authentication encrypted-password "$1$V7oU.Wtz$ferb8fVB.FNBf/kqGSr.V1"
set system login user admin uid 2000
set system login user admin class super-user
set system login user admin authentication encrypted-password "$1$tUJBllq0$XVWiCQYBv5H9KgEwK1Ovj0"
set system services ssh
set system services netconf ssh
set system services web-management http interface ge-0/0/0.0
set system syslog user * any emergency
set system syslog file messages any any
set system syslog file messages authorization info
set system syslog file interactive-commands interactive-commands any
set system license autoupdate url https://ae1.juniper.net/junos/key_retrieval
set interfaces ge-0/0/0 unit 0 family inet address 192.168.100.101/24
set interfaces ge-0/0/1 unit 0
set security zones security-zone MGMT host-inbound-traffic system-services ssh
set security zones security-zone MGMT host-inbound-traffic system-services ping
set security zones security-zone MGMT host-inbound-traffic system-services snmp
set security zones security-zone MGMT host-inbound-traffic system-services netconf
set security zones security-zone MGMT interfaces ge-0/0/0.0
set security zones security-zone BACKBONE host-inbound-traffic system-services ping
set security zones security-zone BACKBONE host-inbound-traffic protocols ospf
set security zones security-zone BACKBONE interfaces ge-0/0/1.0

Using Inventory Variables

A couple changes has been made to the previous inventory. Now we have two hosts entries on the all group, and each host has additional information below its definition.

all:
  hosts:
    "192.168.100.101":
      ipv4_address: 10.10.10.1/24
    "192.168.101.102":
      ipv4_address: 10.10.10.2/24
  vars:
    ansible_connection: netconf
    ansible_network_os: junos
    ansible_user: admin
    ansible_password: Password$1

The new field, ipv4_address, contains information that will be passed to the playbook. Inside the playbook, this change is reflected in the following fashion.

---
- hosts: all
  gather_facts: no

  tasks:

  - name: Config ge-0/0/1
    junos_l3_interfaces:
      config:
        - name: ge-0/0/1
          ipv4:
            - address: "{{ ipv4_address }}"
      state: replaced

Can you see the {{ ipv4_address }} field?

The variable in brackets is assigned with the respective information from the inventory. This means, when Ansible executes the module on a specific host, the ipv4_address value will be as declared on the playbook.

This is a very simple and unefficient method, because the interface is fixed on the playbook, and it could be assigned from the inventory file, but it’s enough for a first approach. Stay tuned for more scenarios with advanced methods.

Another change it that the desired state of the configuration now is replaecd instead of merged. This is idempotency. No matter what you have under ge-0/0/1, Ansible will replace the configuration with what builds from the inventory and the module.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ********************************************************************************************************************

TASK [Config ge-0/0/1] ********************************************************************************************************
[WARNING]: Platform linux on host 192.168.100.101 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.100.101]
[WARNING]: Platform linux on host 192.168.100.102 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.100.102]

TASK [Put ge-0/0/1 on OSPF area 0] ********************************************************************************************
changed: [192.168.100.101]
changed: [192.168.100.102]

PLAY RECAP ********************************************************************************************************************
192.168.100.101            : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
192.168.100.102            : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Ok, we can see that Ansible identified needed changes on both hosts, so we can run the playbook now.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ********************************************************************************************************************

TASK [Config ge-0/0/1] ********************************************************************************************************
[WARNING]:  statement not found

[WARNING]: Platform linux on host 192.168.227.102 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.227.102]
[WARNING]: Platform linux on host 192.168.227.101 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.227.101]

PLAY RECAP ********************************************************************************************************************
192.168.227.101            : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
192.168.227.102            : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The output is the same, but the changes are being effective now. Go ahead and check the configuration on the routers.

admin@VMX01> show configuration interfaces 
ge-0/0/1 {
    unit 0 {
        family inet {
            address 10.10.10.1/24;
        }
        family inet6;
    }
}
fxp0 {
    unit 0 {
        family inet {
            address 192.168.227.101/24;
        }
    }
}

As expected, the ge-0/0/1 is configured now with the correct configuration.

Now, try to add an additional dummy address to VMX1 ge-0/0/1.

admin@VMX02# set interfaces ge-0/0/1 unit 0 family inet address 192.168.77.44/24

admin@VMX02# commit

admin@VMX02> show configuration interfaces ge-0/0/1
ge-0/0/1 {
    unit 0 {
        family inet {
            address 10.10.10.2/24;
            address 192.168.77.44/24;
        }
        family inet6;
    }
}

Run the playbook once again, and see what happens.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ********************************************************************************************************************

TASK [Config ge-0/0/1] ********************************************************************************************************
[WARNING]:  statement not found

[WARNING]: Platform linux on host 192.168.227.101 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

ok: [192.168.227.101]
[WARNING]: Platform linux on host 192.168.227.102 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.227.102]

PLAY RECAP ********************************************************************************************************************
192.168.227.101            : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
192.168.227.102            : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

arturo@arturo-ThinkPad-L440:~/Desktop/ansible$

Ansible tell us that one host has changed, so it is applying the configuration once again. This is idempotency stands for.

Using junos_config

I changed the inventory once again, and it looks like this now.

all:
   hosts:
     "192.168.227.101":
       ipv4_address: 10.10.10.1/24
       ospf_area_0: set protocols ospf area 0.0.0.0 interface ge-0/0/1
     "192.168.227.102":
       ipv4_address: 10.10.10.2/24
       ospf_area_0: set protocols ospf area 0.0.0.0 interface ge-0/0/1
   vars:
     ansible_connection: netconf
     ansible_network_os: junos
     ansible_user: admin
     ansible_password: Password$1

The ospf_area_0 item now is a raw Junos CLI command to run OSPF on area 0 on ge-0/0/1.

The playbook now has a new task which uses the junos_config module to run the previous set command. Notice the lines parameter is a list, so it is possible to run an arbitrary number of configuration commands. There is also a confirm_commit parameter to apply the new configuration.

---
- hosts: all
  gather_facts: no

  tasks:

  - name: Config ge-0/0/1
    junos_l3_interfaces:
      config:
        - name: ge-0/0/1
          ipv4:
            - address: "{{ ipv4_address }}"
      state: replaced
  
  - name: Put ge-0/0/1 on OSPF area 0
    junos_config: 
      lines:
        - "{{ ospf_area_0 }}"
      confirm_commit: yes

Run this new playbook and check the results.

arturo@arturo-ThinkPad-L440:~/Desktop/ansible$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ********************************************************************************************************************

TASK [Config ge-0/0/1] ********************************************************************************************************
[WARNING]: Platform linux on host 192.168.100.101 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.100.101]
[WARNING]: Platform linux on host 192.168.100.102 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.100.102]

TASK [Put ge-0/0/1 on OSPF area 0] ********************************************************************************************
changed: [192.168.100.101]
changed: [192.168.100.102]

PLAY RECAP ********************************************************************************************************************
192.168.100.101            : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
192.168.100.102            : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Well, something has changed.

admin@vSRX01> show interfaces terse | match ge- 
ge-0/0/0                up    up  
ge-0/0/0.0              up    up   inet     192.168.100.101/24
ge-0/0/1                up    up  
ge-0/0/1.0              up    up   inet     10.10.10.1/24   
ge-0/0/2                up    up  
ge-0/0/3                up    up

admin@vSRX01> show ospf neighbor 
Address          Interface              State     ID               Pri  Dead
10.10.10.2       ge-0/0/1.0             Full      10.10.10.2       128    3

Everything worked as expected. The interface is now configured, and OSPF is running on both ge-0/0/1.0 interfaces

Categories
Ansible Juniper Networking

Ansible and Juniper Junos – Interfaces

Previously we had made our first incursions connecting an Ansible control node with a Juniper router. In this post, we’ll see how to retrieve information about the router interfaces, both layer 2 and layer 3, and configure new interfaces.

The official Ansible modules reference will be your main guide for any additional information.
https://docs.ansible.com/ansible/latest/modules/list_of_network_modules.html#junos

If you are interested on this subject, but don’t have access to physical gear, most of it should work on virtual appliances like vMX, vQFX, which you can operate on a stand-alone mode or on a network environment like GNS3 or EVE-NG.

Juniper vLabs will also give you an introduction to the Juniper platform.
https://jlabs.juniper.net/vlabs/portal/index.page

Layer 2 Interfaces

A basic layer 2 interface configuration in Junos looks like this:

ge-0/0/1 {
     description "L2 interface";
     speed 1g;
     unit 0 {
         family ethernet-switching {
             interface-mode access;
             vlan {
                 members vlan30;
             }
         }
     }
 }

This configuration can be written as an Ansible playbook like this:

- name: "Replace provided configuration with device configuration"
  junos_l2_interfaces:
    config:
      - name: ge-0/0/1
        access:
          vlan: v30
    state: merged

Currently, I do not have any EX series or QFX series to decomission and run tests against it, so stay tuned for any updates on this.

The official module documentation is on https://docs.ansible.com/ansible/latest/modules/junos_l2_interfaces_module.html.

Layer 3 Interfaces

A basic layer 3 interface configuration in Junos looks like this:

ge-0/0/1 {
     unit 0 {
         family inet {
             address 192.168.1.10/24;
         }
     }
 }

This configuration can be written as an Ansible playbook like the following, using the same format as the last post.

---
- hosts: all
  gather_facts: no

  tasks:

  - name: Config ge-0/0/1
    junos_l3_interfaces:
      config:
        - name: ge-0/0/1
          ipv4:
            - address: 192.168.1.10/24
      state: merged

Let’s run it and check the result.

$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ********************************************************************************************************************

TASK [Config ge-0/0/1] ********************************************************************************************************
[WARNING]: Platform linux on host 192.168.15.220 is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

changed: [192.168.15.220]

PLAY RECAP ********************************************************************************************************************
192.168.15.220             : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Did you noticed the changed output?
What about the configuration on the router now?

admin> show configuration interfaces
ge-0/0/1 {
    unit 0 {
        family inet {
            address 192.168.1.10/24;
        }
    }
}
fxp0 {
    unit 0 {
        family inet {
            address 192.168.15.220/24;
        }
        family inet6;
    }
}

That’s awesome! We just configured and IP address on ge-0/0/1.

How does Ansible knows what to replace, what to override, and what to delete?

If you take a closer look to the playbook, you will see a line with state: merged. This is a module parameter that specifies the state of the router configuration after the module finishes its job.

The possible values are:

  • merged
  • replaced
  • overriden
  • deleted

In fact, the module matches whatever configuration you build on its parameters, applies a configuration action, and commits the result.

The official module documentation is on https://docs.ansible.com/ansible/latest/modules/junos_l3_interfaces_module.html.

Categories
Ansible Juniper Networking

Ansible and Juniper Junos – First Steps

On the previous post, I proposed a quick and dirty method to provision an Ansible control node, using Vagrant and VirtualBox. But, if you really want to spin the volume up to 11, the best is to work in a dedicated Linux machine.

On this lab, I will be using Linux Mint, which is a Ubuntu derivate, but most examples will work on any Linux distribution.

First, install Ansible if already don’t have it. There are two easy methods.

  • Using pip to install the ansible Python module. You’ll need a working installation of Python2 (altough the latests releases of Ansible works with Python 3 too). Also, pip has to be installed and on path.

    $ pip install ansible
  • Using apt by adding the Ansible ppa repository and offload all the work to the system package manager.

    $ sudo apt-add-repository ppa:ansible/ansible
    $ sudo apt-get update
    $ sudo apt install ansible

I preffer the last one and leave apt handle all the job, because the ppa repository is usually up-to-date with the latest release.

Once installed, check your setup with ansible --version or ansible localhost -m setup.

Ansible manages Junos using NETFCONF over SSH. In order to be able to connect to Junos via Ansible, both SSH and NETCONF services has to be enabled on the remote host.

SSH can be used for sending raw commands using the junos_command module, but NETCONF is definetly more versatile and supports the whose set of Ansible modules, which you can see here.

To use NETCONF, you will need an additional Python module in your Ansible control node.

$ pip install ncclient

And to enable SSH and NETCONF in your Juniper host , do the following. Probably you already have SSH enabled on your router so you can ignore that setting.

admin> edit  
Entering configuration mode
[edit]
admin# set system services ssh 
admin# set system services netconf ssh 
[edit]
admin# commit 
commit complete

Building the Ansible Inventory

Ansible inventory files can use many formats, depending on the plugins you have. The two main formats are INI files, and YAML.

INI-style Inventory

[junos]
192.168.15.194

[junos:vars]
ansible_connection=netconf
ansible_network_os=junos
ansible_user=admin
ansible_password=Password$1

YAML Inventory

all:
  hosts:
    "192.168.15.194":
  vars:
    ansible_connection: netconf
    ansible_network_os: junos
    ansible_user: admin
    ansible_password: Password$1

Both files represent the same set of information on a different style. Personally, I preffer the YAML notation because it is more human-friendly and readeable, and it allows you to learn YAML which is used on many other automation and orchestration tools.

What does this means

all:
  hosts:
    "192.168.15.194":

all: stands for all hosts and its always required. All hosts on an Ansible inventory belongs to at least two groups, all and ungrouped. All groups of course, all hosts, and ungrouped contains all hosts which don’t belong to another specific group besides all.

hosts: specifies the start position for the hosts. Hosts groups can be then written like:

mendoza:
   hosts:
     "192.168.15.194":
     core.thisnetwork.net:

Hosts groups can share a set of variables, which provide information for the Ansible modules. For this example, as the vars section is a direct children of the allgroup, these variables will apply to all hosts in the inventory.

vars:
     ansible_connection: netconf
     ansible_network_os: junos
     ansible_user: admin
     ansible_password: Password$1

The Junos modules for Ansible use the netconf connection, which uses SSH and NETCONF, so ensure to allow TCP/22 and TCP/830 on your rules.

Although a fixed inventory file with manually added hosts can be enough for some users, on future entries we’ll set up a dynamic inventory calling a source of truth like Netbox, or a monitoring system like Zabbix.

Testing our setup

The real fun with Ansible is on ansible-playbook, but first let’s fire up an Ansible module to warm up your Ansible-fu.

$ ansible -m junos_facts -i juniper-hosts.yml all
  • -m junos_facts instructs Ansible to use the junos_facts module
  • juniper-hosts.yml is our previously configured inventory file
  • all tells Ansible the group of hosts inside the inventory to use

If everythig works allright, this will output a long JSON with a bunch of information about your device.

192.168.15.195 | SUCCESS => {
    "ansible_facts": {
        "ansible_net_api": "netconf", 
        "ansible_net_filesystems": [
            "/dev/md0.uzip", 
            "devfs", 
            "/dev/gpt/junos", 
            "devfs", 
            "/dev/md1.uzip", 
            "/dev/md2.uzip", 
            "tmpfs", 
            "procfs", 
            "/dev/ada1s1e", 
            "/dev/ada1s1f", 
            "/dev/md3.uzip", 
            "/dev/md4.uzip", 
            "/dev/md5.uzip", 
            "/dev/md6.uzip", 
            "/dev/md7.uzip", 
            "/dev/md8.uzip", 
            "/dev/md9.uzip", 
            "/dev/md10.uzip", 
            "/dev/md11.uzip", 
            "/packages/mnt/junos-libs-compat32/usr/lib32", 
            "/packages/mnt/os-libs-compat32-10/usr/lib32", 
            "/packages/mnt/os-compat32/libexec", 
            "/var/jails/rest-api", 
            "/dev/md12", 
            "/dev/md13.uzip", 
            "/dev/md14.uzip", 
            "/dev/md15.uzip", 
            "/dev/md16.uzip", 
            "/dev/md17.uzip", 
            "/dev/md18.uzip", 
            "/dev/md19.uzip", 
            "/dev/md20.uzip", 
            "/dev/md21.uzip", 
            "/dev/md22.uzip", 
            "/dev/md23.uzip", 
            "/dev/md24.uzip", 
            "/dev/md25.uzip", 
            "/dev/md26.uzip", 
            "/dev/md27.uzip", 
            "/dev/md28.uzip", 
            "tmpfs", 
            "junosprocfs"
        ], 
        "ansible_net_gather_network_resources": [], 
        "ansible_net_gather_subset": [
            "hardware", 
            "default", 
            "interfaces"
        ], 
        "ansible_net_has_2RE": false, 
        "ansible_net_hostname": "None", 
        "ansible_net_interfaces": {
            ".local.": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Loopback"
            }, 
            "cbp0": {
                "admin-status": "up", 
                "macaddress": "00:05:86:cc:c1:11", 
                "mtu": "9192", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Ethernet"
            }, 
            "demux0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "9192", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Software-Pseudo"
            }, 
            "dsc": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Software-Pseudo"
            }, 
            "em1": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:cf:9b:01", 
                "mtu": "1514", 
                "oper-status": "up", 
                "speed": "1000mbps", 
                "type": "Ethernet"
            }, 
            "esi": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Software-Pseudo"
            }, 
            "fxp0": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:cf:9b:00", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "Unspecified", 
                "type": "Ethernet"
            }, 
            "ge-0/0/0": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:02", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/1": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:03", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/2": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:04", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/3": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:05", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/4": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:06", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/5": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:07", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/6": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:08", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/7": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:09", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/8": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:0a", 
                "mtu": "1514", 
                "oper-status": "down", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "ge-0/0/9": {
                "admin-status": "up", 
                "macaddress": "0c:b8:15:f6:47:0b", 
                "mtu": "1514", 
                "oper-status": "up", 
                "speed": "1000mbps", 
                "type": null
            }, 
            "gre": {
                "admin-status": "up", 
                "macaddress": null, 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "GRE"
            }, 
            "ipip": {
                "admin-status": "up", 
                "macaddress": null, 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "IPIP"
            }, 
            "irb": {
                "admin-status": "up", 
                "macaddress": "00:05:86:cc:c8:f0", 
                "mtu": "1514", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Ethernet"
            }, 
            "jsrv": {
                "admin-status": "up", 
                "macaddress": "00:05:86:cc:c8:c0", 
                "mtu": "1514", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Ethernet"
            }, 
            "lc-0/0/0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "0", 
                "oper-status": "up", 
                "speed": "800mbps", 
                "type": "Unspecified"
            }, 
            "lo0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Loopback"
            }, 
            "lsi": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Software-Pseudo"
            }, 
            "mtun": {
                "admin-status": "up", 
                "macaddress": null, 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Multicast-GRE"
            }, 
            "pfe-0/0/0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "0", 
                "oper-status": "up", 
                "speed": "800mbps", 
                "type": "Unspecified"
            }, 
            "pfh-0/0/0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "0", 
                "oper-status": "up", 
                "speed": "800mbps", 
                "type": "Unspecified"
            }, 
            "pimd": {
                "admin-status": "up", 
                "macaddress": null, 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "PIMD"
            }, 
            "pime": {
                "admin-status": "up", 
                "macaddress": null, 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "PIME"
            }, 
            "pip0": {
                "admin-status": "up", 
                "macaddress": "00:05:86:cc:c8:b0", 
                "mtu": "9192", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "Ethernet"
            }, 
            "pp0": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "1532", 
                "oper-status": "up", 
                "speed": "Unspecified", 
                "type": "PPPoE"
            }, 
            "rbeb": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Software-Pseudo"
            }, 
            "tap": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Software-Pseudo"
            }, 
            "vtep": {
                "admin-status": "up", 
                "macaddress": "Unspecified", 
                "mtu": "Unlimited", 
                "oper-status": "up", 
                "speed": "Unlimited", 
                "type": "Software-Pseudo"
            }
        }, 
        "ansible_net_memfree_mb": 179384, 
        "ansible_net_memtotal_mb": 2058336, 
        "ansible_net_model": "vmx", 
        "ansible_net_modules": [
            {
                "name": "Midplane"
            }, 
            {
                "description": "RE-VMX", 
                "name": "Routing Engine 0"
            }, 
            {
                "description": "VMX SCB", 
                "name": "CB 0"
            }, 
            {
                "description": "VMX SCB", 
                "name": "CB 1"
            }, 
            {
                "chassis_sub_module": null, 
                "description": "Virtual FPC", 
                "name": "FPC 0"
            }
        ], 
        "ansible_net_python_version": "2.7.15+", 
        "ansible_net_routing_engines": {
            "0": {
                "cpu_background": "0", 
                "cpu_background1": "0", 
                "cpu_background2": "0", 
                "cpu_background3": "0", 
                "cpu_idle": "61", 
                "cpu_idle1": "94", 
                "cpu_idle2": "95", 
                "cpu_idle3": "95", 
                "cpu_interrupt": "2", 
                "cpu_interrupt1": "1", 
                "cpu_interrupt2": "1", 
                "cpu_interrupt3": "1", 
                "cpu_system": "27", 
                "cpu_system1": "4", 
                "cpu_system2": "3", 
                "cpu_system3": "3", 
                "cpu_user": "10", 
                "cpu_user1": "1", 
                "cpu_user2": "1", 
                "cpu_user3": "1", 
                "last_reboot_reason": "Router rebooted after a normal shutdown.", 
                "load_average_fifteen": "0.62", 
                "load_average_five": "0.73", 
                "load_average_one": "0.75", 
                "mastership_priority": "master (default)", 
                "mastership_state": "master", 
                "memory_buffer_utilization": "13", 
                "memory_dram_size": "2010 MB", 
                "memory_installed_size": "(2048 MB installed)", 
                "model": "RE-VMX", 
                "slot": "0", 
                "start_time": "2019-11-26 12:06:10 UTC", 
                "status": "OK", 
                "up_time": "11 hours, 46 minutes, 19 seconds"
            }
        }, 
        "ansible_net_serialnum": "VM5DDBEA932E", 
        "ansible_net_system": "junos", 
        "ansible_net_version": "17.1R1.8", 
        "ansible_network_resources": {}, 
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false
}

If you take a closer look, you’ll find out that I am using a vMX appliance, can you spot where is that information?

What if we could use this and more information to gather more useful data about our router, and execute actions of provision new configuration depending on the gathered data?

If Ansible modules are the tools in your workshop, playbooks are your instruction manuals, and your inventory of hosts are your raw material.

The First Playbook

Playbooks are expressed in YAML and are composed of one or more ‘plays’ in a list. The goal of a play is to map a group of hosts to some well defined roles, represented by things ansible calls tasks. At a basic level, a task is nothing more than a call to an ansible module.

Go ahead and create a new file, juniper.yml or whatever you like, and put the following inside:

---
- hosts: all
  gather_facts: no

  tasks:

  - name: Get Configuration
    junos_command:
      commands:
        - show configuration

Going from top to bottom this playbook tells Ansible to:

  • hosts: all, use the all hosts from the inventory
  • gather_facts: no, don’t gather any facts for now. More on this later.
  • tasks:, this is the list of all the tasks I want you to do.
  • - name:, this is the name of the task. It starts with a - because it’s a list, even if it has only a single entry
  • junos_command:use this module for this task, like -m junos_command. This module, like most, supports a set of parameters which you can see here, and they are below!
  • commands: this is the list of commands to execute
  • - show configuration: this is one of the commands

Now, run ansible-playbook juniper.yml -i juniper-hosts.yml. This will run your playbook, using all the hosts on the previously defined juniper-hosts.yml inventory.

$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] ***********************************************************************************************************************************************************************

TASK [Get Configuration] *********************************************************************************************************************************************************
[WARNING]: Platform linux on host 192.168.15.195 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could
change this. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

ok: [192.168.15.195]

PLAY RECAP ***********************************************************************************************************************************************************************
192.168.15.195             : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Well, that was fun but.. there is nothing useful here, besides a warning! In fact, Ansible connected sucessfully to the router, and retrieved the configuration. We didn’t instructed it to show the configuration to us.

Modify your playbook so it looks like this:

---
- hosts: all
  gather_facts: no

  tasks:

  - name: Get Configuration
    junos_command:
      commands:
        - show configuration
    register: config

  - name: Show Config
    debug: var=config

We added an additional line to the first task, register: config, which saves the result of the task in a variable called config. This name can be whatever you like.

There is also an additional task named Show Config, with a debug: var=config instruction, which outputs the config variable.

Run the playbook once again like you did before, and check the result. Of course it will depend on your configuration. On my vMX, which is almost blank, and just using DHCP for a couple interfaces, the result was this.

$ ansible-playbook juniper.yml -i juniper-hosts.yml

PLAY [all] *****************************************************************************

TASK [Get Configuration] ***************************************************************
[WARNING]: Platform linux on host 192.168.15.195 is using the discovered Python
interpreter at /usr/bin/python, but future installation of another Python interpreter
could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html
for more information.

ok: [192.168.15.195]

TASK [Show Config] *********************************************************************
ok: [192.168.15.195] => {
    "config": {
        "ansible_facts": {
            "discovered_interpreter_python": "/usr/bin/python"
        }, 
        "changed": false, 
        "failed": false, 
        "stdout": [
            "## Last changed: 2019-11-26 19:02:46 UTC\nversion 17.1R1.8;\nsystem {\n    root-authentication {\n        encrypted-password \"$6$5LBS/EfQ$tL9utW2Aj4T56SfJUxdnVaF/.RIbaZ65keFn1SbCgOTi6r.LDiGt3FvvoP2WuxuuosVtx0RobNk67obTMeNgF.\";\n    }\n    scripts {\n        inactive: language python;\n    }\n    login {\n        user admin {\n            uid 2000;\n            class super-user;\n            authentication {\n                encrypted-password \"$6$9YynK9hD$Is6rEd7WNnEYGF7q2MqQJoRU/9vGjkQv7Qig.V2WT1905ShVlow4LXKeATM5HR8F1vTwROz2gUpF7z7eCJruo1\";\n            }\n        }\n    }\n    services {\n        ssh;\n        netconf {\n            ssh;\n        }\n    }\n    syslog {\n        user * {\n            any emergency;\n        }\n        file messages {\n            any notice;\n            authorization info;\n        }\n        file interactive-commands {\n            interactive-commands any;\n        }\n    }\n}\ninterfaces {\n    ge-0/0/1 {\n        unit 0 {\n            family inet {\n                dhcp;\n            }\n        }\n    }\n    ge-0/0/9 {\n        unit 0 {\n            family inet {\n                dhcp;\n            }\n        }\n    }\n}"
        ], 
        "stdout_lines": [
            [
                "## Last changed: 2019-11-26 19:02:46 UTC", 
                "version 17.1R1.8;", 
                "system {", 
                "    root-authentication {", 
                "        encrypted-password \"$6$5LBS/EfQ$tL9utW2Aj4T56SfJUxdnVaF/.RIbaZ65keFn1SbCgOTi6r.LDiGt3FvvoP2WuxuuosVtx0RobNk67obTMeNgF.\";", 
                "    }", 
                "    scripts {", 
                "        inactive: language python;", 
                "    }", 
                "    login {", 
                "        user admin {", 
                "            uid 2000;", 
                "            class super-user;", 
                "            authentication {", 
                "                encrypted-password \"$6$9YynK9hD$Is6rEd7WNnEYGF7q2MqQJoRU/9vGjkQv7Qig.V2WT1905ShVlow4LXKeATM5HR8F1vTwROz2gUpF7z7eCJruo1\";", 
                "            }", 
                "        }", 
                "    }", 
                "    services {", 
                "        ssh;", 
                "        netconf {", 
                "            ssh;", 
                "        }", 
                "    }", 
                "    syslog {", 
                "        user * {", 
                "            any emergency;", 
                "        }", 
                "        file messages {", 
                "            any notice;", 
                "            authorization info;", 
                "        }", 
                "        file interactive-commands {", 
                "            interactive-commands any;", 
                "        }", 
                "    }", 
                "}", 
                "interfaces {", 
                "    ge-0/0/1 {", 
                "        unit 0 {", 
                "            family inet {", 
                "                dhcp;", 
                "            }", 
                "        }", 
                "    }", 
                "    ge-0/0/9 {", 
                "        unit 0 {", 
                "            family inet {", 
                "                dhcp;", 
                "            }", 
                "        }", 
                "    }", 
                "}"
            ]
        ], 
        "warnings": [
            "Platform linux on host 192.168.15.195 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information."
        ]
    }
}

PLAY RECAP *****************************************************************************
192.168.15.195             : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

I know we all love show config | display set. Try adding display: set as an additional parameter of junos_command. It should look like this.

    junos_command:
      commands:
        - show configuration
      display: set

Run the playbook again, and your output should be as expected if you ran it on the Junos CLI.

Stay tuned for more posts to learn how to configure Juniper Junos using Ansible.

Categories
Networking

Introduction to NetOps

I am glad to have been invited to participate in the Introduction to NetOps course dictated by the ISOC. This course is a major step getting people involved in network automation and Unix/Linux/BSD technologies.

The Internet Society (ISOC) was founded in 1992 by a number of people involved with the Internet Engineering Task Force (IETF) and provides an organizational home for and financial support for the Internet standards process.

The Internet Society supports the work of the Internet Engineering Task Force (IETF) to create open standards for the Internet.

Categories
MikroTik Networking Projects

Upgrading a MikroTik CHR Cluster

I upgraded a CHR cluster with the main objectives of reduce costs, improve network redundancy and provide an easy administration for CHR instances. As explained in previous posts, CHR can be run on many popular hypervisors, and most users are having great success using Hyper-V Failover clusters or vSphere HA to provide highly available routers without depending on VRRP or other gateway redundancy protocols.

These virtual routers currently provide two main services besides routing for ISP customers. They act as PPPoE concentrator for FTTH users, and provide traffic shaping and policing depending on the customer service plan.

Server Hardware

For this node, I will use a 32 core Dell R730, with 32 GB of RAM, and a 500 GB RAID 10 storage. On future post, new hosts will be added to the cluster.

Unracking the server

Network Conectivity

This server comes with a 4 port Gigabit Ethernet NIC, which could be used without any issues with the ixgbe driver.

First idea was to use two ports in a LACP bundle, and the other two in separate port groups.

I had previous Netflow analysis where I saw a predictable traffic behavior, where most of the bandwidth usage was going from and to a CDN peer of the ISP network. Customers had a mix of public and private addresses of the Class B segment, and they were being moved to CG-NAT ranges. In other words, traffic from a specific set of addresses were going from and to a specific set of addresses.

Why not configure two port-channels, instead of using separate port groups? I tested and due to the nature of the IP addressing on the customer side of the routers, none of the available hashing modes for LACP allowed to achieve a decent distribution on both links of the port-channel.

So, for the purposes of this cluster, I added an Intel X520 dual SFP+ card, providing 20 Gbps conectivity to the CHR instances. Peak bandwidth usage was around 4200 Mbps, so this card is more than enough to allow for future grow.

Installing the Intel X520 NIC

The Intel X520 only supports Intel branded SFP modules, and this behavior can be tuned configuring the kernel module. However, for this particular scenario, where both ports will be connected to a top of rack Dell Force10 S4048-ON switch, I choosed to use DAC cables to keep things simple.

DAC cables on the switch
Connecting the server

The server is using ESXi 6.5 for the hypervisor. After booting, I noticed the NICs were being recognized as vmnic5 and vmnic6, but they were using the ixgbe driver and only establishing links at 1 Gbps.

I downloaded the ixgben driver which is provided by VMware itself here and uploaded it to ESXi via SFTP.

For all my SFTP needs, my tool of choice always is Bitwise SSH client.

Once uploaded, I installed the offline bundle with the following command line. 

[root@esxi] esxcli software vib install -d "/complete/path/to/the/driver/bundle"

Then I followed the KB article to disable the native ixgbe driver and use the new one. First, I placed the host on maintenance mode, and then I executedthe following to disable the driver.

[root@esxi] esxcli system module set --enabled=false --module=ixgbe

After a reboot, the new ixgben driver was loaded, and the NICs were establishing links at 10 Gbps.

I added the new NICs to the previously created virtual switches, checked the correct assignments of port groups, and then migrated the VMs to this host.