00:00 Intro y musiquita
00:40 Bienvenida
01:12 Qué es ingeniería de tráfico
01:58 Porqué ingeniería de tráfico
02:45 Cuando hacer ingeniería de tráfico
05:03 Repaso selección de rutas en BGP
05:29 Ingeniería de tráfico saliente
05:54 Atributo Weight
06:33 Atributo LOCAL_PREF
07:36 Atributo AS_PATH
08:31 Rutas específicas
09:24 ECMP
10:28 UCMP
11:22 Ruta más antigua
11:42 Conclusiones tráfico saliente
12:16 Ingeniería de tráfico entrante
12:57 Atributo AS_PATH
16:33 Atributo AS_PATH en singlehomed
17:15 Atributo MED
17:47 Rutas específicas
19:56 Ingeniería de tráfico entrante con comunidades
20:21 Qué es comunidades en BGP
21:23 Comunidades bien conocidas o well-known
21:58 Comunidad NO_EXPORT
22:35 Comunidad NO_ADVERTISE
23:18 Comunidad BLACKHOLE
24:41 Comunidades no estándares
25:02 Ejemplo AS3356 – Recibidas – Tráfico saliente
25:46 Ejemplo AS3356 – Enviadas – Tráfico entrante
28:05 Outro y musiquita
Características de iBGP y diferencia con eBGP
Funcionamiento de iBGP y su relación con los sistemas autónomos internos
Comparación entre iBGP y eBGP en términos de vecindad, políticas y escalabilidad
00:00 Intro y musiquita
02:25 Repaso breve de eBGP
03:28 RFC1105 implementación original de BGP
05:22 Relaciones entre peers iBGP
06:06 Peering directo y full mesh
06:54 Route reflector
08:12 Confederaciones BGP
09:07 Diferencias principales entre iBGP vs eBGP
10:37 Atributos BGP a considerar en iBGP
12:45 Mecanismo de elección de rutas en iBGP
14:15 Distancia administrativa en iBGP vs eBGP
16:23 NEXT_HOP en iBGP
17:34 LOCAL_PREF y ECMP en iBGP
18:44 Escalabilidad, limitaciones, split horizon, sincronización de rutas
20:28 ¡Laboratorio!
Introducción a BGP
Características técnicas de BGP
Conceptos fundamentales en BGP
00:00 Intro y musiquita que les gusta
01:42 Bienvenida
Introducción a BGP
02:42 ¿Qué es BGP?
03:55 La famosa anécdota de la servilleta
Características técnicas de BGP
05:00 Características técnicas
07:55 Versiones de BGP
10:22 Componentes de una arquitectura BGP
Conceptos fundamentales en BGP
13:18 Conceptos de Sistema Autónomo – AS
15:08 Ejemplos de AS conocidos
15:08 Operación BGP en routers e intercambio de rutas
Soporte para familias en BGP
17:28 AFI / SAFI – Address Familiy Identifier / Subsequent Address Identifier
19:12 AFI más comunes
Principales AFI y SAFI en BGP
20:55 AFI/SAFI IPv4 Unicast e IPv6 Unicast
21:59 AFI/SAFI MPLS VPNv4
23:14 AFI/SAFI MPLS L2VPN
24:12 AFI/SAFI EVPN
Mecanismo de elección de rutas en BGP
25:12 Elección de rutas en BGP
25:32 Atributos de rutas en BGP
Mecanismo de elección de rutas en BGP
28:43 Algoritmo de elección en rutas en BGP – o de la naranja
Laboratorio
31:18 ¡Laboratorio!
41:54 Problemas técnicos :D, pasá de largo al siguiente capítulo
49:55 Laboratorio parte 2
On a previous post we saw an example of a network access topology running anycast default gateways.
The idea is to save IPv4 addresses, without other methods than standard routing protocols. Just plain BGP that can be implemented on most vendors, either via hardaware appliances or virtualized network devices.
For the following examples, I’ll simulate a public /24 prefix using the 198.51.100.0/24 which is reserved by IANA as TEST-NET-2 for documentation.
Lab Network
This lab network comprises 2 AGG/BNG routers, a single core router (which will perform additional aggregation), and a single edge router. All devices are MikroTik RouterOS 6.48.6 CHR instances.
The “edge” router is peering with the “core” through their loopbacks, and just advertising a default to it (or cores, on a future stage), and accepting the entire 198.51.100.0/24.
The “core” is peering with the “edge” of course, and also with two BNGs named as agg-bng-xx. This core is advertising its default to them, and accepting all prefixes within 198.51.100.254/24, with a prefix length up to /29.
If you come from a IOS land, this syntax would be something like this.
ip prefix-list BNG
permit 5 198.51.100.254/24 ge 24 le 29
Finally, the BNGs are peering with the core, accepting a default, and allowing any advertisements from 198.51.100.254/24 from /24 to /29.
Both routers have 198.51.100.254/24 as the anycast default gateway.
If you wonder hor this works, this lab network is similar to the one on the previous post which you can check here.
BNG PPPoE and DHCP
We will start by setting up PPPoE services on our BNGs.
At this point we will work it with local secrets and keeping all the AAA process in the same router, with RADIUS as a future option.
Be aware that RouterOS by default will try its best to adjust the TCP MSS to match the PPPoE interface MTU.
Also, this being PPPoE, we have no restrictions on using the first address on the network as the PPP connection will not care about it being a network address. However, this will have the obvious restrictions and behavior if we run DHCP.
We will also skip the .254 address on the address pool as this is assigned to the ether2 interface on both routers as our anycast default gateway.
Finally, we will add some test subscribers. A dumb switch will act as the last-mile technology which could be xPON, wireless, DSL, you name it. All the subs are running RouterOS 6.48.6, and this is just to have something capable to talk PPPoE. There is also a VPCS 0.8.2 which comes by default with GNS3.
GNS3 Topology
Address me, father
Starting with sub03 VPCS, we will ask DHCP to the BNG.
Welcome to Virtual PC Simulator, version 0.8.2
Dedicated to Daling.
Build time: Aug 23 2021 11:15:00
Copyright (c) 2007-2015, Paul Meng ([email protected])
All rights reserved.
VPCS is free software, distributed under the terms of the "BSD" licence.
Source code and license can be found at vpcs.sf.net.
For more information, please visit wiki.freecode.com.cn.
Press '?' to get help.
Executing the startup file
sub03> ip dhcp
DORA IP 198.51.100.252/24 GW 198.51.100.254
sub03> ping 198.51.100.254
84 bytes from 198.51.100.254 icmp_seq=1 ttl=64 time=1.278 ms
84 bytes from 198.51.100.254 icmp_seq=2 ttl=64 time=1.234 ms
84 bytes from 198.51.100.254 icmp_seq=3 ttl=64 time=0.946 ms
^C
If you pay attention, we did get the .252 address, instead of the .253.
sub4 had probably requested this one before, as RouterOS by default comes with a DHCP client on ether1. Is this the case?
[admin@RouterOS] > /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 D 198.51.100.253/24 198.51.100.0 ether1
Indeed, both are running DHCP. And just for reference, this is how it looks from the BNG.
[admin@agg-bng-02] /ip dhcp-server> lease pr
Flags: X - disabled, R - radius, D - dynamic, B - blocked
# ADDRESS MAC-ADDRESS HOST-NAME SERVER RATE-LIMIT STATUS LAST-SEEN
0 D 198.51.100.253 0C:04:49:87:00:00 RouterOS dhcp1 bound 2m34s
1 D 198.51.100.252 00:50:79:66:68:01 sub03 dhcp1 bound 3m52s
Same is happening with sub01 and sub02, however we’ll remove the DHCP client and add a PPPoE client.
[admin@RouterOS] /interface pppoe-client> add interface=ether1 user=sub1 password=sub1 add-default-route=yes
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
0 X name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> ena 0
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
0 R name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 D 198.51.100.125/32 198.51.100.254 pppoe-out2
[admin@RouterOS] /interface pppoe-client>
This config looks as follows on sub02.
[admin@RouterOS] >
[admin@RouterOS] > /ip dhcp-client
[admin@RouterOS] /ip dhcp-client> remove [find]
[admin@RouterOS] /ip dhcp-client> /
[admin@RouterOS] > /interface pppoe-client
[admin@RouterOS] /interface pppoe-client> add add-default-route=yes disabled=no interface=ether1 name=pppoe-out2 password=sub1 user=sub1
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
0 R name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK INTERFACE
0 D 198.51.100.124/32 198.51.100.254 pppoe-out2
[admin@RouterOS] /interface pppoe-client> /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADS 0.0.0.0/0 pppoe-out2 1
1 ADC 198.51.100.254/32 198.51.100.124 pppoe-out2 0
[admin@RouterOS] /interface pppoe-client>
Alright, we can ping the gateway from both. Can we get beyond it?
sub03> trace 1.1.1.1
trace to 1.1.1.1, 8 hops max, press Ctrl+C to stop
1 198.51.100.254 2.362 ms 1.396 ms 0.958 ms
2 * * *
3 * * *
^C 4
The BNGs is aware of this subscriber, however, we are not advertising anything to the core- yet.
We like connected things
If you recall, on the AGGs, there was a precise out filter on the peering to the core.
Well, the idea is to let the core know about some parts of the subnet, covered by this filter. And the easiest way is to have BGP to
Know there are some hosts running DHCP, probably via static routes pointing to the local interface.
PPPoE subs will already have a dynamic and connected route on the routing table.
Have BGP redistribute connected and statics, in case there are no PPPoE subscribers and we only have DHCP subscribers.
Aggretate all PPPoE interfaces into a supernet, because we are allowing up to /29 This supernet will be a /25, because we created our internet pool from 198.51.100.1-198.51.100.127. Same concept applies for agg02, with the consideration that the aggregate will be 198.51.100.128/25
The include-igp setting will match all IGP routes, like connected routes and iBGP routes.
You can see that the core is aware of a part of the /24 exists on this BNG.
[admin@core-01] > /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADb 0.0.0.0/0 10.0.1.1 200
1 ADC 10.0.0.0/30 10.0.0.2 ether1 0
2 ADo 10.0.1.1/32 10.0.0.1 110
3 ADC 10.1.1.1/32 10.1.1.1 lo0 0
4 ADo 10.10.1.1/32 10.255.255.2 110
5 ADo 10.10.1.2/32 10.255.255.6 110
6 ADC 10.255.255.0/30 10.255.255.1 ether2 0
7 ADC 10.255.255.4/30 10.255.255.5 ether3 0
8 ADb 198.51.100.0/24 10.10.1.1 200
9 ADb 198.51.100.0/25 10.10.1.1 200
However, with this setup, we are still advertising the entire /24 to the core. Let’s adjust the filters on both routers to advertise only anything longer than 24.
And now, from sub4 for example, let’s run a traceroute to it.
[admin@RouterOS] /tool> traceroute 1.1.1.1
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 198.51.100.254 0% 2 1.6ms 2.6 1.6 3.6 1
2 10.255.255.5 0% 2 3.2ms 3.8 3.2 4.3 0.6
3 1.1.1.1 0% 2 4.6ms 4.7 4.6 4.8 0.1
There is a special consideration here if you still haven’t noticed it.
How does agg1 knows about what’s happening on agg2. For example, if from agg1 we try to reach hosts on the 198.51.100.128/25 network, the immediate next hop is agg1 itself, because we have a DAC route pointing to 198.51.100.0/24
[admin@agg-bng-01] > /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADb 0.0.0.0/0 10.0.1.1 200
1 ADo 10.0.0.0/30 10.255.255.1 110
2 ADo 10.0.1.1/32 10.255.255.1 110
3 ADo 10.1.1.1/32 10.255.255.1 110
4 ADC 10.10.1.1/32 10.10.1.1 lo0 0
5 ADo 10.10.1.2/32 10.255.255.1 110
6 ADC 10.255.255.0/30 10.255.255.2 ether1 0
7 ADo 10.255.255.4/30 10.255.255.1 110
8 ADC 198.51.100.0/24 198.51.100.254 ether2 0
9 ADbU 198.51.100.0/25 20
10 ADC 198.51.100.126/32 198.51.100.254 <pppoe-sub1-1> 0
11 ADC 198.51.100.127/32 198.51.100.254 <pppoe-sub1> 0
However, we already have all rhe routing info we need on the core.
The fix is simple and involves of course, filtering, but we’ll cover that on the next post.
Ok, if you have been following the series, you should already know that I equally love and hate UFiber OLTs. They are affordable, deliver a lot of bang for the buck, and have an awful GUI.
Please, be aware that this can change for better or worse in the future, and at the time I’m writing this the latest firmware is v3.1.3. I trust in you UBNT, hope you can sort out this and give us a better product. I’ll keep my fingers crossed.
Python in the middle
I wrote a quick and dirty client which acts as a sort of middleware between the HTTP inteface of the OLT and you.
It allows to provision non existing ONUs, GPON profiles, WiFi profiles, retrieve active ONU status and general configuration.
This is the core of the project. It uses the OLTCLient class to provide a middleware between you and the HTTP interface of the olt.
Initialize a new OLTClient instance with:
client = olt.OLTClient(host, username, password)
The initialization will handle the login for you, altough you can call the login() method manually.
If the OLT is network reacheable, and you have provided the right credentials, and the OLT WEB GUI is alive and well, you should be ready to start.
You can also connect using cli.py:
$ /cli.py
UFiber Client for fw version 3.1.3
UFiber> help
Documented commands (type help <topic>):
========================================
connect help onu quit show
UFiber> connect 10.20.0.101
Username:admin
Password:
Logging to 10.20.0.101 ...
Connection OK
UFiber>
With the advance of cheap MikroTik routers and ready to use CHR instances, setting up a VPN concentrator for remote access has become an easy task. Moving even further, a single router could provide VPN access and dynamic routing to integrate remote networks to the backbone.
I have started a gig as a consultant and sysadmin for a logistics insurance company, and one of my first proposals was to improve the network access for road warriors and remote workers.
The past
There was a Proxmox hypervisor, with some Windows 2012 R2 servers, providing Terminal Services, to execute a locally installed client for an ERP system. Proxmox was also using iptables on its the Debian backend of the to masquerade the VM networks with a public IP address, for Internet connectivity, dstnat rules for a NGINX reverse proxy, and RDP for the Windows servers..
I guess we all know having internet-exposed RDP is not a good idea, even if it is running in a non default port, so the former sysadmin transitioned to a SSH tunnel system, where the users connected to the hypervisor via SSH to establish tunnel to the desired server.
This solution, which I considered not elegant, was the only available at the moment due to networking constraints of the VPS provider, so really it was the best they were able to do, and it worked fine for them.
Over the Proxmox hypervisor, they also had a MikroTik CHR instance, with a P1 license, which was used to make a L2TP tunnel to a RB2011UiAS-rm located on their HQ.
Networks behind the tunnel endpoints were routed with static routers, so I configured a quick multi-area OSPF routing system, with the directly connected networks on area 0, along with the /30 network of the tunnel. I added an additional area on both ends, for the future VPN networks. Once OSPF was working as expected, I remove the static routes.
Securing the tunnels
This interconnection via the L2TP tunnel was just plain ol’ L2TP, without IPsec. This is no bueno, and could be improved. Fortunately, IPsec configuration on MikroTik is trivial. Just select “Use IPsec” on both ends, and use the same IPsec pre-shared key.
Configuring via Winbox
This can of course be configured via CLI. Would you like some RouterOS configuration Ansible on next posts? Let me know in the comments.
/interface l2tp-server server
set authentication=mschap1,mschap2 default-profile=VPN enabled=yes ipsec-secret="PUT_A_SECRET_HERE" use-ipsec=yes
VPN profiles
It’s always a good idea to copy the default-encryption profile, and create a new one based on that template. I set up a local address which was of course, part of the networks announced in a separate area by the OSPF process. I also added a IP pool to be able to provide dynamic addresses for the VPN users.
PPP profile
Maybe you are aware that in the Cisco world, you have to use tcp adjust-mss to adjust the maximum TCP segment size, to advoid fragmentation of packets over the tunnel. Fortunately, this is configured by default on RouterOS.
We don’t want any fragmentation
Finally, to be able to redirect the dial-in to a RADIUS server, we need to instruct the PPP AAA system to use RADIUS, as shown next.
Setting up RADIUS authentication
RADIUS servers are very simple to set up on RouterOS.
Under the RADIUS submenu, add a new server for PPP service, and configure the following parameters.
IP address of the radius server
RADIUS secret
Authentication and accounting ports, usually 1812 and 1813. Some servers use 1645 for accounting. Those are all UDP.
REALM if your server supports that extension
Which source address should the router use for its NAS-IP-Address
Configuring RADIUS
Using Windows NPS as a RADIUS server
NPS can work without a Certificate Authority but if you are working in an Active Directory environment, you’ll save a lot of headaches by installing the CA role.
Installing NPS and CA
Selecting Roles
In my particular scenario, the server was not part of a domain so the certificate generation and association was skipped.
Once the roles have been configured, I headed to the NPS service configuration, and add new RADIUS client.
Make sure to match the RADIUS secret and the source IP address as you configured on the MikroTik side.
New RADIUS Client
Next, the network access policies. I wanted to match the NAS IPv4 address, and the authentication types. If you are not familiar with the RADIUS lang, NAS stands for Network Access Server, which in this case, is the MikroTik router which provides the VPN service.
MikroTik source address
I had to use a CHAP fallback due to some legacy devices withuout MSCHAP support.
Authentication methods
Next, I added a new Access condition, matched the NAS address once again, and selected the local server as point of authentication.
Authenticate on this server
Once everything was properly configured, I set up the VPN client on my side, which looks like as follows. The idea of using NPS as RADIUS was to be able to use my Windows account credentials for the VPN.
VPN client on Windows 10
I verified the successful authentication on the router logs, and the VPN was sucessfully connected.
Previous posts introduced basics connection methods to manage Juniper devices using Ansible playbooks. The inventory files had sensitive information and credentials which should not be accessible to anyone.
SSH and NETCONF over SSH requires client authentication, for example with and username and password, which could looks like this:
admin> show configuration system login
user admin {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$./TeE4CZ$uAMigDedlRuuJgcZx4hYk0"; ## SECRET-DATA
}
}
If you are a frequent SSH user, maybe you are aware that there are other login methods besides using usernames and passwords. By using a key-pair, with public and private keys, a password is no longer needed. The public key is installed on the remote host, and the private key is kept on the control node.
Although by using keys a password is no longer needed, a passphrase can be used with a key, adding an additional security factor to the connection. In fact, using SSH keys with passphrases is considered best practice. However, a private key with a passphrase is less useful for scheduled automation tasks because an operator may not be available to enter the passphrase at the scheduled time.
Creating a Key Pair
A key pair is a set of two cryptographic keys, a public one and a private one. The public key, as its name says, is the one we expose to the public. The private key, must be kept in a secure location.
To create a key pair, lauch ssh-keygen on a new console and follow the prompts. This utility will create two files, which are the public and private keys. Use the -f flag to specify the destination of the output files.
arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ssh-keygen -f ./juniper-hosts.key
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ./juniper-hosts.key.
Your public key has been saved in ./juniper-hosts.key.pub.
The key fingerprint is:
SHA256:Kc/MZ11dLlXpcrK9PKO4L6XzaTrdczaek1ydzaFTFXw arturo@arturo-ThinkPad-L440
The key's randomart image is:
+---[RSA 2048]----+
| ..+|
| oE|
| . =|
| . o O.|
| . S @.B|
| * . * +=|
| = o = = +|
| o =.o.%+|
| +X=o+B|
+----[SHA256]-----+
arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ls
juniper-hosts.key juniper-hosts.key.pub
Now that we have created the key pair, let’s examine them to find out how a key looks like.
This is the private key, which in fact is a plain text file with a RSA key inside it.
This keypair can authenticate an user which connects via SSH.
Installing the public key on Junos
We already know that a basic authentication schema on Junos looks like this.
admin> show configuration system login
user admin {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$kYNQ.bg0$4T3W7GAPuXwsX3nbbsRCb/"; ## SECRET-DATA
}
}
The main idea of using SSH keys, is to avoid user interaction, by trusting the keys instead of a credentials combination.
As seen above, the keys are plain text files. We need to install the public key on the Junos configuration, either configuring it manually, or using Ansible to configure it.
Manually configuring the key
I added a new user called ansible, set its class as super-user and configured its authentication as ssh-rsa.
[edit]
admin# show system login | display set
set system login user admin uid 2000
set system login user admin class super-user
set system login user admin authentication encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."
set system login user ansible uid 2001
set system login user ansible class super-user
set system login user ansible authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfRe11M8nFYdAd5aLwjBI4a2yBbBbfPzpn3V50fR0FLpJYPOax5ayFJMPY90PwRTSYZpzVX36tYglgjDRVmWlN4QqI2dL7X994fGWf5LQsvCf3UTp+BVG3qQT/25O/bXs9rl4/kcts+5LA+xUzBGB0IkvWlggVqAkvKuxqQNYTSoO0FdnR96d2ZSvo2usIuh+McGREBK+In0ThW/Hhiqsb1qT7aNfbWDQtE3Fn+cW/a4fBV4iCJsL7UmJn8gZoFI0Ki8XXfXBvUBTIublnkM28zqG7YLr5wxM01Dl+IF+AymvJuhbj4xUIYlDfUS2HIUTHRc+COiz8RxL0+njfo6mn arturo@arturo-ThinkPad-L440"
The SSH public key is copied and pasted between double quotes.
Using Ansible to configure the key
Altough the key can be configured manually on the remote hosts, what if we have hundreds, or thousands of hosts to configure?
The idea behind this series of posts is to use Ansible whenever possible, so, let’s write a quick playbook to automate the key configuration.
rturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ cat junos-install-ssh-key.yaml
---
- hosts: all
gather_facts: no
vars:
- auth_key: "{{lookup('file', '{{ key_file }}')}}"
tasks:
- name: Install SSH key on remote host
junos_config:
lines:
- set system login user ansible authentication ssh-rsa "{{ auth_key }}"
- set system login user ansible class super-user
The playbook starts as usual, matching all hosts in the inventory, and without gathering facts, just for the sake of speed.
On vars, we are using the lookup plugin to read from a file and store its contents on a variable. Lookup can retrieve data from multiple sources, for example, take a secret from Hashicorp’s Vault. In this scenario, it will read a file which name is take from the key_file variable from the inventory.
It is possible to set a fixed file name on the playbook, but by taking the filename as a variable from the inventory, it gives us more flexibility. We could have multiple keys and rotate them by just changing the file name on the inventory, or use different keys per host group, and still apply the playbook to the full inventory, while using the proper key for each group.
The inventory for this playbook looks like the following. Notice the key_file variable which tells the playbook where to look for the key.
admin> show configuration system login
user admin {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."; ## SECRET-DATA
}
}
Let’s run the playbook to apply the new configuration which will create the ansible user, set ssh-rsa authentication for it, and set its class as super-user.
arturo@arturo-ThinkPad-L440:~/Desktop/ansible-01$ ansible-playbook junos-install-ssh-key.yaml -i junos-hosts.yaml
PLAY [all] *****************************************************************************
TASK [Install SSH key on remote host] **************************************************
changed: [192.168.227.101]
PLAY RECAP *****************************************************************************
192.168.227.101 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Ok, the playbook executed with no errors, and Ansible says there is 1 changed host, which is what we expected.
Let’s check the router configuration again.
admin> show configuration system login
user admin {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$MExZQJdK$lLhnzSw.CLSMQg5bdIiws."; ## SECRET-DATA
}
}
user ansible {
uid 2001;
class super-user;
authentication {
ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfRe11M8nFYdAd5aLwjBI4a2yBbBbfPzpn3V50fR0FLpJYPOax5ayFJMPY90PwRTSYZpzVX36tYglgjDRVmWlN4QqI2dL7X994fGWf5LQsvCf3UTp+BVG3qQT/25O/bXs9rl4/kcts+5LA+xUzBGB0IkvWlggVqAkvKuxqQNYTSoO0FdnR96d2ZSvo2usIuh+McGREBK+In0ThW/Hhiqsb1qT7aNfbWDQtE3Fn+cW/a4fBV4iCJsL7UmJn8gZoFI0Ki8XXfXBvUBTIublnkM28zqG7YLr5wxM01Dl+IF+AymvJuhbj4xUIYlDfUS2HIUTHRc+COiz8RxL0+njfo6mn arturo@arturo-ThinkPad-L440"; ## SECRET-DATA
}
}
There is a new user called ansible, with all the parameters we specified. That’s great!
Authenticating using keys
I wrote another playbook to show the system uptime
---
- hosts: all
gather_facts: no
tasks:
- name: Get uptime
junos_command:
commands:
- show system uptime
register: uptime
- name: Show uptime
debug: var=uptime
There is no plain text password, but instead, by setting up the ansible_ssh_private_key_file variable, we are instructing Ansible to authenticate using the private key.
This is great, now Ansible authenticates using the SSH key. You maybe are thinking:
“Do i have to edit my inventory every time i want to use keys?”
The answer is, no, and in the next post we will set a interactive prompt to connect using user and password to run the first playbook, which will configure the key, and then we will run all the other playbooks connecting with this key.
I upgraded a CHR cluster with the main objectives of reduce costs, improve network redundancy and provide an easy administration for CHR instances. As explained in previous posts, CHR can be run on many popular hypervisors, and most users are having great success using Hyper-V Failover clusters or vSphere HA to provide highly available routers without depending on VRRP or other gateway redundancy protocols.
These virtual routers currently provide two main services besides routing for ISP customers. They act as PPPoE concentrator for FTTH users, and provide traffic shaping and policing depending on the customer service plan.
Server Hardware
For this node, I will use a 32 core Dell R730, with 32 GB of RAM, and a 500 GB RAID 10 storage. On future post, new hosts will be added to the cluster.
Unracking the server
Network Conectivity
This server comes with a 4 port Gigabit Ethernet NIC, which could be used without any issues with the ixgbe driver.
First idea was to use two ports in a LACP bundle, and the other two in separate port groups.
I had previous Netflow analysis where I saw a predictable traffic behavior, where most of the bandwidth usage was going from and to a CDN peer of the ISP network. Customers had a mix of public and private addresses of the Class B segment, and they were being moved to CG-NAT ranges. In other words, traffic from a specific set of addresses were going from and to a specific set of addresses.
Why not configure two port-channels, instead of using separate port groups? I tested and due to the nature of the IP addressing on the customer side of the routers, none of the available hashing modes for LACP allowed to achieve a decent distribution on both links of the port-channel.
So, for the purposes of this cluster, I added an Intel X520 dual SFP+ card, providing 20 Gbps conectivity to the CHR instances. Peak bandwidth usage was around 4200 Mbps, so this card is more than enough to allow for future grow.
Installing the Intel X520 NIC
The Intel X520 only supports Intel branded SFP modules, and this behavior can be tuned configuring the kernel module. However, for this particular scenario, where both ports will be connected to a top of rack Dell Force10 S4048-ON switch, I choosed to use DAC cables to keep things simple.
DAC cables on the switch
Connecting the server
The server is using ESXi 6.5 for the hypervisor. After booting, I noticed the NICs were being recognized as vmnic5 and vmnic6, but they were using the ixgbe driver and only establishing links at 1 Gbps.
I downloaded the ixgben driver which is provided by VMware itself here and uploaded it to ESXi via SFTP.
Then I followed the KB article to disable the native ixgbe driver and use the new one. First, I placed the host on maintenance mode, and then I executedthe following to disable the driver.
[root@esxi] esxcli system module set --enabled=false --module=ixgbe
After a reboot, the new ixgben driver was loaded, and the NICs were establishing links at 10 Gbps.
I added the new NICs to the previously created virtual switches, checked the correct assignments of port groups, and then migrated the VMs to this host.
MikroTik Cloud Hosted Router (CHR) is a RouterOS version intended to be used as a virtual machine instance.
It runs on x86-64-bit architecture and can be deployed on most hypervisors such as:
VMWare, ESXi, Player and Workstation
Microsoft Hyper-V
Oracle VirtualBox
KVM
And others, like Xen, but I haven’t tested it yet
Some special requeriments apply depending on the subyacent hypervisor.
ESXi
Network adapters must be vmxnet3 or E1000. Just use vmxnet3 to get the most. Disks must be IDE, VMware paravirtual SCSI, LSI Logic SAS or LSI Logic Parallel.
Hyper-V
Network adapters must be Network adapter or Legacy Network adapter .Disks IDE or SCSI.
Qemu/KVM
Virtio, E1000 or vmxnet3 NICs. IDE, Sata or Virtio disks.
VirtualBox
Networking using E1000 or rtl8193, and disks with IDE, SATA, SCSI or SAS interfaces.
Licensing
The CHR images have full RouterOS features enabled by default, but they use a different licensing model than other RouterOS versions.
Paid licenses
p1
p1 (perpetual-1), which allows CHR to run indefinitely. It comes with a limit of 1Gbps upload per interface. All the rest of the features provided by CHR are available without restrictions. It can be upgraded p1 to p10 or p-unlimited.
p10
p10 (perpetual-10), which also allows CHR to run indefinitely, with a 10Gbps upload limit per interface. All features are available without restrictions. It can be upgraded to p-unlimited.
p-unlimited (really?)
The p-unlimited (perpetual-unlimited) license level allows CHR to run indefinitely. It is the highest tier license and it has no enforced limitations.
Free licenses (yay!)
There are two ways to use and try CHR free of charge.
free
The free license level allows CHR to run indefinitely, with a limit of 1Mbps upload per interface. All the rest of the features have no restrictions. This level comes activated by default on all images.
60-day trial
Th p1/p10/pU licenses can be tested with a 60 days trial.
Cool. How can i try it?
The easiest way to spin up a working instance of CHR is using the OVA appliance provided by MikroTik.
Once downloaded, the OVA can be used to deploy a new instance. I’ll be using ESXi on this example. The OVA comes preconfigured with a single network adapter, but more interfaces can be added on a later stage.
Creating new VM from OVA templateSetting VM name, and uploading OVA fileI’ll use local storage for itThin provisioned disks, and a previously configured VM networkReview everything, and deploy
Initial Configuration
After the VM boots, log in via CLI with the default credentials:
Username: admin
Password: none
CHR comes with a free licence by default, limited to 1Mbps upload limit. This is handy for lab purposes, or low traffic scenarios like stand-alone DHCP servers.
A DHCP client is enabled by default on the single existing ether1 interface. Use any of the following methods to find out the adquired address.
/ip dhcp-client print
/ip address print
Let’s get a trial licence. You will need the credentials for your MikroTik account. If you don’t have a MikroTik account, get one here.
The CHR instance will also need Internet access, so be sure to connect the virtual NIC to a VM network where it can make its way to the outside.
