j2ipaddr

Jinja2 filters for IP addresses, the easy way

Why

On networking and network automation, we need to extract info about IP addresses as a combination of two values:

  • a host address
  • a subnet mask

For 10.10.10.5/24, the host address is 10.10.10.5 and the subnet mask is 255.255.255.0, and its prefix length is 24.

There is additional information we can infer from this single item, as its network address, broadcast address.

Useful data for network engineers are wildcards or hostmasks, network size, class, type, and so on.

Jinja2 provides several integrated filters to work with, however it can be complicated to use complex data types.

Ansible provides a way to work this on its ansible.utils.ipaddr collection.

However, probably you won’t need the entire Ansible package just to be able to use it.

This package intends to provide a set of filters and handler to the Python 3 netaddr module, on a way that is hopefully easy and lightweight to use.

What

Included filters are the following:

ip_address(addr)

Returns an IP address for a combination of IP address and subnet mask

ip_address('10.10.10.5/24')
> 10.10.10.5
{{ '10.10.10.5/24 | ip_address }}
> 10.10.10.5

ip_prefixlen(addr)

Returns a prefix length for a combination of IP address and subnet mask

ip_prefixlen('10.10.10.5/24')
> 24
{{ '10.10.10.5/24 | ip_prefixlen }}
> 24

ip_netmask(addr)

Returns a subnet mask for a combination of IP address and subnet mask

ip_netmask('10.10.10.5/24')
> 255.255.255.0
{{ '10.10.10.5/24 | ip_netmask }}
> 255.255.255.0

ip_hostmask(addr)

Returns a wilcard or hostmask for a combination of IP address and subnet mask

ip_hostmask('10.10.10.5/24')
> 0.0.0.255
{{ '10.10.10.5/24 | ip_hostmask }}
> 0.0.0.255

ip_wildcard(addr)

Alias for ip_hostmask(addr)

ip_wildcard('10.10.10.5/24')
> 0.0.0.255
{{ '10.10.10.5/24 | ip_wildcard }}
> 0.0.0.255

ip_network(addr)

Returns a network address for a combination of IP address and subnet mask

ip_network('10.10.10.5/24')
> 10.10.10.0
{{ '10.10.10.5/24 | ip_network_hosts_size }}
> 10.10.10.0

ip_broadcast(addr)

Returns a broadcast address for a combination of IP address and subnet mask

ip_broadcast('10.10.10.5/24')
> 10.10.10.255
{{ '10.10.10.5/24 | ip_broadcast }}
> 10.10.10.255

ip_network_hosts_size(addr)

Returns the size of the subnet for a combination of IP address and subnet mask

ip_network_hosts_size('10.10.10.5/24')
> 255
{{ '10.10.10.5/24 | ip_network_hosts_size }}
> 255

ip_network_first(addr)

Returns the first usable address in network address for a combination of IP address and subnet mask

ip_network('10.10.10.5/24')
> 10.10.10.1
{{ '10.10.10.5/24 | ip_network_hosts_size }}
> 10.10.10.1

ip_network_last(addr)

Returns the last usable address in network address for a combination of IP address and subnet mask

ip_network('10.10.10.5/24')
> 10.10.10.254
{{ '10.10.10.5/24 | ip_network_hosts_size }}
> 10.10.10.254

How

Simply install with pip.

$ pip install j2ipaddr

To insert the filters on your Jinja2 processor, simply use the following syntax. The filter name can be changed by adjusting the dict key name.

import jinja2
import j2ipaddr.filters
jinja2.filters.FILTERS['ip_prefixlen'] = filters.ip_prefixlen

Or, probably an easier way, use the following one-liner to load all the filters into your Jinja2 filters

import jinja2
import j2ipaddr.filters
jinja2.filters.FILTERS = {**jinja2.filters.FILTERS, **filters.load_all()}

On your templates, you can do this as an example:

Variables

host:
  interfaces:
    Te1/0/1:
      ipv4_addresses:
        - 10.10.10.5/24

Template

router ospf 10
  network {{host.interfaces.Te1/0/1.ipv4_addresses[0] | ip_network }} {{host.interfaces.Te1/0/1.ipv4_addresses[0] | ip_wildcard  }} area 0.0.0.0

The output would looks like this:

router ospf 10
  network 10.0.0.0 0.0.0.255 area 0.0.0.0

Where

You can find this project on

Anycast with multiple BNGs

On a previous post we saw an example of a network access topology running anycast default gateways.

The idea is to save IPv4 addresses, without other methods than standard routing protocols. Just plain BGP that can be implemented on most vendors, either via hardaware appliances or virtualized network devices.

For the following examples, I’ll simulate a public /24 prefix using the 198.51.100.0/24 which is reserved by IANA as TEST-NET-2 for documentation.

Lab Network

This lab network comprises 2 AGG/BNG routers, a single core router (which will perform additional aggregation), and a single edge router. All devices are MikroTik RouterOS 6.48.6 CHR instances.

Lab network

edge-01

/interface bridge
add name=lo0

/ip address
add address=10.0.1.1 interface=lo0 network=10.0.1.1
add address=10.0.0.1/30 interface=ether1 network=10.0.0.0

/routing ospf instance
set [ find default=yes ] router-id=10.0.1.1
/routing ospf interface
add passive=yes
add interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.0/30
add area=backbone network=10.0.1.1/32

/routing bgp instance
set default as=65000 router-id=10.0.1.1
/routing bgp peer
add default-originate=always in-filter=core-01-in name=core-01 out-filter=core-01-out remote-address=10.1.1.1 remote-as=65000 update-source=lo0
/routing filter
add action=accept chain=core-01-in prefix=198.51.100.0/24
add action=discard chain=core-01-in
add action=accept chain=core-01-out prefix=0.0.0.0/0
add action=discard chain=core-01-out

/system identity
set name=edge-01

The “edge” router is peering with the “core” through their loopbacks, and just advertising a default to it (or cores, on a future stage), and accepting the entire 198.51.100.0/24.

core-01

/interface bridge
add name=lo0

/ip address
add address=10.1.1.1 interface=lo0 network=10.1.1.1
add address=10.0.0.2/30 interface=ether1 network=10.0.0.0

add address=10.255.255.1/30 interface=ether2 network=10.255.255.0
add address=10.255.255.5/30 interface=ether3 network=10.255.255.4

/routing bgp instance
set default as=65000 cluster-id=10.1.1.1 router-id=10.1.1.1
/routing bgp peer
add in-filter=edge-01-in name=edge-01 out-filter=edge-01-out remote-address=10.0.1.1 remote-as=65000 update-source=lo0
/routing bgp peer
add in-filter=edge-01-in name=edge-01 out-filter=edge-01-out remote-address=10.0.1.1 remote-as=65000 update-source=lo0
add default-originate=if-installed in-filter=agg-bng-in name=agg-bng-01 out-filter=agg-bng-out remote-address=10.10.1.1 remote-as=65000 route-reflect=yes update-source=lo0
add default-originate=if-installed in-filter=agg-bng-in name=agg-bng-02 out-filter=agg-bng-out remote-address=10.10.1.2 remote-as=65000 route-reflect=yes update-source=lo0

/routing filter
add action=accept chain=edge-01-in prefix=0.0.0.0/0
add action=discard chain=edge-01-in
add action=accept chain=edge-01-out prefix=198.51.100.0/24
add action=discard chain=edge-01-out

/routing filter
add action=accept chain=agg-bng-in prefix=198.51.100.0/24 prefix-length=24-29
add action=discard chain=agg-bng-in
add action=accept chain=agg-bng-out prefix=0.0.0.0/0
add action=discard chain=agg-bng-out

/routing ospf instance
set [ find default=yes ] router-id=10.1.1.1
/routing ospf interface
add passive=yes
add interface=ether1 network-type=point-to-point
add interface=ether2 network-type=point-to-point
add interface=ether3 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.0/30
add area=backbone network=10.1.1.1/32
add area=backbone network=10.255.255.0/30
add area=backbone network=10.255.255.4/30

/system identity
set name=core-01

The “core” is peering with the “edge” of course, and also with two BNGs named as agg-bng-xx. This core is advertising its default to them, and accepting all prefixes within 198.51.100.254/24, with a prefix length up to /29.

If you come from a IOS land, this syntax would be something like this.

ip prefix-list BNG
 permit 5 198.51.100.254/24 ge 24 le 29

agg-bng-01

/interface bridge
add name=lo0

/ip address
add address=10.10.1.1 interface=lo0 network=10.10.1.1
add address=10.255.255.2/30 interface=ether1 network=10.255.255.0
add address=198.51.100.254/24 interface=ether2 network=198.51.100.0

/routing ospf instance
set [ find default=yes ] router-id=10.10.1.1
/routing ospf interface
add passive=yes
add interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=10.255.255.0/30
add area=backbone network=10.10.1.1/32

/routing bgp instance
set default as=65000 router-id=10.10.1.1
/routing bgp peer
add in-filter=core-01-in name=core-01 out-filter=core-01-out remote-address=10.1.1.1 remote-as=65000 route-reflect=yes update-source=lo0
/routing filter
add action=accept chain=core-01-in prefix=0.0.0.0/0
add action=discard chain=core-01-in
add action=accept chain=core-01-out prefix=198.51.100.0/24 prefix-length=24-29
add action=discard chain=core-01-out

/system identity
set name=agg-bng-01

agg-bng-02

/interface bridge
add name=lo0

/ip address
add address=10.10.1.2 interface=lo0 network=10.10.1.2
add address=10.255.255.6/30 interface=ether1 network=10.255.255.0
add address=198.51.100.254/24 interface=ether2 network=198.51.100.0

/routing ospf instance
set [ find default=yes ] router-id=10.10.1.2
/routing ospf interface
add passive=yes
add interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=10.255.255.4/30
add area=backbone network=10.10.1.2/32

/routing bgp instance
set default as=65000 router-id=10.10.1.2
/routing bgp peer
add in-filter=core-01-in name=core-01 out-filter=core-01-out remote-address=10.1.1.1 remote-as=65000 route-reflect=yes update-source=lo0
/routing filter
add action=accept chain=core-01-in prefix=0.0.0.0/0
add action=discard chain=core-01-in
add action=accept chain=core-01-out prefix=198.51.100.0/24 prefix-length=24-29
add action=discard chain=core-01-out

/system identity
set name=agg-bng-02

Finally, the BNGs are peering with the core, accepting a default, and allowing any advertisements from 198.51.100.254/24 from /24 to /29.

Both routers have 198.51.100.254/24 as the anycast default gateway.

If you wonder hor this works, this lab network is similar to the one on the previous post which you can check here.

BNG PPPoE and DHCP

We will start by setting up PPPoE services on our BNGs.

At this point we will work it with local secrets and keeping all the AAA process in the same router, with RADIUS as a future option.

Be aware that RouterOS by default will try its best to adjust the TCP MSS to match the PPPoE interface MTU.

Also, this being PPPoE, we have no restrictions on using the first address on the network as the PPP connection will not care about it being a network address. However, this will have the obvious restrictions and behavior if we run DHCP.

We will also skip the .254 address on the address pool as this is assigned to the ether2 interface on both routers as our anycast default gateway.

agg-bng-01

/ip pool
add name=internet ranges=198.51.100.0-198.51.100.127

/ppp profile
add local-address=198.51.100.254 name=internet remote-address=internet

/ppp secret
add name=sub1 password=sub1 profile=internet
add name=sub2 password=sub2 profile=internet

/interface pppoe-server server
add default-profile=internet interface=ether2 disabled=no

agg-bng-02

/ip pool
add name=internet ranges=198.51.100.128-198.51.100.253

/ppp profile
add local-address=198.51.100.254 name=internet remote-address=internet

/ppp secret
add name=sub3 password=sub3 profile=internet
add name=sub4 password=sub4 profile=internet

/interface pppoe-server server
add default-profile=internet interface=ether2 disabled=no

For DHCP, we will reuse the same previously created address pool. The following config applies to both routers.

/ip dhcp-server
add address-pool=internet disabled=no interface=ether2 name=dhcp1
/ip dhcp-server network
add address=198.51.100.0/24 gateway=198.51.100.254 netmask=24

Finally, we will add some test subscribers. A dumb switch will act as the last-mile technology which could be xPON, wireless, DSL, you name it. All the subs are running RouterOS 6.48.6, and this is just to have something capable to talk PPPoE. There is also a VPCS 0.8.2 which comes by default with GNS3.

GNS3 Topology

Address me, father

Starting with sub03 VPCS, we will ask DHCP to the BNG.

Welcome to Virtual PC Simulator, version 0.8.2
Dedicated to Daling.
Build time: Aug 23 2021 11:15:00
Copyright (c) 2007-2015, Paul Meng (mirnshi@gmail.com)
All rights reserved.

VPCS is free software, distributed under the terms of the "BSD" licence.
Source code and license can be found at vpcs.sf.net.
For more information, please visit wiki.freecode.com.cn.

Press '?' to get help.

Executing the startup file


sub03> ip dhcp
DORA IP 198.51.100.252/24 GW 198.51.100.254

sub03> ping 198.51.100.254

84 bytes from 198.51.100.254 icmp_seq=1 ttl=64 time=1.278 ms
84 bytes from 198.51.100.254 icmp_seq=2 ttl=64 time=1.234 ms
84 bytes from 198.51.100.254 icmp_seq=3 ttl=64 time=0.946 ms
^C

If you pay attention, we did get the .252 address, instead of the .253.

sub4 had probably requested this one before, as RouterOS by default comes with a DHCP client on ether1. Is this the case?

[admin@RouterOS] > /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
 #   ADDRESS            NETWORK         INTERFACE                                                                                                                                                                                         
 0 D 198.51.100.253/24  198.51.100.0    ether1                 

Indeed, both are running DHCP. And just for reference, this is how it looks from the BNG.

[admin@agg-bng-02] /ip dhcp-server> lease pr
Flags: X - disabled, R - radius, D - dynamic, B - blocked
 #   ADDRESS                                       MAC-ADDRESS       HOST-NAME                             SERVER                             RATE-LIMIT                             STATUS  LAST-SEEN
 0 D 198.51.100.253                                0C:04:49:87:00:00 RouterOS                              dhcp1                                                                     bound   2m34s
 1 D 198.51.100.252                                00:50:79:66:68:01 sub03                                 dhcp1                                                                     bound   3m52s

Same is happening with sub01 and sub02, however we’ll remove the DHCP client and add a PPPoE client.

[admin@RouterOS] /interface pppoe-client> add interface=ether1 user=sub1 password=sub1 add-default-route=yes
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
 0 X  name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
      dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> ena 0
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
 0  R name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
      dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
 #   ADDRESS            NETWORK         INTERFACE
 0 D 198.51.100.125/32  198.51.100.254  pppoe-out2
[admin@RouterOS] /interface pppoe-client>

This config looks as follows on sub02.

[admin@RouterOS] >
[admin@RouterOS] > /ip dhcp-client
[admin@RouterOS] /ip dhcp-client> remove [find]
[admin@RouterOS] /ip dhcp-client> /
[admin@RouterOS] > /interface pppoe-client
[admin@RouterOS] /interface pppoe-client> add add-default-route=yes disabled=no interface=ether1 name=pppoe-out2 password=sub1 user=sub1
[admin@RouterOS] /interface pppoe-client> pr
Flags: X - disabled, I - invalid, R - running
 0  R name="pppoe-out2" max-mtu=auto max-mru=auto mrru=disabled interface=ether1 user="sub1" password="sub1" profile=default keepalive-timeout=10 service-name="" ac-name="" add-default-route=yes default-route-distance=1
      dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
[admin@RouterOS] /interface pppoe-client> /ip ad pr
Flags: X - disabled, I - invalid, D - dynamic
 #   ADDRESS            NETWORK         INTERFACE
 0 D 198.51.100.124/32  198.51.100.254  pppoe-out2
[admin@RouterOS] /interface pppoe-client> /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                          pppoe-out2                1
 1 ADC  198.51.100.254/32  198.51.100.124  pppoe-out2                0
[admin@RouterOS] /interface pppoe-client>

Alright, we can ping the gateway from both. Can we get beyond it?

sub03> trace 1.1.1.1
trace to 1.1.1.1, 8 hops max, press Ctrl+C to stop
 1   198.51.100.254   2.362 ms  1.396 ms  0.958 ms
 2     *  *  *
 3     *  *  *
^C 4

The BNGs is aware of this subscriber, however, we are not advertising anything to the core- yet.

We like connected things

If you recall, on the AGGs, there was a precise out filter on the peering to the core.

Well, the idea is to let the core know about some parts of the subnet, covered by this filter. And the easiest way is to have BGP to

  • Know there are some hosts running DHCP, probably via static routes pointing to the local interface.
  • PPPoE subs will already have a dynamic and connected route on the routing table.
  • Have BGP redistribute connected and statics, in case there are no PPPoE subscribers and we only have DHCP subscribers.
  • Aggretate all PPPoE interfaces into a supernet, because we are allowing up to /29
    This supernet will be a /25, because we created our internet pool from 198.51.100.1-198.51.100.127. Same concept applies for agg02, with the consideration that the aggregate will be 198.51.100.128/25
[admin@agg-bng-01] /routing bgp aggregate> add prefix=198.51.100.0/25 instance=default
[admin@agg-bng-01] /routing bgp aggregate> pr
Flags: X - disabled, A - active
 #   PREFIX               INSTANCE
 0   198.51.100.0/25      default
[admin@agg-bng-01] /routing bgp aggregate> set include-igp=yes
[admin@agg-bng-01] /routing bgp aggregate> ..instance
[admin@agg-bng-01] /routing bgp instance> set redistribute-connected=yes redistribute-static=yes
[admin@agg-bng-01] /routing bgp> advertisements pr
PEER     PREFIX               NEXTHOP          AS-PATH                                                                                                                                                                  ORIGIN     LOCAL-PREF
core-01  198.51.100.0/25      10.10.1.1                            

The include-igp setting will match all IGP routes, like connected routes and iBGP routes.

You can see that the core is aware of a part of the /24 exists on this BNG.

[admin@core-01] > /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  0.0.0.0/0                          10.0.1.1                200
 1 ADC  10.0.0.0/30        10.0.0.2        ether1                    0
 2 ADo  10.0.1.1/32                        10.0.0.1                110
 3 ADC  10.1.1.1/32        10.1.1.1        lo0                       0
 4 ADo  10.10.1.1/32                       10.255.255.2            110
 5 ADo  10.10.1.2/32                       10.255.255.6            110
 6 ADC  10.255.255.0/30    10.255.255.1    ether2                    0
 7 ADC  10.255.255.4/30    10.255.255.5    ether3                    0
 8 ADb  198.51.100.0/24                    10.10.1.1               200
 9 ADb  198.51.100.0/25                    10.10.1.1               200

However, with this setup, we are still advertising the entire /24 to the core. Let’s adjust the filters on both routers to advertise only anything longer than 24.

[admin@agg-bng-01] /routing filter> pr
Flags: X - disabled
 0   chain=core-01-in prefix=0.0.0.0/0 invert-match=no action=accept set-bgp-prepend-path=""

 1   chain=core-01-in invert-match=no action=discard set-bgp-prepend-path=""

 2   chain=core-01-out prefix=198.51.100.0/24 prefix-length=24-29 invert-match=no action=accept set-bgp-prepend-path=""

 3   chain=core-01-out invert-match=no action=discard set-bgp-prepend-path=""
[admin@agg-bng-01] /routing filter> set prefix-length=25-29 2

[admin@agg-bng-01] /routing filter> pr
Flags: X - disabled
 0   chain=core-01-in prefix=0.0.0.0/0 invert-match=no action=accept set-bgp-prepend-path=""

 1   chain=core-01-in invert-match=no action=discard set-bgp-prepend-path=""

 2   chain=core-01-out prefix=198.51.100.0/24 prefix-length=25-29 invert-match=no action=accept set-bgp-prepend-path=""

 3   chain=core-01-out invert-match=no action=discard set-bgp-prepend-path=""
[admin@agg-bng-01] /routing filter> ..
[admin@agg-bng-01] /routing> bgp ad pr
PEER     PREFIX               NEXTHOP          AS-PATH                                                                                                                                                                  ORIGIN     LOCAL-PREF
core-01  198.51.100.0/25      10.10.1.1                                                                                                                                                                                 

And now, the routing table on our core looks as follows.

[admin@core-01] > ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  0.0.0.0/0                          10.0.1.1                200
 1 ADC  10.0.0.0/30        10.0.0.2        ether1                    0
 2 ADo  10.0.1.1/32                        10.0.0.1                110
 3 ADC  10.1.1.1/32        10.1.1.1        lo0                       0
 4 ADo  10.10.1.1/32                       10.255.255.2            110
 5 ADo  10.10.1.2/32                       10.255.255.6            110
 6 ADC  10.255.255.0/30    10.255.255.1    ether2                    0
 7 ADC  10.255.255.4/30    10.255.255.5    ether3                    0
 8 ADb  198.51.100.0/25                    10.10.1.1               200
 9 ADb  198.51.100.128/25                  10.10.1.2               200
[admin@core-01] >

Going out and beyond

For the sake of examples, I’m adding a lo1 interface on the edge with 1.1.1.1/32 to simulate an external host.

[admin@edge-01] > /inte bridge add name=lo1
[admin@edge-01] > /ip address add address=1.1.1.1/32 interface=lo1

And now, from sub4 for example, let’s run a traceroute to it.

[admin@RouterOS] /tool> traceroute 1.1.1.1
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS
 1 198.51.100.254                     0%    2   1.6ms     2.6     1.6     3.6       1
 2 10.255.255.5                       0%    2   3.2ms     3.8     3.2     4.3     0.6
 3 1.1.1.1                            0%    2   4.6ms     4.7     4.6     4.8     0.1

There is a special consideration here if you still haven’t noticed it.

How does agg1 knows about what’s happening on agg2. For example, if from agg1 we try to reach hosts on the 198.51.100.128/25 network, the immediate next hop is agg1 itself, because we have a DAC route pointing to 198.51.100.0/24

[admin@agg-bng-01] > /ip ro pr
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  0.0.0.0/0                          10.0.1.1                200
 1 ADo  10.0.0.0/30                        10.255.255.1            110
 2 ADo  10.0.1.1/32                        10.255.255.1            110
 3 ADo  10.1.1.1/32                        10.255.255.1            110
 4 ADC  10.10.1.1/32       10.10.1.1       lo0                       0
 5 ADo  10.10.1.2/32                       10.255.255.1            110
 6 ADC  10.255.255.0/30    10.255.255.2    ether1                    0
 7 ADo  10.255.255.4/30                    10.255.255.1            110
 8 ADC  198.51.100.0/24    198.51.100.254  ether2                    0
 9 ADbU 198.51.100.0/25                                             20
10 ADC  198.51.100.126/32  198.51.100.254  <pppoe-sub1-1>            0
11 ADC  198.51.100.127/32  198.51.100.254  <pppoe-sub1>              0

However, we already have all rhe routing info we need on the core.

The fix is simple and involves of course, filtering, but we’ll cover that on the next post.

UFiber v4 Python Client

Yeah, I finally remembered to make a post about this. I know it will like as a copy-paste of the previous one, because, in fact it is.

Ok, if you have been following the series, you should already know that I equally love and hate UFiber OLTs. They are affordable, deliver a lot of bang for the buck, and have an awful GUI.

Well, the GUI is lovely on v4.

Python in the middle

I wrote a quick and dirty client which acts as a sort of middleware between the HTTP inteface of the OLT and you.

It allows to provision non existing ONUs, GPON profiles, WiFi profiles, retrieve active ONU status and general configuration.

Take a look to it on https://github.com/baldoarturo/ufiber-client-4, and feel free to contribute if you want to.

How to help

It would be awesome to have docs 😀

Are you a pydoc master? Let’a add docstrings.

Do you have an OLT for me to test? Ping me and we can set up a VPN.

olt.py

This is the core of the project. It uses the OLTClient class to provide a middleware between you and the HTTP interface of the OLT.

Initialize a new OLTClient instance with:

client = OLTClient(host='192.168.1.1', username='ubnt', password='ubnt', debug_level=logging.DEBUG)

Required params are only host, and credentials.

The initialization will handle the login for you, altough you can call the login() method manually.

If the OLT is network reachable, and you have provided the right credentials, and the OLT GUI is alive and well, you should be ready to start.

What changes on v4

Well, UBNT got rid of the GPON profiles. 🙁

This software is intented to give you an alternative by keeping profiles as JSON in the ./profiles folder.

You can copy the template.json file and make your way using it as a starting point. It should be self descriptive.

There is an schema.json which validates your profile before pushing changes into the OLT.

UFiber Python Client

Ok, if you have been following the series, you should already know that I equally love and hate UFiber OLTs. They are affordable, deliver a lot of bang for the buck, and have an awful GUI.

Please, be aware that this can change for better or worse in the future, and at the time I’m writing this the latest firmware is v3.1.3. I trust in you UBNT, hope you can sort out this and give us a better product. I’ll keep my fingers crossed.

Python in the middle

I wrote a quick and dirty client which acts as a sort of middleware between the HTTP inteface of the OLT and you.

It allows to provision non existing ONUs, GPON profiles, WiFi profiles, retrieve active ONU status and general configuration.

Take a look to it on https://github.com/baldoarturo/ufiber-client, and feel free to contribute if you want to.

Edited on Aug 15 2020: I did the same for firmware version 4, which is cleaner and fixes a lot of bugs. Stay tuned!

ufiber-client

This is a quick dirty project built to provide a quick dirty client for Ubiquiti UFiber OLTs, using firmware version 3.x

There is also a CLI attempt, but I couldn’t find any ready to use packages to build a decent CLI.

More info about what am I doing this is on the following entries:

olt.py

This is the core of the project. It uses the OLTCLient class to provide a middleware between you and the HTTP interface of the olt.

Initialize a new OLTClient instance with:

client = olt.OLTClient(host, username, password)

The initialization will handle the login for you, altough you can call the login() method manually.

If the OLT is network reacheable, and you have provided the right credentials, and the OLT WEB GUI is alive and well, you should be ready to start.

You can also connect using cli.py:

$ /cli.py
UFiber Client for fw version 3.1.3
UFiber> help

Documented commands (type help <topic>):
========================================
connect  help  onu  quit  show

UFiber> connect 10.20.0.101
Username:admin
Password:
Logging to 10.20.0.101 ...
Connection OK
UFiber>

UFiber OLT API

In a previous post we took a quick look to the Ubiquiti UFiber OLT. As always, UBNT tries to offer a non expensive solution to provide last-mile conectivity for end users. I am using non-expensive because UBNT gear is not cheap. Yeah, it can be affordable, but you only get what you pay for.

We saw that the command line is very limited, even when the software is a fork of Vyatta. There is no way to get ONUs provisioned from the command line, so forgot about Ansible (we love Ansible), netmiko, and other SSH clients tools to ease your life.

UBNT wants you to use the web GUI, period. They offer a dockerized management system called UNMS, which really comes handy after you have provisioned your customers.

Both you and me, as network operators, know that provisioning customers is one of the more boring tasks, but is is still a critical one. Fast and precise provisioning translates in more customers, more stability, faster troubleshooting, and peace of mind.

Yeah, SONAR exists, but not all operators can work with their pricing and technology supports. And don’t even think to integrate billing if you are using electronic invoicing with AFIP in Argentina.

If you are still here, don’t give up. If there is a will, there is a way.

Under the hood

The OLT has a web GUI served by HTTPS, with a self-signed certificate, on port 443. There is no easy way to use a proper certificate here, but well, it’s something.

If you are not authenticated, this is what waits for you in the URL root.

Looks like a login screen

I want to know if this is a standard HTML form. And indeed it is.

<form id="LoginForm" method="post" class="ui-form">
    <input id="Username" name="username" class="text-input" type="text" placeholder="Username" autocapitalize="off" autocorrection="off">
    <input id="Password" name="password" class="text-input" type="password" placeholder="Password">
    <input id="LoginButton" class="submit-input ui-button ui-widget ui-state-default ui-corner-all" type="submit" value="Login" role="button" aria-disabled="false">
</form>

What happens when we log in? I’m using Chrome version 81 and something, let’s open devtools to see the network activity.

General

Request URL:
https://x.x.x.x/
Request Method:
POST
Status Code:
303 See Other
Remote Address:
x.x.x.x:443

Request Headers

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: es,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 32
Content-Type: application/x-www-form-urlencoded
Cookie: PHPSESSID=5da99950e9f74ad8b727f219c9e41d76; X-CSRF-TOKEN=9f0c78e2ea8994b39834e0241466c21b68a28df59bf98364ece91dcd183bdab5; beaker.session.id=29fdb5243db8446f81f75587c9c2a722
DNT: 1
Host: x.x.x.x
Origin: https://x.x.x.x
Referer: https://x.x.x.x/
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36

Response

Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Fri, 02 Jan 2015 08:54:30 GMT
Location: https://x.x.x.x/
Server: Server
Set-Cookie: PHPSESSID=a24b5cbbd6874a1eb09c2d086a93efc6
Set-Cookie: X-CSRF-TOKEN=6f13035a0b7aa4b375e6798c7c60f12e805ecea8c74a3306da81c710e6a3701b
Set-Cookie: beaker.session.id=a24b5cbbd6874a1eb09c2d086a93efc6; httponly; Path=/; secure

Form Data

username: ubnt
password: ubnt

So, this is a standard POST. And we got a cookie.

This can be translated to Python by using the request module.

host = 'olt.ubnt'
url = 'https://{host}'.format(host=host)

USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'

HEADER_FORM_URLENCODED = {
    'Content-Type': 'application/x-www-form-urlencoded',
    'User-Agent': USER_AGENT,
}

form_data = {
    'username': username,
    'password': password,
}
response = requests.post(
    verify=False,
    url=url,
    headers=HEADER_FORM_URLENCODED,
    data=form_data
)

Good enough for me. Of course this should be into a try/catch structure, but ymmv.

Shut up and take my ONUs

So, we are logged in. What about ONU configuration? The GUI allows to update firware, but this is done automatically starting from software version 3.1.3. We can also set many parameters of the ONU configuration, and provision them via OMCI, but the ONU should already exist in the ONU list.

Configuring existing ONUs

There is no way to add non-existing ONUs to the configuration, which makes pre-provisioning impossible.

What happens when we click on save?

A POST request is made, passing an interesting payload to the /api/edge/batch.json endpoint. Let’s see how does it looks like.

This POST puts a payload with all the ONU configuration, and a bit more. It uses a JSON structure which looks like this.

{
    "SET": {
        "onu-list": {
            "UBNTffffffff": {
                "disable": "false",
                "profile": "profile-2",
                "name": "ARTURO TEST",
                "wifi": {
                    "provisioned": false,
                    "enabled": true,
                    "ssid": "UBNT-ONU",
                    "hide-ssid": false,
                    "auth-mode": "wpa2psk",
                    "wpapsk": "",
                    "channel": "auto",
                    "channel-width": "20/40",
                    "tx-power": "100"
                },
                "pppoe-mode": "auto",
                "pppoe-user": "ARTURO",
                "pppoe-password": "ARTURO",
                "wan-address": "null",
                "port-forwards": []
            }
        }
    }
}

Of course this is a fake ONU with a UBNTffffffff serial number. Yeah, we can fool the GUI and send whatever values we want.

This comes real handy because you can pre-provision ONUs before they show up in the PON port.

As an ISP, this means a time saver, because you can deliver ONUs ready to plug and play, and the OLT will hand out all the configuration without further intervention.

The web GUI is handy (altough not so stable), but it really doesn’t makes sense to have to manually provision each ONU when they are connected in the PON port.
Technicians have to spend to a lot time on customers houses waiting for the NOC to configure each new customer. This is no-bueno in pandemic times. You want to install as many new customers as possible, as fast as possible, and staying in foreign homes as less as possible.

It seems it’s time for me to code something.

If you are reading this, Robert Pera, please make me a CLI.

Digging into Ubiquiti’s UFiber OLT

As some of you might know, currently I’m working as a network engineer on a medium size ISP. The company had a long history working as a WISP, and in later times they moved into FTTH, trying several vendors among the lead players of the industry.

As some of you might also know, Argentina has a history of economic meltdowns, currency devaluations and import restrictions. Considering this, the best solution to implement a network here is usually the one you can afford, which can provide the performance you need, and over all things, the one you will be able to keep buying in the future.

So, considering all these factors, when planning for a GPON network for a medium size operator…while trying to keep costs low for both the company and customers:

  • It really doesn’t matter if Calix supports XGS-PON technologies…
  • Or if Huawei gear is compatible with almost everything…
  • Or if Furukawa Electric has some great management software…

The real questions to ask were:

  • Can the company afford the OLTs, and the ONUs for the planned customer base?
  • Will they be in the market in the years to come?

Enter Ubiquiti UFiber

UFiber offers internet and telecom service providers a cost‑effective fiber optic delivery system for Triple Play Services (data, voice, IPTV/VoD) with speeds of up to 2.488 Gbps downstream and 1.244 Gbps upstream.

OLTs come with dual hot-swap power supplies, 4 and 8 PON ports versions. Every PON port supports 128 CPEs, 20 Km maximum range. The uplinks are two SFP+, which can with in LACP.

The ONUs options, at the time when I’m writing, are:

  • UFiber Nano – one PON (of course), one Gigabit Ethernet, a fancy LCD display. Passive PoE powered.
  • UFiber Loco – a PON, a Giga Eth, passive PoE powered or external micro USB power.
  • UFiber Wifi, like above, but with 4 Giga Ethernet ports, and a 802.11n interface.
  • UFiber Instant, a nice SFP ONU.

Ok, sounds nice. How do we manage them? There is a web GUI…

Hey, the GUI looks like EdgeOS !

Once logged in, the GUI has a nice dashboard which looks like this. And it crashes from time to time.

They tried

But this not EdgeOS, the OLT is a different product! Let’s ssh into it to get the real feel.

ssh admin@olt
The authenticity of host 'olt (olt)' can't be established.
ECDSA key fingerprint is SHA256:thnWRB2bImsdNuu1ar74GryFwv5r7PoHJsHhJOkHnCQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'olt' (ECDSA) to the list of known hosts.
Welcome to EdgeOS
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
admin@olt's password:
Linux olt 4.4.159+ #1 SMP Fri Feb 22 15:28:22 UTC 2019 mips
Welcome to EdgeOS
Last login: Tue May 26 15:50:16 2020 from 190.211.80.70
admin@olt:~$

Ok, this is definitely EdgeOS. So we got a fully featured command line interface, with commands similar to Juniper JunOS.

admin@olt:~$ show configuration | display set
-vbash: display: command not found

Ok, maybe the command line is not so-fully-featured. No worries, I’ll write Ansible playbooks to manage the OLTs anyway. Most of the configuration is Juniper-like, so all I need at this moment is find out how to configure GPON profiles, and provision ONUs.

admin@olt:~$ show configuration | match onu
admin@olt:~$ show configuration | match profile
admin@olt:~$ show configuration | match gpon
gpon {

There you are! Let’s go into configuration mode.

admin@olt:~$ configure
[edit]
admin@olt# show system gpon
isolation enable
mtu 1518
[edit]
admin@olt#

Well, this is awkward. There is nothing about GPON in the command line. Neither in the working configuration, and of course being Ubiquiti, there are no command line manuals.

Love the smell of undocumented commands in the morning

So, I saw two interesting lines before: Linux olt 4.4.15, and -vbash: display: command not found, which tells me this is Linux, not BSD as in Junos, and we have bash.

admin@olt:~$
Possible completions:
  add           Add an object to a service
  clear         Clear system information
  configure     Enter configure mode
  connect       Establish a connection
  copy          Copy data
  delete        Delete a file
  disconnect    Take down a connection
  generate      Generate an object
  initial-setup Enter initial configuration dialog
  no            Disable or reset operational variable
  ping          Send Internet Control Message Protocol (ICMP) echo request
  ping6         Send IPv6 Internet Control Message Protocol (ICMP) echo request
  reboot        Reboot the system
  release       Release specified variable
  rename        Re-name something.
  renew         Renew specified variable
  reset         Reset a service
  restart       Restart a service
  set           Set system or shell options
  show          Show system information
  shutdown      Shutdown the system
  telnet        Telnet to <hostname|IPv4 address>
  terminal      Control terminal behaviors
  traceroute    Track network path to <hostname|IPv4 address>
  traceroute6   Track network path to <hostname|IPv6 address>

No signs of bourne again shells in the horizon. Does my magic have any power here?

admin@olt:~$ sh
sh-4.4$ whoami
admin
sh-4.4$ sudo su
root@olt:/home/admin#

Finally a decent shell. Which world is this?

root@olt:~# uname -a
Linux olt 4.4.159+ #1 SMP Fri Feb 22 15:28:22 UTC 2019 mips GNU/Linux
root@olt:~# ls -l /etc/ | grep apt
drwxr-xr-x 6 root root 117 Feb 22 2019 apt

We have apt, so this is a Debian world. I checked on /etc/apt/ and there are no repositories, but I am sure I could run cowsay on this. But the fun can wait.

Where is my GPON configuration? It should say “onu” somewhere.

root@olt:/# grep -r "onu" / | more
grep: /proc/sys/net/ipv4/route/flush: Permission denied
/config/onu_config.json: "onu-policies": {
/config/onu_config.json: "onu-list": {
/config/onu_config.json: "onu-profiles": {
/home/admin/.history:show configuration | match onu
/home/admin/.history:show configuration | match onu
Binary file /lib/mipsel-linux-gnu/libbsd.so.0.8.3 matches
Binary file /lib/mipsel-linux-gnu/libnss_hesiod-2.24.so matches
Binary file /lib/udev/hwdb.bin matches
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Monument Labs, Inc.
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Optical Zonu Corporation
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Presonus Corporation
/lib/udev/hwdb.d/20-usb-vendor-model.hwdb: ID_VENDOR_FROM_DATABASE=PreSonus Audio Electronics, Inc.
Binary file /opt/bcm68620/bcm68620_appl.bin matches
Binary file /opt/bcm68620/bcm_dev_ctrl_linux.ko matches
Binary file /opt/bcm68620/bcm_user_appl matches
/opt/vyatta/share/vyatta-cfg/templates/system/gpon/logging/module/node.def:syntax:expression: $VAR(@) in "main", "oltsys", "onu", "session", "events", "mon_th", "sdk"
/opt/vyatta/share/vyatta-cfg/templates/system/gpon/logging/module/node.def:allowed: echo main oltsys onu session events mon_th sdk

I bolded the interesting information.

  • There is a /config directory, which has a JSON file called onu_config.json
  • The operating system, is in fact, Vyatta.

If you are curious, this is the content of /config. We will dig deeper on the next article.

root@olt:/# ls -l /config
total 200
-rw-rw-r-- 1 root vyattacfg 3336 Jan 1 2015 2020
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 auth
-rw-rw-r-- 1 root vyattacfg 3882 May 26 11:59 config.boot
-rw-r----- 1 root vyattacfg 2402 Dec 31 2014 config.boot.2015-01-01-0001.pre-migration
-rw-r----- 1 root vyattacfg 3151 Apr 13 2015 config.boot.2015-04-14-0130.pre-migration
-rw------- 1 root vyattacfg 187285 May 26 16:14 onu_config.json
drwxrwsr-x 1 root vyattacfg 232 Feb 22 2019 scripts
drwxr-sr-x 2 root vyattacfg 232 Dec 31 2014 snmp
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 support
drwxr-xr-x 1 root root 160 Oct 29 2018 udapi-bridge
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 user-data
drwxr-sr-x 3 www-data vyattacfg 224 Dec 31 2014 wizard

Creating passwordless logins with Ansible

What kind of users? Well, a special user called Ansible, which will use SSH keys to login into remote devices, allowing for full automation on playbooks.

Creating a new key

If you have been following the series, maybe you remember that we already created keys on the Juniper Junos SSH Keys post.

To create a new key, let’s issue the ssh-keygen command as follows. The -f flag tells the output path, and the -C flags specifies a comment.

$ ssh-keygen -f ansible.key -C ansible-login-passwordless

This should output two files, ansible.key and ansible.key.pub.

The public key should look something like this.

$ cat ansible.key.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtJUPfzJY6vKqLUssPPQe+LD7qRmIPbVhb/1i4Qab7T0Vf3x+ItfJyV4Ej4FsnRSU8iMU8J5eIdcetGQfsmwIZAm8glB0T6En5F9lvq2Yd+3RKIvxM3UlrIH6EaRedhsRUyV96CHfIO2nVqS9dmFfgrOJMIOwfTWIiRDNczUPw7aqw0FExslw9ZC0FO/1A6hYgofkGLrdIu9gK/WkNg5BE1EUCYPqbDBEHnnhv3C33LqiSJZnXJyqu53qz+jlv+1LZxerNHuovMGZMkjQsBo2f3r9Gk/9HqBmT0rcLr5prm4CqqryJ3S9VyVVlF599BlqYMuMjj+fCj277R8kSnLxl ansible-login-passwordless

Of course we need an inventory to use, which has the following content.

$ cat inventory.yml 
---
all:
  hosts:
    vars:
      ansible_ssh_user: ansible
      ansible_ssh_private_key_file: ansible.key
      ansible_python_interpreter: auto_silent
    hosts:
      localhost:

This inventory only has one host, localhost, and uses three main variables:

  • ansible_ssh_user, which tell Ansible to use the user ansible
  • ansible_ssh_private_key_file, which indicates the key for this user
  • ansible_python_interpreter, just to avoid non needed logs

The playbook will looks like this. Notice we don’t need to gather_facts here, and we will instruct ansible to use become to gain privileges on the destination host.

---
- hosts: all
  become: yes

  tasks:

    - name: Make sure we have a "wheel" group
      group:
        name: wheel
        state: present

    - name: Allow 'wheel' group to have passwordless sudo
      lineinfile:
        dest: /etc/sudoers
        state: present
        regexp: '^%wheel'
        line: '%wheel ALL=(ALL) NOPASSWD: ALL'
        validate: 'visudo -cf %s'
        
    - name: Create "ansible" user
      user:
        name: ansible
        comment: Ansible Automation User
        groups: wheel

    - name: Add ssh key
      authorized_key:
        user: ansible
        state: present
        key: "{{ lookup('file', './ansible.key.pub') }}"

First, we want to make sure there is a group called wheel which will group users with administrative privileges.

Then, the /etc/sudoers file will be edited by allowing the wheel group to gain privileges, with a failsafe using a visudo validation.

Once the group has been created, the new user will be created, and a SSH key will be added to it.

It seems allright, but, how should we run the playbook, if the default user is ansible and this user does not exists yet? Let’s give it a try.

$ ansible-playbook create-user.yml -i inventory.yml 

PLAY [all] ************************************************************************

TASK [Gathering Facts] ************************************************************
fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ansible@localhost: Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************
localhost                  : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   

It fails, as expected, because the ansible user does not exists in the host.

Well, there is a way to provide a one-time password by connecting a as different user. You will need to install sshpass with your favourite package manager, like apt install sshpass.

One installed, run the playbook once again with the following arguments:

  • -e “ansible_ssh_user=xxxxx”, where xxxxx is a valid user on the remote host
  • -kK, which tell Ansible to ask for a login and a sudo password
$ ansible-playbook create-user.yml -i inventory.yml -e "ansible_ssh_user=arturo" -kK
SSH password: 
BECOME password[defaults to SSH password]: 

PLAY [all] ************************************************************************

TASK [Gathering Facts] ************************************************************
ok: [localhost]

TASK [Make sure we have a "wheel" group] ******************************************
changed: [localhost]

TASK [Allow 'wheel' group to have passwordless sudo] ******************************
changed: [localhost]

TASK [Create "ansible" user] ******************************************************
changed: [localhost]

TASK [Add ssh key] ****************************************************************
changed: [localhost]

PLAY RECAP ************************************************************************
localhost                  : ok=5    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 

Awesome, we have sucessfully created a new user!

Let’s try to connect using the ansible user with its key, as defined in the playbook.

$ ansible -m ping -i inventory.yml all
localhost | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "ping": "pong"
}

Stay tuned for more automation using Ansible.

Quick Ansible control node with Vagrant

Ansible is an IT automation tools which can configure systems, deploy files and software and orchestrate almost every possible task you can imagine.

Unlike other configuration management tools like Chef or Puppet, Ansible is agent-less, and does all its magic over SSH. Forget about keeping up client daemons updated and additional certificates. Just use your existing SSH keys, or usernames and passwords.

Due to it agent-less nature, we just need a host to initiate SSH sessions towards the managed hosts. This node is called a control node in the Ansible jargon.

On this post, we’ll catch up with the basics of Vagrant and Ansible. The repository for this post is on https://github.com/baldoarturo/ansible-control-node.

Download VirtualBox

VirtualBox is a general-purpose full virtualizer for x86 hardware, targeted at server, desktop and embedded use. Get it from here.

Get Vagrant

We’ll use Vagrant to quickly spin up a control node, based on a ubuntu/xenial64 box. If you are not familiar with Vagrant, you just need to know it is a tool capable to provision VMs on different hypervisors, using a Vagrantfile a VM template.

First, get Vagrant for your system here, https://www.vagrantup.com/downloads.html, and install it.

Use Git

In order to make everything easier, we’ll init a new git repository to keep track of all the configuration changes. Also, we can push this repository to a remote and clone from wherever we need it, making a portable Ansible control node.

C:\Users\Arturo\Desktop\ansible-control-node>git init
Initialized empty Git repository in C:/Users/Arturo/Desktop/ansible-control-node/.git/

C:\Users\Arturo\Desktop\ansible-control-node>git config user.name "Arturo Baldo"

C:\Users\Arturo\Desktop\ansible-control-node>git config user.email "baldoarturo@gmail.com"

C:\Users\Arturo\Desktop\ansible-control-node>

Although everything can be done from a shell, I prefer to use the integrated terminal on VSCode, and make use of the universe of plugins. There is support for Ansible and Ansible Playbooks, Vagrant, a lot more!

Also, even if this post uses a Windows system, everything is reproducible on Linux platforms because all the tools are multiplatform.

Init the Vagrant environment

The Vagrant CLI is self explanatory and easy to understand.

C:\Users\Arturo\Desktop\ansible-control-node>vagrant
Usage: vagrant [options] <command> [<args>]

    -v, --version                    Print the version and exit.
    -h, --help                       Print this help.

Common commands:
     box             manages boxes: installation, removal, etc.
     cloud           manages everything related to Vagrant Cloud
     destroy         stops and deletes all traces of the vagrant machine
     global-status   outputs status Vagrant environments for this user
     halt            stops the vagrant machine
     help            shows the help for a subcommand
     init            initializes a new Vagrant environment by creating a Vagrantfile
     login
     package         packages a running vagrant environment into a box
     plugin          manages plugins: install, uninstall, update, etc.
     port            displays information about guest port mappings
     powershell      connects to machine via powershell remoting
     provision       provisions the vagrant machine
     push            deploys code in this environment to a configured destination
     rdp             connects to machine via RDP
     reload          restarts vagrant machine, loads new Vagrantfile configuration
     resume          resume a suspended vagrant machine
     snapshot        manages snapshots: saving, restoring, etc.
     ssh             connects to machine via SSH
     ssh-config      outputs OpenSSH valid configuration to connect to the machine
     status          outputs status of the vagrant machine
     suspend         suspends the machine
     up              starts and provisions the vagrant environment
     upload          upload to machine via communicator
     validate        validates the Vagrantfile
     version         prints current and latest Vagrant version
     winrm           executes commands on a machine via WinRM
     winrm-config    outputs WinRM configuration to connect to the machine

First, run vagrant intit to initialize a new environment.

C:\Users\Arturo\Desktop\ansible-control-node>vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Wait! You have a new file on your git repository. Make it the first commit.

C:\Users\Arturo\Desktop\ansible-control-node>git add .

C:\Users\Arturo\Desktop\ansible-control-node>git commit -m "First commit"
[master (root-commit) 1416f75] First commit
 1 file changed, 70 insertions(+)
 create mode 100644 Vagrantfile

C:\Users\Arturo\Desktop\ansible-control-node>

On the newly create Vagrantfile, we can set configuration and provisioning options for the VM.

First, replace config.vm.box = "base" with config.vm.box = "ubuntu/xenial64" to use this box.

If you want to see the VM, config vb.gui = true. By default this setting is commented out, so the VM will start in headless mode.

Feel free to customize the VM memory by commenting out the vb.memory = "1024" setting and adjusting it to your needs.

Once you are done, save your changes, commit, execute vagrant up, and watch Vagrant create a brand new VM for you.

C:\Users\Arturo\Desktop\ansible-control-node>vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/xenial64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/xenial64' version '20191114.0.0' is up to date...
==> default: Setting the name of the VM: ansible-control-node_default_1574353915423_60685
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection reset. Retrying...
    default: Warning: Connection aborted. Retrying...
    default: Warning: Remote connection disconnect. Retrying...
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default:
    default: Guest Additions Version: 5.1.38
    default: VirtualBox Version: 6.0
==> default: Mounting shared folders...
    default: /vagrant => C:/Users/Arturo/Desktop/ansible-control-node

Well, how do we login into the new VM? Try vagrant ssh

C:\Users\Arturo\Desktop\ansible-control-node>vagrant ssh
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-169-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage


0 packages can be updated.
0 updates are security updates.

New release '18.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


vagrant@ubuntu-xenial:~$ whoami
vagrant
vagrant@ubuntu-xenial:~$ 

Type exit and you will return to your system prompt.

You’ll see that you have new items on your directory. A .log file, with the output from Vagrant, and a .vagrant/ directory.

To exclude them from the repository, create a .gitignore, with the following contents.

.vagrant/
*.log

Save and commit.

Provisioning the VM

Return to the Vagrantfile, and add a section like this:

  config.vm.provision "shell", inline: <<-SHELL
    apt-get update
    apt-get install software-properties-common -y
    apt-add-repository --yes --update ppa:ansible/ansible
    apt-get install ansible -y
  SHELL

This block instructs Vagrant to execute commands on the shell. First, the package list is updated, then the software-properties-common package is installed. To install Ansible, the ppa:ansible/ansible repository is added, and then Ansible is installed.

Notice the usage of -y to avoid interactive prompts.

Save the file, commit, and run vagrant provision. This will re-provision the VM with the new instructions from the Vagrantfile.

Once Vagrant finishes, connect once again with vagrant ssh, and run ansible --version.

vagrant@ubuntu-xenial:~$ ansible --version
ansible 2.9.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Oct  8 2019, 14:14:10) [GCC 5.4.0 20160609]

That’s great! We have just installed Ansible on our new VM.

Seek and Destroy

You might wonder, where is this VM? Head to the VirtualBox Manager, and you’ll find it inside.

Here it is!

Without asking anyone.. kill the machine and delete it.

Hard power off
Delete all files

Head back to the console, and you will see that the SSH session has died. Of course, this makes perfect sense as the VM no longer exists.

vagrant@ubuntu-xenial:~$ Connection to 127.0.0.1 closed by remote host.
 Connection to 127.0.0.1 closed.

If you try to run vagrant provision, it will fail, as there is no VM to provision.

C:\Users\Arturo\Desktop\ansible-control-node>vagrant provision
==> default: VM not created. Moving on...

But what happens if you run vagrant up once again? Surprise! The VM will be recreated with all the previous configuration settings.

Using Ansible

Well, the VM is ready, Ansible is installed.. now what? Let’s write an inventory file and see what can we do.

I’ll create a simple file to connect to a testing docker instance on 192.168.85.253. No need to worry about ssh keys now, as we are testing, username and password will do the trick.

vagrant@ubuntu-xenial:~$ cat docker
[all:vars]
ansible_connection=ssh
ansible_user=MY_USERNAME
ansible_password=MY_PASSWORD

[docker]
192.168.85.253

The [all:vars] section configures the connection settings for all hosts in the file. The [docker] section contains the lists of hosts, in this case just 192.168.85.253.

Let’s run: ansible -m ping -i docker all, where -m is the module to run, -i is the inventory file, and all is the section of the inventory file which contains the hosts. Notice this is a very special ping.

vagrant@ubuntu-xenial:~$ ansible -m ping -i docker all
[DEPRECATION WARNING]: Distribution Ubuntu 16.04 on host 192.168.15.253 should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release
 will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed   
in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
192.168.15.253 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false, 
    "ping": "pong"
}

Whoa! That definetly doesn’t look like a ICMP ping. First, this is an old box about to be decomissioned, so it has been very unmantained. Ansible warns us about an old version of Python, and a old version of Ansible which was installed previously on the remote hosts.

Now, try ansible -m setup -i docker all. Be aware this will throw a ton of data to you. The setup module gathers data about the host.

What it does and how does it, is documented here.

101 Labs – Linux LPIC1 is out

I am the Technical Editor for the new Paul Browning book, 101 Labs – Linux LPIC1.

Experts agree that we retain only 10% of what we read but 90% of what we do. Perhaps this explains why the global pass rate for most IT exams is a ghastly 40%.

101 Labs’ mission is to turn you into an IT expert by doing instead of reading. Using free software and free trials, the labs take you by the hand and walk you through every aspect of the protocols and technologies you will encounter in your IT career.

Configuration tips and tricks are shared with you as well as how to avoid the common mistakes many novice engineers make, which can quickly become career-ending.

You can get a free preview or buy the ebook below.

101 Labs – Linux LPIC1 takes you through the beginner exam syllabus, the LPI Linux Essentials in case you are a novice and want to learn the basics of Linux or simply just brush up your skills. You then tackle all the main LPIC1 topics broken into the two exams 101 and 102.

Upgrading a MikroTik CHR Cluster

I upgraded a CHR cluster with the main objectives of reduce costs, improve network redundancy and provide an easy administration for CHR instances. As explained in previous posts, CHR can be run on many popular hypervisors, and most users are having great success using Hyper-V Failover clusters or vSphere HA to provide highly available routers without depending on VRRP or other gateway redundancy protocols.

These virtual routers currently provide two main services besides routing for ISP customers. They act as PPPoE concentrator for FTTH users, and provide traffic shaping and policing depending on the customer service plan.

Server Hardware

For this node, I will use a 32 core Dell R730, with 32 GB of RAM, and a 500 GB RAID 10 storage. On future post, new hosts will be added to the cluster.

Unracking the server

Network Conectivity

This server comes with a 4 port Gigabit Ethernet NIC, which could be used without any issues with the ixgbe driver.

First idea was to use two ports in a LACP bundle, and the other two in separate port groups.

I had previous Netflow analysis where I saw a predictable traffic behavior, where most of the bandwidth usage was going from and to a CDN peer of the ISP network. Customers had a mix of public and private addresses of the Class B segment, and they were being moved to CG-NAT ranges. In other words, traffic from a specific set of addresses were going from and to a specific set of addresses.

Why not configure two port-channels, instead of using separate port groups? I tested and due to the nature of the IP addressing on the customer side of the routers, none of the available hashing modes for LACP allowed to achieve a decent distribution on both links of the port-channel.

So, for the purposes of this cluster, I added an Intel X520 dual SFP+ card, providing 20 Gbps conectivity to the CHR instances. Peak bandwidth usage was around 4200 Mbps, so this card is more than enough to allow for future grow.

Installing the Intel X520 NIC

The Intel X520 only supports Intel branded SFP modules, and this behavior can be tuned configuring the kernel module. However, for this particular scenario, where both ports will be connected to a top of rack Dell Force10 S4048-ON switch, I choosed to use DAC cables to keep things simple.

DAC cables on the switch
Connecting the server

The server is using ESXi 6.5 for the hypervisor. After booting, I noticed the NICs were being recognized as vmnic5 and vmnic6, but they were using the ixgbe driver and only establishing links at 1 Gbps.

I downloaded the ixgben driver which is provided by VMware itself here and uploaded it to ESXi via SFTP.

For all my SFTP needs, my tool of choice always is Bitwise SSH client.

Once uploaded, I installed the offline bundle with the following command line.

[root@esxi] esxcli software vib install -d "/complete/path/to/the/driver/bundle"

Then I followed the KB article to disable the native ixgbe driver and use the new one. First, I placed the host on maintenance mode, and then I executedthe following to disable the driver.

[root@esxi] esxcli system module set --enabled=false --module=ixgbe

After a reboot, the new ixgben driver was loaded, and the NICs were establishing links at 10 Gbps.

I added the new NICs to the previously created virtual switches, checked the correct assignments of port groups, and then migrated the VMs to this host.