Categories
Networking Projects Ubiquiti

UFiber v4 Python Client

Yeah, I finally remembered to make a post about this. I know it will like as a copy-paste of the previous one, because, in fact it is.

Ok, if you have been following the series, you should already know that I equally love and hate UFiber OLTs. They are affordable, deliver a lot of bang for the buck, and have an awful GUI.

Well, the GUI is lovely on v4.

Python in the middle

I wrote a quick and dirty client which acts as a sort of middleware between the HTTP inteface of the OLT and you.

It allows to provision non existing ONUs, GPON profiles, WiFi profiles, retrieve active ONU status and general configuration.

Take a look to it on https://github.com/baldoarturo/ufiber-client-4, and feel free to contribute if you want to.

How to help

It would be awesome to have docs 😀

Are you a pydoc master? Let’a add docstrings.

Do you have an OLT for me to test? Ping me and we can set up a VPN.

olt.py

This is the core of the project. It uses the OLTClient class to provide a middleware between you and the HTTP interface of the OLT.

Initialize a new OLTClient instance with:

client = OLTClient(host='192.168.1.1', username='ubnt', password='ubnt', debug_level=logging.DEBUG)

Required params are only host, and credentials.

The initialization will handle the login for you, altough you can call the login() method manually.

If the OLT is network reachable, and you have provided the right credentials, and the OLT GUI is alive and well, you should be ready to start.

What changes on v4

Well, UBNT got rid of the GPON profiles. 🙁

This software is intented to give you an alternative by keeping profiles as JSON in the ./profiles folder.

You can copy the template.json file and make your way using it as a starting point. It should be self descriptive.

There is an schema.json which validates your profile before pushing changes into the OLT.

Categories
Networking Projects Ubiquiti

UFiber Python Client

Ok, if you have been following the series, you should already know that I equally love and hate UFiber OLTs. They are affordable, deliver a lot of bang for the buck, and have an awful GUI.

Please, be aware that this can change for better or worse in the future, and at the time I’m writing this the latest firmware is v3.1.3. I trust in you UBNT, hope you can sort out this and give us a better product. I’ll keep my fingers crossed.

Python in the middle

I wrote a quick and dirty client which acts as a sort of middleware between the HTTP inteface of the OLT and you.

It allows to provision non existing ONUs, GPON profiles, WiFi profiles, retrieve active ONU status and general configuration.

Take a look to it on https://github.com/baldoarturo/ufiber-client, and feel free to contribute if you want to.

Edited on Aug 15 2020: I did the same for firmware version 4, which is cleaner and fixes a lot of bugs. Stay tuned!

ufiber-client

This is a quick dirty project built to provide a quick dirty client for Ubiquiti UFiber OLTs, using firmware version 3.x

There is also a CLI attempt, but I couldn’t find any ready to use packages to build a decent CLI.

More info about what am I doing this is on the following entries:

olt.py

This is the core of the project. It uses the OLTCLient class to provide a middleware between you and the HTTP interface of the olt.

Initialize a new OLTClient instance with:

client = olt.OLTClient(host, username, password)

The initialization will handle the login for you, altough you can call the login() method manually.

If the OLT is network reacheable, and you have provided the right credentials, and the OLT WEB GUI is alive and well, you should be ready to start.

You can also connect using cli.py:

$ /cli.py
UFiber Client for fw version 3.1.3
UFiber> help

Documented commands (type help <topic>):
========================================
connect  help  onu  quit  show

UFiber> connect 10.20.0.101
Username:admin
Password:
Logging to 10.20.0.101 ...
Connection OK
UFiber>
Categories
Networking Projects Ubiquiti

UFiber OLT API

In a previous post we took a quick look to the Ubiquiti UFiber OLT. As always, UBNT tries to offer a non expensive solution to provide last-mile conectivity for end users. I am using non-expensive because UBNT gear is not cheap. Yeah, it can be affordable, but you only get what you pay for.

We saw that the command line is very limited, even when the software is a fork of Vyatta. There is no way to get ONUs provisioned from the command line, so forgot about Ansible (we love Ansible), netmiko, and other SSH clients tools to ease your life.

UBNT wants you to use the web GUI, period. They offer a dockerized management system called UNMS, which really comes handy after you have provisioned your customers.

Both you and me, as network operators, know that provisioning customers is one of the more boring tasks, but is is still a critical one. Fast and precise provisioning translates in more customers, more stability, faster troubleshooting, and peace of mind.

Yeah, SONAR exists, but not all operators can work with their pricing and technology supports. And don’t even think to integrate billing if you are using electronic invoicing with AFIP in Argentina.

If you are still here, don’t give up. If there is a will, there is a way.

Under the hood

The OLT has a web GUI served by HTTPS, with a self-signed certificate, on port 443. There is no easy way to use a proper certificate here, but well, it’s something.

If you are not authenticated, this is what waits for you in the URL root.

Looks like a login screen

I want to know if this is a standard HTML form. And indeed it is.

<form id="LoginForm" method="post" class="ui-form">
    <input id="Username" name="username" class="text-input" type="text" placeholder="Username" autocapitalize="off" autocorrection="off">
    <input id="Password" name="password" class="text-input" type="password" placeholder="Password">
    <input id="LoginButton" class="submit-input ui-button ui-widget ui-state-default ui-corner-all" type="submit" value="Login" role="button" aria-disabled="false">
</form>

What happens when we log in? I’m using Chrome version 81 and something, let’s open devtools to see the network activity.

General

Request URL:
https://x.x.x.x/
Request Method:
POST
Status Code:
303 See Other
Remote Address:
x.x.x.x:443

Request Headers

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: es,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 32
Content-Type: application/x-www-form-urlencoded
Cookie: PHPSESSID=5da99950e9f74ad8b727f219c9e41d76; X-CSRF-TOKEN=9f0c78e2ea8994b39834e0241466c21b68a28df59bf98364ece91dcd183bdab5; beaker.session.id=29fdb5243db8446f81f75587c9c2a722
DNT: 1
Host: x.x.x.x
Origin: https://x.x.x.x
Referer: https://x.x.x.x/
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36

Response

Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Fri, 02 Jan 2015 08:54:30 GMT
Location: https://x.x.x.x/
Server: Server
Set-Cookie: PHPSESSID=a24b5cbbd6874a1eb09c2d086a93efc6
Set-Cookie: X-CSRF-TOKEN=6f13035a0b7aa4b375e6798c7c60f12e805ecea8c74a3306da81c710e6a3701b
Set-Cookie: beaker.session.id=a24b5cbbd6874a1eb09c2d086a93efc6; httponly; Path=/; secure

Form Data

username: ubnt
password: ubnt

So, this is a standard POST. And we got a cookie.

This can be translated to Python by using the request module.

host = 'olt.ubnt'
url = 'https://{host}'.format(host=host)

USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'

HEADER_FORM_URLENCODED = {
    'Content-Type': 'application/x-www-form-urlencoded',
    'User-Agent': USER_AGENT,
}

form_data = {
    'username': username,
    'password': password,
}
response = requests.post(
    verify=False,
    url=url,
    headers=HEADER_FORM_URLENCODED,
    data=form_data
)

Good enough for me. Of course this should be into a try/catch structure, but ymmv.

Shut up and take my ONUs

So, we are logged in. What about ONU configuration? The GUI allows to update firware, but this is done automatically starting from software version 3.1.3. We can also set many parameters of the ONU configuration, and provision them via OMCI, but the ONU should already exist in the ONU list.

Configuring existing ONUs

There is no way to add non-existing ONUs to the configuration, which makes pre-provisioning impossible.

What happens when we click on save?

A POST request is made, passing an interesting payload to the /api/edge/batch.json endpoint. Let’s see how does it looks like.

This POST puts a payload with all the ONU configuration, and a bit more. It uses a JSON structure which looks like this.

{
    "SET": {
        "onu-list": {
            "UBNTffffffff": {
                "disable": "false",
                "profile": "profile-2",
                "name": "ARTURO TEST",
                "wifi": {
                    "provisioned": false,
                    "enabled": true,
                    "ssid": "UBNT-ONU",
                    "hide-ssid": false,
                    "auth-mode": "wpa2psk",
                    "wpapsk": "",
                    "channel": "auto",
                    "channel-width": "20/40",
                    "tx-power": "100"
                },
                "pppoe-mode": "auto",
                "pppoe-user": "ARTURO",
                "pppoe-password": "ARTURO",
                "wan-address": "null",
                "port-forwards": []
            }
        }
    }
}

Of course this is a fake ONU with a UBNTffffffff serial number. Yeah, we can fool the GUI and send whatever values we want.

This comes real handy because you can pre-provision ONUs before they show up in the PON port.

As an ISP, this means a time saver, because you can deliver ONUs ready to plug and play, and the OLT will hand out all the configuration without further intervention.

The web GUI is handy (altough not so stable), but it really doesn’t makes sense to have to manually provision each ONU when they are connected in the PON port.
Technicians have to spend to a lot time on customers houses waiting for the NOC to configure each new customer. This is no-bueno in pandemic times. You want to install as many new customers as possible, as fast as possible, and staying in foreign homes as less as possible.

It seems it’s time for me to code something.

If you are reading this, Robert Pera, please make me a CLI.

Categories
Networking Projects Ubiquiti

Digging into Ubiquiti’s UFiber OLT

As some of you might know, currently I’m working as a network engineer on a medium size ISP. The company had a long history working as a WISP, and in later times they moved into FTTH, trying several vendors among the lead players of the industry.

As some of you might also know, Argentina has a history of economic meltdowns, currency devaluations and import restrictions. Considering this, the best solution to implement a network here is usually the one you can afford, which can provide the performance you need, and over all things, the one you will be able to keep buying in the future.

So, considering all these factors, when planning for a GPON network for a medium size operator…while trying to keep costs low for both the company and customers:

  • It really doesn’t matter if Calix supports XGS-PON technologies…
  • Or if Huawei gear is compatible with almost everything…
  • Or if Furukawa Electric has some great management software…

The real questions to ask were:

  • Can the company afford the OLTs, and the ONUs for the planned customer base?
  • Will they be in the market in the years to come?

Enter Ubiquiti UFiber

UFiber offers internet and telecom service providers a cost‑effective fiber optic delivery system for Triple Play Services (data, voice, IPTV/VoD) with speeds of up to 2.488 Gbps downstream and 1.244 Gbps upstream.

OLTs come with dual hot-swap power supplies, 4 and 8 PON ports versions. Every PON port supports 128 CPEs, 20 Km maximum range. The uplinks are two SFP+, which can with in LACP.

The ONUs options, at the time when I’m writing, are:

  • UFiber Nano – one PON (of course), one Gigabit Ethernet, a fancy LCD display. Passive PoE powered.
  • UFiber Loco – a PON, a Giga Eth, passive PoE powered or external micro USB power.
  • UFiber Wifi, like above, but with 4 Giga Ethernet ports, and a 802.11n interface.
  • UFiber Instant, a nice SFP ONU.

Ok, sounds nice. How do we manage them? There is a web GUI…

Hey, the GUI looks like EdgeOS !

Once logged in, the GUI has a nice dashboard which looks like this. And it crashes from time to time.

They tried

But this not EdgeOS, the OLT is a different product! Let’s ssh into it to get the real feel.

ssh admin@olt
The authenticity of host 'olt (olt)' can't be established.
ECDSA key fingerprint is SHA256:thnWRB2bImsdNuu1ar74GryFwv5r7PoHJsHhJOkHnCQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'olt' (ECDSA) to the list of known hosts.
Welcome to EdgeOS
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
admin@olt's password:
Linux olt 4.4.159+ #1 SMP Fri Feb 22 15:28:22 UTC 2019 mips
Welcome to EdgeOS
Last login: Tue May 26 15:50:16 2020 from 190.211.80.70
admin@olt:~$

Ok, this is definitely EdgeOS. So we got a fully featured command line interface, with commands similar to Juniper JunOS.

admin@olt:~$ show configuration | display set
-vbash: display: command not found

Ok, maybe the command line is not so-fully-featured. No worries, I’ll write Ansible playbooks to manage the OLTs anyway. Most of the configuration is Juniper-like, so all I need at this moment is find out how to configure GPON profiles, and provision ONUs.

admin@olt:~$ show configuration | match onu
admin@olt:~$ show configuration | match profile
admin@olt:~$ show configuration | match gpon
gpon {

There you are! Let’s go into configuration mode.

admin@olt:~$ configure
[edit]
admin@olt# show system gpon
isolation enable
mtu 1518
[edit]
admin@olt#

Well, this is awkward. There is nothing about GPON in the command line. Neither in the working configuration, and of course being Ubiquiti, there are no command line manuals.

Love the smell of undocumented commands in the morning

So, I saw two interesting lines before: Linux olt 4.4.15, and -vbash: display: command not found, which tells me this is Linux, not BSD as in Junos, and we have bash.

admin@olt:~$
Possible completions:
  add           Add an object to a service
  clear         Clear system information
  configure     Enter configure mode
  connect       Establish a connection
  copy          Copy data
  delete        Delete a file
  disconnect    Take down a connection
  generate      Generate an object
  initial-setup Enter initial configuration dialog
  no            Disable or reset operational variable
  ping          Send Internet Control Message Protocol (ICMP) echo request
  ping6         Send IPv6 Internet Control Message Protocol (ICMP) echo request
  reboot        Reboot the system
  release       Release specified variable
  rename        Re-name something.
  renew         Renew specified variable
  reset         Reset a service
  restart       Restart a service
  set           Set system or shell options
  show          Show system information
  shutdown      Shutdown the system
  telnet        Telnet to <hostname|IPv4 address>
  terminal      Control terminal behaviors
  traceroute    Track network path to <hostname|IPv4 address>
  traceroute6   Track network path to <hostname|IPv6 address>

No signs of bourne again shells in the horizon. Does my magic have any power here?

admin@olt:~$ sh
sh-4.4$ whoami
admin
sh-4.4$ sudo su
root@olt:/home/admin#

Finally a decent shell. Which world is this?

root@olt:~# uname -a
Linux olt 4.4.159+ #1 SMP Fri Feb 22 15:28:22 UTC 2019 mips GNU/Linux
root@olt:~# ls -l /etc/ | grep apt
drwxr-xr-x 6 root root 117 Feb 22 2019 apt

We have apt, so this is a Debian world. I checked on /etc/apt/ and there are no repositories, but I am sure I could run cowsay on this. But the fun can wait.

Where is my GPON configuration? It should say “onu” somewhere.

root@olt:/# grep -r "onu" / | more
grep: /proc/sys/net/ipv4/route/flush: Permission denied
/config/onu_config.json: "onu-policies": {
/config/onu_config.json: "onu-list": {
/config/onu_config.json: "onu-profiles": {
/home/admin/.history:show configuration | match onu
/home/admin/.history:show configuration | match onu
Binary file /lib/mipsel-linux-gnu/libbsd.so.0.8.3 matches
Binary file /lib/mipsel-linux-gnu/libnss_hesiod-2.24.so matches
Binary file /lib/udev/hwdb.bin matches
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Monument Labs, Inc.
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Optical Zonu Corporation
/lib/udev/hwdb.d/20-OUI.hwdb: ID_OUI_FROM_DATABASE=Presonus Corporation
/lib/udev/hwdb.d/20-usb-vendor-model.hwdb: ID_VENDOR_FROM_DATABASE=PreSonus Audio Electronics, Inc.
Binary file /opt/bcm68620/bcm68620_appl.bin matches
Binary file /opt/bcm68620/bcm_dev_ctrl_linux.ko matches
Binary file /opt/bcm68620/bcm_user_appl matches
/opt/vyatta/share/vyatta-cfg/templates/system/gpon/logging/module/node.def:syntax:expression: $VAR(@) in "main", "oltsys", "onu", "session", "events", "mon_th", "sdk"
/opt/vyatta/share/vyatta-cfg/templates/system/gpon/logging/module/node.def:allowed: echo main oltsys onu session events mon_th sdk

I bolded the interesting information.

  • There is a /config directory, which has a JSON file called onu_config.json
  • The operating system, is in fact, Vyatta.

If you are curious, this is the content of /config. We will dig deeper on the next article.

root@olt:/# ls -l /config
total 200
-rw-rw-r-- 1 root vyattacfg 3336 Jan 1 2015 2020
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 auth
-rw-rw-r-- 1 root vyattacfg 3882 May 26 11:59 config.boot
-rw-r----- 1 root vyattacfg 2402 Dec 31 2014 config.boot.2015-01-01-0001.pre-migration
-rw-r----- 1 root vyattacfg 3151 Apr 13 2015 config.boot.2015-04-14-0130.pre-migration
-rw------- 1 root vyattacfg 187285 May 26 16:14 onu_config.json
drwxrwsr-x 1 root vyattacfg 232 Feb 22 2019 scripts
drwxr-sr-x 2 root vyattacfg 232 Dec 31 2014 snmp
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 support
drwxr-xr-x 1 root root 160 Oct 29 2018 udapi-bridge
drwxrwsr-x 1 root vyattacfg 160 Feb 22 2019 user-data
drwxr-sr-x 3 www-data vyattacfg 224 Dec 31 2014 wizard