Using Zabbix API for Custom Reports

Zabbix is an open source monitoring tool for diverse IT components, including networks, servers, virtual machines (VMs) and cloud services. It provides monitoring metrics, among others network utilization, CPU load and disk space consumption. Data can be collected in a agent-less fashion using SNMP, ICMP, or with an multi-platform agent, available for most operating systems.

Even when it is considered one of the best NMS on the market, its reporting capabilities are very limited. For example, this is an availability report created with PRTG.

Image result for prtg reports

And this is a Zabbix Report. There is no graphs, no data tables, and it is difficult to establish a defined time span for the data collection.

My client required an executive report with the following information.

  • Host / Service Name
  • Minimum SLA for ICMP echo request monitoring
  • Achieved SLA for ICMP echo request monitoring
  • Memory usage graph, if host is being SNMP-monitored
  • Main network interface graph, if host is being SNMP-monitored
  • And storage usage graph, also if the host is being SNMP-monitored

Using the Zabbix API

To do call the API, we need to send HTTP POST requests to the api_jsonrpc.php file located in the frontend directory. For example, if the Zabbix frontend is installed under http://company.com/zabbix, the HTTP request to call the apiinfo.version method may look like this:

POST http://company.com/zabbix/api_jsonrpc.php HTTP/1.1
Content-Type: application/json-rpc
{
    "jsonrpc":"2.0",
    "method":"apiinfo.version",
    "id":1,
    "auth":null,
    "params":
        {
        }
}

The request must have the Content-Type header set to one of these values: application/json-rpc, application/json or application/jsonrequest.

Before access any data, it’s necessary to log in and obtain an authentication token. The user.login method is used for this.

{
    "jsonrpc": "2.0",
    "method": "user.login",
    "params": {
        "user": "Admin",
        "password": "zabbix"
    },
    "id": 1,
    "auth": null
}

If the authentication request succeeds, the API response will look like this.

{
    "jsonrpc": "2.0",
    "result": "0424bd59b807674191e7d77572075f33",
    "id": 1
}

The result field is the authentication token, which will be sent on subsequent requests.

Instead of reinvent the wheel, let’s use a existing library to call the API.

Using jqzabbix jQuery plugin for the Zabbix API

GitHub user kodai provides a nice JavaScript client, in a form of a jQuery plugin. You can get it on https://github.com/kodai/jqzabbix.

The usage is quite forward, first, include both jQuery and jqzabbix.js on your HTML file. I using Cloudflare to link jQuery.

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js">/script>
<script type="text/javascript" charset="utf-8" src="jqzabbix.js"></script>

An object has to be created to initialize the client. I prefer to set url, username, and password dynamically, with data provided by the end user, so no credentials are stored here.

server = new $.jqzabbix({
	url: url,  			// URL of Zabbix API
	username: user,   	// Zabbix login user name
	password: pass,  	// Zabbix login password
	basicauth: false,   // If you use basic authentication, set true for this option
	busername: '',      // User name for basic authentication
	bpassword: '',      // Password for basic authentication
	timeout: 5000,      // Request timeout (milli second)
	limit: 1000,        // Max data number for one request
});

As told before, the first step is to authenticate with the API, and save the authorization token. This is handled by the jqzabbix library by first making a request to get the API version, and then authenticating.

server.getApiVersion();
server.userLogin();

If the authentication procedure is completed properly, the API version and authentication ID are stored as properties of the server object. The userlogin() method allows to set callbacks for both success and error.

var success = function() { console.log('Success!'); }
var error = function() { console.error('Error!'); }

server.userLogin(null, success, error)

Once authenticated, the Zabbix API methods are called in the following fashion with the sendAjaxRequest method.

server.sendAjaxRequest(method, params, success, error)

Retrieving Hosts

I set a global array hosts to store the hosts information.
Another global array called SEARCH_GROUPS is used to define which hosts groups should considered on the API request. By setting the selectHosts parameter to true, the hosts on the host groups are retrieved too on the response.

On success, the result is stored on the hosts array, and the get_graphs function is called. If there is an error, the default error callback is fired.

hosts = [];
function get_hosts() {
	// Get hosts
	server.sendAjaxRequest(
		"hostgroup.get",
		{
			"selectHosts": true,
			"filter": {
				"name": SEARCH_GROUPS
			}
		},
		function (e) {
			e.result.forEach(group => {
				group.hosts.forEach(host => {
					hosts.push(host);
				});
			});
			get_graphs();
		},
		error,
	);
}

Retrieving Graphs

Previously, user defined graphs were configured on Zabbix, to match the client requeriments of specific information. All names for the graphs that should be included on the report were terminated the ” – Report” suffix.

This function retrieves all those graphs, and by setting the selectHosts to true, the hosts linked to each graph are retrieved too.

On success, the result is stored on the graphs array, and the render function is called. If there is an error, the default error callback is fired.

graphs = [];
function get_graphs() {
	server.sendAjaxRequest(
		"graph.get",
		{
			"selectHosts": "*",
			"search": {
				name: "- Report"
			}
		},
		function (e) {
			graphs = e.result;
			render();
		},
		error
	)
}

Retrieving Graphs Images Instead of Graph Data

By this time you should have noticed that the Zabbix API allows to retrieve values for the graphs, but no images. An additional PHP file will be stored with the HTML and JS files, as a helper to call the web interface by using php_curl.

You can get it on https://zabbix.org/wiki/Get_Graph_Image_PHP. I made a couple modifications to it in order to pass username and password on the URL query, with parameters for the graph ID, the timespan, and the image dimensions.

<?php
//////////
// GraphImgByID v1.1 
// (c) Travis Mathis - travisdmathis@gmail.com
// It's free use it however you want.
// ChangeLog:
// 1/23/12 - Added width and height to GetGraph Function
// 23/7/13 - Zabbix 2.0 compatibility
// ERROR REPORTING
error_reporting(E_ALL);
set_time_limit(1800);


$graph_id = filter_input(INPUT_GET,'id');
$period= filter_input(INPUT_GET,'period');
$width= filter_input(INPUT_GET,'width');
$height = filter_input(INPUT_GET,'height');
$user = filter_input(INPUT_GET,'user');
$pass = filter_input(INPUT_GET,'pass');

//CONFIGURATION
$z_server = 'zabbix_url'; //set your URL here
$z_user = $user;
$z_pass = $pass;
$z_img_path = "/usr/local/share/zabbix/custom_pages/tmp_images/";

//NON CONFIGURABLE
$z_tmp_cookies = "";
$z_url_index = $z_server . "index.php";
$z_url_graph = $z_server . "chart2.php";
$z_url_api = $z_server . "api_jsonrpc.php";

// Zabbix 1.8
// $z_login_data  = "name=" .$z_user ."&password=" .$z_pass ."&enter=Enter";
// Zabbix 2.0
$z_login_data = array('name' => $z_user, 'password' => $z_pass, 'enter' => "Sign in");

// FUNCTION
function GraphImageById($graphid, $period = 3600, $width, $height) {
    global $z_server, $z_user, $z_pass, $z_tmp_cookies, $z_url_index, $z_url_graph, $z_url_api, $z_img_path, $z_login_data;
    // file names
    $filename_cookie = $z_tmp_cookies . "zabbix_cookie_" . $graphid . ".txt";
    $image_name = $z_img_path . "zabbix_graph_" . $graphid . ".png";

    //setup curl
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $z_url_index);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $z_login_data);
    curl_setopt($ch, CURLOPT_COOKIEJAR, $filename_cookie);
    curl_setopt($ch, CURLOPT_COOKIEFILE, $filename_cookie);
    // login
    curl_exec($ch);
    // get graph
    curl_setopt($ch, CURLOPT_URL, $z_url_graph . "?graphid=" . $graphid . "&width=" . $width . "&height=" . $height . "&period=" . $period);
    $output = curl_exec($ch);
    curl_close($ch);
    // delete cookie
    header("Content-type: image/png");
    unlink($filename_cookie);
    /*
      $fp = fopen($image_name, 'w');
      fwrite($fp, $output);
      fclose($fp);
      header("Content-type: text/html");
     */
    return $output;
}

echo GraphImageById($graph_id, $period, $width, $height);

Quick and Dirty Frontend

You should be able to customize this small frontend to your needs.

<html>

<head>
	<link rel="stylesheet" href="https://unpkg.com/chota@latest">
	<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
	<script src="https://cdn.jsdelivr.net/npm/js-cookie@2/src/js.cookie.min.js"></script>
	<script src="jqzabbix.js"></script>
	<style>
		.host-container {
			margin-bottom: 3em;
		}
		@media print {
			.host-container {
				page-break-before: auto;
				page-break-after: auto;
				page-break-inside: avoid;
			}
			img {
				display: block;
				page-break-before: auto;
				page-break-after: auto;
				page-break-inside: avoid;
			}
		}
	</style>
</head>

<body>
	<div id="container" class="container">

		<div class="row" style="margin-bottom: 3em">
			<div class="col">
				<h2>Services and Availability Report</h2>
				<table id="table" class="bg-dark">
					<thead>
						<th>Host Name</th>
						<th>Target</th>
						<th class="is-text-center">Availibilty</th>
						<th class="is-text-center">Availabilty Status</th>
						<th class="is-text-center">Total Availability</th>
					</thead>
				</table>
			</div>
		</div>


		<div id="template" style="display: none">
			<div class="host-container">
				<div class="row bg-dark">
					<div class="col-12">
						<span id="host-HOST_ID-name">Service Name</span>
					</div>
				</div>
				<div class="row bg-light">
					<div class="col-3">
						Status
					</div>
					<div class="col-3">
						SLA Minimum
					</div>
					<div class="col-3">
						SLA
					</div>
				</div>
				<div class="row bg-primary">
					<div class="col-3">
						<span id="host-HOST_ID-status"></span>OK</span>
					</div>
					<div class="col-3">
						<span id="host-HOST_ID-sla"></span>99.9%
					</div>
					<div class="col-3">
						<span id="host-HOST_ID-sla-value"></span>100%
					</div>
				</div>
				<div class="row is-text-center" id="host-HOST_ID-graphs">
				</div>
			</div>
		</div>

	</div>

	<script src="ui.js"></script>

</body>

</html>

Result

The final page is a complete report, including a briefing table which resumes the services status and SLA compliance.

Customizing NetBox Templates

NetBox is an IP address management (IPAM) and data center infrastructure management (DCIM) tool. Initially conceived by the network engineering team at DigitalOcean, NetBox was developed specifically to address the needs of network and infrastructure engineers.

Image result for netbox device types

When I started using NetBox on my daily job, I planned to use it as a replacement for all the spreadsheets I had for switch configurations, IP address management, secrets, and VLAN assignments. NetBox can handle all of this and more, but the interface didn’t suit my needs.

NetBox is built using the Python Django framework, which I have used for another projects. I used Visual Studio Code to clone the repository and debug, as it has native support for the Django template language.

I keep a copy of the repository on my local machine for ease of modifications. Prior, I have set DEBUG=TRUE on netbox/configuration.py, and allowed localhost and my local network to access the development server. Also, I set the correct settings to connect to the existing postgresql database.

Connecting the existing DB to my local development server

This environment works for test purposes, but the best you can do is to set up separated development and production environments, and commit your changes to production once everything is tested.

Using VSCode to debug Django

The URL definition for the single device view is around line #147 of the netbox/dcim/urls.py file, and it looks like this.

 url(r'^devices/(?P<pk>\d+)/$', views.DeviceView.as_view(), name='device'),

Heading to the DeviceView view, I put a breakpoint on the interfaces
QuerySet of the view definition, and launched the debugger. The default location is at http://localhost:8000.

Setting up the debugger
Breakpoints

I headed to http://localhost:8000/dcim/devices/570/, where I had defined a switch with several VLANs, to hit the breakpoint and find out if the
QuerySet had information about the VLANs, or if they were queried in a per-interface basis, on the interface view.

QuerySet returns this

Lucky me, the QuerySet recovered all the information I needed, and it is passed to the template via a render() call.

All the information I want is rendered on this table. This is the power of the Django framework. I added line #513 as an additional header for the VLANs column.

This table has a for loop which iterates for each interface of the device, so I edited the included template file at dcim/inc/interface.html.

Both tagged and untagged VLANs groups have a bolded title, and the VID and VLAN name is shown after it. I used the dictsort filter, which is part of the Django framework, to sort all the VLANs by their VID.

dcim/inc/interface.html

The final result looks like the following image, and it allows to keep track of all the VLANs on all ports, at first sight. This is easier and more user friendly than getting that information interface per interface, or making a new custom view.

New Template Rendering

Running NetBox in Docker

NetBox is an IP address management (IPAM) and data center infrastructure management (DCIM) tool. Initially conceived by the network engineering team at DigitalOcean, NetBox was developed specifically to address the needs of network and infrastructure engineers.

A quick way to get it working is to use the Docker stack provided at https://github.com/ninech/netbox-docker.

Installing

First, I cloned the repository.

$ git clone -b master https://github.com/ninech/netbox-docker.git
$ cd netbox-docker

Once cloned, I used docker-compose to pull the images

$ docker-compose pull

And then I started the stack with

$ docker-compose up -d

The service will be up and running after a few minutes. Once ready, you need to find where to connect to with

$ docker-compose port nginx 8080

Or use this snippet

$ echo "http://$(docker-compose port nginx 8080)/"

Here I use Portainer as a gui to manage Docker, and Traefik as a reverse proxy to enable FQDN access to the services behind. I added an entry on my DNS to route netbox.arturo.local to the Docker IP address, on the exposed port for Nginx.

Spiceworks Customization

Andrew Foster at Topland Communications reached me via Upwork looking to customize and fine tune a existing Spiceworks installation.

After a quick inspection, I decided to tackle the project by compacting the DB in first place. Spiceworks keeps a lot of logs regarding the system activity, which are located on C:\Program Files\Spiceworks\Log. In order to clean them, the first step is to stop Spiceworks service.

Logs are stored in two main locations:

  • C:\Program Files\Spiceworks\Log, for the Spiceworks service
  • C:\Program Files\Spiceworks\httpd\log\, where the Apache server keeps them

Once the logs are cleaned, I compacted the DB to increase the performance, and I started the service again.

Ticket rules were configured to auto assign support tickets, thus saving time to the support operators.

And the user portal was customized to match the company colors and logo.

Axis CCTV and Video Management System

A security company contacted me thorough Upwork, searching for support on a brand new installation of an Axis Camera Station System on a educational institution. This company, Coyote Cabling from New Mexico, US, was on charge of a 52 camera installation, with an option to add 32 existing cameras on a later stage.

After a research, they decided to use Axis S1148 servers, which really are re-branded Dell servers. The S1148 comes with a ready to use Windows Server 2012 OS, and with the Axis Camera Station preinstalled. This vendor supported hardware allowed to reduce licensing costs, because they are included on the server price, and avoid any incompatibilities