InfluxDB

Install and Setup TIG Stack on Ubuntu 20.04

by

In this tutorial, we are going to learn how to install and setup TIG Stack on Ubuntu 20.04.

TIG stack is a group of powerful open-source monitoring tools, Telegraf, InfluxDB and Grafana where;

  • Telegraf is an open-source server agent for collecting and sending metrics and events from databases, systems, and IoT sensors.
  • InfluxDB is an open-source time series database and provides datastore for metrics, events, and real-time analytics.
  • Grafana is a data visualization and monitoring tool and supports time series datastores such as Graphite, InfluxDB, Prometheus, Elasticsearch.

TIG stack can be used for monitoring system metrics such as memory, disk, logged in users, system load, swap usage, system uptime, system processes. 

Installing and Setup TIG Stack on Ubuntu 20.04

In order to install TIG stack, you need to install and setup each individual component of the stack, Telegraf, InfluxDB, Grafana.

Run System Update
To begin with, ensure that your system packages are up-to-date;

apt update
apt upgrade

Install Telegraf on Ubuntu 20.04

You can install Telegraf on Ubuntu 20.04 either by downloading the DEB package file or directly from the InfluxData repos.

Install Telegraf using DEB Package File

To install Telegraf using the DEB binary, grab the binary installer from the InfluxData downloads page. You can simply obtain the link to binary installer and pull it with wget;

wget https://dl.influxdata.com/telegraf/releases/telegraf_1.14.3-1_amd64.deb

Once you have the binary downloaded, you can install it as follows

Read more

Share

Install InfluxDB on Ubuntu 20

by

InfluxDB is a time-series database licensed under open source. It helps to collect metric data from devices such as servers, switches, virtual servers, ups and plot graphs in realtime. It is written in Go language and optimized for fast and high availability.

Enabling Repository

Install InfluxDB by importing the key and enable the required repository.

$ wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
$ source /etc/lsb-release
$ echo "deb https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

Install InfluxDB

Right after enabling the repository, Install Influxdb packages by running package manager for the respective operating system.

Use apt for installing on Ubuntu Linux distributions

Installing on Ubuntu

To install the influxdb on Ubuntu use the apt.

$ sudo apt-get update
$ sudo apt-get install influxdb

Enable the Service

Right after installing the package enable the service to start persistently, however don’t start it.

Read more

Share

Linux Monitoring using Grafana InfluxDB and Telegraf on Debian 10

by

The basic installation of Grafana InfluxDB and Telegraf is described in my other post here-

Install Grafna, InfluxDB, Telegraf for Jitsi Video Meet Monitoring on Debian 10

All is needed is to create a telegraf configuration file:

nano /etc/telegraf.d/dashboard.conf

# Global tags can be specified here in key="value" format.
[global_tags]
# dc = "us-east-1" # will tag all metrics with dc=us-east-1
# rack = "1a"
## Environment variables can be used as tags, and throughout the config file
# user = "$USER"

# Configuration for telegraf agent
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
debug = false
quiet = false
hostname = ""
omit_hostname = false

### OUTPUT

# Configuration for influxdb server to send metrics to
[[outputs.influxdb]]
urls = ["http://your_host:8086"]
database = "telegraf_metrics"

## Retention policy to write to. Empty string writes to the default rp.
retention_policy = ""
## Write consistency (clusters only), can be: "any", "one", "quorum", "all"
write_consistency = "any"

## Write timeout (for the InfluxDB client), formatted as a string.
## If not provided, will default to 5s. 0s means no timeout (not recommended).
timeout = "5s"
# username = "telegraf"
# password = "2bmpiIeSWd63a7ew"
## Set the user agent for HTTP POSTs (can be useful for log differentiation)
# user_agent = "telegraf"
## Set UDP payload size, defaults to InfluxDB UDP Client default (512 bytes)
# udp_payload = 512

# Read metrics about cpu usage
[[inputs.cpu]]
## Whether to report per-cpu stats or not
percpu = true
## Whether to report total system cpu stats or not
totalcpu = true
## Comment this line if you want the raw CPU time metrics
fielddrop = ["time_*"]

# Read metrics about disk usage by mount point
[[inputs.disk]]
## By default, telegraf gather stats for all mountpoints.
## Setting mountpoints will restrict the stats to the specified mountpoints.
# mount_points = ["/"]

## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually
## present on /run, /var/run, /dev/shm or /dev).
ignore_fs = ["tmpfs", "devtmpfs"]

# Read metrics about disk IO by device
[[inputs.diskio]]
## By default, telegraf will gather stats for all devices including
## disk partitions.
## Setting devices will restrict the stats to the specified devices.
# devices = ["sda", "sdb"]
## Uncomment the following line if you need disk serial numbers.
# skip_serial_number = false

# Get kernel statistics from /proc/stat
[[inputs.kernel]]
# no configuration

# Read metrics about memory usage
[[inputs.mem]]
# no configuration

# Get the number of processes and group them by status
[[inputs.processes]]
# no configuration

# Read metrics about swap memory usage
[[inputs.swap]]
# no configuration

# Read metrics about system load & uptime
[[inputs.system]]
# no configuration

# Read metrics about network interface usage
[[inputs.net]]
# collect data only about specific interfaces
# interfaces = ["eth0"]

[[inputs.netstat]]
# no configuration

[[inputs.interrupts]]
# no configuration

[[inputs.linux_sysctl_fs]]
# no configuration

Update the necessary paramaters accordign to your installations.

Import the Grafana Dashboard Template from here-

https://grafana.com/grafana/dashboards/928

Share

Install Grafna, InfluxDB, Telegraf for Jitsi Video Meet Monitoring on Debian 10

by

Step 1: Install InfluxDB

apt update && apt install -y gnupg2 curl wget
wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian buster stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
apt update && apt install influxdb -y
systemctl enable --now influxdb
systemctl status influxdb

If you run a firewall (i.e. ufw) on this server, open the port for influxdb and grafana webserver:

ufw allow 8086/tcp
ufw allow 3000/tcp

Step 2: Install Grafana to display stats dashboards

curl https://packages.grafana.com/gpg.key | sudo apt-key add -
add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
apt update && apt install grafana -y
systemctl enable --now grafana-server
systemctl status grafana-server

Step 3: Install & configure telegraf

wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian buster stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
apt update && apt install telegraf -y
mv /etc/telegraf/telegraf.conf /etc/telegraf/telegraf.conf.original
nano /etc/telegraf/telegraf.conf

Enter following contents in telegraf.conf:

[global_tags]

###############################################################################
# GLOBAL #
###############################################################################

[agent]
interval = "10s"
debug = false
hostname = "jitsi_host"
round_interval = true
flush_interval = "10s"
flush_jitter = "0s"
collection_jitter = "0s"
metric_batch_size = 1000
metric_buffer_limit = 10000
quiet = false
logfile = ""
omit_hostname = false
nano /etc/telegraf/telegraf.d/jitsi.conf

Enter following contents in jitsi.conf:

###############################################################################
# INPUTS #
###############################################################################
[[inputs.http]]
name_override = "jitsi_stats"
urls = [
"http://116.203.231.172:8080/colibri/stats"
]
data_format = "json"
###############################################################################
# OUTPUTS #
###############################################################################
[[outputs.influxdb]]
urls = ["http://localhost:8086"]
database = "jitsi"
timeout = "0s"
retention_policy = ""

We enable start on boot and start telegraf now on server “jitsi”:

systemctl enable --now telegraf
systemctl status telegraf

(Mind: We will not create a database as Telegraf will create our database if it does not find one)

Server: "jitsi"

Step 4: Adapt Jitsi onfiguration to expose stats

nano /etc/jitsi/videobridge/config

Make sure to configure the jvb options:

JVB_OPTS="--apis=rest,xmpp"

and

nano /etc/jitsi/videobridge/sip-communicator.properties

Here we configure colibri statistics:

org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri
service jitsi-videobridge2 restart

Check output in the terminal on the jitsi server:

curl -v http://127.0.0.1:8080/colibri/stats
Response: {"inactive_endpoints":0,"inactive_conferences":0,"total_ice_succeeded_relayed":0,"total_loss_degraded_participant_seconds":0,"bit_rate_download":0,"muc_clients_connected":1,"total_participants":0,"total_packets_received":0,"rtt_aggregate":0.0,"packet_rate_upload":0,"p2p_conferences":0,"total_loss_limited_participant_seconds":0,"octo_send_bitrate":0,"total_dominant_speaker_changes":0,"receive_only_endpoints":0,"total_colibri_web_socket_messages_received":0,"octo_receive_bitrate":0,"loss_rate_upload":0.0,"version":"2.1.169-ga28eb88e","total_ice_succeeded":0,"total_colibri_web_socket_messages_sent":0,"total_bytes_sent_octo":0,"total_data_channel_messages_received":0,"loss_rate_download":0.0,"total_conference_seconds":0,"bit_rate_upload":0,"total_conferences_completed":0,"octo_conferences":0,"num_eps_no_msg_transport_after_delay":0,"endpoints_sending_video":0,"packet_rate_download":0,"muc_clients_configured":1,"conference_sizes":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"total_packets_sent_octo":0,"conferences_by_video_senders":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"videostreams":0,"jitter_aggregate":0.0,"total_ice_succeeded_tcp":0,"octo_endpoints":0,"current_timestamp":"2020-04-17 23:14:38.468","total_packets_dropped_octo":0,"conferences":0,"participants":0,"largest_conference":0,"total_packets_sent":0,"total_data_channel_messages_sent":0,"total_bytes_received_octo":0,"octo_send_packet_rate":0,"conferences_by_audio_senders":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"total_conferences_created":0,"total_ice_failed":0,"threads":37,"videochannels":0,"total_packets_received_octo":0,"graceful_shutdown":false,"octo_receive_packet_rate":0,"total_bytes_received":0,"rtp_loss":0.0,"total_loss_controlled_participant_seconds":0,"total_partially_failed_conferences":0,"endpoints_sending_audio":0,"total_bytes_sent":0,"mucs_configured":1,"total_failed_conferences":0,"mucs_joined":1}

With this response we see that the rest api responds and can be used for our purpose! To make sure that the colibri rest endpoint can be accessed by telegraf:

ufw allow 8080/tcp

Configure dashboards in Grafana

Open Grafana in browser: http://<Your-IP>:3000 31 and prepare your admin account

Add datasource.

We will add an InfluxDB datasource and set it to be the default.

Name: InfluxDB
Default: On
HTTP URL: http://localhost:8086 14
HTTP Access: Server (default)
Database: jitsi

Download a Dashboard.

https://grafana.com/grafana/dashboards/11969

A collection of all fields from the jitsi rest api response is available in the json attached for convenience.

Src: https://community.jitsi.org/t/how-to-to-setup-grafana-dashboards-to-monitor-jitsi-my-comprehensive-tutorial-for-the-beginner/38696

Share