data_exporter - Export raw event and performance data
Introduction
The Data-Exporter Addon is a daemon which collects ever occurring events (such as ALERTS, NOTIFICATIONS; DOWNTIMES, …) and all performance data. The collected data is saved locally on the SONARPLEX device. The data will then be uploaded to the configured FTP server. All the configuration happens through "Host Properties".
The data export format can be either XML or CSV, it can be configured (see below).
The Data-Exporter enables you to import historical data in on-premise reporting tools for in-deep analyzation.
Installation
- Open the Administration Web Interface> Configuration > My Plugins
- Choose Browse and select the new extension file and click
to start the upload - The installation was successful if no errors are shown in red and the last status line is OK
You will see that a host object plus service objects for the Self-Monitoring are automatically created during installation. See Self-Monitoring for reference.
On this page:
Configuration
Setting a unique Location Identifier
The default location name is set to local. It is recommended to change it as it helps identifying one SONARPLEX in a setup of multiple ones and the exported archives include the location identifier.
- Open the Administration Web Interface > Configuration > My Properties and scroll down to the bottom
- Open the resource LOCATION
- Set Ressource Value to the desired location name, e,g, Cologne
- Click
to save the configuration
Configuring the Data Export Settings
Version previous to 1.0.437 used Host Properties for the basic configuration. Versions starting from 1.0.437 and higher offer all Settings as part of the Administration Web Interface which makes the setup easier. Make sure to use the latest version.
The destination server was automatically created as the host DATA_EXPORT_SERVER. So the first steps is to change the ip address of this host object to match your environment. The new setting is loaded by the Data Exporter as soon as the Monitor process is restarted. The next step is to set the configuration details for the export.
The configuration can be found in Open the Administration Web Interface> Configuration > System > Data Export Configuration
A minimum configuration requires the following:
- Open the Administration Web Interface> Configuration > System > Data Export Configuration
- Set the FTP Username and FTP Password
- Set the FTP Directory and make sure it exists on the server, it will default to the users login directory if not set
- Choose
to save the configuration and 
to apply it immediately, the Data Exporter will be restarted and a first export will happen shortly
All available settings are explained in detail below.
Setting | Description |
|---|---|
| FTP server address | Loaded automatically from the host object DATA_EXPORT_SERVER |
| Data Export Interval in minutes | 1-9999 (upload interval in minutes) |
| Data Export Format | XML / CSV (default CSV) |
| Data Export ItemType Blacklist | perfdata/events (excludes configured itemtype from export) |
| FTP Username | Username |
| FTP Password | Password |
| FTP Port | UDP Port, defaults to 21 |
| FTP Directory | Upload directory on the server Make sure this directory exists on the FTP server, otherwise the upload will fail. |
| FTP Timeout in seconds | 5-30 (defaults to 30) |
| Suppress Soft Events | Yes / No, activate to only log HARD events and suppress SOFT ones. |
| FTP Debug level | Debug level |
Troubleshooting
Enable the Logging
All problems are logged in the data_exporter.log, that can be found in Administration Web Interface > Status > Logs > Data Exporter [data_exporter.log].
Make sure you enabled logging for the Data Exporter:
- Open the Administration Web Interface > Configuration > System > Data Export Configuration
- Set the FTP Debug Level to at least INFO
- Choose to save the configuration and to apply it immediately, the logfiles will from then on be written with information
Always make sure to disable the logging when you're done with troubleshooting as writing logfiles has significant impact on the overall I/O performance, especially for less powerful hardware.
Self-Monitoring
The data exporter features so called self-monitoring. During installation the configured hosts are checked if one is considered to be a data export server (_data_export_server property is set). If not a new host object is created with two new service checks, which are explained below. Of course the service checks are attched to an existing host if found.
Data Export Server Ping
This service check pings the ip address of the configured Data Export server.
Data Export Server Problems
The service check shows all processing and upload problems as soon as those occurr, such as ftp login problems, non existing ftp upload directories or wrong ftp ports.
- style