Introduction

This page will go into the technical depths about the timing of data acquisition in general.Some abbreviations will be used for the sake of simplicity.

DA = Data Acquisition

Publish strategy

Passive modules without polling and timeout

These modules are not polling and are also not waiting up to a timeout time. They just listen and forward a result, when present.

DA moduledescription
trapdSNMP trap daemon
gps_tcp_clientlisten to GPS NMEA packet received from a TCP connection
gps_serial_clientlisten to GPS NMEA packet received from a serial port






Modules with interval and timeout

http_server

The http_server works passive regarding incoming notifications (no polling, no timeout) and is driven by actions to request images. These requests have a configurable timeout. If this timeout is exceeded the action ends up with an error.

unitminmaxdefault
seconds1sundefined60s

GPIO DIO

The GPIO DIO waits for input changes with a timeout parameter.  The timeout for that is configurable with the attribute polling_interval, with a default of 10000ms (10s). If the corresponding sensor_gateway is configured to  publish_strategy="always" then a raw_result of the GPIO input/output states will be generated all "polling_interval" milliseconds.

polling_interval

unitminmaxdefault

milliseconds

1 msundefined10 000 ms

da_external_checker

The da_external_checker is a data acquisition module that periodically invokes an external script.

interval

Mandatory parameter with the period in milliseconds to wait until try to launch the script again. This value is expected to be bigger or equal to 1000 (1 second) 

unitminmaxdefault

milliseconds

1000 msundefinedundefined

timeout

Mandatory parameter with the amount of time in milliseconds that a script could run.

unitminmaxdefault

milliseconds

1 msundefinedundefined

ble_scanner

The ble_scanner is a data acquisition module which listen to advertisements sent by BLE (Bluetooth Low Energy) devices.

alive_timeout

Mandatory parameter with the amount of time in seconds that a BLE device should be kept in memory and marked as alive (or in the neighborhood).

unitminmaxdefault
seconds1sundefinedundefined

AP (wifi) scanner

The ap_scanner is a data acquisition module which scans Wireless Access Points on the neighborhood.

interval

Mandatory parameter which defines the interval in milliseconds between each scan.

unitminmaxdefault

milliseconds

1000 msundefined

undefined

timeout

Mandatory parameter which defines the maximum amount of time that the scan is allowed to run, in milliseconds.

unitminmaxdefault

milliseconds

1000 msundefinedundefined

PLC engine interface

poll_interval

If not configured, the default is taken.

unitminmaxdefault

milliseconds

50 msundefined1000 ms

In difference to the other DA modules the poll_interval is configured for the complete module but not separated in the sensor gateways.

Modules with scheduled polling

The modules, listed in this section use a configuration schema to control the scheduling, which is defined in the XML schema definition (XSD).

Schema

This complex schema definition is currently only partially used in the DA modules, which are listed in this section. Different DA modules use this definition in a different way. 


The timing resolution of these modules is limited to a second. Even though the units are milliseconds the values get rounded internally to seconds.


retry_algorithm

is a value of following enumeration

retry algorithmdescription
linearlinear timing steps
binary_expbinary exponential timing steps, like the following set: 1,2,4,8,16,32,64,...


Only the linear retry algorithm (which is the default) is used for now. As linear is the default, there is no need to put it to the configuration.

da_rest_client, da_icmp, snmpgetd

These three DA modules use the schema definition above in the same way.The used parameters are:

polling_interval

time between each query

unitminmaxdefaultremark

milliseconds

1000 ms 1
undefined

undefined

rounded to seconds

retry_interval_before_alert

time for which a query could be retried, after that an error is reported and it need to wait for the next query cycle.

unitminmaxdefaultremark

milliseconds

1000 ms 1
undefined

undefined

rounded to seconds

fixed_timeout

timeout for each query try

unitminmaxdefault

remark

milliseconds

1000 ms 1
undefined

undefined

rounded to seconds

1 It is possible to configure a minimum value >= 0 milliseconds. This misconfiguration is intercepted by the DA modules.

 These three DA modules essentially calculate and perform the query up to trials times until they either get the result or specify the error of the query. This is done each polling_interval cycle.

Example

The following configuration example for the scheduling configuration

...
            <scheduling>
              <polling_interval>15000</polling_interval>
				<error_handling>
					<retry retry_interval_before_alert="9000"/>
					<timeout_handling>
						<fixed_timeout>3000</fixed_timeout>
					</timeout_handling>
				</error_handling>
            </scheduling>
...

means: All 15 seconds the module tries 3 times to do the query. Each trial is limited to 3 seconds.

modbusd

The DA module modbusd uses the parameters

with the same resolution (rounded to seconds) and limitations as in the tables above in this section. Usually the modbusd is configured with a polling_interval only. The difference to the other three DA modules of this section is the error handling, if configured. In case of a failed primary query in the polling_interval query cycle the modbusd continues to poll with the retry_polling_interval until the retry_interval_before_alert is reached.

Configuration of the data export to the cloud

Example

[data_store]
...
# default upload strategy (never, on_change, interval, always)
upload_strategy = interval
# upload interval to server in seconds
upload_interval = 60
...

The SiteController.cfg is on default regarding the upload_interval (60s) and the upload_strategy (interval).


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<component_template config_version="1.0" schema_version="1.5.16-Aspirator">
  <description>DA rest client</description>
  <class>multi-sensor</class>
  <vendor>azeti</vendor>
  <version>1</version>
  <sensors>
    <!-- REST sensors -->
    <sensor sensor_id="rest_temperatures_sensor/cooler">
      <sensor_class>unknown</sensor_class>
      <sensor_gateway sensor_gateway_id="rest_temperatures_gw">
        <demux>
          <keys>
            <key>cooler</key>
          </keys>
        </demux>
      </sensor_gateway>
    </sensor>
    <sensor sensor_id="rest_pressures_sensor/tire">
      <sensor_class>unknown</sensor_class>
      <upload>always</upload>
      <sensor_gateway sensor_gateway_id="rest_pressures_gw">
        <demux>
          <keys>
            <key>tire</key>
          </keys>
        </demux>
      </sensor_gateway>
    </sensor>
  </sensors>
  <devices>
    <device device_id="rest_server">
      <rest_client_device>
        <host>http://192.168.118.26:7088</host>
        <form_auth>
          <uri>/rest/login.php</uri>
          <form_data>
            <form_key_value>
              <pair_key>username</pair_key>
              <pair_value>operator</pair_value>
            </form_key_value>
            <form_key_value>
              <pair_key>password</pair_key>
              <pair_value>LetMeIn</pair_value>
            </form_key_value>
            <form_key_value>
              <pair_key>t</pair_key>
              <pair_value>OK</pair_value>
            </form_key_value>
          </form_data>
          <method>post</method>
        </form_auth>
      </rest_client_device>
      <sensor_gateways>
        <sensor_gateway publish_strategy="always" sensor_gateway_id="rest_temperatures_gw">
          <rest_client>
            <uri>/rest/temperature.php</uri>
            <scheduling>
              <polling_interval>15000</polling_interval>
              <error_handling>
                <retry retry_interval_before_alert="9000"/>
                <timeout_handling>
                  <fixed_timeout>3000</fixed_timeout>
                </timeout_handling>
              </error_handling>
            </scheduling>
          </rest_client>
        </sensor_gateway>
        <sensor_gateway publish_strategy="always" sensor_gateway_id="rest_pressures_gw">
          <rest_client>
            <uri>/rest/pressure.php</uri>
            <scheduling>
              <polling_interval>15000</polling_interval>
              <error_handling>
                <retry retry_interval_before_alert="9000"/>
                <timeout_handling>
                  <fixed_timeout>3000</fixed_timeout>
                </timeout_handling>
              </error_handling>
            </scheduling>
          </rest_client>
        </sensor_gateway>
      </sensor_gateways>
    </device>
  </devices>
</component_template>

In the example configuration above are 2 sensors configured, each with an own sensor gateway at the data acquisition device. Usually more then one sensors are using the same sensor gateway, which would be possible also here.Both sensor gateways are configured to have the publish_strategy always (default is on_change).

In the example configuration the sensor rest_pressures_sensor/tire is configured to have the upload_strategy always. The other sensor rest_temperatures_sensor/cooler remains with the default upload_strategy interval.

Both gateways of the device 'rest_server' emit their raw results all 15 seconds, regardless if there was a change in the values, because both gateways are configured with the publish_strategy="always"

The sensor rest_pressures_sensor/tire is configured with the upload strategy <upload>always</upload>. This way in the cloud the sensor data are present with a difference in their timestamps of 15 seconds.

The sensor rest_temperatures_sensor/cooler has no specific upload strategy in its configuration, so it uses the default upload strategy interval which is set to 60 seconds (see the Example snippet of the SiteController.cfg above). Herewith the sensor data are present in the cloud with a difference in their timestamps of  60 seconds.

In both cases additional values are uploaded to cloud, if there is a change in the value.