The prodos memodata processing

Owned by Christopher Priem (Deactivated)
Last updated: Sept 27, 2022 by Carsten Fenneker
4 min read

This document describes the configuration and the data flow of the prodos memodata processing. The prodos webserver offers an URI that triggers the download of a memodata zip file from the prodos control. New data are available after the completion of a dosing cycle. Specific parts of the memodata have to be extracted and stored at the SiteController.

The SiteController uses its module da_rest_client to interact with the prodos webserver by REST requests.

1. Determine the completion of a dosing cycle

The sensor globalstates_dosing_mode represents the current state of the dosing cycle:

Configuration of the sensor for the dosing mode
    <sensor sensor_id="globalstates_dosing_mode">
    <sensor_class>code</sensor_class>
      <state_evaluation_expressions>
        <state_evaluation_expression>
          <expression>True</expression>
            <true>%value%</true>
        </state_evaluation_expression>
      </state_evaluation_expressions>
      <sensor_gateway sensor_gateway_id="globalstates_gw">
      <demux>
        <keys>
          <key>dms</key>
          </keys>
        </demux>
      </sensor_gateway>
    </sensor>

This sensor requires the sensor gateway globalstates_gw:

Configuration of the sensor gateway
<sensor_gateway publish_strategy="on_change" sensor_gateway_id="globalstates_gw">
   <rest_client>
      <uri>/fcgi/data/globalstates/1</uri>
      <scheduling>
         <polling_interval>3000</polling_interval>
         <error_handling>
            <retry retry_interval_before_alert="2000"/>
            <timeout_handling>
              <fixed_timeout>1000</fixed_timeout>
            </timeout_handling>
         </error_handling>
      </scheduling>
   </rest_client>
</sensor_gateway>

With this scheduling parameters (3 seconds interval ) all changes of the dosing mode state are acquired. If the sensor globalstates_dosing_mode has the value 'AUTO Dosing _ END' or  'Dosing _ END' the dosing cycle is completed and new memodata could be downloaded.

2. Download of the memodata

An AutomationController rule detects the state change to the dosing cycle completion

Configuration of the download trigger rule
 	<rule rule_id="Prodos download trigger">
 	  <triggers>
 	    <trigger value_name="dosing_mode" value_type="string" trigger_topic="calibrated_result" sensor_id="globalstates_dosing_mode"/>
 	  </triggers>
 	  <conditions>
 	    <condition expr="dosing_mode == 'AUTO Dosing _ END' or dosing_mode == 'Dosing _ END'">
 	      <true>
 	        <action_list>
 	          <action action_id="download_memodata" command="download"/>
 	        </action_list>
 	      </true>
 	    </condition>
 	  </conditions>
 	</rule>

Please refer to the da_rest_client action to get more information about the download action.

The action itself is configured this way:

Configuration of the download action
    <action action_id="download_memodata" device_id="rest_test">
      <commands>
        <command command_id="download">download('/fcgi/control/memodata_download')</command>
      </commands>
    </action>

3. Determine the completion of memodata download

As described in the da_rest_client action the da_rest_client is able to represent the download completion and the download time in a specific sensor gateway (the sensor gateway id is identical to the device id of the da_rest_client).

Configuration of a sensor to indcate the download
    <sensor sensor_id="rest_download_file">
      <sensor_class>unknown</sensor_class>
      <upload>always</upload>
      <error_severity>200</error_severity>
      <state_evaluation_expressions>
        <state_evaluation_expression>
          <expression>True</expression>
            <true>%value%</true>
        </state_evaluation_expression>
      </state_evaluation_expressions>
      <sensor_gateway sensor_gateway_id="rest_test">
        <demux>
          <keys>
            <key>dl_file</key>
          </keys>
        </demux>
      </sensor_gateway>
    </sensor>

For a valid XML configuration it is required to add the sensor gateway id to the device configuration of the da_rest_client:

   <device device_id="rest_test">
     <rest_client_device>
       <host>...</host>
     </rest_client_device>
      <sensor_gateways>
      	<sensor_gateway sensor_gateway_id="rest_test">
      	  <simulator/>
      	</sensor_gateway>
      </sensor_gateways>
   </device>

4. Processing of the downloaded memodata

An AutomationController rule detects the completion of a download

Configuration of a processing trigger rule
 	<rule rule_id="Prodos log processing">
 	  <triggers>
 	    <trigger value_name="dl_file" value_type="string" trigger_topic="calibrated_result" sensor_id="rest_download_file"/>
 	  </triggers>
 	  <conditions>
 	    <condition expr="dl_file is not None">
 	      <true>
 	        <action_list>
 	          <action action_id="process_memodata" command="go"/>
 	        </action_list>
 	      </true>
 	    </condition>
 	  </conditions>
 	</rule>

The action itself is configured this way:

Configuration of a memodata processing action
    <action action_id="process_memodata" device_id="VirtualSensorProvider">
      <commands>
        <command command_id="go">launch_exec_job('process_memodata')</command>
      </commands>
    </action>

Please refer to the Prodos log processor to get detailed information about the processing. For completeness the minimal configuration in the SiteController.cfg to launch the processing is shown here:

[remote_exec_calls]
process_memodata=/opt/azeti/SiteController/src/prodos_log_processor.py