Ability to Collect custom field(s)to timeseries database - DQ collect Part 2

Big Picture User Experience

Enhance attributes collected with the value into the historical DB. Primarily, user defined fields that represent DQ driven by 3rd party applications and/or Krunch pipeline.

Detailed Description

With the expanded capability of Enterprise SCADA with the ability to extend tables and introduction of Krunch pipeline. Customers and 3rd party integrators will be building solutions to meet customer needs. These solutions will need to support custom Data Qualities that are driven by these applications. These custom DQ will reside outside of the flag structure and will be defined as boolean fields.
For example, a customer have developed application that enables to define a point as downgraded. This is a boolean value that is added to a telemetry point and if true will provide a visual indication to the SCADA controller. While this point is in the "downgrade" condition, field values is still being krunched and processed by the Enterprise SCADA system. Coupled with the krunch, these points can also be collected into historical DB and later rendered on Trend object within SCADA HMI application. When a SCADA controller inspects the trends, it is prominent for him see the how the value is trending overtime and not less importantly to see the DQ that is applicable for each sample. The DQ provides additional context to the trended values and informs SCADA Controller the important of the value.With these DQs, it is important to trend the custom DQ as well.As such, it is requested to enhance Enteprise SCADA to:

1. Enterprise SCADA to collect the custom boolean field(s) (aka custom DQ) in addition to the baseline DQ attributes collected.

2. For SCADA Administrator to define the precedence/priority of the DQ that will be stored within the DB. So that at render time, these DQ can be prioritized in with the baseline precedence configuration.

3. Support N number of custom boolean fields (Need to discuss with the team on what is possible, hopefully, 8.)

4. A 3rd party application queries the collected data and this DQ is returned per the configured precedence.

5. The custom DQ roll up into hourly, daily and monthly aggregates of the data as follows:
   

   For the Min, Max values:

   Summary will record all DQ present at the time of the max or min value into the encoded status integer.

   For the Value value:

   Summary will record all DQ present at the time of the value field.

   For the Average value:

   Summary will record all DQ present during the hour will be reflected in the hour’s record:

   Collect Time	Realtime Value	Realtime DQ Present	Timeseries Record	Hour Record
   09:00	10	TestMode	Val=10, DQ= pack(TM)	Max: 100 @09:45, DQ=pack(T,F) Min: 10 @09:15, DQ=pack(TM, S) Val: 100 @09:45, DQ=pack(TM, F) Avg: 52.5 @09:00, DQ=pack(TM,S,A,T,F)
   09:15	10	TestMode, Stale	Val=10, DQ=pack(TM, S)
   09:30	90	TestMode, Alarm	Val=90, DQ=pack(TM, A)
   09:45	100	Telemetered, Fresh	Val=100, DQ=pack(T, F)

Business Value

Improve SCADA Controllers situational awareness by presenting the most important points’ states on ezXOS Trend. This will allow the SCADA controller a faster review of point behavior through history rather than analyzing events presented in tabular format.

Improve the precision of the recorded data quality, by recording them all and giving control of what is trended to the SCADA administrator.

Customer, Project, and Deadline Details

• Number of customers (names/projects) that will consume this

  • Shell (SPLC) will use this feature. SE-WWW have also stated a need for this.

• Deadlines and Commitments

  • Shell is expected to be the first consumer of this feature.

  • Shell project kick-off is Q2-2022.

  • Project team may need to start with ES2022-R2, and upgrade mid-flight to ES2023 in Q1-2023

• Commercial and/or Contractual Impacts

  • Commitments to SPLC have been made to have this feature.

Out of Scope

• TimeSeries database only. Timeline is now in End of Life and not accepting new features.

• Handling custom flags in realtime, and how the HMI gets access to these for realtime data is out of scope for this requst.

• Presentation of a de-normalised view of the packed data qualities for use by a third party. This is a simple view that can be developed by the projects as needed.

• The “questionable” bit mentioned as one of the baseline DQs for historical doesn’t yet have a realtime feature to populate it, and is reserved for future use.

Assumptions

• It is assumed that the value of the points will be collected along with all the relevant data quality flags in accordance to the collection type configured for the field, in one operation. This is to avoid “data smear”.

• The requested enhancement is aimed to provide comparable (if not similar) functionality to the customers’ current implementation. As the customer using baseline HMI application, the changes requested in this document should integrate and work with baseline HMI in the same manner as it works on current customer’s system. No changes to HMI were identified to meet this goal. Please see appendix A for more details.

• The enhancement is targeted for Flexible Chart/Plot objects.

Dependencies

• Productization/documentation of extended configuration of color and line style for "Data Quality" on trends (This is will be done by HMI team in PI 39).

• The HMI product expects a consistent data contract between the backend and the front end. FCO object functionality to render multiple DQ will be reviewed and re-designed if needed to meet desired functionality outlined in this PMFR.

• The HMI team need to provide guidance on the “contracted” data format expected by FCO object after the design phase of FCO. (see ADO - 1006723) - First step was done. During implementation the two team will need to collaborate.

PSR - Performance, Scalability, Resilience

• The collection process should be able process and store these DQ for the permitted number of collection points without impacting system performance.

• Rendering of the points on Trend and Plot object shall not be degraded (within reason) with additional DQ.

NFR – Non-Functional Requirements

1. Properly documented in the following documentation

   • Realtime Admin guide for configuring collect for custom fields

   • Historical admin guide for configuring DQ precedence

2. Provide documentation on how to add a custom DQ in the RT table.

Risks/Mitigations

Most of the states are baseline states that are defined either in flags, Hilo or tankHilow structures. The custom DQ is user-defined.

Link to full doc (appendix incl):https://aveva.sharepoint.com/sites/MidStreamPM/Shared%20Documents/Customers/Shell%20Pipeline/Rationalization/PMFR/Historical%20Collection/PMFR%20-%20Historical%20Collection.docx?web=1

AHA Feature:https://aveva.aha.io/features/OASYS-172
Class 5 estimate:

Corresponding HMI Feature:https://dev.azure.com/AVEVA-VSTS/OASyS/_workitems/edit/1006723