Description

Osisoft PI is the most used historian in the Oil&Gas industry. EdgeX needs to be capable to export data to such a platform and the target is to deliver this feature as part of California Release (June 2018).

Requirements

• Use PI Web API (restful) for Data Ingestion
• Authenticate the EdgeX node through Basic Authentication
• Create new tags in PI Archive
• Export throughput up to 1,000 events/second from each EdgeX node
• Local buffering and retry mechanism with resilience to network downtime
• Local logging of error messages & events through EdgeX Log service

Design

This capability can be implemented by cloning the current EdgeX HTTPS REST export feature and modifying it accordingly to send messages in a format accepted by PI Web API. This is an approach that has been adopted by other IIoT platforms as well.

• Assumptions
  • There is a 1-to-1 relationship between EdgeX measurements and PI Tags. A table to map the datatypes shall be defined.
  • There is a 1-to-1 relationship between EdgeX events and PI Tag. A table to map the datatypes shall be defined.
  • The corresponding PI Tag will have a name that is obtained by concatenating the three following pieces (with "." in between)
    • Tag Prefix as defined in the next section "Step 1 - Configuration"
    • Device Name (as what is specified in EdgeX Metadata - this must be unique)
    • Measurement/Event Name (as per what is specified in EdgeX Metadata - this must be unique within the same device)
  • An example of PI Tag Name will be "deployment001.deviceABC.measurement005"
• Step 1 - Configuration
  • Parameters that define the configuration of the export service and should be set through Export Client API calls:
    • PI Web API Endpoint (e.g. https://mypiwebapiendpoint/piwebapi)
    • PI Archive Name (e.g. mypiarchive)
    • Export interval in milliseconds (e.g. 1000) - as an alternative this could be batch size
    • Username (e.g. piwebapiuser)
    • Password (e.g. piwebapipassword)
    • Tag Prefix (e.g. deployment0001)
    • Datapoint(s) that will be exported to corresponding PI tag(s)
  • As part of the registration to export client, it would be appropriate to verify connectivity/authentication. 
    • Make a GET request to https://mypiwebapiendpoint/piwebapi/dataservers with the basic authentication header as per piwebapiuser/piwebapipassword
    • The result will look like the below. If there is an entry with "Name":"mypiarchive" then the registration can be considered successful and the WebId of the archive should be stored in EdgeX for usage in following steps

    • { "Links": {}, "Items": [ { "WebId": "s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA", "Id": "7ab0ec2c-1daf-41e7-b310-f7d13aed28a9", "Name": "10.125.23.143", "Path": "\\\\PIServers[10.125.23.143]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA/points", "EnumerationSets": "mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA/enumerationsets" } }, { "WebId": "s0AAAAAAAAAAD_____vIRf4ATkRQ", "Id": "00000000-0000-0000-ffff-ffffbc845fe0", "Name": "mypiarchive", "Path": "\\\\PIServers[mypiarchive]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ/points", "EnumerationSets": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ/enumerationsets" } }, { "WebId": "s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE", "Id": "581bbc60-958f-49f2-8873-f10f849f5df8", "Name": "pisrv002", "Path": "\\\\PIServers[pisrv002]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE/points", "EnumerationSets": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE/enumerationsets" } } ] }

    • One of the following 3 errors may occur during this verification:
      • Unable to reach the endpoint (may be due to different reasons such as connectivity, firewall blocking port 443, DNS resolution...). In this case a timeout is necessary to pop out an error after X amounts of seconds/retries
      • 401 Unauthorized (Wrong credentials, either username or password)
      • mypiarchive does not exist in the list of registered PI Archives (or it may be that the server is registered with the hostname while the specified endpoint is an IP address)
        
        

• Step 2 - Initialization
  • The following steps need to be executed after the client registration or any time the configuration changes.
    • Verify if the tags are already existing in the PI Archive. 
      • For each configured tag, the following query must be executed.
      • GET https://mypiwebapiendpoint/piwebapi/search/query?q=name:tagname&scope=pi:mypiarchive

      • If the tag already exists, the reply will look like:

        { "TotalHits": 1, "Links": { "Next": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10&start=10", "First": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10", "Last": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10&start=0" }, "Errors": [], "Items": [ { "Name": "tagname", "Description": "Some Description", "MatchedFields": [ { "Field": "name" } ], "ItemType": "pipoint", "AFCategories": [], "UniqueID": "\\\\{044de274-8d72-4cb6-839b-9e415e793dd8}\\?3", "WebId": "P0dOJNBHKNtkyDm55BXnk92AAwAAAAU1JWR0RZUExNT1NJRDAzXENEVDE1OA", "UoM": "deg. c", "DataType": "float32", "Links": { "Self": "https://mypiwebapiendpoint:443/piwebapi/points/P0dOJNBHKNtkyDm55BXnk92AAwAAAAU1JWR0RZUExNT1NJRDAzXENEVDE1OA" }, "Score": 13.62607 } ] }

        
        In this case the WebId of the tag must be read and stored in EdgeX in order to be used in further iterations. If the tag does not exist, the reply will look something like:

      • { "Links": { "Next": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3ACDT15&scope=pi%3Asrvgdyplmosid03&count=10&start=10", "First": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3ACDT15&scope=pi%3Asrvgdyplmosid03&count=10", "Last": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3ACDT15&scope=pi%3Asrvgdyplmosid03&count=10&start=0" }, "Errors": [], "Items": [] }

        
        

      • In this case the tag will have to be created with the following command

      • POST https://mypiwebapiendpoint/piwebapi//{webId}/points where WebId is riferred to the target dataserver. The payload of the POST request should look like the following:

      • { "Name": "PointName", "Descriptor": "Some description", "PointClass": "classic", "PointType": "Float32", "DigitalSetName": "", "EngineeringUnits": "", "Step": false, "Future": false }

mypiwebapiendpoint

• Step 3 - Data Export
  • TBD

Possible improvements for a later version:

• Support Kerberos authentication
• Ability to send data to multiple PI Nodes concurrently (up to 3) for High Availability in the Cloud
• High throughput up to 100,000 events/second from each EdgeX node by using an ad-hoc OMF connector to send messages to PI Connector Relay with OMF
  
  

Code

To be updated.

Current status

Waiting for Go Export Client & Distribution to support HTTPS REST