Export Service to Osisoft PI

Description

Osisoft PI is the most used historian in the Oil&Gas industry. EdgeX needs to be capable to export data to such a platform. 

Requirements

• Authenticate the edge node through Basic Authentication
• Create new tags in PI Archive
• Export throughput up to 1,000 events/second from each EdgeX node
• Local buffering and retry mechanism with resilience to network downtime
• Local logging of error messages & events through EdgeX Log service

Design

This capability can be implemented by cloning the current EdgeX HTTPS REST export feature and modifying it accordingly to send messages in a format accepted by PI Web API. This is an approach that has been adopted by other IIoT platforms as well.

• Assumptions
  • There is a 1-to-1 relationship between EdgeX measurements and PI Tags. A table to map the datatypes shall be defined.
  • There is a 1-to-1 relationship between EdgeX events and PI Tag. A table to map the datatypes shall be defined.
  • The corresponding PI Tag will have a name that is obtained by concatenating the three following pieces (with "." in between, example: "deployment0001.deviceA.measurement05")
    • Tag Prefix as defined in the next section "Tag Configuration"
    • Device Name (as per how it was configured in EdgeX - this must be unique)
    • Measurement/Event Name (as per how it was configured in EdgeX - this must be unique within the same device)
• Step 1 - Configuration
  • Parameters that define the configuration of the export service and should be set through Export Client API calls:
    • PI Web API Endpoint (e.g. https://mypiwebapiendpoint/piwebapi)
    • PI Archive Name (e.g. mypiarchive)
    • Export interval in milliseconds (e.g. 1000) - as an alternative this could be batch size
    • Username (e.g. piwebapiuser)
    • Password (e.g. piwebapipassword)
    • Tag Prefix (e.g. deployment0001)
    • Datapoint(s) that will be exported to corresponding PI tag(s)
  • After registering to export client, it would be appropriate to verify connectivity/authentication. 
    • Make a GET request to https://mypiwebapiendpoint/piwebapi/dataservers with the basic authentication header as per piwebapiuser/piwebapipassword
    • The result will look like the below. If there is an entry with "Name":"mypiarchive" then the registration can be considered successful.

    • { "Links": {}, "Items": [ { "WebId": "s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA", "Id": "7ab0ec2c-1daf-41e7-b310-f7d13aed28a9", "Name": "10.125.23.143", "Path": "\\\\PIServers[10.125.23.143]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA/points", "EnumerationSets": "mypiwebapiendpoint/piwebapi/dataservers/s0LOyweq8d50GzEPfROu0oqQMTAuMTI3LjkyLjEzNA/enumerationsets" } }, { "WebId": "s0AAAAAAAAAAD_____vIRf4ATkRQ", "Id": "00000000-0000-0000-ffff-ffffbc845fe0", "Name": "mypiarchive", "Path": "\\\\PIServers[mypiarchive]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ/points", "EnumerationSets": "https://mypiwebapiendpoint/piwebapi/dataservers/s0AAAAAAAAAAD_____vIRf4ATkRQ/enumerationsets" } }, { "WebId": "s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE", "Id": "581bbc60-958f-49f2-8873-f10f849f5df8", "Name": "pisrv002", "Path": "\\\\PIServers[pisrv002]", "IsConnected": false, "ServerVersion": "", "Links": { "Self": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE", "Points": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE/points", "EnumerationSets": "https://mypiwebapiendpoint/piwebapi/dataservers/s0YLwbWI-V8kmIc_EPhJ9d-AVVNBLUgwTDY1UzE/enumerationsets" } } ] }

    • Otherwise  one of the following 3 errors may occur:
      • Unable to reach the endpoint (may be due to different reasons such as connectivity, firewall blocking port 443, DNS resolution...). In this case a timeout is necessary to pop out an error after X amounts of seconds/retries
      • 401 Unauthorized (Wrong credentials, either username or password)
      • mypiarchive does not exist in the list of registered PI Archives (or it may be that the server is registered with the hostname while the specified endpoint is an IP address)
        
        

• Step 2 - Initialization
  • The following steps need to be executed after the client registration.
    • Verify if the tags are already existing in the PI Archive. 
      • For each configured tag, the following query must be executed.
      • GET https://mypiwebapiendpoint/piwebapi/search/query?q=name:tagname&scope=pi:mypiarchive

      • If the tag already exists, the reply will look like:

        { "TotalHits": 1, "Links": { "Next": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10&start=10", "First": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10", "Last": "https://mypiwebapiendpoint/piwebapi/search/query?q=name%3Atagname&scope=pi%3Amypiarchive&count=10&start=0" }, "Errors": [], "Items": [ { "Name": "tagname", "Description": "Some Description", "MatchedFields": [ { "Field": "name" } ], "ItemType": "pipoint", "AFCategories": [], "UniqueID": "\\\\{044de274-8d72-4cb6-839b-9e415e793dd8}\\?3", "WebId": "P0dOJNBHKNtkyDm55BXnk92AAwAAAAU1JWR0RZUExNT1NJRDAzXENEVDE1OA", "UoM": "deg. c", "DataType": "float32", "Links": { "Self": "https://mypiwebapiendpoint:443/piwebapi/points/P0dOJNBHKNtkyDm55BXnk92AAwAAAAU1JWR0RZUExNT1NJRDAzXENEVDE1OA" }, "Score": 13.62607 } ] }

        
        In this case the WebId of the tag must be obtained in order to be used in further iterations. If the tag does not exist, the reply will look something like:

      • 
        

      • In this case the tag will have to be created 
        
        
        

• Step 3 - Data Export
  • TBD

Possible improvements for a later version:

• Support Kerberos authentication
• Ability to send data to multiple PI Nodes concurrently (up to 3) for High Availability in the Cloud
• High throughput up to 100,000 events/second from each EdgeX node by using an ad-hoc OMF connector to send messages to PI Connector Relay with OMF
  
  

Code

To be updated.

Current status

Waiting for Go Export Client & Distribution to support HTTPS REST