You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Current »

Description

Due to the massive amount of time series managed at the edge and the disconnected nature of operations, a native time series database is necessary as part of EdgeX. MongoDB is not an optimal fit for huge amount of time series data.

The TSDB is useful for local storage and shall be easily interfaced with open source visualization tools (e.g. Grafana) and open source analytics platforms (e.g. Kapacitor). 

InfluxDB is an open source TSDB distributed under MIT License which can be integrated as part of the solution.


Requirements

  • Ability to send data to a local or remote InfluxDB instance via InfluxDB http(s) REST API or InfluxDB Go Client
  • Authentication with username/password
  • Create new measurements in InfluxDB
  • Export throughput up to 1,000 events/second from each EdgeX node
  • Local buffering and retry mechanism with resilience to network downtime
  • Local logging of error messages & events through EdgeX Log service

Design

Although the best option would be to modify Core Data Service in order to rely on InfluxDB instead of MongoDB, a first step to achieve integration with influx is to modify the existing Export Service in order to enable a new delivery via InfluxDB http(s) REST API or InfluxDB Go Client.

Assumptions

    • Whenever a new subscription is created through the EdgeX Export Client, the target InfluxDB Database name and Measurement name will be specified
    • There is a 1-to-1 relationship between EdgeX measurements and a tag-field pair. A table to map the datatypes shall be defined.
    • The corresponding Tag will have a name that is obtained by concatenating the three following pieces (with "." in between)
      • Tag Prefix as defined in the next section "Step 1 - Configuration"
      • Device Name (as what is specified in EdgeX Metadata - this must be unique)
      • Measurement/Event Name (as per what is specified in EdgeX Metadata - this must be unique within the same device)
    • An example of Tag Name will be "deployment001.deviceABC.measurement005"


Code

To be updated.

Current status

Waiting for Go Export Client & Distribution to be finalized by the appointed group before making the contribution

  • No labels