Delivery:  ~June 2018

The main theme for the California release of EdgeX is to provide a solid open source foundation for commercialization and deployment in a wide variety of Industrial IoT edge use cases. 

Key to this is the first implementation of priority APIs and reference microservices for security and manageability.  Another key theme for the California release is improving overall performance and lowering the baseline footprint of the code base.  Work is currently underway for drop-in alternatives for key microservices (e.g. Core Services, Export and Device Services) based in Go Lang with a stretch goal being select implementations in C.  Overall, the California release will improve EdgeX ease of use among a larger, more polyglot, development community.  

Planned features to be delivered (as of January 2018 TSC Face-to-Face meeting).

Release Themes and Objectives

• Deliver top priority security and system management APIs and reference implementations of supporting microservices (e.g. key management)
• Deliver on promise for a performant, reliable IoT edge platform
  • Reduce overall footprint by an order of magnitude through alternative microservice implementations in Go Lang and possibly C
  • Enable near real-time performance (see targets below)
    
• Improved and overhauled documentation set - moving developer documentation to GitHub
• Device Service SDKs in Go Lang and C/C++
• Blackbox tests for the entire EdgeX API set
• Arm native testing - with continuous integration processes extended to produce artifacts and support the native testing
• Performance tests on startup time, request/response times on all APIs, latency to actuation from device service collection, through core data, to rules engine, command back to a device service.
  • Performance metric testing will include CPU and memory usage statistics
• Not MVP, but additional contributions sought for
  • Provide additional reference connectivity
    • Export Services (e.g. AWS/Greengrass)
    • Device Services (e.g. OPC-UA)
  • Demonstration of EdgeX in real-world POC/test bed, including through possible collaboration with the IIC

General Release Tasks and Notes

• Expand hardware support - ARM
  EdgeX has been built to be cross platform.  Some micro services (such as those written in Go) require compiling to the native platforms (cross compiling).  Once the program artifacts for the platform have been created, they also need to be containerized (in Docker) and some of the containerization is platform dependent.  Finally, it is also desired that the micro services be tested on their native platform.  While virtual machines and simulation environments can allow functional tests to occur, some issues and performance metrics for that platform can only be identified when running on the target platform.  This task therefore includes:
  • CI process that includes cross compiling for Intel/Arm platform artifacts
  • Micro service Docker container images produced for Intel and Arm platforms
  • Native platform testing of the micro services on Intel and Arm platforms
    
    
• Not MVP, but additional contributions sought for
  • Alternate (from Docker/Docker Compose) deployment and orchestration options
    Today, while the community and users of EdgeX are free to deploy EdgeX as they see fit based on their use case/needs, the EdgeX community provides Docker container images and a docker-compose.yml file to help get and deploy EdgeX.  Going forward, it is anticipated that the community will seek and even need alternate means of deploying, managing, and orchestrating the EdgeX micro services.  Options include using Kubernetes, Swarm, Mesos, Nomad, to name a few.  Additional support may be offered by system management tools or facilities.  At this time, the community welcomes input and contributions in this area from the community and user groups, but has no firm plan to do work in this area by the California release.

Micro Service Tasks and Notes

• Meet all performance targets
  While more requirements are needed, the overall performance targets for EdgeX are listed at the bottom of this page and include running all the core, support and application micro services in < 1GB or RAM on a 64 bit CPU, requiring less than 32GB of disk storage space, start (collectively) in under 1 minute and actuate from sensor collection to device trigger in < 1 second.

  In order to accomplish these targets, it is believed the following work must be accomplished for the California release

  • Provide 100% compatible and complete Go Lang micro services for the existing Java core, support and application micro services to include core-data, core-metadata, core-command, core-consul (with seed), support-logging,  support-notifications, support-scheduler, support-rulesengine, export-client, export-distro.
  • Provide device service SDKs and example device services in languages like Go and C/C++
• Implement security & system management API hooks
  Depending on the implementation needs of security (see below) and system management (see below), there may be a need to make changes to the existing micro service code in support of these facets.  It is likely that each micro service will implement some sort of base service to provide data and system management functionality to the system management agent.  It is hoped that security work will not require micro service changes but this could change depending on design and implementation needs.
• Not MVP, but additional contributions sought for
  • Improved and more dynamic configuration
    Consul today provides for centralized configuration of the micro services.  The configuration seed service is used to initialize Consul with configuration from each of the micro services.  This process is often error prone given the multiple configuration files in each micro service (and often multiple for the various environments such as dockerized or non-dockerized).  Can the initialization of Consul be improved, simplified, and made less error prone?
    Changes to configuration in Consul are immediately made available to applications.  However, applications must implement a “watcher” to see a configuration change and then call to get that change (the core-config-watcher in GitHub was meant to demonstrate how to do this but was never implemented or used in any service).  Further, even if micro services are made to be more dynamic in watching for and using new/updated configuration, some of the configuration changes would only work after a restart (ex: the REST endpoint port number of the micro service).  Is there a way to signify which configuration is allowed to be changed at runtime and which can only take affect after a restart of the service.
  • Better Export Services flexibility/SDK-like elements
    In the export distribution service (Java or Go), facilities to provide endpoints with EdgeX data are statically programmed into the pipeline of this service today.  That is, when data comes to export distribution, based on a client’s registration, it is filtered, transformed/formatted, compressed, and then encrypted.  Some options are provided (like allowing formatting in JSON versus XML), but these options are again statically build into the export distribution pipe.  If a user wishes to add new filters, new transformations (example adding a CSV format), new encryption or compression routines, then the user must get the export-distro code base, fork it and make their own copy.  Is there a way to create plugins or modules into the pipe-filter architecture of the export-distro so that it is more flexible to meet new needs going forward?  Is it possible to make some sort of SDK for the export-distro service so users can essentially create a custom export facility with the options for filtering, transforming, enriching, etc. the EdgeX data as they see fit without having to fork and rebuild the entire micro service on their own?

Device Service SDK and Device Service Tasks and Notes

• Provide alternative SDK language support in Go Lang and possibly C
  In addition to lowering the footprint and improving performance of the device service micro services, drawing in more sensors/devices to the EdgeX platform can be achieved by providing more alternative language device service SDKs – primarily those in C/++ and Go.
• Implement security & system management API hooks
  Depending on the implementation needs of security (see below) and system management (see below), there may be a need to make changes to the SDKs and existing micro service code in support of these facets.  It is likely that each micro service will implement some sort of base service to provide data and system management functionality to the system management agent.  As device services are typically created from an SDK, the same boilerplate code for the base service needs to be added to the SDK(s).  It is hoped that security work will not require micro service changes but this could change depending on design and implementation needs.

• Not MVP, but additional contributions sought for
  
  • Provide SDK Tool plugins to facility developers
    

    The Java Device Service SDK today is operated by command line execution.  That is, in order to create a new device service, the user must edit configuration settings in a file, then go to the command line and execute an SDK operation, and then finally manually import the new generated project into the tool of choice.  In the future, for the Java SDK as well as other language versions of the SDK, it is preferred to have some sort of plugin (example: Eclipse Plugin) or UI driven tool (“wizard”) to assist developers by providing a more user-friendly facility to capture their configuration options, generate the new device service and load it into to development tool of choice.

    The Device Service SDK today is available through GitHub.  Developers not working on the actual open source products may not be familiar with GitHub and/or may prefer not to have to use Git tools to pull down and work with the SDK.  As an alternative, we could make the SDK (and any necessary elements like libraries) available through a Web site for download.

  • Provide new and/or updated Device Services

    Many of the original device services where created with driver stacks that are suit for purpose on all platforms, are no longer supported, are not homogeneous in their make up (i.e. having some parts Java while other elements in Python), or are using stacks that are not consider the best option for the protocol today.  The BLE and BACnet device services fall under this categorization.

    Several of the device services were created to prove, conceptually, how to connect to a device using that protocol, but it may not be a full implementation of the protocol API.  For example, the SNMP device service implements enough to drive a Patlite and a few other devices, but does not understand all of SNMP.

    Many of these devices services need to be updated, made more consistent in makeup and more complete in terms of protocol implementation.

  • Additional DS (i.e. Zigbee, OPC-UA, CANBus, …)

    EdgeX will never provide all the device services necessary to implement every use case and to connect to every type of device or sensor.  However, EdgeX would like to offer reference implementation device services for the most common of IIoT protocols.  EdgeX offers many of those today to include BACNet, Modbus, BLE, and MQTT.  But we need reference implementation device services in OPC-UA, Zigbee, Zwave, CANBus and a others as well.

  • Reduce/optimize Java DS (remove Spring Framework, etc.)

    The Java Device Service SDK is a bit cumbersome/complex to use, tightly coupled in some cases, and hard to maintain or update.  A refactor of the SDK would improve its ease of use, decouple common elements (such as scheduling) from the SDK so that this code does not have to be replicated with each corresponding device service, and improve its overall quality.  The SDK also relies heavily on the Spring Framework which makes it particularly heavy for edge systems.

Security Tasks and Notes

• A OS reverse proxy will be selected and integrated to EdgeX

• Integrate with AAA service (see below)

• With no (or very few) changes to existing micro services at this time

• Currently looking at Traefik for dev/test and possibly NginX for prod

  • This will be for HTTP/S only (meaning security for MQTT and other protocols will not yet be provided)

• An open source Authentication and Authorization server (s). (E.G., Keycloak, Dex, Hydra, …) will be selected and integrated to EdgeX

• Prod deployment models will require integration with centralized AAA.  Initial solution will be to integrate with reverse proxy

• Data Protection Services via HashiCorps’s Vault will be integrated with EdgeX

• This will provide Key Management, Certificate Services, Encryption

• API abstraction to allow 3rd party implementation/extensions

• Some additional research/work needs to be finalized for California around license, external CA support, local crypto library, bootstrap provisioning, local secure store for bootstrap credentials, unattended startup

• Security testing framework

System Management Tasks and Notes

• System management API (action, alerts, metric) as discussed and outlined here

• System management Agent

Target Performance

• The target is to run all of EdgeX (all the services to include the virtual device service) on a Raspberry Pi 3 (1 GB RAM, 64bit CPU, at least 32GB storage space, running Raspbian Linux distro)

  • This is not an endorsement of the platform as our favorite or otherwise endorsed platform

  • It only suggests the general characteristics of a “developer community” target platform

  • This may not be entirely feasible without all Go replacements, but is the target and the development community will report back when/where this is not possible

  • For example, it is unlikely the target security implementation will fit on this size platform

• Additional “developer community” targets

• Startup in 1 minute or less (post OS boot – from micro service start to all services up and available)

• Throughput for one piece of data (with one IP device connected by hard wire – versus WiFi) from data ingestion at the Device Service layer, through Core Data, to Rules Engine and back down through Command Service finally to Device Service for actuation will be < 1 second