Page History

...

The Delhi release is slated to provide the first system management capabilities while also implementing additional features in the security.  Delhi will continue to show improvements in performance and include tests to monitor and report on potential performance issues.  The release will also establish the certification program within the EdgeX project for both commercial instances of the full platform and drop-in "EdgeX-compliant" microservicesby offering Device Service SDKs in Go and C.  Improved testing at all levels (unit, blackbox, performance, etc.) to ensure the quality of the system going forward while also providing a means to check backward compatibility.  Finally, this release will also start to look at EdgeX in a wider, distributed context - that is exploring EdgeX instances (or services) spread across a fog deployment.Key release themes and planned features to be implemented will be detailed after the mid-year TSC Face-to-face meeing (~June 2018)include design and implementation plans (scheduled for Edinburgh release and beyond) for the replacement of MongoDB as the reference implementation database, and a replacement to the export services to allow for better scale and flexibility at the northbound layer.  the scope of this release is smaller given the development cycle for the prior California release was made longer to accommodate all the Go Lang refactoring.

Release Themes and Objectives

• Deliver system management APIs in all the microservices and provide a system management agent for orchestrating multi-service system management commands and communications to 3rd party system management tools/platforms.
• Go /C replacement services for any remaining Java services.
  
  • Provide new and improved device services in Go and C.
  • Notification, Scheduler, Rules Engine replacement services.
• Improved performance to continually lower footprint, data collection and actuation latency, lower startup time, etc. which began with the California Release and offer performance testing to validate performance targets are being met. 
• Implementation of additional priority features for security and manageability
• Potentially, implementation of specific services and SDKs in C to support deterministic real time use cases (see section below for more details)
• Certification process for EdgeX-compliant microservices to enable a plug-and-play ecosystem 
  
  • Various levels of certification are being considered, from micro service replacement certification (validating alternate or commercial implementations of EdgeX micro services satisfy API requirements along with performance metrics and quality checks) to full EdgeX deployments (for commercial versions of EdgeX)
  • Additional certification processes may be developed around particular cross cutting features such as security.
• Arm 32 support and native testing - with continuous integration processes extended to produce artifacts and support the native testing
• Facilitate East/West capability
  • Micro service load balancing, failover, scale-over, ...
  • Device from EdgeX A triggers action on device on EdgeX B
• Address data privacy concerns
  • EU laws and affirmation about data use/storage/etc.
  • HIP-A
• Not MVP, but additional contributions sought for
  • Provide additional reference connectivity
    • Export Services (e.g. AWS/Greengrass)
    • Device Services (e.g. OPC-UA)
  • Demonstration of EdgeX in real-world POC/test bed, including through possible collaboration with the IIC

General Tasks and Notes

• Certification Process - want to have outlined by Delhi

• Probably will include different levels of certification (micro service, versus “box”, etc.)

• ARM 64 support is being provided with the California release.  In this release, ARM 32 support is handled.
  • CI process that includes cross compiling for Arm 32 platform artifacts
  • Micro service Docker container images produced for Arm 32 platforms
  • Native platform testing of the micro services on Arm 32 platforms

• Provide roadmap around technical debt/refactor (what and when)

  • Example:  Device service rewrites (ex: BLE, Bacnet); allow multiple instances of any microservice (for future load balancing and scaling efforts - today only single instances are allowed)
    

• Improving EdgeX resiliency in face of issues or non-availability of some resources/services/etc.

• Potentially, provide EdgeX User Interface(s)

Micro Service Tasks and Notes

• and C device service SDKs with at least one device service implementation for example sake.
• The next wave of security features to include access control lists to grant access to appropriate services, and improved security service bootstrapping.
• Refactored and improved Go Lang microservices
• Options and implementation plan for database replacement
• Design and implementation plans for export service replacement with application services
• Provide an EdgeX UI suitable for use in exploring several instances of EdgeX

General Tasks and Notes

• Implement a performance test harness

  To establish a baseline of performance characteristics that can be used to understand EdgeX resource utilization, data throughput, etc.

• Support binary data

  Allow the EdgeX system (from device services through export services) to support binary data such as images, chunks of video or audio.  This will require device services, core data, and the export distribution to appropriately handle this type of data.  Local edge analytics may be fed binary data and create intelligence from it to allow for actuation at the edge based on the binary data (example: examine an image an detect the presence of a person of interest).

• Explore the use of Swagger to document the EdgeX REST API
  EdgeX currently uses RAML to disseminate information on microservice APIs.  Swagger offers more fidelity while also offering better code generation going forward.
• Provide an initial EdgeX user interface
  This Javascript-based user interface will allow users to explore the sensor data collected by EdgeX as well as provide some provisioning and management capability.  Used for prototyping, demonstration and management of small clusters of EdgeX instances.
• Update all services to return proper HTTP status codes on HTTP requrests

Core & Supporting Services Tasks and Notes

• Refactoring

  Refactor the services to implement data drive design, improves structure and organization of the code, improves ability to abstract infrastructure needs (database, messaging, etc.) to allow for replacements later, and allow services to bootstrap without artificial sleep mechanisms

• Implemented more structured / formatted logging
  Allows for better query and inspection of logs going forward.Meet all performance targets
  While more requirements are needed, the overall performance targets for EdgeX are listed at the bottom of this page and include running all the core, support and application micro services in < 1GB or RAM on a 64 bit CPU, requiring less than 32GB of disk storage space, start (collectively) in under 1 minute and actuate from sensor collection to device trigger in < 1 second.

  In order to accomplish these targets, it is believed the following work must be accomplished for the Delhi release

• Provide 100% compatible and complete Go Lang micro services for the existing Java core, support and application micro services to include core-data, core-metadata, core-command, core-consul (with seed), support-logging,  support-notifications, support-scheduler, support-rulesengine, export-client, export-distro.
Provide device service SDKs and example device services in languages like Go and C/C++

• Implement security & system management API hooks

  Depending on the implementation needs of security and system management (see below), there may be a need to make changes to the existing micro service code in support of these facets.  It is likely that each micro service will implement some sort of base service to provide data and system management functionality to the system management agent.  In support of access to services based on access control list security, additional micro service change could be required.

• Refresh (to latest releases) and improve north side connectors (like Azure IoT Hub connector)

• Support additional backend integration(s) such as Watson, AWS/Greengrass, …

• Support additional export feature(s)

  • Support additional formats (Ex: Haystack, OPC-UA, …)

  • Support additional endpoint types (Ex:  DDS, AMQP, …)

  • Provide enrichment services

• How best to facilitate client command requests/actuation

• Expose command information north bound (export of actuation information and APIs)

• More dynamic configuration

• Potentially, alternate message infrastructure between some or all the servicesAccess control to the service APIs will also be managed by the security services and code added to the service to require authentication before accessing the services will be needed.

Device Services and SDK Tasks and Notes

• Implement security & system management API hooks

  Depending on the implementation needs of security and system management, there may be a need to make changes to the SDKs and existing micro service code in support of these facets.  It is likely that each micro service will implement some sort of base service to provide data and system management functionality to the system management agent.  As device services are typically created from an SDK, the same boilerplate code for the base service needs to be added to the SDK(s).  Additional changes may be required in support of security needs (like granting access by access control list).

  device service SDKs in Go Lang and C.
  The device service SDKs and resulting device services will allow the EdgeX community to retire the last of the Java micro services and complete the dramatic performance improvement of EdgeX.  

• A stretch goal includes providing the existing device services in Modbus, BACNet, BLE, etc. in either Go or C.  

  Provide new and/or updated Device Services

  Many of the original device services where created with driver stacks that are suit for purpose on all platforms, are no longer supported, are not homogeneous in their make up (i.e. having some parts Java while other elements in Python), or are using stacks that are not consider the best option for the protocol today.  The BLE and BACnet device services fall under this categorization.

  Several of the device services were created to prove, conceptually, how to connect to a device using that protocol, but it may not be a full implementation of the protocol API.  For example, the SNMP device service implements enough to drive a Patlite and a few other devices, but does not understand all of SNMP.

  Many of these devices services need to be updated, made more consistent in makeup and more complete in terms of protocol implementation.

• Additional DS (i.e. Zigbee, OPC-UA, CANBus, …)
  EdgeX will never provide all the device services necessary to implement every use case and to connect to every type of device or sensor.  However, EdgeX would like to offer reference implementation device services for the most common of IIoT protocols.  EdgeX offers many of those today to include BACNet, Modbus, BLE, and MQTT.  But we need reference implementation device services in OPC-UA, Zigbee, Zwave, CANBus and a others as well.

• Implement security & system management API hooks

  Depending on the implementation needs of security and system management, there may be a need to make changes to the SDKs and existing micro service code in support of these facets.  It is likely that each micro service will implement some sort of base service to provide data and system management functionality to the system management agent.  As device services are typically created from an SDK, the same boilerplate code for the base service needs to be added to the SDK(s).  Access control to the service APIs will also be managed by the security services and code added to the service to require authentication before accessing the services will be needed

  Not MVP, but additional contributions sought for
  
• Provide SDK Tool plugins to facility developers
• The Java Device Service SDK today is operated by command line execution.  That is, in order to create a new device service, the user must edit configuration settings in a file, then go to the command line and execute an SDK operation, and then finally manually import the new generated project into the tool of choice.  In the future, for the Java SDK as well as other language versions of the SDK, it is preferred to have some sort of plugin (example: Eclipse Plugin) or UI driven tool (“wizard”) to assist developers by providing a more user-friendly facility to capture their configuration options, generate the new device service and load it into to development tool of choice.
The Device Service SDK today is available through GitHub.  Developers not working on the actual open source products may not be familiar with GitHub and/or may prefer not to have to use Git tools to pull down and work with the SDK.  As an alternative, we could make the SDK (and any necessary elements like libraries) available through a Web site for download

.

Security Tasks and Notes

• 
  
• California release was focused on initially securing the EdgeX “gateway” and remains the priority
  • This work will be extended / augmented in this release (looking at alternative authentication, adding MQTT and other protocol reverse proxy, etc.)
    
• Code signing – how to certify integrity of the system
• The release will start to explore, potentially, securing EdgeX’s devices

• Explore potential use of hyperledger

...

• System management API (action, alerts, metric) as discussed and outlined here

• System management Agent (see same document above outlining the agent functionality and duties)

• Potentially explore "Gateway management" to include:
  
  • Demonstrate basic management of EdgeX via select 3rd party console (ex: VMware Pulse IoT Center, System Center, …)
  • Demonstrate EdgeX software updates
  • Updates to non-EdgeX components (drivers, end-devices)
• Explore alternate deployment/orchestration options (e.g. Kubernetes)

Testing/QA

• Performance tests on startup time, request/response times on all APIs, latency to actuation from device service collection, through core data, to rules engine, command back to a device service.
  • Performance metric testing will include CPU and memory usage statistics
  Security testing framework
• Improved unit and integration testing across all EdgeX microservices
  • Returning unit test coverage to ~80%
  • Requires some Go code refactoring and decoupling among resources (like the database, logging, etc.)
  • Use of mock objects to isolate functions under test.

...