Page History

...

1. To smooth out any outliers or “warm-up” affects each individual performance test result will typically be calculated by taking the average over a number of repeated test iterations (e.g. 1000, but may vary depending on sample rate for each test – more for low sample rates and less for high sample rates).
2. Will require a configurable Virtual Device Service to drive many of the tests listed above. The current Virtual Device Service is written in Java but to be able to support performance testing on other platforms (e.g. 32 but ARM) then it is likely that Virtual Device Service implementations in either Go or C (or both) will be required.
3. It is expected that a test harness/framework that can satisfy the requirements and support the scope of test outlined above will have to be created specifically for this task.   However, there are many 3^rd party performance testing technologies that may form some part of the overall framework. These includes basic command line utilities such as Top (Unix) or Docker (stats) for providing CPU and Memory consumption data. There are also more sophisticated commercial container monitoring tools such as DataDog with integrated dashboarding and reporting. To support API level load testing there are both open source tools such as bender Bender a Go-based framework for load testing services using protocols such as HTTTP and Thrift or commercial tools such as Load Impact that can be used with existing Postman test collections (Postman is currently used for EdgeX blackbox testing) to create realistic load scenarios with multiple clients issuing REST request simultaneously. Where appropriate use of these types of tool should be considered to complement any tests/scripts that have to be written specifically to support the implementation of the test cases listed in this document.
4. May require an emulated Cloud server (a dummy server that runs on the local test node and that be used to help log performance data) as an Export Service target that can be used to support the Latency and Throughput tests listed previously.