Skip to end of metadata
Go to start of metadata

This process was approved by TSC vote on 7/22/20.

The process to objectively assess the security risk of 3rd party open source components or dependencies is outlined with consideration of the legacy way of performing the assessment, as well as the new process discussed within the project during the Hanoi development time frame.

The Process

The process should take into consideration relevant data such as the project's age, popularity / maturity, evidence of security practices, recent commit history, diversity of committers, established CVE practices, or other observable evidence.  In terms of licensing compliance, ideal process should also consider the license associated to the component as well.

Additionally, the ideal process should take into account the following scenarios:

  1. Existing Code (Skeletons in the Closet)
  2. Code in Holding (Analysis of code before it is accepted)
  3. Pull Request with new dependency

Developer Process for Each Use Case

Use CaseProcessToolsApplicability
1.) Existing Code
(Skeletons in the Closet)

Automated scan within build automation via Snyk CLI 

Scans of the published Docker images via Snyk  with notifications to SIR Team members / Snyk Administrators

Working groups should review any issues identified via the build automation tools and address within the context of the working group reviews.

Snyk

Community Bridge Advanced Snyk Reports

Clair

Scan automation occurs within the build - PR merge to master


2.) Code in Holding
(Analysis of code before it is accepted)

Review catalog of approved packages in this wiki.  Compare that list to the list being submitted.  Provide the summary of differences to include the list of new packages and why they are needed.

Complete the paper study for each package. 

See paper study process as written for use case 3.

When considering code that is under consideration for moving into the main EdgeX Foundry Org, out of holding


3.) Pull Request with new dependency

Submitter of a Pull Request (PR) will complete the Pull Request template to include any new changes that introduce dependency changes (e.g. imports or go module dependencies)

The standard Pull Request template includes a question that asks  - "Are there any new imports or modules? If so, what are they used for and why?"

Submitter of the PR will add a dependency label to the pull request.

If the dependency is security related, the submitter will add the security-review label to the PR so a member of the Security WG can help review.

Submitter should include scan results which include consideration of compliance (license) as well as security vulnerability (e.g. CVE) data, that can be reviewed by a Security WG member.

Note: Reviewers will see one of the changed files is go.mod for Go projects.

Run this command at the root of your repo

GO111MODULE=on go list -m all 

GO111MODULE=on go mod graph

For a PR with new dependencies, the submitter of the PR will complete a manual paper study to collect the following data points for review:

  • Total increase in new imports: (count)
    Does the new import introduce additional import dependencies, if so, how many?
    • Ensure that every one of the new dependencies is checked for the same criteria.
  • Releases/Tags: (count)
    • We should avoid new imports that have never had a release and/or tag. How many is too few, this is a judgement call and probably involves also considering how long ago the last release was, and how far apart releases have been done.
  • Contributors: (count)
  • License - what is the license, and is it Apache 2.0 compatible?
  • Stars/Forks/Watchers: (count)
    • These are all indications of how wide-spread the package is used.
  • godoc.org metrics: (count)
    • The individual godoc pages hosted by godocs.org include metrics at the base of the page which indicate how many packages import the package
  • Subjective opinion of the reviewers – at the end of the day, we rely on our reviewers to vet new code. Reviewers should give thought to whether the code is improving our project, whether we'd be better off to implement the functionality ourselves, and at the same time considering whether this new import itself comes with too many dependencies (e.g. go-kit).

    When submitting the PR, complete the PR template and set the labels using both - dependency , security-review (security components only)
  • On approval, notify the working group chair to update the catalog of approved packages if required.



On a Pull Request, whenever there's a new dependency introduced as shown through changes to the go.mod



Approved Go Modules (those in Red are being investigated for replacement - avoid them if possible

ModuleLicenseAvoidNotes
bitbucket.org/bertimus9/systemstat v0.0.0-20180207000608-0eeff89b0690MITYesdynamically versioned and EdgeX exploring possible replacements
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214eMITYesdynamically versioned and EdgeX exploring possible replacements
Researched and found to be part of hashicorp serf which is the component used by consul for service discovery and orchestration.
Avoid use beyond Consul inclusion.  See https://github.com/edgexfoundry/edgex-go/issues/2615 for background.
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da
No
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310
No
github.com/bgentry/speakeasy v0.1.0
No
github.com/BurntSushi/toml v0.3.1
No
github.com/cenkalti/backoff v2.2.1+incompatible
No
github.com/cloudflare/gokey v0.1.0
No
github.com/davecgh/go-spew v1.1.1
No
github.com/dgrijalva/jwt-go v3.2.0+incompatible
No
github.com/diegoholiveira/jsonlogic v1.0.1-0.20200220175622-ab7989be08b9
No
github.com/eclipse/paho.mqtt.golang v1.1.1
No
github.com/eclipse/paho.mqtt.golang v1.2.0
No
github.com/edsrzf/mmap-go v1.0.0
No
github.com/faceterteam/onvif4go v0.4.0
No
github.com/fatih/color v1.7.0
No
github.com/fsnotify/fsnotify v1.4.7
No
github.com/fxamacker/cbor/v2 v2.2.0
No
github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
No
github.com/go-kit/kit v0.8.0
No
github.com/golang/mock v1.2.0
No
github.com/golang/protobuf v1.3.2
No
github.com/golang/snappy v0.0.1
No
github.com/go-logfmt/logfmt v0.4.0author'sYesdynamically versioned and EdgeX exploring possible replacements
github.com/gomodule/redigo v2.0.0+incompatible
No
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
No
github.com/google/uuid v1.1.0
No
github.com/google/uuid v1.1.1
No
github.com/go-redis/redis/v7 v7.2.0
No
github.com/gorilla/mux v1.7.1
No
github.com/gorilla/mux v1.7.4
No
github.com/go-stack/stack v1.8.0
No
github.com/hashicorp/consul/api v1.1.0
No
github.com/hashicorp/consul/sdk v0.1.1
No
github.com/hashicorp/errwrap v1.0.0
No
github.com/hashicorp/go-cleanhttp v0.5.1
No
github.com/hashicorp/go-immutable-radix v1.0.0
No
github.com/hashicorp/golang-lru v0.5.0
No
github.com/hashicorp/golang-lru v0.5.1
No
github.com/hashicorp/go-msgpack v0.5.3
No
github.com/hashicorp/go-multierror v1.0.0
No
github.com/hashicorp/go.net v0.0.1
No
github.com/hashicorp/go-rootcerts v1.0.0
No
github.com/hashicorp/go-sockaddr v1.0.0
No
github.com/hashicorp/go-sockaddr v1.0.1
No
github.com/hashicorp/go-syslog v1.0.0
No
github.com/hashicorp/go-uuid v1.0.1
No
github.com/hashicorp/logutils v1.0.0
No
github.com/hashicorp/mdns v1.0.0
No
github.com/hashicorp/memberlist v0.1.3
No
github.com/hashicorp/serf v0.8.2
No
github.com/hpcloud/tail v1.0.0
No
github.com/imdario/mergo v0.3.6
No
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
No
github.com/kr/pretty v0.1.0
No
github.com/kr/pty v1.1.1
No
github.com/kr/text v0.1.0
No
github.com/mattn/go-colorable v0.0.9
No
github.com/mattn/go-isatty v0.0.3
No
github.com/miekg/dns v1.0.14
No
github.com/miekg/dns v1.1.4
No
github.com/mitchellh/cli v1.0.0
No
github.com/mitchellh/consulstructure v0.0.0-20190329231841-56fdc4d2da54MITYesdynamically versioned and EdgeX exploring possible replacements
Researched and found to be used by consul in order to update [Writable] service configurations in real time.
Avoid use beyond Consul inclusion.  See https://github.com/edgexfoundry/edgex-go/issues/2617 for background.
github.com/mitchellh/copystructure v1.0.0
No
github.com/mitchellh/go-homedir v1.0.0
No
github.com/mitchellh/go-homedir v1.1.0
No
github.com/mitchellh/go-testing-interface v1.0.0
No
github.com/mitchellh/go-wordwrap v1.0.0
No
github.com/mitchellh/gox v0.4.0
No
github.com/mitchellh/iochan v1.0.0
No
github.com/mitchellh/mapstructure v1.1.2
No
github.com/mitchellh/reflectwalk v1.0.0
No
github.com/OneOfOne/xxhash v1.2.5
No
github.com/OneOfOne/xxhash v1.2.6
No
github.com/onsi/ginkgo v1.10.1
No
github.com/onsi/gomega v1.7.0
No
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913cauthor'sYesdynamically versioned and EdgeX exploring possible replacements
Used by Consul and must be retained for now.  See https://github.com/edgexfoundry/edgex-go/issues/2618 for details
github.com/pebbe/zmq4 v1.0.0
No
github.com/pelletier/go-toml v1.2.0
No
github.com/pkg/errors v0.8.1
No
github.com/pmezard/go-difflib v1.0.0
No
github.com/posener/complete v1.1.1
No
github.com/remyoudompheng/bigfft v0.0.0-20190321074620-2f0d2b0e0001
No
github.com/robfig/cron v0.0.0-20180505203441-b41be1df6967
No
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028fMITYesdynamically versioned and EdgeX exploring possible replacements
Used by Consul and must be retained for now.  See https://github.com/edgexfoundry/edgex-go/issues/2619 for details
github.com/ryanuber/columnize v2.1.0+incompatible
No
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529MITYesdynamically versioned and EdgeX exploring possible replacements
Used by Consul and must be retained for now.  See https://github.com/edgexfoundry/edgex-go/issues/2620 for details
github.com/soniah/gosnmp v1.21.0
No
github.com/spf13/cast v1.3.0
No
github.com/stretchr/objx v0.1.0
No
github.com/stretchr/objx v0.2.0
No
github.com/stretchr/testify v1.5.1
No
github.com/stretchr/testify v1.6.1
No
github.com/x448/float16 v0.8.4
No
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3
No
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
No
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4
No
golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392
No
golang.org/x/net v0.0.0-20181201002055-351d144fa1fc
No
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd
No
golang.org/x/net v0.0.0-20190228165749-92fc7df08ae7
No
golang.org/x/net v0.0.0-20190921015927-1a5e07d1ff72
No
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553
No
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4
No
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
No
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5
No
golang.org/x/sys v0.0.0-20190411185658-b44545bcd369
No
golang.org/x/sys v0.0.0-20190922100055-0a153f010e69
No
golang.org/x/sys v0.0.0-20191010194322-b09406accb47
No
golang.org/x/text v0.3.0
No
golang.org/x/text v0.3.2
No
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
No
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
No
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
No
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15
No
gopkg.in/eapache/queue.v1 v1.1.0
No
gopkg.in/fsnotify.v1 v1.4.7
No
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce
No
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
No
gopkg.in/yaml.v2 v2.2.8
No
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
No
modernc.org/b v1.0.0
No
modernc.org/db v1.0.0
No
modernc.org/fileutil v1.0.0
No
modernc.org/file v1.0.0
No
modernc.org/golex v1.0.0
No
modernc.org/internal v1.0.0
No
modernc.org/lldb v1.0.0
No
modernc.org/mathutil v1.0.0
No
modernc.org/ql v1.0.0
No
modernc.org/sortutil v1.0.0
No
modernc.org/strutil v1.0.0
No
modernc.org/zappy v1.0.0
No
github.com/spf13/cobra v0.0.5Apache 2YesTo be used with CLI only (edgex-cli); approved 7/29/20 by TSC
github.com/spf13/viper v1.3.2MITYesTo be used with CLI only (edgex-cli); approved 7/29/20 by TSC
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8eMITYesTo be used with CLI only (edgex-cli); approved 7/29/20 by TSC

Process Research

Explore Documentation: Issue-1947



  • No labels

8 Comments

  1. James/Tony - some points for consideration...
    On the current process, I think the list is good, but do we need to provide some guidance on the criteria - if not some #'s - then some suggestions on example modules that would fail and those that would pass based on that criteria set?
    Also, the license has to be Apache or equivalent and we should define what those are (MIT, BSD, etc.)

    Are we saying the Ideal process is our target?  All of it or just parts? 

  2. Jim - thanks for the feedback. 

    Per the our meeting yesterday, I understood Tony would try to simplify the acceptance criteria from the laundry list of questions I had complied in my explore work

    As I noted the license criteria as follows:

    Note License Acceptance Criteria has been defined within the project.

  3. A good example of something that should fail acceptance and would be considered as rejected for use would be something like this one:

    Package: github.com/goburrow/modbus:
    evidence: https://github.com/goburrow/modbus/pulse/monthly


  4. A good example of a dependency that should pass acceptance and would be considered as accepted for use would be something like this one: 

    Package: github.com/hashicorp/vault:
    evidence: https://github.com/hashicorp/vault/pulse

    or 

    Package: github.com/Kong/kong
    evidence: https://github.com/Kong/kong/pulse

  5. In terms of Ideal process.. the TSC can make the call on conditional acceptance of a component that does not measure up to the acceptance criteria. 

    1. Perhaps a simplified process for making the decision to accept or reject an open source dependency (use case 2 / use case 3 only) could be the following:

      • Review the GitHub pulse data - evidence of lack of maintenance on a project (no active commits, no recent releases, low number of committers, stale activity with PRs and Issues management etc.) equates to rejection for use within the project
      • Review Security vulnerability data - evidence of lack of a process to address reported security vulnerabilities and / or evidence of a publicly reported vulnerability without any course of action to address equates to rejection for use within the project
      • Review License - lack of evidence that the license fits per the criteria outlined for acceptance, equates to rejection for use within the project


  6. James, as promised, here's my take on the simplified requirements for vetting a new 3rd party import:

    • Total increase in new imports: does the new import introduce additional import dependencies, if so, how many?
      • Ensure that every one of the new dependencies is checked for the same criteria.
    • Releases/Tags: count
      • We should avoid new imports that have never had a release and/or tag. How many is too few, this is a judgement call and probably involves also considering how long ago the last release was, and how far apart releases have been done.
    • Contributors: count
    • License - what is the license, and is it Apache 2.0 compatible?
    • Stars/Forks/Watchers: counts
      • These are all indications of how wide-spread the package is used.
    • godoc.org metrics: count
      • The individual godoc pages hosted by godocs.org include metrics at the base of the page which indicate how many packages import the package
    • Subjective opinion of the reviewers – at the end of the day, we rely on our reviewers to vet new code. Reviewers should give thought to whether the code is improving our project, whether we'd be better off to implement the functionality ourselves, and at the same time considering whether this new import itself comes with too many dependencies (e.g. go-kit).

    I think we need to be careful about being too prescriptive about things like security vulnerability processes (or lack thereof), as I would wager that very few of our existing package imports include published security vulnerability processes. Likewise, if we want folks to do a CVE search on the imports, we should provide a link and instructions on how to do this. I think a better ask would be to ask reviewers to consider the existing bug/issue list of the project as a part of the overall process.

  7. Only comment I have is about the formatting of the approved Go modules. I liked the way it was initially presented in table format with more detail about each module. Not sure if all those details would be helpful here or not.

    Example:

    Module NameLicenseAVOIDNotesEtc..

    bitbucket.org/bertimus9/systemstat v0.0.0-20180207000608-0eeff89b0690

    ...YESToo old...

    github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e

    ...YES...
    github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da...NO...