Policy Evaluation in Source Control Management

Policy Evaluation in Source Control Management

Lifecycle for Source Control Management (SCM) is a set of features that lets developers gain early insight into code changes by working in tandem with continuous integration to push policy information about an application’s components directly into their SCM.

Lifecycle for SCM has the following features:

Early Feedback and Automation

In today’s software development lifecycle (SDLC), developers are asked to release complex software at an increasingly faster pace. Because of this, devs are relying more and more on OSS, utilizing openly available, popular components rather than writing unique code. Consequently, Sonatype’s State of the Software Supply Chain found that open-source software (OSS) components make up more than 80% of most modern applications. With this, the traditional capacity of a developer simply writing code has shifted to a holistic role that includes assembling and integrating code that supports both your software and business operations.

While assembling code, developers often use source control management systems, like GitHub, GitLab, and Bitbucket. SCMs are often the first place where a piece of code gets shared and reviewed by both humans and machines. Lifecycle for SCM uses Lifecycle to establish a relevant policy configuration. Scan evaluations in SCM are run against that policy configuration so that as developers build applications, they receive accurate and timely information to take action on reported violations.

The main value of SCM systems is their ease of collaboration. Developers are able to push quality control of their application development to a source control platform, where this collaboration exists in code reviews that contain both commits and pull requests.

Building on this, Lifecycle for SCM provides suggested remediations for policy violations directly into the source control repository. We do this by automatically creating pull requests with the changes to an application’s component manifest. Using this type of early feedback and automation, we reduce rework and keep development teams focused on contributing business value rather than managing application component risk.

How Lifecycle for SCM Works

To use Lifecycle for SCM, you first need to configure the IQ Server to allow access to your company’s Source Control Management platform. For large organizations, we recommend enabling automatic source control which lets CI and CLI integrations configure application source control connections when running from a locally cloned repository (a common practice in CI systems).

Once configured, commits will immediately receive automated commit feedback. If enabled and appropriately configured, applications will also start seeing automated pull requests for any new policy violation with suggested remediation, and pull request comments that detail the summary of violations, affected components, and a description of the violations that were introduced in the PR.

A Continuous Integration system or build agent is required for these features to work. For example, if Jenkins is pulling and running the scan, Lifecycle can pick up the source control context and use that as a source. This eases control and gets policy information closer to developers.

We suggest using CI against all feature branches in your SCM, letting you see early feedback on the quality of your code as it’s written. For Lifecycle to provide feedback to feature branches during development, CI must be configured to run a policy evaluation against these branches. We recommend that evaluations against the main branch use the ‘build’ stage in Lifecycle, and that feature branch evaluations are performed against the ‘develop’ stage. The ‘develop’ stage is intended for policy evaluation snapshots where changes to application dependencies and any associated policy violations are not tracked linearly. Because feature branches are volatile and vary across the features developed, using the ‘develop’ stage helps reduce noise by not showing these evaluations on the dashboard or your reports.

When deploying Lifecycle for SCM, you should authorize Lifecycle at the Root Organization level to communicate with your SCM tool (example: GitHub). Then, when you do any evaluation, we will look at the context in modern CI systems—meaning your Git repo gets cloned, then we run an evaluation in that directory, and we can get that information and see what was cloned. This lets us associate GitHub projects to applications in Lifecycle.

Example Use Case: Automatic Pull Requests for npm

Using this example, you will be able to:

Prerequisites

Note: Automated pull requests only work on private repositories. Currently, we support Auto PRs on GitHub, Bitbucket Server, and Bitbucket Cloud.

Configuration

  1. Configure Lifecycle:

    • Configure the Root organization with a GitHub token and then enable pull requests.

    • Create a new application in Lifecycle with an application ID of npm-example.

  2. Configure a Jenkins Continuous Integration pipeline:

    • Install the latest version of the Sonatype Platform Plugin in your Jenkins server.

    • Add credentials to authenticate with Lifecycle.

    • Add credentials to authenticate with GitHub.

    • Configure the Sonatype Platform Plugin to communicate with Lifecycle.

    • Create a new multi-branch pipeline job for the demo GitHub repository created above. The sample pipeline will: a. Build and scan the repository on the main branch and create a pull request. b. Build the Lifecycle for SCM created pull request, execute an npm install to automatically apply the change in the package.json to package-lock.json, then automatically merge changes to the main branch.

    • Note that the checkout to local branch option is used to match the structure of our sample Jenkinsfile.

Workflow

Once you’ve finished up the configuration, it’s time to kick off a build of your npm application. When you do so, the following workflow takes place:

pipeline {
        agent {
                docker {
                        // Docker image with node and git installed image
                        'tarampampam/node:13-alpine'
                }
        }
        environment {
                HOME = '.'
        }
        stages {
                stage('Build') {
                        steps {
                                sh 'npm install'
                        }
                }
                stage('Test') {
                        steps {
                                echo "Run acceptance tests to ensure that IQ remediation changes function as expected in the application"
                        }
                }
                stage('Policy Evaluation') {
                        // evaluation takes place against the merge branch we intend to merge to
                        when {
                                branch 'main'
                        }
                        steps {
                                sh 'npm run build'
                                // build script using webpack and the copy-modules-webpack-plugin for easy scanning
                                nexusPolicyEvaluation
                                        iqStage: 'build',
                                        iqApplication: 'npm-example',
                                        iqScanPatterns: [[scanPattern: 'webpack-modules']],
                                        failBuildOnNetworkError: true
                        }
                }
}

Note

Note: Verify your Jenkinsfile contains the environment {HOME = '.'} shown in the example above. Your docker npm commands may fail without this.

2. A policy violation is found for the react-dom 16.0.0 component. We also see that there is a newer version available we can replace it with.
  1. Lifecycle automatically creates a pull request on GitHub, replacing the affected component with the newer, no-violation version. Details of the vulnerability and a link to the full Lifecycle report are provided directly in the pull request description.
  1. Lifecycle suggests the smallest change possible to remediate the problem and preserves the formatting of your package.json file.
  1. Our Jenkinsfile recognizes it as a Lifecycle generated pull request based on the branch name. If the build successfully verifies the update, the package-lock.json file is updated and pushed back to GitHub. Note that this only occurs if a change was made to the package-lock.json.
stage('Incorporate IQ remediation changes to package-lock.json') {
when {
    // Only work with branches we recognize as created by Sonatype IQ
    branch pattern: ".*-to-.*", comparator: "REGEXP"
}
steps {
    script {
      sh 'git status'
      // Look to see if package-lock.json file was updated as part of the build process
      def dirty = sh(script: 'git diff HEAD package-lock.json', returnStdout: true).trim().size() > 0
      if (dirty) {
        echo 'detected changes to package-lock.json, merging now'
        withCredentials([\
            [$class          : 'UsernamePasswordMultiBinding',\
             credentialsId   : 'github-user',\
             usernameVariable: 'GITHUB_USERNAME', passwordVariable: 'GITHUB_PASSWORD'\
            ]\
        ]) {
          // Configure committer details
          sh 'git config --global user.email sonatype-ci@sonatype.com'
          sh 'git config --global user.name Sonatype CI'

// Only necessary for https urls, can use other mechanisms to authenticate with github as desired
          def updatedUrl = env.GIT_URL.replaceAll('github.com', "${GITHUB_USERNAME}:${GITHUB_PASSWORD}@github.com")
          sh "git config remote.origin.url ${updatedUrl}"

// By default Jenkins will only checkout the specific branch so we configure to pull the branch we
          // desire to merge to
          sh "git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'"
          sh 'git fetch origin main'

// Commit changes to package-lock.json file, merge and push to the remote branch
          sh 'git commit -m "committing IQ remediation changes" package-lock.json'
          sh 'git checkout main'
          sh "git merge ${GIT_BRANCH}"

// This should automatically close the open PR when pushed
          sh 'git push origin main'
        }
      }
      else {
        echo 'no changes detected in package-lock.json, skipping merge'
      }
    }
}
}
  1. Now the pull request, which contains both the package.json update from Lifecycle and the package-lock.json update from CI, can be merged.
  1. Merging the changes into the main branch triggers another Jenkins CI build and an associated Lifecycle analysis. The new scan shows the updated component version.

Lifecycle for SCM always bumps to a specific version, and pins it in your manifest. Only pinned versions give you control on remediation through security and quality policies. For example, version 1.3.7 might pass policy, but version 1.3.8 might not. A version range of ~1.3.0 does not allow this type of control—we don’t want to replace one security violation with another.

The example workflow is shown here using GitHub, Jenkins CI, and an npm project. We encourage you to utilize other systems you have in place to work well with our software.

Measuring Success

Success can be measured by a reduction in mean time to resolution (MTTR) which can be seen in our success metrics. Lifecycle for SCM provides the ability to prevent new violations from entering your main branch by informing development teams of risk as soon as they introduce it. We use automation to fix any violations as they occur by providing pull requests with the version change, and both of these practices should reduce MTTR.

If you’re ready to start using Lifecycle for SCM features, reach out to your customer success representative for guidance.