Preparing to Run npm Application Analysis

JavaScript projects rely on build tools and dependency managers that can significantly alter source files before deployment. These transformations improve performance and security, but they also make it harder for analysis tools to identify components accurately. Following the right scanning approach ensures that dependency data remains complete, current, and aligned with your final application.

This topic outlines how to prepare your project for scanning, when to run scans during the build process, and how to include the information needed for precise results.

When to Scan

JavaScript dependencies are often transformed during build steps such as bundling, minification, or obfuscation. These modifications can remove identifying information from files.

To achieve the most accurate results, follow these best practices:

Delete the existing node_modules before scanning to remove any cached projects that will not be included in the final application
Run the scan after installing dependencies (npm install, yarn install, or pnpm install)
Run the scan before bundling, minification, or obfuscation
The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis
You can scan the project directly from source control if other methods are not possible
For the most precise results, you should include a package.json, package-lock.json, and the new node_modules folder that is created when running the install command described in the sections below

If scans are performed after obfuscation, IQ may not be able to identify components using ABF.

Scanning JavaScript Files

The following guidance explains how to set up your environment for reliable results when scanning JavaScript files.

Remove Unused Dependencies

Delete the node_modules directory before running a fresh installation to avoid scanning unused or stale packages.

node_modules Considerations

If node_modules is included in a scan, all files inside of it will be evaluated. This can mean that stale, unused, or dev dependencies could be included, introducing noise into your analysis. This is why you should delete the existing node_modules file; a new one will be created when you run the install command explained below.

For Node projects, use the --production flag to exclude dev dependencies in the package.json:

npm install --production

For Yarn:

yarn install --prod

Lock Files Required

npm requires package-lock.json for accurate results. Scan after the npm or yarn install command has run and the package-lock.json file is created.
Scanning only package.json does not produce results under ABF.
Both package.json and package-lock.json are needed to generate a dependency hierarchy.

Include Source Code When Needed

To match embedded or bundled JavaScript, include the application’s own JavaScript files in the scan context path.
Any declared packages are matched to discovered JavaScript target files.
The scan will match any packages present in the scan context path.

Evaluating Manifests Without JavaScript Files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm.

If any *.js file is detected, IQ automatically switches to ABF.

Lock Files and Dependency Information

Manifest files alone do not include transitive dependencies. Sometimes, they do not include the specific direct versions that will be used in the final application. Therefore, we highly recommend including the lock files in your analysis for the best results.

To include dependency information (i.e., Direct vs. Transitive), the package.json will need to include an auto-generated lock file along with the manifest files.

If only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded. This package.json must contain a dependencies section.

Only the packages specified in the dependencies section will be considered production dependencies. If the dependencies section is empty, then no dependencies will be included.

Evaluate a File Feature Note

To produce a dependency tree using the Evaluate a File option from the Lifecycle user interface, put the lock file in an archive along with its corresponding package.json file (e.g., ZIP them). Then, scan this archive instead.

Handling devDependencies and optionalDependencies

By default, dev and optional dependencies are excluded from manifest-based analysis.

To include them, do one of the following:

Scan node_modules directly, or
Enable the scan-npm-dev-and-opt-dependencies feature using the Feature Configuration REST API.
When this feature is enabled, analysis only looks at devDependencies and optionalDependencies defined in the package.json file for the direct dependencies.
- Transitive dependencies brought in from the package-lock.json will not be included in the analysis.

Evaluating npm Packages by Lock File

npm packages can be identified by scanning the following named lock files:

Filename	Generated by
`package-lock.json` `npm-shrinkwrap.json`	npm package manager v6 npm package manager v7 Preference is given to a `npm-shrinkwrap.json` file when both files are present in a scan

name and version fields from dependencies (or packages for npm v7) will be evaluated.

Example

name: ansi-regex
version: 3.0.0
name: wordwrap
version: 0.0.3

{ "requires": true, "lockfileVersion": 1, "dependencies": { "ansi-regex": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=" }, "wordwrap": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc=" } }}

Evaluating Yarn Packages by Lock File

Yarn packages can be identified by scanning the following named lock files:

Filename	Generated by
`yarn.lock`	Yarn package manager v1 Yarn package manager v2 Yarn package manager v3 Yarn package manager v4

name and version fields will be evaluated.

Example

name:@dangl/angular-material-shared
version: 2.0.0

@dangl/angular-material-shared@2.0.0:
  version "2.0.0"
@progress/kendo-theme-material@0.3.2:
  version "0.3.2"
@angular@0.0.1:
  version "0.0.1"

Evaluating pnpm Packages by Lock File

pnpm packages can be identified by scanning the following named lock files:

Filename	Generated by
`pnpm-lock.yaml`	pnpm package manager

name and version fields from packages and dependencies will be evaluated.

Example

name: @angular-devkit/schematics
version: 8.3.26

lockfileVersion: 5.3

specifiers:
  '@angular-devkit/schematics': 8.3.26

dependencies:
  '@angular-devkit/schematics': 8.3.26

packages:
  /@angular-devkit/schematics/8.3.26:
    resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==}
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'}
    dependencies:
      '@angular-devkit/core': 8.3.26
      rxjs: 6.4.0
      dev: false