Understanding Sparse Checkout for SCM Scanning

When Sonatype Lifecycle performs Source Control Management (SCM) scans, it uses sparse checkout to efficiently analyze repositories without checking out all repository files into the working directory.

What is Sparse Checkout?

Sparse checkout is a Git feature that allows Lifecycle to check out only the specific files needed for component analysis, rather than every file in your repository. This approach:

Reduces disk usage by materializing only necessary files in the working directory
Speeds up scanning by avoiding checkout of unnecessary files
Can improve performance for repositories with large codebases

Significant Scan Files

Lifecycle identifies open source components by examining a limited set of significant scan files, rather than all files in a repository.

Manifest and Lock Files : These files are used during component analysis.

Examples include:

pom.xml, build.gradle
package.json, package-lock.json, yarn.lock
requirements.txt, Pipfile.lock
go.mod, go.sum

Binary and Archive Files : These files can also be relevant for identifying open source components.

Examples include:

*.jar, *.war
*.zip, *.tgz
*.dll, *.rpm
*.js

Note

Lifecycle may include files beyond manifests, such as archives and .js files, because they can be used for hash-based identification of open source components.

Why JavaScript Files Are Included

JavaScript files (*.js) are included for the same reason archive files such as *.jar are included: they can be used for hash-based identification of distributed code artifacts.

In the npm ecosystem, artifacts are distributed as .tgz packages that contain .js files. These files are extracted into the node_modules directory of a project. Lifecycle can use hash-based identification against .js files or .tgz packages.

How Sparse Checkout Works

Execution Flow

Partial clone fetches only the file contents needed for component analysis when the available Git version is 2.27 or higher
Shallow clone fetches only the latest commit
Sparse checkout materializes only the files required for scanning in the working directory
Component scan analyzes the checked-out files for component detection

Note

When partial clone is not available, Git may fetch and store compressed contents for all files in the latest commit, while sparse checkout materializes only the files needed for component analysis.

Requirements

Sparse checkout requires:

Native Git client
Git version 2.16.0 or higher for sparse checkout
Git version 2.27 or higher for partial clone
Configured in IQ Server settings (see IQ Server Configuration)

Note

JGit (Java implementation of Git) does not support sparse checkout and will perform a full clone if used instead of the native Git client.

Best Practices

For Self-Hosted Installations

Sonatype strongly recommends configuring IQ Server with a native Git client to take advantage of sparse checkout. See Source Control Integration Best Practices for configuration details.

For Lifecycle Cloud (SaaS)

Lifecycle Cloud uses partial clone when available and sparse checkout for SCM scanning. Instead of materializing all repository files in the working directory, it materializes only the files required for component analysis.

Troubleshooting

Empty Scan Results

If sparse checkout finds no files to check out, you may see this error:

Sparse checkout leaves no entry on working directory

Resolution

Verify the repository contains supported manifest or other significant scan files
Check the SCM configuration's scan target path
Ensure the branch being scanned contains the expected files

Unexpected Components Missing

If expected components are not being detected:

Verify relevant files are committed since uncommitted changes are not scanned
Check file extensions to ensure supported files follow standard naming conventions
Review scan target configuration to confirm the correct directory is being scanned