Hugging Face Model Analysis

Hugging Face Model Analysis

Hugging Face Ecosystem

IQ Server Hugging Face application analysis supports all AI/ML models hosted on the Hugging Face (HF) platform. These include a variety of popular AI/ML models in use by the developer community, e.g. Large Language Models (LLMs), image classification, object detection, speech recognition etc.

What is Supported

You can scan applications that include any model from the Hugging Face platform using the Sonatype IQ CLI, Jenkins plugin or the IQ Server UI “Evaluate a File” action, and then reports can then be viewed in the IQ UI.

The table below lists the formats and extensions for the models supported by Sonatype CLI:

Format Extension
Pytorch .bin, .pt, .pth, .pkl, .pickle
Safetensors .safetensors
TensorFlow .h5
TensorFlow.js .bin
OpenVino .bin
MLC-LLM .bin
GGUF .gguf
Rust .ot
Transformers Pytorch .bin.index.json
Transformers Safetensors .safetensors.index.json
Transformers TensorFlow .h5.index.json
ONNX .onnx
GGML .bin
Transformers Flax .msgpack.index.json
Flax/Jax .msgpack
FastText .bin

Ways to trigger a Hugging Face scan

You can initiate analysis of Hugging Face model files using following ways:

  1. Sonatype IQ CLI - Scan a directory containing Hugging Face model files.

  2. Sonatype IQ Jenkins Plugin - Run an IQ Policy Evaluation build step on a workspace that includes the model files.

  3. IQ Server UI - Evaluate a File by uploading an individual model file (for example, consolidated.00.pth) using the Evaluate a File action in an application.

Types of Repositories Supported

Models on the HF platform could be organized in different folder structures inside a single repositories.

Sonatype CLI can scan:

  1. Repositories with a single model as one single file.

  2. Repositories with a single model which is sharded (split across multiple files).

  3. Repositories with multiple models (different formats) in multiple non-sharded files.

  4. Repositories with multiple models (different formats) in different folders (directories).

Using Git LFS

Models on the HF platform are hosted as Git repositories.

Enabling Git Large File Storage (git-lfs) may be necessary for cloning the models. This will ensure that the git clone commands actually download the model files on the disk.

Starting release 189, you can scan the model files without enabling git-lfs and downloading the complete model file.

Steps To Analyze

  1. Clone/download the repository from the HF platform. The name of the model is also the name of the repository.

Example:

Should I enable Git LFS?

Before release 189, enabling Git LFS was required. So, if you are on an older IQ version and not using the CLI, you should enable Git LFS.

Starting with release 189, enabling Git LFS is more optional as you can now scan both of the following:

  1. Invoke Sonatype IQ CLI to scan the model files. The model files are evaluated against policy conditions and the link for a detailed evaluation report is generated.

  2. Go to Sonatype Lifecycle > Reports to view the application report.