---
title: Monthly Quality Report for SBOM Tools (October 2023)
description: This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools. Using https://sbom.sh
image: https://codenotary.com/hubfs/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape-1.jpg
---

**$ protect --distro linux --machines 25 --free**

[Start now](https://apps.codenotary.com/linux)

[![cn-logo-black-nobg](https://codenotary.com/hubfs/cn-logo-black-nobg.svg)](https://codenotary.com/)

- Product
  
  #### [![AgentMon Start](https://codenotary.com/hubfs/AgentMon%20Start.svg) **AgentMon Start** Organization-wide AI agent spend, security and device fleet TRY NOW →](https://apps.codenotary.com/agentmon-start)
  
  #### [![AgentMon for Enterprise](https://codenotary.com/hubfs/AgentMon%20for%20Enterprise.svg) **AgentMon** Currently monitors more \> 7 million agent interactions/day. TRY NOW →](https://codenotary.com/agentmon)
  
  #### [![AgentX](https://codenotary.com/hubfs/AgentX.svg) **AgentX** Agentic network control middleware. TRY NOW →](https://codenotary.com/agent-network-control)
  
  #### [![Autonomous Security](https://codenotary.com/hubfs/Autonomous%20Security.svg) **Autonomous Security** AI Agents keep your servers secure. TRY NOW →](https://codenotary.com/trust)
- Use Cases
  
  #### [**AI Agent Risk Monitoring** Continuous oversight of autonomous agents across every environment.](https://codenotary.com/use-cases#risk)
  
  #### [**Autonomous Security Operations** Self-healing defenses that detect, contain, and remediate threats.](https://codenotary.com/use-cases#agentops)
  
  #### [**AI Coding Governance & Performance Monitoring** AI-generated code reviewed, tracked, and held to quality standards.](https://codenotary.com/use-cases#performance)
  
  #### [**AI Tool Cost & Usage Optimization** Spend and consumption optimized across every AI service in use.](https://codenotary.com/use-cases#cost)
  
  #### [**AI Tool Security & Policy Enforcement** Approved AI usage enforced with guardrails and policy controls.](https://codenotary.com/use-cases#security#security)
  
  #### [**Shadow AI Governance** Unsanctioned AI tools discovered, surfaced, and brought under control.](https://codenotary.com/use-cases#shadowit)
- [Blog](https://codenotary.com/blog)
- [Press](https://codenotary.com/press)
- Resources
  
  #### [**Integrations** Connect with your favorite tools and platforms. LEARN MORE →](https://codenotary.com/integrations)
  
  #### [**Support** Get help from our dedicated support team. GET HELP →](https://support.codenotary.com)
  
  #### [**Success Stories** Read how customers achieve their goals. READ MORE →](https://codenotary.com/success)
  
  #### [**Learn** Access documentation and learning resources. EXPLORE →](https://codenotary.com/learn)

[Login](https://apps.codenotary.com/auth/login)

[All posts](https://codenotary.com/blog/all)

 Oct 23, 2023

# Monthly Quality Report for SBOM Tools (October 2023)

 By  [Mahrukh](https://codenotary.com/blog/author/mahrukh)  ·   2 minute read

## Introduction

In our continuous effort to enhance the quality of Software Bill of Materials (SBOM) content, we are excited to present our monthly SBOM quality report. This report aims to provide insights into the performance of various SBOM tools, shedding light on their capabilities, strengths, and weaknesses. For the October edition, we have conducted extensive testing using a diverse set of tools to analyze the SBOM quality, license detection, vulnerability scanning, and pipeline support.

This time we use the new docker container image that contains already many of the SBOM and vulnerability scanner tools: [codenotary/sbom.sh:v1.3 ](https://hub.docker.com/r/codenotary/sbom.sh)

The nice thing about that container image is, that it automatically ships the output to [https://sbom.sh](https://sbom.sh) where the result is visualized and can be shared using a unique URL.

**Tools**

- - Aqua Trivy 0.46.0
    - Anchore Syft 0.93.0
    - Anchore Grype 0.71.0

In this blog post, we check the container image of Jenkinfis: jenkins/jenkins

## Goal

The primary objective of this report is to create a baseline and gain comprehensive understanding of how various SBOM tools function. Additionally, we aim to assess the accuracy and comparability of their results. For this purpose, we have chosen a popular Maven Java Open Source application, Apache Pulsar, to test the SBOM tools against. Our evaluation goes beyond SBOM generation to include advanced functionality such as license information, vulnerability scanning, and pipeline support.

## Overall Result

![Brown Peach Illustrative GreetingsSlogans Banner Landscape-1](https://codenotary.com/hs-fs/hubfs/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape-1.jpg?width=6912&height=3456&name=Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape-1.jpg)

**Findings:**

1. There is a dependency gap between all used products. vcn has the lowest number as duplicates are not shown. Therefore, the number is misleading.
2. There is a vulnerability gap between Grype and other products.
3. CycloneDX exhibits the best SBOM quality, followed by vcn.
4. None of the tools differentiate between direct and transitive dependencies; all treat them as flat. That's an important area we work on as well.
5. None of the tools produce results similar to mvn `dependency:tree`, which may be acceptable in certain cases.

## Testcase

For our evaluation, we selected the [Jenkins](https://www.jenkins.io/)container image, version 2.414.3. We aimed to establish a baseline using the Maven dependency tree, covering both used and unused but existing dependencies.

We assessed SBOM quality using [SBOM Quality Score](https://github.com/interlynk-io/sbomqs).

Important: the quality score measures the overall structure of the SBOM file, not the completeness!

#### Aqua Trivy 0.46.0

**Command:**

- Generate SBOM with the command: `docker run -it --rm codenotary/sbom.sh:v1.3 trivyimage vulnscan jenkins/jenkins`

**Result:**

- 172 components
- 58 vulnerabilities
- SBOM quality 6.3/10
- Command duration: 41 seconds

[View SBOM](https://sbom.sh/279bf08f-d6bb-4df9-b40d-a21e16031836)

#### Anchore Syft 0.93.0

**Command:**

- Generate SBOM with the command: `docker run -it --rm codenotary/sbom.sh:v1.3 syftimage jenkins/jenkins`

**Result:**

- 479 components
- 0 vulnerabilities
- SBOM quality 7.56/10
- Command duration: 17 seconds

[View SBOM](https://sbom.sh/e6545de4-81a1-43a5-b212-076a02b32dc5)

#### Anchore Grype 0.71.0

**Integration:**

- Download and install using `wget` and `dpkg`.

**Command:**

- Generate SBOM with the command: `docker run -it --rm codenotary/sbom.sh:v1.3 grypeimage jenkins/jenkins`

**Result:**

- 478 components
- 123 vulnerabilities
- SBOM quality 7.11/10
- Command duration: 18 seconds

[View SBOM](https://sbom.sh/61124625-e14e-495b-8b36-c2ea8cce360e)

*Note: Grype and Syft work best in combination, and the Grype result was used.*

### ![12-min](https://codenotary.com/hs-fs/hubfs/12-min.jpg?width=6912&height=3456&name=12-min.jpg)

 

## Conclusion

This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools. It is essential to choose the right tool based on your specific requirements, as each tool comes with its own strengths and limitations. That's where [Trustcenter](https://codenotary.com/products/trustcenter/) comes in with its ability to consume all of the tools we looked into for this report. 

We encourage organizations to prioritize SBOM generation and leverage these reports to make informed decisions about tool selection and integration into their software development pipelines.

[![Share on twitter](https://4059529.fs1.hubspotusercontent-na1.net/hub/4059529/hubfs/01-marketplace/twitter-color.png?width=35&height=35&name=twitter-color.png)](https://twitter.com/intent/tweet?original_referer=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0&utm_medium=social&utm_source=twitter&url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0&utm_medium=social&utm_source=twitter&source=tweetbutton&text=) [![Share on facebook](https://4059529.fs1.hubspotusercontent-na1.net/hub/4059529/hubfs/01-marketplace/facebook-color.png?width=35&height=35&name=facebook-color.png)](http://www.facebook.com/share.php?u=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0&utm_medium=social&utm_source=facebook) [![Share on linkedin](https://4059529.fs1.hubspotusercontent-na1.net/hub/4059529/hubfs/01-marketplace/linkedin-color.png?width=35&height=35&name=linkedin-color.png)](http://www.linkedin.com/shareArticle?mini=true&url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0&utm_medium=social&utm_source=linkedin) [![Share on pinterest](https://4059529.fs1.hubspotusercontent-na1.net/hub/4059529/hubfs/pinterest.jpg?width=35&height=35&name=pinterest.jpg)](http://pinterest.com/pin/create/button/?url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0&utm_medium=social&utm_source=pinterest&media=)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Mahrukh",
    "url" : "https://codenotary.com/blog/author/mahrukh"
  },
  "dateModified" : "2023-10-23T15:28:32.358Z",
  "datePublished" : "2023-10-23T15:28:32.000Z",
  "headline" : "Monthly Quality Report for SBOM Tools (October 2023)",
  "image" : [ "https://codenotary.com/hubfs/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape-1.jpg" ],
  "mainEntityOfPage" : {
    "@id" : "https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://codenotary.com/hubfs/logo-light.svg"
    },
    "name" : "Codenotary, Inc."
  }
}
```

```json
{
  "@context" : "http://schema.org",
  "@type" : "Article",
  "author" : {
    "@type" : "Person",
    "name" : [ "Mahrukh" ],
    "url" : "https://codenotary.com/blog/author/mahrukh"
  },
  "datePublished" : "2023-10-23T15:28:32+0000",
  "description" : "This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools. Using https://sbom.sh",
  "headline" : "Monthly Quality Report for SBOM Tools (October 2023)",
  "image" : "https://23873599.fs1.hubspotusercontent-na1.net/hubfs/23873599/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape-1.jpg",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://cdn2.hubspot.net/hubfs/23873599/logo-light.svg"
    },
    "name" : ""
  },
  "url" : "https://codenotary.com/blog/monthly-quality-report-for-sbom-tools-0"
}
```