---
title: Monthly Quality Report for SBOM Tools
description: This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools.
image: https://codenotary.com/hubfs/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape%20(7).jpg
---
**$ protect --distro linux --machines 25 --free**
[Start now](https://apps.codenotary.com/linux)
[](https://codenotary.com/)
- Product
#### [ **AgentMon Start** Organization-wide AI agent spend, security and device fleet TRY NOW →](https://apps.codenotary.com/agentmon-start)
#### [ **AgentMon** Currently monitors more \> 7 million agent interactions/day. TRY NOW →](https://codenotary.com/agentmon)
#### [ **AgentX** Agentic network control middleware. TRY NOW →](https://codenotary.com/agent-network-control)
#### [ **Autonomous Security** AI Agents keep your servers secure. TRY NOW →](https://codenotary.com/trust)
- Use Cases
#### [**AI Agent Risk Monitoring** Continuous oversight of autonomous agents across every environment.](https://codenotary.com/use-cases#risk)
#### [**Autonomous Security Operations** Self-healing defenses that detect, contain, and remediate threats.](https://codenotary.com/use-cases#agentops)
#### [**AI Coding Governance & Performance Monitoring** AI-generated code reviewed, tracked, and held to quality standards.](https://codenotary.com/use-cases#performance)
#### [**AI Tool Cost & Usage Optimization** Spend and consumption optimized across every AI service in use.](https://codenotary.com/use-cases#cost)
#### [**AI Tool Security & Policy Enforcement** Approved AI usage enforced with guardrails and policy controls.](https://codenotary.com/use-cases#security#security)
#### [**Shadow AI Governance** Unsanctioned AI tools discovered, surfaced, and brought under control.](https://codenotary.com/use-cases#shadowit)
- [Blog](https://codenotary.com/blog)
- [Press](https://codenotary.com/press)
- Resources
#### [**Integrations** Connect with your favorite tools and platforms. LEARN MORE →](https://codenotary.com/integrations)
#### [**Support** Get help from our dedicated support team. GET HELP →](https://support.codenotary.com)
#### [**Success Stories** Read how customers achieve their goals. READ MORE →](https://codenotary.com/success)
#### [**Learn** Access documentation and learning resources. EXPLORE →](https://codenotary.com/learn)
[Login](https://apps.codenotary.com/auth/login)
[All posts](https://codenotary.com/blog/all)
Sep 21, 2023
# Monthly Quality Report for SBOM Tools
By [Mahrukh](https://codenotary.com/blog/author/mahrukh) · 3 minute read
## Introduction
In our continuous effort to enhance the quality of Software Bill of Materials (SBOM) content, we are excited to present our monthly SBOM quality report. This report aims to provide insights into the performance of various SBOM tools, shedding light on their capabilities, strengths, and weaknesses. For this edition, we have conducted extensive testing using a diverse set of tools to analyze the SBOM quality, license detection, vulnerability scanning, and pipeline support.
**Tools**
- - Aqua Trivy 0.45.0
- CycloneDX Core (Java) 7.3.2
- CycloneDX Maven Plugin 2.7.9
- Anchore Syft 0.89.0
- Anchore Grype 0.66.0
- vcn version v1.0.5
- CycloneDX cdxgen 9.8.0
In this report, we delve into the world of SBOM generation tools, evaluating their performance in generating accurate and reliable SBOMs. This initiative is part of our ongoing commitment to promoting transparency, security, and the adoption of best practices in software development.
## Goal
The primary objective of this SBOM quality report is to create a baseline and gain a comprehensive understanding of how various SBOM tools function. Additionally, we aim to assess the accuracy and comparability of their results. For this purpose, we have chosen a popular Maven Java Open Source application, Apache Pulsar, to test the SBOM tools against. Our evaluation goes beyond SBOM generation to include advanced functionality such as license information, vulnerability scanning, and pipeline support.
## Overall Result
.jpg?width=6912&height=3456&name=Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape%20(7).jpg)
**Findings:**
1. There is a dependency gap between all used products. vcn has the lowest number as duplicates are not shown. Therefore, the number is misleading.
2. There is a vulnerability gap between Grype and other products.
3. CycloneDX exhibits the best SBOM quality, followed by vcn.
4. None of the tools differentiate between direct and transitive dependencies; all treat them as flat. That's an important area we work on as well.
5. None of the tools produce results similar to mvn `dependency:tree`, which may be acceptable in certain cases.
### Testcase
For our evaluation, we selected the Apache Pulsar project, available at [https://github.com/apache/pulsar](https://github.com/apache/pulsar). We aimed to establish a baseline using the Maven dependency tree, covering both used and unused but existing dependencies.
The command used to obtain the baseline:
`mvn dependency:tree | grep -Eo '[^ ]+:[^ ]+:[^:]+:[^:]+:[^:]+$' | sort | uniq | wc -l`
Result: 2585
We assessed SBOM quality using [SBOM Quality Score](https://github.com/interlynk-io/sbomqs). Important: the quality score measures the overall structure of the SBOM file not the completeness!
### Tools
We evaluated the following SBOM generation tools:
#### Aqua Trivy 0.45.0
**Integration:**
- Download and install using `wget` and `dpkg`.
**Command:**
- Generate SBOM with the command: `trivy fs . --timeout 5m -f cyclonedx --scanners vuln --output sboms/trivy-sbom.json` inside the application directory.
**Result:**
- 1209 components
- 32 vulnerabilities
- 2016 licenses
- SBOM quality issues
- 2 errors in Maven constraints detected
- Command duration: 2 minutes
[View SBOM](https://sbom.sh/ac51e1b8-e3f9-4300-bdcc-c7019eb19cbf)
#### CycloneDX Core (Java) 7.3.2
**Integration:**
- Add the following section to `pom.xml`:
` org.cyclonedx cyclonedx-core-java 7.3.2 `
**Command:**
- Generate SBOM with the command: `mvn org.cyclonedx:cyclonedx-maven-plugin:makeBom` inside the application.
**Result:**
- 16 components
- 0 vulnerabilities
- Incomplete CycloneDX format (no serial)
- Command duration: 15 minutes
[View SBOM](https://sbom.sh/eddc712c-4d5f-4361-a147-66bc0f2ed9ff)
#### CycloneDX Maven Plugin 2.7.9
**Integration:**
- Add the following section to `pom.xml`:
` org.cyclonedx cyclonedx-maven-plugin `
**Command:**
- Generate SBOM with the command: `mvn org.cyclonedx:cyclonedx-maven-plugin:makeBom` inside the application.
**Result:**
- 2 components (makeBom), 1031 (makeAggregateBom)
- 0 vulnerabilities
- 1102 licenses (makeAggregateBom)
- Incomplete CycloneDX format (no serial)
- Command duration: 6 minutes (makeBom), 32 seconds (makeAggregateBom)
[View makeAggregateBom SBOM](https://sbom.sh/5afecf76-d09d-4120-bacb-dff6e7eb96f0)
#### Anchore Syft 0.89.0
**Integration:**
- Download and install using `wget` and `dpkg`.
**Command:**
- Generate SBOM with the command: `syft . -o cyclonedx-json > sboms/syft-sbom.json` inside the application directory.
**Result:**
- 977 components
- 0 vulnerabilities
- Good SBOM quality
- Command duration: 5 seconds
[View SBOM](https://sbom.sh/e6545de4-81a1-43a5-b212-076a02b32dc5)
#### Anchore Grype 0.66.0
**Integration:**
- Download and install using `wget` and `dpkg`.
**Command:**
- Generate SBOM with the command: `syft . -o cyclonedx-json > sboms/syft-sbom.json` inside the application directory.
**Result:**
- 977 components
- 2085 vulnerabilities
- Good SBOM quality
- Command duration: 5 seconds
[View SBOM](https://sbom.sh/14fdf11a-8972-4210-a7da-3455e8ec4c03)
*Note: Grype and Syft work best in combination, and the Grype result was used.*
#### vcn version v1.0.5
**Integration:**
- Downloaded from the [Trustcenter/Enterprise](https://codenotary.com/products/trustcenter/) customer portal.
**Command:**
- Generate SBOM with the command: `vcn bom . --experimental --vuln-scan --bom-cdx-json vcnbom.json` inside the application directory.
**Result:**
- 597 components
- 0 vulnerabilities
- Good SBOM quality
- Command duration: 30 seconds
[View SBOM](https://sbom.sh/67a935b4-58be-4a25-bfb8-cbf3159853f8)
#### CycloneDX cdxgen 9.8.0
**Integration:**
- Can be downloaded from [https://github.com/CycloneDX/cdxgen](https://github.com/CycloneDX/cdxgen) or run as a Docker application.
**Command:**
- Generate SBOM with the Docker command: `docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app -o /app/sboms/cdxgen-bom.json` inside the application directory.
**Result:**
- 1394 components
- 0 vulnerabilities
- 1411 licenses
- Parse error
- Command duration: 10 minutes
[View SBOM](https://sbom.sh/3767a9fb-f816-4f7c-a0b2-2a4f703557ee)
### 
### Conclusion
This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools. It is essential to choose the right tool based on your specific requirements, as each tool comes with its own strengths and limitations. That's where [Trustcenter](https://codenotary.com/products/trustcenter/) comes in with its ability to consume all of the tools we looked into for this report.
We encourage organizations to prioritize SBOM generation and leverage these reports to make informed decisions about tool selection and integration into their software development pipelines.
[](https://twitter.com/intent/tweet?original_referer=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools&utm_medium=social&utm_source=twitter&url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools&utm_medium=social&utm_source=twitter&source=tweetbutton&text=) [](http://www.facebook.com/share.php?u=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools&utm_medium=social&utm_source=facebook) [](http://www.linkedin.com/shareArticle?mini=true&url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools&utm_medium=social&utm_source=linkedin) [](http://pinterest.com/pin/create/button/?url=https://codenotary.com/blog/monthly-quality-report-for-sbom-tools&utm_medium=social&utm_source=pinterest&media=)
```json
{
"@context" : "https://schema.org",
"@type" : "BlogPosting",
"author" : {
"@type" : "Person",
"name" : "Mahrukh",
"url" : "https://codenotary.com/blog/author/mahrukh"
},
"dateModified" : "2023-09-21T19:54:08.768Z",
"datePublished" : "2023-09-21T14:35:12.000Z",
"headline" : "Monthly Quality Report for SBOM Tools",
"image" : [ "https://codenotary.com/hubfs/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape%20(7).jpg" ],
"mainEntityOfPage" : {
"@id" : "https://codenotary.com/blog/monthly-quality-report-for-sbom-tools",
"@type" : "WebPage"
},
"publisher" : {
"@type" : "Organization",
"logo" : {
"@type" : "ImageObject",
"url" : "https://codenotary.com/hubfs/logo-light.svg"
},
"name" : "Codenotary, Inc."
}
}
```
```json
{
"@context" : "http://schema.org",
"@type" : "Article",
"author" : {
"@type" : "Person",
"name" : [ "Mahrukh" ],
"url" : "https://codenotary.com/blog/author/mahrukh"
},
"datePublished" : "2023-09-21T14:35:12+0000",
"description" : "This monthly SBOM quality report has provided valuable insights into the performance of various SBOM generation tools.",
"headline" : "Monthly Quality Report for SBOM Tools",
"image" : "https://23873599.fs1.hubspotusercontent-na1.net/hubfs/23873599/Brown%20Peach%20Illustrative%20GreetingsSlogans%20Banner%20Landscape%20%287%29.jpg",
"publisher" : {
"@type" : "Organization",
"logo" : {
"@type" : "ImageObject",
"url" : "https://cdn2.hubspot.net/hubfs/23873599/logo-light.svg"
},
"name" : ""
},
"url" : "https://codenotary.com/blog/monthly-quality-report-for-sbom-tools"
}
```