reverse-engineer-docker-files-and-visualize-docker-compose-files

Docker-compose files can become quite complex and in some cases you would like to document not just the docker-compose.yml content, but also visualize how the different docker-compose service communicate. Furthermore, it can be quite interesting to reverse-engineer docker files to get a better understanding, what they are build of.

There are different ways to do so and I want to dig into some of the most popular ones.

dfimage

Starting with dfimage, a reverse-engineering project for Docker files, that simplifies that task a lot.

You can simply use the available dockerhub image, if you like, or of course build the project yourself.

To run the image against a local (already pulled) docker image, simply execute:

docker images # to pick your image id

or pull an image

docker pull grafana/grafana #as an example

docker run -v /var/run/docker.sock:/var/run/docker.sock laniksj/dfimage imageID

docker run -v /var/run/docker.sock:/var/run/docker.sock laniksj/dfimage 808a15f85914

The project needs access to the docker.sock to access the image details, therefore run it with -v /var/run/docker.sock …

The output for the Grafana image looks like that and can give you a pretty good idea, how the image was build:

FROM grafana/grafana:latest
ADD file:fe64057fbb83dccb960efabbf1cd8777920ef279a7fa8dbca0a8801c651bdf7c in /
CMD ["/bin/sh"]
ARG GF_UID=472
ARG GF_GID=472
ENV PATH=/usr/share/grafana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin GF_PATHS_CONFIG=/etc/grafana/grafana.ini GF_PATHS_DATA=/var/lib/grafana GF_PATHS_HOME=/usr/share/grafana GF_PATHS_LOGS=/var/log/grafana GF_PATHS_PLUGINS=/var/lib/grafana/plugins GF_PATHS_PROVISIONING=/etc/grafana/provisioning
WORKDIR /usr/share/grafana
RUN |2 GF_GID=472 GF_UID=472 /bin/sh -c apk add –no-cache ca-certificates bash
&& apk add –no-cache –upgrade –repository=http://dl-cdn.alpinelinux.org/alpine/edge/main openssl musl-utils
RUN |2 GF_GID=472 GF_UID=472 /bin/sh -c if [ `arch` = "x86_64" ]; then apk add –no-cache –virtual phantomjs-utils curl
&& cd /tmp
&& curl -Ls https://github.com/dustinblackman/phantomized/releases/download/2.1.1/dockerized-phantomjs.tar.gz | tar xz
&& cp -R lib lib64 /
&& cp -R usr/lib/x86_64-linux-gnu /usr/lib
&& cp -R usr/share/fonts /usr/share
&& cp -R etc/fonts /etc
&& rm -rf /tmp/*
&& apk del –no-cache phantomjs-utils; fi
COPY dir:44ba505a8722a860c8980bd5fc72dc9df74b7364a65579eaf9df86a41f1492e4 in /usr/share/grafana
RUN |2 GF_GID=472 GF_UID=472 /bin/sh -c mkdir -p "$GF_PATHS_HOME/.aws"
&& addgroup -S -g $GF_GID grafana
&& adduser -S -u $GF_UID -G grafana grafana
&& mkdir -p "$GF_PATHS_PROVISIONING/datasources" "$GF_PATHS_PROVISIONING/dashboards" "$GF_PATHS_PROVISIONING/notifiers" "$GF_PATHS_LOGS" "$GF_PATHS_PLUGINS" "$GF_PATHS_DATA"
&& cp "$GF_PATHS_HOME/conf/sample.ini" "$GF_PATHS_CONFIG"
&& cp "$GF_PATHS_HOME/conf/ldap.toml" /etc/grafana/ldap.toml
&& chown -R grafana:grafana "$GF_PATHS_DATA" "$GF_PATHS_HOME/.aws" "$GF_PATHS_LOGS" "$GF_PATHS_PLUGINS" "$GF_PATHS_PROVISIONING"
&& chmod -R 777 "$GF_PATHS_DATA" "$GF_PATHS_HOME/.aws" "$GF_PATHS_LOGS" "$GF_PATHS_PLUGINS" "$GF_PATHS_PROVISIONING"
EXPOSE 3000
COPY file:3e1dfb34fa6281634e9860cf1caea6384f6978cb513eb33b07f04752b4879694 in /run.sh
USER grafana
ENTRYPOINT ["/run.sh"]

dive docker image

The project can be found here: https://github.com/LanikSJ/dfimage

dive

dive is a tool to explore every single layer of a docker image. It can not just help to audit a docker image, but also to find ways to shrink your image size.

As always you can either build the dive command, download a release or use a docker container. I use the debian release v.0.8.1 in that example:

wget https://github.com/wagoodman/dive/releases/download/v0.8.1/dive_0.8.1_linux_amd64.deb
sudo dpkg -i dive_0.8.1_linux_amd64.deb

check the Grafana image

dive 808a15f85914
Fetching image… (this can take a while with large images)
Parsing image…
Analyzing image…
Building cache…

dive usage demo Dockerfile

The project demo video

Our Grafana image looks more like that 

dive grafana image

Another nice dive feature is the integration into a CI build process, by setting the CI variable to true

CI=true dive

The release downloads can be found here: https://github.com/wagoodman/dive/releases/tag/v0.8.1

The dive project can be found here:

https://github.com/wagoodman/dive

Docker-Compose-Viz

The last project I want to cover is the visualization of a docker-compose file. That’s very convenient, when you want to visualize the communication between ports or different services inside your docker-compose file. It’s using the graphviz project to create a image file. Just make sure, you’re running the following command within the same directory as the docker-compose.yml file and make sure you have write permissions. 

After running the command you’ll find the docker-compose.png file within that directory as well.

docker run –rm -it –name dcv -v $(pwd):/input pmsipilot/docker-compose-viz render -m image docker-compose.yml

This example is a simple prest/postgres project based on the following docker-compose.yml file:

version: "3"
services:
prest:
image: prest/prest
links:

  • "postgres:postgres"
    environment:
  • PREST_DEBUG=true # remove comment for enable DEBUG mode (disable JWT)
  • PREST_PG_HOST=postgres
  • PREST_PG_USER=prest

    – PREST_PG_PASS=prest

  • PREST_PG_DATABASE=prest
  • PREST_PG_PORT=5432
  • PREST_JWT_DEFAULT=false # remove if need jwt
    depends_on:
  • postgres
    ports:
  • "13000:3000"
    postgres:
    image: postgres:latest
    volumes:
  • "./data:/var/lib/postgresql/data"
    environment:
  • POSTGRES_USER=prest
  • POSTGRES_DB=prest

    – POSTGRES_PASSWORD=prest

    ports:
  • "15432:5432"

docker-compose prest visualize

or dgraph:

version: "3.2"
services:
zero:
image: dgraph/dgraph:latest
volumes:

  • type: volume
    source: dgraph
    target: /dgraph
    volume:
    nocopy: true
    ports:
  • 5080:5080
  • 6080:6080
    restart: on-failure
    command: dgraph zero –my=zero:5080
    server:
    image: dgraph/dgraph:latest
    volumes:
  • type: volume
    source: dgraph
    target: /dgraph
    volume:
    nocopy: true
    ports:
  • 18080:8080
  • 19080:9080
    restart: on-failure
    command: dgraph alpha –my=server:7080 –lru_mb=2048 –zero=zero:5080
    ratel:
    image: dgraph/dgraph:latest
    volumes:
  • type: volume
    source: dgraph
    target: /dgraph
    volume:
    nocopy: true
    ports:
  • 8000:8000
    command: dgraph-ratelvolumes:
    dgraph:

dgraph viz

Depending on the amount and complexity of the services, you can get some decent sized images generated – so make sure to have a big screen. 

The docker-compose viz project can be found here: https://github.com/pmsipilot/docker-compose-viz

CNIL
Metrics and Logs

(formerly, Opvizor Performance Analyzer)

VMware vSphere & Cloud
PERFORMANCE MONITORING, LOG ANALYSIS, LICENSE COMPLIANCE!

Monitor and Analyze Performance and Log files:
Performance monitoring for your systems and applications with log analysis (tamperproof using immudb) and license compliance (RedHat, Oracle, SAP and more) in one virtual appliance!

Subscribe to Our Newsletter

Get the latest product updates, company news, and special offers delivered right to your inbox.

Subscribe to our newsletter

Use Case - Tamper-resistant Clinical Trials

Goal:

Blockchain PoCs were unsuccessful due to complexity and lack of developers.

Still the goal of data immutability as well as client verification is a crucial. Furthermore, the system needs to be easy to use and operate (allowing backup, maintenance windows aso.).

Implementation:

immudb is running in different datacenters across the globe. All clinical trial information is stored in immudb either as transactions or the pdf documents as a whole.

Having that single source of truth with versioned, timestamped, and cryptographically verifiable records, enables a whole new way of transparency and trust.

Use Case - Finance

Goal:

Store the source data, the decision and the rule base for financial support from governments timestamped, verifiable.

A very important functionality is the ability to compare the historic decision (based on the past rulebase) with the rulebase at a different date. Fully cryptographic verifiable Time Travel queries are required to be able to achieve that comparison.

Implementation:

While the source data, rulebase and the documented decision are stored in verifiable Blobs in immudb, the transaction is stored using the relational layer of immudb.

That allows the use of immudb’s time travel capabilities to retrieve verified historic data and recalculate with the most recent rulebase.

Use Case - eCommerce and NFT marketplace

Goal:

No matter if it’s an eCommerce platform or NFT marketplace, the goals are similar:

  • High amount of transactions (potentially millions a second)
  • Ability to read and write multiple records within one transaction
  • prevent overwrite or updates on transactions
  • comply with regulations (PCI, GDPR, …)


Implementation:

immudb is typically scaled out using Hyperscaler (i. e. AWS, Google Cloud, Microsoft Azure) distributed across the Globe. Auditors are also distributed to track the verification proof over time. Additionally, the shop or marketplace applications store immudb cryptographic state information. That high level of integrity and tamper-evidence while maintaining a very high transaction speed is key for companies to chose immudb.

Use Case - IoT Sensor Data

Goal:

IoT sensor data received by devices collecting environment data needs to be stored locally in a cryptographically verifiable manner until the data is transferred to a central datacenter. The data integrity needs to be verifiable at any given point in time and while in transit.

Implementation:

immudb runs embedded on the IoT device itself and is consistently audited by external probes. The data transfer to audit is minimal and works even with minimum bandwidth and unreliable connections.

Whenever the IoT devices are connected to a high bandwidth, the data transfer happens to a data center (large immudb deployment) and the source and destination date integrity is fully verified.

Use Case - DevOps Evidence

Goal:

CI/CD and application build logs need to be stored auditable and tamper-evident.
A very high Performance is required as the system should not slow down any build process.
Scalability is key as billions of artifacts are expected within the next years.
Next to a possibility of integrity validation, data needs to be retrievable by pipeline job id or digital asset checksum.

Implementation:

As part of the CI/CD audit functionality, data is stored within immudb using the Key/Value functionality. Key is either the CI/CD job id (i. e. Jenkins or GitLab) or the checksum of the resulting build or container image.

White Paper — Registration

We will also send you the research paper
via email.

CodeNotary — Webinar

White Paper — Registration

Please let us know where we can send the whitepaper on CodeNotary Trusted Software Supply Chain. 

Become a partner

Start Your Trial

Please enter contact information to receive an email with the virtual appliance download instructions.

Start Free Trial

Please enter contact information to receive an email with the free trial details.