Repository too big, failing github action aquasecurity/setup-trivy due timeout

[tomas-lipensky]

Description

Even the size of files in the repo is about 100MB, the size of repository i more than 1Gb. The issue is probably cause by having binary files in git repository. I mean archives in pkg/fanal/test/testdata. Every update of the archive means, that the old one is still stored in git history.

Can it be redesigned and deleted old gzip archives from the history?

Impact: github action aquasecurity/setup-trivy takes a lot of time, timeouts and can cause space issue during the GitHub action runs.

Desired Behavior

Reffer pkg/fanal/test/testdata archives sources from artifact repository, not from code repository. Delete pkg/fanal/test/testdata archives from history

Actual Behavior

gzip files in pkg/fanal/test/testdata are making repository size too big (repository history). git clone makes transfer of more than 1gb data, it is slow and action aquasecurity/setup-trivy is failing

Reproduction Steps

git clone https://github.com/aquasecurity/trivy

it produce big data transfer and it takes several minutes (depends on transfer rate)

Target

Git Repository

Scanner

None

Output Format

None

Mode

None

Debug Output

git clone https://github.com/aquasecurity/trivy
Cloning into 'trivy'...
remote: Enumerating objects: 154891, done.
remote: Counting objects: 100% (434/434), done.
remote: Compressing objects: 100% (356/356), done.
remote: Total 154891 (delta 268), reused 78 (delta 78), pack-reused 154457 (from 5)
Receiving objects: 100% (154891/154891), 906.28 MiB | 2.87 MiB/s, done.
Resolving deltas: 100% (82742/82742), done.

Operating System

Linux, MacOS

Version

not depending

Checklist

• Run trivy clean --all
• Read the troubleshooting

[DmitriyLewen]

Hello @tomas-lipensky

The setup-trivy action should take only the contrib directory:
https://github.com/aquasecurity/setup-trivy/blob/e6c2c5e321ed9123bda567646e2f96565e34abe1/action.yaml#L66-L67
Also, it doesn’t fetch the full history:
https://github.com/aquasecurity/setup-trivy/blob/e6c2c5e321ed9123bda567646e2f96565e34abe1/action.yaml#L69

The setup-trivy action should take only the contrib directory:
https://github.com/aquasecurity/setup-trivy/blob/e6c2c5e321ed9123bda567646e2f96565e34abe1/action.yaml#L66-L67

[tomas-lipensky]

Maybe I do something wrongly with sparse checkout, but it is 0.9 GB. It seems that it really takes just the listed files, but the GIT database is complete and big:

lipenskyt@E1068775 Downloads % mkdir trivy
lipenskyt@E1068775 Downloads % cd trivy
lipenskyt@E1068775 trivy % git remote add origin https://github.com/aquasecurity/trivy.git
fatal: not a git repository (or any of the parent directories): .git
lipenskyt@E1068775 trivy % git init
Initialized empty Git repository in /Users/lipenskyt/Downloads/trivy/.git/
lipenskyt@E1068775 trivy % git remote add origin https://github.com/aquasecurity/trivy.git
lipenskyt@E1068775 trivy % git config core.sparseCheckout true
lipenskyt@E1068775 trivy % git sparse-checkout set contrib
lipenskyt@E1068775 trivy % cat .git/info/sparse-checkout
contrib
lipenskyt@E1068775 trivy % git config core.sparseCheckout true
lipenskyt@E1068775 trivy % git pull
remote: Enumerating objects: 154852, done.
remote: Counting objects: 100% (469/469), done.
remote: Compressing objects: 100% (384/384), done.
remote: Total 154852 (delta 290), reused 85 (delta 85), pack-reused 154383 (from 2)
Receiving objects: 100% (154852/154852), 905.85 MiB | 2.61 MiB/s, done.
Resolving deltas: 100% (82746/82746), done.
From https://github.com/aquasecurity/trivy
..

• [new tag] v0.9.2 -> v0.9.2
  There is no tracking information for the current branch.
  Please specify which branch you want to merge with.
  See git-pull(1) for details.

  git pull

If you wish to set tracking information for this branch you can do so with:

git branch --set-upstream-to=origin/<branch> main

lipenskyt@E1068775 trivy % du -smx .
919 .