TestGitHash and TestGitRefs are failing on Gitlab-CI since 2025-08-18
(first runtime-test failures report for master) [1] due a new Gitlab-CI
security settings that prevents the any use of localhost (127.0.0.1) on
the Gitlab-CI runner [2]:
"To prevent exploitation of insecure internal web services, all webhook
and integration requests to the following local network addresses are
not allowed:
Private network addresses, including 127.0.0.1, ::1, 0.0.0.0,
10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, and IPv6 site-local
(ffc0::/10) addresses."
Gitlab suggest using a second container as a Services [3] since each
containers have access to one another and can communicate when running
the job. But this is really not practical to run git daemon since the
service is started before the Buildroot git tree is fetched and git
daemon needs git-remote directory content.
See [4]:
"If you have your service that tries to write to CI_PROJECT_DIR
immediately after it's started - it will not work and fail with an
error like '/builds/nknapp/gitlab-ci-test/': No such file or directory
error. Because that directory is most probably not there yet, as the
job step that creates it is yet to be executed!"
Also, using Gitlab-CI services requires to handle GitTestBase setUp
differently on Gitlab-CI than local use of run-tests.
So Gitlab-CI services is not an option.
We could move support/testing/tests/download/git-remote to our Gitlab
group (https://gitlab.com/buildroot.org) but since it's only intended
for testing the git download backend, we don't really want to do that.
So the only remaining choice to clone the repository from the Buildroot
git tree (local machine).
To do so, we have to do the following changes:
- Replace all "git://localhost:$(GITREMOTE_PORT_NUMBER)" by
"$(GITREMOTE_DIR)". GITREMOTE_DIR will contain the absolute path to
tests/download/git-remote directory.
- Force to use git download backend using "SITE_METHOD = git" otherwise
the download infra will fall back using wget download backend.
- Cloning a git repository with submodules using the "file://" protocol
needs to be allowed from git config. It's disabled by default:
"fatal: transport 'file' not allowed"
Use a local gitconfig file generated by GitTestBase before starting
TestGitHash and TestGitRefs with:
[protocol "file"]
allow = always
This is considered safe since we are using a git repository from the
Buildroot git tree.
- Running TestGitHash and TestGitRefs on Gitlab-CI requires additional
git config settings to disable ownership check of the git tree [5].
Thanks to Arnout for advices about GIT_CONFIG_GLOBAL.
Runtime tested:
https://gitlab.com/kubu93/buildroot/-/jobs/11428504450 (TestGitHash)
https://gitlab.com/kubu93/buildroot/-/jobs/11428490035 (TestGitRefs)
[1] https://lore.kernel.org/buildroot/68a3ee65.500a0220.397724.30db@mx.google.com/
[2] https://docs.gitlab.com/security/webhooks/#allow-requests-to-the-local-network-from-webhooks-and-integrations
[3] https://docs.gitlab.com/ci/services/
[4] https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25982#note_2139685797
[5] a016b693f7
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/11271124174 (TestGitHash)
https://gitlab.com/buildroot.org/buildroot/-/jobs/11271124173 (TestGitRefs)
Cc: Arnout Vandecappelle <arnout@rnout.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit a367362af0)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
eaa8114191