support/scripts/pkg-stats: check_url_status(): use HEAD requests to limit server load

The gnu.org admins have been blocking the IP address of machines running
pkg-stats as the GET requests for the (many) packages with gnu.org URLs are
seen as abusive.

The resource body is not used, so use a HTTP HEAD request instead of a GET
to limit server load and bandwidth use.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit is contained in:
Peter Korsgaard
2025-12-02 21:21:58 +01:00
committed by Julien Olivain
parent 2b4ee4e72f
commit f9f3e6ccc6

View File

@@ -469,7 +469,7 @@ async def check_url_status(session, pkg, npkgs, retry=True, verbose=False):
global check_url_count
try:
async with session.get(pkg.url) as resp:
async with session.head(pkg.url) as resp:
if resp.status >= 400:
pkg.status['url'] = ("error", "invalid {}".format(resp.status))
check_url_count += 1