feat: synchronize GitHub releases with Gitea

Gitea pull mirrors preserve Git refs but leave release notes and uploaded artifacts outside the backup. Without a separate synchronization path, restoring a repository would still depend on GitHub for important release context and binaries.

README.md

@@ -10,15 +10,15 @@ Mirrors **all** GitHub repositories of a user into a self-hosted Gitea instance
 as pull mirrors — including repositories created later.
 
 Gitea can already mirror a single repository through its web UI. What it cannot
-do is watch an account and pick up new repositories on its own. That gap is the
+do is watch an account and pick up new repositories on its own, or continuously
-only reason this script exists: a daily cron job asks the GitHub API which
+mirror GitHub release metadata and uploaded assets. A cron job closes both gaps.
-repositories exist and creates the mirrors that are missing. Keeping their
+Keeping the repositories' *git content* up to date remains Gitea's job.
-*content* up to date remains Gitea's job.
 
 | Task | Handled by | Frequency |
 | --- | --- | --- |
 | Discover **new** repositories, create mirrors | this script (cron) | daily |
 | Pull new commits into **existing** mirrors | Gitea itself | `MIRROR_INTERVAL` |
+| Synchronize release metadata and uploaded assets | this script (cron) | daily |
 
 ### Requirements
 
@@ -74,6 +74,8 @@ python3 sync_mirrors.py                # run for real
 | `MIRROR_INTERVAL` | Go duration, e.g. `24h`, `8h`, `30m` |
 | `INCLUDE_FORKS` | Mirror forks too (default `false`) |
 | `INCLUDE_ARCHIVED` | Mirror archived repositories (default `true`) |
+| `SYNC_RELEASES` | Synchronize GitHub releases on every run (default `true`) |
+| `SYNC_RELEASE_ASSETS` | Download uploaded release assets too (default `true`) |
 | `DRY_RUN` | `true` = show what would happen, change nothing |
 
 Every key can be overridden by an environment variable of the same name, which
@@ -107,13 +109,20 @@ remaining validity, warns from 14 days out and exits non-zero once expired:
 
 ### Scope and limits
 
-- Mirrored is **git data only**: all branches and tags. Issues, pull requests,
+- Gitea mirrors **git data** itself: all branches and tags. The script separately
-  releases and the wiki are excluded on purpose — a pull mirror never syncs
+  synchronizes GitHub release titles, notes, draft/prerelease state and uploaded
-  those, so importing them once would leave a snapshot that silently goes
+  assets on every run. Issues, pull requests and the wiki remain excluded.
-  stale. To change that, flip the flags in `create_mirror()`.
+- A release is synchronized only after its git tag has reached Gitea. If the tag
+  is still missing, the release is logged and retried by the next cron run.
+- Source archives generated automatically for a tag are not copied because
+  Gitea generates its own. Assets explicitly uploaded to GitHub are copied.
 - The script **never deletes** anything in Gitea. If a repository disappears or
   is renamed on GitHub, its mirror stays and is reported as an orphan in the
-  log — for a backup that is the right direction.
+  log — for a backup that is the right direction. The same applies to releases
+  and assets deleted upstream. An asset with the same name but a different size
+  is reported, while the existing Gitea copy is retained.
+- GitHub author identities, original release timestamps and download counters
+  cannot be reproduced through Gitea's regular release API.
 - A Gitea repository that exists but is **not** a mirror is left untouched, so
   a real working repository can never be overwritten.
 - Mirrors are read-only in Gitea. Push to GitHub, not to the mirror.
@@ -122,13 +131,22 @@ remaining validity, warns from 14 days out and exits non-zero once expired:
 
 | File | Purpose |
 | --- | --- |
-| `sync_mirrors.py` | Creates missing mirrors, aligns the mirror interval |
+| `sync_mirrors.py` | Creates mirrors, aligns intervals and synchronizes releases |
+| `tests/` | Unit tests for repeatable release synchronization |
 | `run_sync.sh` | Cron wrapper: appends to `sync.log`, trims it to 1000 lines |
 | `install.sh` | Installs scripts, config and cron entry |
 | `mirror.env.example` | Configuration template |
 | `mirror.env` | Real config with secrets — git-ignored, `chmod 600` |
 | `sync.log` | Last ~2000 log lines |
 
+### Tests
+
+Run the unit tests from the repository root:
+
+```bash
+python3 -m unittest discover -s tests -v
+```
+
 ### Troubleshooting
 
 ```bash
@@ -141,6 +159,8 @@ tail -f ~/gitea-github-mirror/sync.log
 | `GitHub API 403` | Rate limit, or token lacks repository access |
 | `migrate failed (HTTP 409)` | Name already taken in Gitea |
 | `... exists in Gitea but is not a mirror` | Real repo with that name — rename one of them |
+| `release ... waits for its tag` | Gitea has not fetched the GitHub tag yet; the next run retries it |
+| `asset ... differs` | An upstream asset changed; the retained Gitea backup is not overwritten |
 | `Missing config value(s)` | `mirror.env` not filled in |
 | `no private repositories` despite a token | Token's repository access is "Public repositories", or no repository permission is set at all |
 
@@ -167,14 +187,15 @@ selbst gehostete Gitea-Instanz — auch Repositories, die erst später entstehen
 
 Ein einzelnes Repository kann Gitea bereits über die Weboberfläche spiegeln.
 Was Gitea nicht kann: einen Account beobachten und neue Repositories von selbst
-aufnehmen. Genau dafür gibt es dieses Skript — ein täglicher Cron-Job fragt die
+aufnehmen oder GitHub-Release-Metadaten und hochgeladene Assets fortlaufend
-GitHub-API, welche Repositories existieren, und legt die fehlenden Mirrors an.
+spiegeln. Ein Cron-Job schließt beide Lücken. Die *Git-Inhalte* aktuell zu halten
-Deren *Inhalt* aktuell zu halten bleibt Giteas Aufgabe.
+bleibt Giteas Aufgabe.
 
 | Aufgabe | Erledigt von | Häufigkeit |
 | --- | --- | --- |
 | **Neue** Repositories finden, Mirror anlegen | dieses Skript (Cron) | täglich |
 | Commits in **bestehende** Mirrors holen | Gitea selbst | `MIRROR_INTERVAL` |
+| Release-Metadaten und hochgeladene Assets synchronisieren | dieses Skript (Cron) | täglich |
 
 ### Voraussetzungen
 
@@ -230,6 +251,8 @@ python3 sync_mirrors.py                # echter Lauf
 | `MIRROR_INTERVAL` | Go-Duration, z. B. `24h`, `8h`, `30m` |
 | `INCLUDE_FORKS` | Forks mitspiegeln (Standard `false`) |
 | `INCLUDE_ARCHIVED` | Archivierte Repositories spiegeln (Standard `true`) |
+| `SYNC_RELEASES` | GitHub-Releases bei jedem Lauf synchronisieren (Standard `true`) |
+| `SYNC_RELEASE_ASSETS` | Auch hochgeladene Release-Assets laden (Standard `true`) |
 | `DRY_RUN` | `true` = nur anzeigen, nichts ändern |
 
 Jeder Schlüssel lässt sich durch eine gleichnamige Umgebungsvariable
@@ -266,13 +289,22 @@ Ablauf mit Exit-Code 1:
 
 ### Umfang und Grenzen
 
-- Gespiegelt werden **nur Git-Daten**: alle Branches und Tags. Issues, Pull
+- Gitea selbst spiegelt die **Git-Daten**: alle Branches und Tags. Das Skript
-  Requests, Releases und Wiki bleiben bewusst außen vor — ein Pull-Mirror
+  synchronisiert zusätzlich bei jedem Lauf GitHub-Release-Titel, Beschreibung,
-  synchronisiert das nie, ein einmaliger Import würde also stillschweigend
+  Draft-/Prerelease-Status und hochgeladene Assets. Issues, Pull Requests und
-  veralten. Umstellbar über die Flags in `create_mirror()`.
+  Wiki bleiben außen vor.
+- Ein Release wird erst synchronisiert, wenn sein Git-Tag in Gitea angekommen
+  ist. Fehlt der Tag noch, wird das protokolliert und beim nächsten Cron-Lauf
+  erneut versucht.
+- Automatisch für Tags erzeugte Quellcodearchive werden nicht kopiert, weil
+  Gitea eigene erzeugt. Explizit auf GitHub hochgeladene Assets werden kopiert.
 - Das Skript **löscht nie** etwas in Gitea. Verschwindet ein Repository auf
   GitHub oder wird umbenannt, bleibt der Mirror bestehen und wird im Log als
-  verwaist gemeldet — für ein Backup ist das die richtige Richtung.
+  verwaist gemeldet — für ein Backup ist das die richtige Richtung. Gleiches
+  gilt für upstream gelöschte Releases und Assets. Hat ein gleichnamiges Asset
+  eine andere Größe, wird dies gemeldet und die Gitea-Kopie behalten.
+- GitHub-Autoren, ursprüngliche Release-Zeitpunkte und Downloadzähler können
+  über die reguläre Gitea-Release-API nicht identisch übernommen werden.
 - Ein Gitea-Repository, das existiert, aber **kein** Mirror ist, wird nicht
   angefasst; ein echtes Arbeits-Repository kann so nie überschrieben werden.
 - Mirrors sind in Gitea schreibgeschützt. Gepusht wird nach GitHub, nicht in
@@ -282,13 +314,22 @@ Ablauf mit Exit-Code 1:
 
 | Datei | Zweck |
 | --- | --- |
-| `sync_mirrors.py` | Legt fehlende Mirrors an, gleicht das Mirror-Intervall ab |
+| `sync_mirrors.py` | Legt Mirrors an, gleicht Intervalle ab und synchronisiert Releases |
+| `tests/` | Unit-Tests für die wiederholbare Release-Synchronisierung |
 | `run_sync.sh` | Cron-Wrapper: schreibt nach `sync.log`, kürzt es auf 1000 Zeilen |
 | `install.sh` | Installiert Skripte, Konfiguration und Cron-Eintrag |
 | `mirror.env.example` | Konfigurationsvorlage |
 | `mirror.env` | Echte Konfiguration mit Secrets — git-ignoriert, `chmod 600` |
 | `sync.log` | Die letzten ~2000 Log-Zeilen |
 
+### Tests
+
+Die Unit-Tests im Wurzelverzeichnis des Repositorys starten:
+
+```bash
+python3 -m unittest discover -s tests -v
+```
+
 ### Fehlersuche
 
 ```bash
@@ -301,6 +342,8 @@ tail -f ~/gitea-github-mirror/sync.log
 | `GitHub API 403` | Rate Limit, oder Token hat keinen Repository-Zugriff |
 | `migrate failed (HTTP 409)` | Name in Gitea bereits vergeben |
 | `... exists in Gitea but is not a mirror` | Echtes Repo gleichen Namens — eines umbenennen |
+| `release ... waits for its tag` | Gitea hat den GitHub-Tag noch nicht geholt; der nächste Lauf versucht es erneut |
+| `asset ... differs` | Ein Upstream-Asset wurde geändert; das Gitea-Backup wird nicht überschrieben |
 | `Missing config value(s)` | `mirror.env` nicht ausgefüllt |
 | `no private repositories` trotz Token | Repository access steht auf „Public repositories", oder es ist gar keine Repository-Permission gesetzt |
 

mirror.env.example

@@ -22,5 +22,9 @@ GITEA_OWNER=your-gitea-username
 MIRROR_INTERVAL=24h
 INCLUDE_FORKS=false
 INCLUDE_ARCHIVED=true
+# Gitea's native mirror only pulls git data. The script separately keeps
+# GitHub release metadata and uploaded assets in sync on every cron run.
+SYNC_RELEASES=true
+SYNC_RELEASE_ASSETS=true
 # Set to true to preview without changing anything.
 DRY_RUN=false

sync_mirrors.py

@@ -1,18 +1,23 @@
 #!/usr/bin/env python3
 """Mirror all GitHub repositories of a user into Gitea as pull mirrors.
 
-Existing mirrors are left to Gitea's own cron; this script only creates the
+Existing git mirrors are left to Gitea's own cron. This script creates the ones
-ones that are missing and keeps the mirror interval aligned with the config.
+that are missing, aligns their interval, and synchronizes GitHub release
-It never deletes anything on the Gitea side -- a mirror that vanished upstream
+metadata and assets because Gitea's native mirror does not. It never deletes
-is exactly what a backup is supposed to keep.
+anything on the Gitea side -- retained history is the point of a backup.
 """
 
 import json
+import http.client
+import io
 import os
 import re
 import sys
+import tempfile
 import urllib.error
+import urllib.parse
 import urllib.request
+import uuid
 from datetime import datetime, timezone
 
 KNOWN_KEYS = (
@@ -24,6 +29,8 @@ KNOWN_KEYS = (
     "MIRROR_INTERVAL",
     "INCLUDE_FORKS",
     "INCLUDE_ARCHIVED",
+    "SYNC_RELEASES",
+    "SYNC_RELEASE_ASSETS",
     "DRY_RUN",
 )
 
@@ -54,6 +61,8 @@ def load_config():
         "MIRROR_INTERVAL": "24h",
         "INCLUDE_FORKS": "false",
         "INCLUDE_ARCHIVED": "true",
+        "SYNC_RELEASES": "true",
+        "SYNC_RELEASE_ASSETS": "true",
         "DRY_RUN": "false",
     }
     if os.path.exists(CONFIG_PATH):
@@ -102,6 +111,109 @@ def api(url, token=None, scheme="token", method="GET", payload=None):
         return resp.status, (json.loads(body) if body.strip() else None)
 
 
+def path_part(value):
+    """Quote one URL path component, including slashes used inside tag names."""
+    return urllib.parse.quote(str(value), safe="")
+
+
+class SafeRedirectHandler(urllib.request.HTTPRedirectHandler):
+    """Do not forward API credentials to a cross-host download redirect."""
+
+    def redirect_request(self, req, fp, code, msg, headers, newurl):
+        redirected = super().redirect_request(req, fp, code, msg, headers, newurl)
+        if redirected is None:
+            return None
+        old_host = urllib.parse.urlsplit(req.full_url).netloc.lower()
+        new_host = urllib.parse.urlsplit(newurl).netloc.lower()
+        if old_host != new_host:
+            redirected.remove_header("Authorization")
+        return redirected
+
+
+safe_download_opener = urllib.request.build_opener(SafeRedirectHandler())
+
+
+def download_github_asset(cfg, asset):
+    """Download a release asset into a temporary file without buffering it in RAM."""
+    headers = {
+        "Accept": "application/octet-stream",
+        "User-Agent": "gitea-github-mirror/1.0",
+    }
+    if cfg["GITHUB_TOKEN"]:
+        headers["Authorization"] = "Bearer %s" % cfg["GITHUB_TOKEN"]
+    req = urllib.request.Request(asset["url"], headers=headers)
+    tmp = tempfile.TemporaryFile()
+    size = 0
+    try:
+        with safe_download_opener.open(req, timeout=180) as resp:
+            while True:
+                chunk = resp.read(1024 * 1024)
+                if not chunk:
+                    break
+                tmp.write(chunk)
+                size += len(chunk)
+        expected = int(asset.get("size") or 0)
+        if expected and size != expected:
+            raise IOError("downloaded %d bytes, expected %d" % (size, expected))
+        tmp.seek(0)
+        return tmp, size
+    except Exception:
+        tmp.close()
+        raise
+
+
+def upload_release_asset(cfg, repo_name, release_id, asset, source, size):
+    """Stream a temporary file to Gitea as multipart/form-data."""
+    query = urllib.parse.urlencode({"name": asset["name"]})
+    url = ("%s/api/v1/repos/%s/%s/releases/%s/assets?%s"
+           % (cfg["GITEA_URL"], path_part(cfg["GITEA_OWNER"]), path_part(repo_name),
+              release_id, query))
+    parsed = urllib.parse.urlsplit(url)
+    boundary = "----gitea-github-mirror-%s" % uuid.uuid4().hex
+    safe_name = str(asset["name"]).replace("\\", "_").replace('"', "_")
+    safe_name = safe_name.replace("\r", "_").replace("\n", "_")
+    content_type = asset.get("content_type") or "application/octet-stream"
+    content_type = str(content_type).replace("\r", "").replace("\n", "")
+    preamble = (
+        "--%s\r\n"
+        "Content-Disposition: form-data; name=\"attachment\"; filename=\"%s\"\r\n"
+        "Content-Type: %s\r\n\r\n" % (boundary, safe_name, content_type)
+    ).encode("utf-8")
+    ending = ("\r\n--%s--\r\n" % boundary).encode("ascii")
+    request_path = parsed.path + (("?" + parsed.query) if parsed.query else "")
+
+    if parsed.scheme == "https":
+        conn = http.client.HTTPSConnection(parsed.hostname, parsed.port, timeout=180)
+    elif parsed.scheme == "http":
+        conn = http.client.HTTPConnection(parsed.hostname, parsed.port, timeout=180)
+    else:
+        raise ValueError("Unsupported Gitea URL scheme: %s" % parsed.scheme)
+
+    try:
+        conn.putrequest("POST", request_path)
+        conn.putheader("Authorization", "token %s" % cfg["GITEA_TOKEN"])
+        conn.putheader("Accept", "application/json")
+        conn.putheader("Content-Type", "multipart/form-data; boundary=%s" % boundary)
+        conn.putheader("Content-Length", str(len(preamble) + size + len(ending)))
+        conn.putheader("User-Agent", "gitea-github-mirror/1.0")
+        conn.endheaders()
+        conn.send(preamble)
+        while True:
+            chunk = source.read(1024 * 1024)
+            if not chunk:
+                break
+            conn.send(chunk)
+        conn.send(ending)
+        response = conn.getresponse()
+        body = response.read()
+        if response.status >= 400:
+            raise urllib.error.HTTPError(
+                url, response.status, response.reason, response.headers, io.BytesIO(body))
+        return json.loads(body.decode("utf-8")) if body.strip() else None
+    finally:
+        conn.close()
+
+
 def github_repos(cfg):
     """All repos owned by the user; private ones included when a token is set."""
     token = cfg["GITHUB_TOKEN"]
@@ -229,6 +341,229 @@ def align_interval(cfg, name, existing, dry_run):
     log("OK", "interval of %s set to %s" % (name, cfg["MIRROR_INTERVAL"]))
 
 
+def github_releases(cfg, repo_name):
+    releases = []
+    page = 1
+    while True:
+        url = ("https://api.github.com/repos/%s/%s/releases?per_page=100&page=%d"
+               % (path_part(cfg["GITHUB_USER"]), path_part(repo_name), page))
+        _, batch = api(url, token=cfg["GITHUB_TOKEN"], scheme="Bearer")
+        if not batch:
+            break
+        releases.extend(batch)
+        if len(batch) < 100:
+            break
+        page += 1
+    return releases
+
+
+def github_release_assets(cfg, repo_name, release_id):
+    assets = []
+    page = 1
+    while True:
+        url = ("https://api.github.com/repos/%s/%s/releases/%s/assets"
+               "?per_page=100&page=%d"
+               % (path_part(cfg["GITHUB_USER"]), path_part(repo_name), release_id, page))
+        _, batch = api(url, token=cfg["GITHUB_TOKEN"], scheme="Bearer")
+        if not batch:
+            break
+        assets.extend(batch)
+        if len(batch) < 100:
+            break
+        page += 1
+    return assets
+
+
+def gitea_releases(cfg, repo_name):
+    releases = []
+    page = 1
+    while True:
+        url = ("%s/api/v1/repos/%s/%s/releases?limit=50&page=%d"
+               % (cfg["GITEA_URL"], path_part(cfg["GITEA_OWNER"]),
+                  path_part(repo_name), page))
+        _, batch = api(url, token=cfg["GITEA_TOKEN"])
+        if not batch:
+            break
+        releases.extend(batch)
+        if len(batch) < 50:
+            break
+        page += 1
+    return releases
+
+
+def gitea_tags(cfg, repo_name):
+    tags = set()
+    page = 1
+    while True:
+        url = ("%s/api/v1/repos/%s/%s/tags?limit=50&page=%d"
+               % (cfg["GITEA_URL"], path_part(cfg["GITEA_OWNER"]),
+                  path_part(repo_name), page))
+        _, batch = api(url, token=cfg["GITEA_TOKEN"])
+        if not batch:
+            break
+        tags.update(tag["name"] for tag in batch)
+        if len(batch) < 50:
+            break
+        page += 1
+    return tags
+
+
+def release_payload(release):
+    return {
+        "tag_name": release["tag_name"],
+        "target_commitish": release.get("target_commitish") or "",
+        "name": release.get("name") or "",
+        "body": release.get("body") or "",
+        "draft": bool(release.get("draft")),
+        "prerelease": bool(release.get("prerelease")),
+    }
+
+
+def release_needs_update(source, target):
+    """Return true for fields Gitea's PATCH endpoint can actually change."""
+    wanted = release_payload(source)
+    if bool(target.get("draft")) != wanted["draft"]:
+        return True
+    if bool(target.get("prerelease")) != wanted["prerelease"]:
+        return True
+    # Gitea ignores empty strings in release PATCH requests. Retaining old text
+    # also follows this script's policy of never deleting backup information.
+    if wanted["name"] and (target.get("name") or "") != wanted["name"]:
+        return True
+    if wanted["body"] and (target.get("body") or "") != wanted["body"]:
+        return True
+    return False
+
+
+def release_update_payload(source):
+    wanted = release_payload(source)
+    payload = {
+        "draft": wanted["draft"],
+        "prerelease": wanted["prerelease"],
+    }
+    for key in ("target_commitish", "name", "body"):
+        if wanted[key]:
+            payload[key] = wanted[key]
+    return payload
+
+
+def sync_release_assets(cfg, repo_name, source_release, target_release, dry_run):
+    copied = preserved = 0
+    source_assets = github_release_assets(cfg, repo_name, source_release["id"])
+    target_assets = {asset["name"]: asset for asset in target_release.get("assets", [])}
+
+    for asset in source_assets:
+        name = asset["name"]
+        existing = target_assets.get(name)
+        if existing is not None:
+            source_size = int(asset.get("size") or 0)
+            target_size = int(existing.get("size") or 0)
+            if source_size and target_size and source_size != target_size:
+                preserved += 1
+                log("WARN", "%s release %s asset %s differs (%d vs %d bytes) - "
+                    "Gitea copy left untouched"
+                    % (repo_name, source_release["tag_name"], name,
+                       source_size, target_size))
+            continue
+
+        if dry_run:
+            log("DRY", "would copy %s release %s asset %s (%d bytes)"
+                % (repo_name, source_release["tag_name"], name,
+                   int(asset.get("size") or 0)))
+            copied += 1
+            continue
+
+        tmp = None
+        try:
+            tmp, size = download_github_asset(cfg, asset)
+            upload_release_asset(cfg, repo_name, target_release["id"], asset, tmp, size)
+            copied += 1
+            log("OK", "copied %s release %s asset %s (%d bytes)"
+                % (repo_name, source_release["tag_name"], name, size))
+        except urllib.error.HTTPError as exc:
+            detail = exc.read().decode("utf-8", "replace")[:300]
+            fail("%s release %s asset %s failed (HTTP %s): %s"
+                 % (repo_name, source_release["tag_name"], name, exc.code, detail))
+        except Exception as exc:
+            fail("%s release %s asset %s failed: %s"
+                 % (repo_name, source_release["tag_name"], name, exc))
+        finally:
+            if tmp is not None:
+                tmp.close()
+    return copied, preserved
+
+
+def sync_releases(cfg, repo_name, dry_run):
+    stats = {"created": 0, "updated": 0, "assets": 0, "preserved": 0, "waiting": 0}
+    source_releases = github_releases(cfg, repo_name)
+    target_releases = gitea_releases(cfg, repo_name)
+    by_tag = {release["tag_name"]: release for release in target_releases}
+    tags_to_check = [
+        release["tag_name"] for release in source_releases
+        if not release.get("draft")
+        and (release["tag_name"] not in by_tag
+             or by_tag[release["tag_name"]].get("draft"))
+    ]
+    available_tags = gitea_tags(cfg, repo_name) if tags_to_check else set()
+
+    source_tags = {release["tag_name"] for release in source_releases}
+    orphaned = sorted(tag for tag in by_tag if tag not in source_tags)
+    if orphaned:
+        log("WARN", "%s Gitea release(s) with no GitHub counterpart left untouched: %s"
+            % (repo_name, ", ".join(orphaned)))
+
+    for source in source_releases:
+        tag = source["tag_name"]
+        target = by_tag.get(tag)
+        needs_mirrored_tag = (not source.get("draft")
+                              and (target is None or target.get("draft")))
+        if needs_mirrored_tag and tag not in available_tags:
+            stats["waiting"] += 1
+            log("WARN", "%s release %s waits for its tag to reach the Gitea mirror"
+                % (repo_name, tag))
+            continue
+
+        try:
+            if target is None:
+                if dry_run:
+                    target = {"id": source["id"], "tag_name": tag, "assets": []}
+                    log("DRY", "would create %s release %s" % (repo_name, tag))
+                else:
+                    _, target = api(
+                        "%s/api/v1/repos/%s/%s/releases"
+                        % (cfg["GITEA_URL"], path_part(cfg["GITEA_OWNER"]),
+                           path_part(repo_name)),
+                        token=cfg["GITEA_TOKEN"], method="POST",
+                        payload=release_payload(source))
+                    log("OK", "created %s release %s" % (repo_name, tag))
+                stats["created"] += 1
+            elif release_needs_update(source, target):
+                if dry_run:
+                    log("DRY", "would update %s release %s" % (repo_name, tag))
+                else:
+                    _, target = api(
+                        "%s/api/v1/repos/%s/%s/releases/%s"
+                        % (cfg["GITEA_URL"], path_part(cfg["GITEA_OWNER"]),
+                           path_part(repo_name), target["id"]),
+                        token=cfg["GITEA_TOKEN"], method="PATCH",
+                        payload=release_update_payload(source))
+                    log("OK", "updated %s release %s" % (repo_name, tag))
+                stats["updated"] += 1
+
+            if truthy(cfg["SYNC_RELEASE_ASSETS"]):
+                copied, preserved = sync_release_assets(
+                    cfg, repo_name, source, target, dry_run)
+                stats["assets"] += copied
+                stats["preserved"] += preserved
+        except urllib.error.HTTPError as exc:
+            detail = exc.read().decode("utf-8", "replace")[:300]
+            fail("%s release %s failed (HTTP %s): %s"
+                 % (repo_name, tag, exc.code, detail))
+        except Exception as exc:
+            fail("%s release %s failed: %s" % (repo_name, tag, exc))
+    return stats
+
+
 def main():
     cfg = load_config()
     dry_run = truthy(cfg["DRY_RUN"])
@@ -272,6 +607,8 @@ def main():
                     "not 'Public repositories'")
 
     created = skipped = 0
+    release_totals = {"created": 0, "updated": 0, "assets": 0,
+                      "preserved": 0, "waiting": 0}
     for repo in sorted(repos, key=lambda r: r["name"].lower()):
         name = repo["name"]
         try:
@@ -281,14 +618,30 @@ def main():
             continue
 
         if existing is None:
+            mirror_created = False
             try:
                 create_mirror(cfg, repo, dry_run)
                 created += 1
+                mirror_created = True
             except urllib.error.HTTPError as exc:
                 detail = exc.read().decode("utf-8", "replace")[:300]
                 fail("%s: migrate failed (HTTP %s): %s" % (name, exc.code, detail))
             except Exception as exc:
                 fail("%s: migrate failed: %s" % (name, exc))
+
+            if mirror_created and truthy(cfg["SYNC_RELEASES"]):
+                try:
+                    if dry_run:
+                        count = len(github_releases(cfg, name))
+                        if count:
+                            log("DRY", "would synchronize %d release(s) after creating %s"
+                                % (count, name))
+                    else:
+                        stats = sync_releases(cfg, name, dry_run)
+                        for key in release_totals:
+                            release_totals[key] += stats[key]
+                except Exception as exc:
+                    fail("%s: could not synchronize releases: %s" % (name, exc))
             continue
 
         skipped += 1
@@ -300,6 +653,18 @@ def main():
         except Exception as exc:
             fail("%s: could not update mirror interval: %s" % (name, exc))
 
+        if truthy(cfg["SYNC_RELEASES"]):
+            try:
+                stats = sync_releases(cfg, name, dry_run)
+                for key in release_totals:
+                    release_totals[key] += stats[key]
+            except urllib.error.HTTPError as exc:
+                detail = exc.read().decode("utf-8", "replace")[:300]
+                fail("%s: release sync failed (HTTP %s): %s"
+                     % (name, exc.code, detail))
+            except Exception as exc:
+                fail("%s: release sync failed: %s" % (name, exc))
+
     github_names = {r["name"].lower() for r in repos}
     try:
         page, orphans = 1, []
@@ -323,7 +688,12 @@ def main():
     except Exception as exc:
         log("WARN", "Could not check for orphaned mirrors: %s" % exc)
 
-    log("INFO", "Done: %d created, %d already present, %d error(s)" % (created, skipped, errors))
+    log("INFO", "Done: %d mirror(s) created, %d already present; releases: "
+        "%d created, %d updated, %d asset(s) copied, %d preserved, "
+        "%d waiting for tags; %d error(s)"
+        % (created, skipped, release_totals["created"], release_totals["updated"],
+           release_totals["assets"], release_totals["preserved"],
+           release_totals["waiting"], errors))
     return 1 if errors else 0
 
 

tests/__init__.py

@@ -0,0 +1 @@
+"""Tests for gitea-github-mirror."""

tests/test_sync_mirrors.py

@@ -0,0 +1,188 @@
+import io
+import unittest
+from unittest import mock
+
+import sync_mirrors as mirror
+
+
+def config(**overrides):
+    values = {
+        "GITHUB_USER": "octocat",
+        "GITHUB_TOKEN": "github-token",
+        "GITEA_URL": "http://gitea.invalid",
+        "GITEA_TOKEN": "gitea-token",
+        "GITEA_OWNER": "backup",
+        "SYNC_RELEASE_ASSETS": "true",
+    }
+    values.update(overrides)
+    return values
+
+
+def github_release(**overrides):
+    values = {
+        "id": 101,
+        "tag_name": "v1.0.0",
+        "target_commitish": "main",
+        "name": "Version 1.0.0",
+        "body": "Release notes",
+        "draft": False,
+        "prerelease": False,
+    }
+    values.update(overrides)
+    return values
+
+
+class ReleaseSyncTests(unittest.TestCase):
+    def setUp(self):
+        mirror.errors = 0
+
+    @mock.patch.object(mirror, "upload_release_asset")
+    @mock.patch.object(mirror, "download_github_asset")
+    @mock.patch.object(mirror, "github_release_assets")
+    @mock.patch.object(mirror, "gitea_tags")
+    @mock.patch.object(mirror, "gitea_releases")
+    @mock.patch.object(mirror, "github_releases")
+    @mock.patch.object(mirror, "api")
+    def test_creates_release_and_copies_missing_asset(
+            self, api, github_releases, gitea_releases, gitea_tags,
+            github_assets, download_asset, upload_asset):
+        source = github_release()
+        asset = {
+            "id": 501,
+            "name": "program.zip",
+            "size": 3,
+            "url": "https://api.github.com/assets/501",
+            "content_type": "application/zip",
+        }
+        github_releases.return_value = [source]
+        gitea_releases.return_value = []
+        gitea_tags.return_value = {"v1.0.0"}
+        github_assets.return_value = [asset]
+        download_asset.return_value = (io.BytesIO(b"zip"), 3)
+        api.return_value = (201, {"id": 901, "tag_name": "v1.0.0", "assets": []})
+
+        stats = mirror.sync_releases(config(), "project", False)
+
+        self.assertEqual(1, stats["created"])
+        self.assertEqual(1, stats["assets"])
+        self.assertEqual(0, mirror.errors)
+        self.assertEqual("POST", api.call_args.kwargs["method"])
+        self.assertEqual("v1.0.0", api.call_args.kwargs["payload"]["tag_name"])
+        upload_asset.assert_called_once()
+
+    @mock.patch.object(mirror, "github_release_assets")
+    @mock.patch.object(mirror, "gitea_tags")
+    @mock.patch.object(mirror, "gitea_releases")
+    @mock.patch.object(mirror, "github_releases")
+    @mock.patch.object(mirror, "api")
+    def test_second_run_does_not_write_matching_release(
+            self, api, github_releases, gitea_releases, gitea_tags, github_assets):
+        source = github_release()
+        target = {
+            "id": 901,
+            "tag_name": "v1.0.0",
+            "name": "Version 1.0.0",
+            "body": "Release notes",
+            "draft": False,
+            "prerelease": False,
+            "assets": [{"name": "program.zip", "size": 3}],
+        }
+        github_releases.return_value = [source]
+        gitea_releases.return_value = [target]
+        github_assets.return_value = [{"name": "program.zip", "size": 3}]
+
+        stats = mirror.sync_releases(config(), "project", False)
+
+        self.assertEqual(0, stats["created"])
+        self.assertEqual(0, stats["updated"])
+        self.assertEqual(0, stats["assets"])
+        api.assert_not_called()
+        gitea_tags.assert_not_called()
+
+    @mock.patch.object(mirror, "gitea_tags", return_value=set())
+    @mock.patch.object(mirror, "gitea_releases", return_value=[])
+    @mock.patch.object(mirror, "github_releases", return_value=[github_release()])
+    @mock.patch.object(mirror, "api")
+    def test_waits_when_mirror_has_not_fetched_release_tag(
+            self, api, github_releases, gitea_releases, gitea_tags):
+        stats = mirror.sync_releases(config(), "project", False)
+
+        self.assertEqual(1, stats["waiting"])
+        self.assertEqual(0, stats["created"])
+        api.assert_not_called()
+
+    @mock.patch.object(mirror, "gitea_tags")
+    @mock.patch.object(mirror, "gitea_releases", return_value=[])
+    @mock.patch.object(
+        mirror, "github_releases", return_value=[github_release(draft=True)])
+    @mock.patch.object(mirror, "api")
+    def test_creates_draft_without_requiring_mirrored_tag(
+            self, api, github_releases, gitea_releases, gitea_tags):
+        api.return_value = (
+            201,
+            {"id": 901, "tag_name": "v1.0.0", "draft": True, "assets": []},
+        )
+
+        stats = mirror.sync_releases(
+            config(SYNC_RELEASE_ASSETS="false"), "project", False)
+
+        self.assertEqual(1, stats["created"])
+        self.assertEqual(0, stats["waiting"])
+        gitea_tags.assert_not_called()
+        self.assertTrue(api.call_args.kwargs["payload"]["draft"])
+
+    @mock.patch.object(mirror, "gitea_releases")
+    @mock.patch.object(mirror, "github_releases")
+    @mock.patch.object(mirror, "api")
+    def test_updates_metadata_without_touching_assets(
+            self, api, github_releases, gitea_releases):
+        source = github_release(body="New notes", prerelease=True)
+        target = {
+            "id": 901,
+            "tag_name": "v1.0.0",
+            "name": "Version 1.0.0",
+            "body": "Old notes",
+            "draft": False,
+            "prerelease": False,
+            "assets": [],
+        }
+        github_releases.return_value = [source]
+        gitea_releases.return_value = [target]
+        api.return_value = (200, dict(target, body="New notes", prerelease=True))
+
+        stats = mirror.sync_releases(
+            config(SYNC_RELEASE_ASSETS="false"), "project", False)
+
+        self.assertEqual(1, stats["updated"])
+        self.assertEqual("PATCH", api.call_args.kwargs["method"])
+        self.assertEqual("New notes", api.call_args.kwargs["payload"]["body"])
+        self.assertTrue(api.call_args.kwargs["payload"]["prerelease"])
+
+    @mock.patch.object(mirror, "download_github_asset")
+    @mock.patch.object(mirror, "github_release_assets")
+    @mock.patch.object(mirror, "gitea_releases")
+    @mock.patch.object(mirror, "github_releases")
+    def test_keeps_changed_asset_in_gitea(
+            self, github_releases, gitea_releases, github_assets, download_asset):
+        source = github_release()
+        target = {
+            "id": 901,
+            "tag_name": "v1.0.0",
+            "name": "Version 1.0.0",
+            "body": "Release notes",
+            "draft": False,
+            "prerelease": False,
+            "assets": [{"name": "program.zip", "size": 2}],
+        }
+        github_releases.return_value = [source]
+        gitea_releases.return_value = [target]
+        github_assets.return_value = [{"name": "program.zip", "size": 3}]
+
+        stats = mirror.sync_releases(config(), "project", False)
+
+        self.assertEqual(1, stats["preserved"])
+        download_asset.assert_not_called()
+
+
+if __name__ == "__main__":
+    unittest.main()