Vishesh 'ironeagle' Bangotra
322741e00c
All checks were successful
continuous-integration/drone/push Build is passing
Add the exact pip/npm registry endpoints and secret names, the scope-scoped (not global) registry requirement given Verdaccio's missing uplink, the host-scoped _auth rule, the per-item Drone shell caveat, and the Verdaccio storage uid/permissions gotcha behind the confusing "404 no such package available" error on publish.
8 lines
7.7 KiB
JSON
8 lines
7.7 KiB
JSON
{
|
||
"module": "drone_pipelines",
|
||
"content": {
|
||
"path": "drone_pipelines",
|
||
"docstring": "# Drone Pipelines\n\n## Summary\n\nEvery Aetos repository ships a Drone CI definition at `.drone.yml`. There is\nexactly **one standard per repository**, and there are **three types of file**,\nchosen by what the repository produces:\n\n| Type | File | Produces | Repos |\n|------|------|----------|-------|\n| **A — PyPI package** | `templates/drone/python-lib.yaml` | A wheel + sdist on the private index | `dagpipe`, `doc-forge`, `hexa`, `mail-intake`, `mongo-ops`, `omniread`, `openapi-first`, `py-jwt` |\n| **B — npm package** | `templates/drone/npm-lib.yaml` | A package published to the private npm registry | `doc-weaver`, `react-auth`, `react-openapi` |\n| **C — Docker deployment** | `templates/drone/service-docker.yaml` | An image pushed to the registry and run as a container | `auth-server` |\n\nAll templates run on `platform: linux / arm64` with the workspace at\n`/drone/src`. Copy the matching template to `<repo>/.drone.yml`; the Type A and\nType B files need no edits, the Type C file needs its placeholders filled in\n(below).\n\n> Conventions: `.drone.yml` (not `.yaml`), one file per repo, secrets injected\n> with `from_secret` — never commit credentials.\n\n---\n\n## Type A — PyPI package (`python-lib.yaml`)\n\nFor internal Python libraries. The file contains **two pipelines**:\n\n1. **`build-and-publish-pypi`** — triggered on `tag`:\n - `check-version` — **guard**: fail unless `$DRONE_TAG` equals the version in\n `pyproject.toml`; then read `name` / `version` and query\n `$PIP_REPO_URL/simple/<package>/`; if that version already exists, exit\n `78` and skip the whole build (see [91 – Versioning & Changelog](91_versioning_and_changelog.md)).\n - `quality-gate` — install the package with its `[dev]` extras from the\n private index, then `black --check`, `ruff check`, `mypy`, `pydoclint`,\n `pytest`.\n - `build-package` — `python -m build` into `dist/`.\n - `upload-to-private-pypi` — `twine upload` to `$PIP_REPO_URL`.\n2. **`backfill-pypi-from-tags`** — triggered on `custom`:\n - `fetch-tags` — `git fetch --tags --force`.\n - `build-and-upload-missing` — walk every tag in version order, skip the ones\n already on the index, `git checkout` the rest, verify `tag == pyproject\n version` at that commit, build, and upload. This is the disaster-recovery /\n replay path.\n\nSecrets: `PIP_REPO_URL`, `PIP_USERNAME`, `PIP_PASSWORD`.\n\n- **Ship as-is.** Package name and version come from `pyproject.toml`, so the\n same file is reused verbatim across every library.\n\n---\n\n## Type B — npm package (`npm-lib.yaml`)\n\nFor internal npm libraries (the React packages). The file contains **two\npipelines**:\n\n1. **`build-and-publish-npm`** — triggered on `tag`:\n - `check-version` — **guard**: fail unless `$DRONE_TAG` equals the version in\n `package.json`; query `$NPM_REPO_URL` for `name@version`; if it already\n exists, exit `78` and skip the whole build.\n - `quality-gate` — write `.npmrc` from the secrets, `npm install`, then\n `npm run quality-gate`.\n - `build-package` — `npm install && npm run build` into `dist/`.\n - `publish-to-private-npm` — `npm publish --access restricted` to\n `$NPM_REPO_URL`.\n2. **`backfill-npm-from-tags`** — triggered on `custom`:\n - `fetch-tags` — `git fetch --tags --force`.\n - `build-and-publish-missing` — walk every tag in version order, skip the\n ones already on the registry, `git checkout` the rest, verify\n `tag == package.json version` at that commit, build, and publish. This is\n the disaster-recovery / replay path.\n\nSecrets: `NPM_REPO_URL`, `NPM_USERNAME`, `NPM_PASSWORD`. Auth is a generated\n`.npmrc` written from secrets at runtime. It sets a **scope-scoped** registry,\nnever a global one — `@aetoskia:registry=$NPM_REPO_URL` — plus a host-scoped\nauth line `//<host>/:_auth=<base64(user:pass)>`. The scope and host are derived\nfrom `package.json` and `NPM_REPO_URL` respectively. A global `registry=` would\npush public transitive deps through Verdaccio (which has no uplink) and 404\n`npm install`; a bare `_auth=` is rejected by modern npm. See\n[93 – Library Registry](93_library_registry.md).\n\n- **Ship as-is.** Name and version come from `package.json`, so the same file\n is reused verbatim across every npm package.\n\n> **Registry credentials are shared.** The private npm registry (Verdaccio at\n> `npm.aetoskia.com`) authenticates against the **same `htpasswd` file** as the\n> private PyPI index and the Docker registry (`/auth/htpasswd`), with\n> registration disabled (`max_users: -1`). `NPM_USERNAME` / `NPM_PASSWORD` are\n> therefore the **same credentials as `PIP_USERNAME` / `PIP_PASSWORD`** — no\n> separate npm account exists. Verdaccio's htpasswd plugin accepts **bcrypt**\n> hashes (`$2y$…`); non-bcrypt entries that pypiserver tolerates will not\n> authenticate against npm. See [93 – Library Registry](93_library_registry.md).\n\n---\n\n## Type C — Docker deployment (`service-docker.yaml`)\n\nFor a service that ships a container image — a long-running API/worker, e.g.\n`auth-server`. The file contains **one pipeline**, `build-and-deploy-service`,\ntriggered on `tag`:\n\n- `fetch-tags` — resolve the latest tag into `LATEST_TAG.txt`; fail if there\n are no tags.\n- `check-version` — **guard**: fail unless `$DRONE_TAG` equals the repo-root\n `VERSION` file (see [91 – Versioning & Changelog](91_versioning_and_changelog.md)).\n- `check-remote-image` — if `aetos/<app>:<tag>` already exists locally, exit\n `78` and skip the build.\n- `quality-gate` — install from `requirements.txt` and run\n `black` / `ruff` / `mypy` / `pytest`.\n- `build-image` — `docker build` with the private-index build args, tagged\n `aetos/<app>:<tag>` and `:latest`.\n- `push-image` — login, re-tag with `$REGISTRY_HOST`, push both tags.\n- `stop-old` — `docker rm -f <app>`.\n- `run-container` — `docker run -d --name <app> -p <host>:<container>\n --restart always` with the service's `-e` environment.\n\nSecrets: `PIP_REPO_URL`, `PIP_USERNAME`, `PIP_PASSWORD` (service build args),\n`REGISTRY_HOST`, `REGISTRY_USER`, `REGISTRY_PASS`, plus the service runtime env\n(for example `MONGO_*`, `JWT_SECRET`).\n\nPlaceholders to fill in per repo:\n\n| Placeholder | Meaning | Example |\n|-------------|---------|---------|\n| `<app>` | service image / container name | `auth-server` |\n| `aetos/<app>` | image repository on the registry | `aetos/auth-server` |\n| `<host-port>` / `<container-port>` | published : in-image port | `9003:8000` |\n| `<ENV_VAR>` | runtime environment variable | `MONGO_HOST` |\n\n---\n\n## What not to do\n\n- Do not hand-roll a fourth shape. A repo is exactly one of Type A, B, or C,\n never combinations.\n- Do not inline secrets; always use `from_secret`.\n- Do not drop the `check-version` / `check-remote-image` skip — it guards the\n index and the running container against duplicate publishes.\n- Do not drop the tag↔version guard. A Python package needs its\n `pyproject.toml` version to equal the tag; an npm package needs its\n `package.json` version; a service needs a root `VERSION` file. See\n [91 – Versioning & Changelog](91_versioning_and_changelog.md).\n- Do not rename the pipelines; tooling and runbooks key off\n `build-and-publish-pypi`, `build-and-publish-npm`,\n `build-and-deploy-service`, and their `backfill-*-from-tags` companions.\n\n---\n\n## Related\n\n- [01 – Build Recipe](01_build_recipe.md) · [30 – Infra Adapters](30_infra_adaptors.md) · [91 – Versioning & Changelog](91_versioning_and_changelog.md) · [93 – Library Registry](93_library_registry.md)\n",
|
||
"objects": {}
|
||
}
|
||
} |