304 lines
8.6 KiB
Markdown
304 lines
8.6 KiB
Markdown
# 🏗️ Private DevOps Stack — Aetoskia Infrastructure
|
||
|
||
This repository defines a private, self-hosted DevOps stack running on Raspberry Pi nodes, consisting of:
|
||
|
||
* **Gitea** for Git hosting
|
||
* **Drone** for CI/CD automation
|
||
* **Docker Registry** with **Registry UI**
|
||
* **Drone Runner** for build execution
|
||
* **PyPI Server** for private Python package hosting
|
||
|
||
---
|
||
|
||
## 🧬 Services Overview
|
||
|
||
| Service | Role | Port(s) | Depends On |
|
||
| ---------------- | ---------------------------------------------------- | --------- | ------------ |
|
||
| **gitea** | Git server and web UI for repositories | 6002, 222 | — |
|
||
| **drone-server** | CI/CD orchestration linked with Gitea | 6003 | gitea |
|
||
| **drone-runner** | Executes Drone pipelines on local Docker host | — | drone-server |
|
||
| **registry** | Private Docker registry | 6005 | — |
|
||
| **registry-ui** | Web UI for browsing and managing the registry images | 6001 | registry |
|
||
| **pypi-server** | Private Python package index and repository | 6006 | — |
|
||
|
||
---
|
||
|
||
## ⚙️ Service Details
|
||
|
||
---
|
||
|
||
### 🐳 **Registry**
|
||
|
||
Private Docker registry storing all container images built and deployed across your network.
|
||
|
||
#### Ports
|
||
|
||
* **6005 → 5000** — Exposes the registry API to clients.
|
||
|
||
#### Mounts
|
||
|
||
* `/home/aetos/registry:/var/lib/registry` — Persistent image storage.
|
||
* `/home/aetos/registry/auth:/auth (ro)` — Authentication credentials.
|
||
* `./registry/config.yml:/etc/docker/registry/config.yml (ro)` — Custom registry configuration.
|
||
|
||
#### Environment
|
||
|
||
* **REGISTRY_AUTH:** Enables authentication using `htpasswd`.
|
||
* **REGISTRY_AUTH_HTPASSWD_REALM:** Realm name for authentication.
|
||
* **REGISTRY_AUTH_HTPASSWD_PATH:** Path to the `htpasswd` file.
|
||
|
||
#### Extra Config Required
|
||
|
||
1. Generate authentication file:
|
||
|
||
```bash
|
||
mkdir -p /home/aetos/registry/auth
|
||
docker run --rm httpd:2.4 htpasswd -Bbn <username> <password> > /home/aetos/registry/auth/htpasswd
|
||
```
|
||
2. Set correct permissions:
|
||
|
||
```bash
|
||
chmod 644 /home/aetos/registry/auth/htpasswd
|
||
```
|
||
3. Verify config file (`registry/config.yml`) includes your chosen storage and auth setup.
|
||
|
||
---
|
||
|
||
### 🦯 **Registry UI**
|
||
|
||
Web interface for visualizing and managing images stored in the internal registry.
|
||
|
||
#### Ports
|
||
|
||
* **6001 → 80** — Accessible on port 6001 via browser.
|
||
|
||
#### Mounts
|
||
|
||
* `/home/aetos/registry/auth:/auth (ro)` — Shared authentication file with the registry.
|
||
|
||
#### Environment
|
||
|
||
* **REGISTRY_TITLE:** UI header title.
|
||
* **REGISTRY_URL:** Backend registry endpoint (`https://dcr.aetoskia.com`).
|
||
* **REGISTRY_AUTH:** Enables `htpasswd` login.
|
||
* **REGISTRY_AUTH_HTPASSWD_PATH:** Location of `htpasswd`.
|
||
* **SINGLE_REGISTRY:** Limits UI to a single registry source.
|
||
|
||
#### Extra Config Required
|
||
|
||
* Ensure DNS or local `/etc/hosts` includes:
|
||
|
||
```
|
||
192.168.1.111 dcr.aetoskia.com
|
||
```
|
||
* Login using credentials from `/auth/htpasswd`.
|
||
|
||
---
|
||
|
||
### 🦉 **Gitea**
|
||
|
||
Self-hosted Git service used as Drone’s SCM backend and code hosting platform.
|
||
|
||
#### Ports
|
||
|
||
* **6002 → 3000** — Web UI.
|
||
* **222 → 22** — Git SSH access.
|
||
|
||
#### Mounts
|
||
|
||
* `/home/aetos/gitea:/data` — Persistent Gitea data (repos, users, configs).
|
||
|
||
#### Environment
|
||
|
||
* **USER_UID / USER_GID:** Runs Gitea under the specified user/group IDs.
|
||
|
||
#### Extra Config Required
|
||
|
||
1. Access Gitea at [https://git.aetoskia.com](https://git.aetoskia.com).
|
||
2. Log in as the **`aetos`** admin user (repository owner).
|
||
3. Verify Drone OAuth App is registered with:
|
||
|
||
```
|
||
Application Name: Drone CI
|
||
Redirect URI: https://drone.aetoskia.com/login
|
||
```
|
||
4. Confirm Gitea has these extra host mappings:
|
||
|
||
```
|
||
192.168.1.111 drone.aetoskia.com
|
||
```
|
||
|
||
---
|
||
|
||
### 🚀 **Drone Server**
|
||
|
||
Core CI/CD engine that connects to Gitea and manages build pipelines.
|
||
|
||
#### Ports
|
||
|
||
* **6003 → 80** — Drone web UI and API endpoint.
|
||
|
||
#### Mounts
|
||
|
||
* `/home/aetos/drone:/data` — Persistent Drone data and SQLite database.
|
||
* `/home/aetos/certs:/usr/local/share/ca-certificates (ro)` — Trusted CA certificates.
|
||
|
||
#### Environment
|
||
|
||
* **DRONE_GITEA_CLIENT_ID / SECRET:** OAuth credentials from git.
|
||
* **DRONE_GITEA_SERVER:** Gitea URL (`https://git.aetoskia.com`).
|
||
* **DRONE_RPC_SECRET:** Shared secret for secure runner communication.
|
||
* **DRONE_SERVER_HOST / PROTO:** External access configuration.
|
||
|
||
#### Extra Config Required
|
||
|
||
1. Register the OAuth app in Gitea → Admin → Applications:
|
||
|
||
* Redirect URI: `https://drone.aetoskia.com/login`
|
||
2. Stop Drone and make admin user (if needed):
|
||
|
||
```bash
|
||
docker stop drone-server
|
||
sqlite3 /home/aetos/drone/database.sqlite "UPDATE users SET user_admin=1 WHERE user_login='aetos';"
|
||
docker start drone-server
|
||
```
|
||
3. Use admin token for CLI:
|
||
|
||
```bash
|
||
export DRONE_SERVER=https://drone.aetoskia.com
|
||
export DRONE_TOKEN=<admin_token>
|
||
```
|
||
4. Mark repositories as trusted:
|
||
|
||
```bash
|
||
drone repo update apps/homepage --trusted=true
|
||
```
|
||
|
||
---
|
||
|
||
### 🧮 **Drone Runner**
|
||
|
||
Executes pipeline steps on the Docker host using the Drone RPC secret.
|
||
|
||
#### Ports
|
||
|
||
* *(none)* — Internal runner; connects to Drone Server API via HTTPS.
|
||
|
||
#### Mounts
|
||
|
||
* `/var/run/docker.sock:/var/run/docker.sock` — Allows pipeline containers to spawn Docker builds.
|
||
|
||
#### Environment
|
||
|
||
* **DRONE_RPC_PROTO / HOST / SECRET:** Connects to Drone Server via shared key.
|
||
* **DRONE_RUNNER_CAPACITY:** Number of parallel jobs supported.
|
||
* **DRONE_RUNNER_NAME / OS / ARCH:** Runner identification.
|
||
* **DRONE_LOGS_DEBUG / TRACE:** Enables detailed logging for troubleshooting.
|
||
|
||
#### Extra Config Required
|
||
|
||
1. Ensure the RPC secret matches the one defined in `drone-server`.
|
||
2. Validate connectivity:
|
||
|
||
```bash
|
||
docker logs drone-runner
|
||
```
|
||
|
||
Look for:
|
||
|
||
```
|
||
successfully pinged the remote drone server
|
||
```
|
||
|
||
---
|
||
|
||
### 🐍 **PyPI Server**
|
||
|
||
Private Python package index for hosting internal packages and dependencies.
|
||
|
||
#### Ports
|
||
|
||
* **6006 → 8080** — Accessible via browser or pip client.
|
||
|
||
#### Mounts
|
||
|
||
* `/mnt/omnissiah-vault/data/pypiserver:/data` — Package storage directory.
|
||
* `/mnt/omnissiah-vault/data/pypiserver/auth:/auth (ro)` — Authentication credentials.
|
||
|
||
#### Environment
|
||
|
||
* **PYPISERVER_AUTH=update,download,list** — Enables controlled access.
|
||
|
||
#### Command
|
||
|
||
```yaml
|
||
command:
|
||
- -P
|
||
- /auth/htpasswd
|
||
- -a
|
||
- update,download,list
|
||
- /data/packages
|
||
```
|
||
|
||
#### Usage
|
||
|
||
To upload packages:
|
||
|
||
```bash
|
||
pip install twine
|
||
TWINE_USERNAME=<username> TWINE_PASSWORD=<password> twine upload --repository-url https://pip.aetoskia.com dist/*
|
||
```
|
||
|
||
To install packages:
|
||
|
||
```bash
|
||
pip install --index-url https://<username>:<password>@pip.aetoskia.com/simple <package-name>
|
||
```
|
||
|
||
#### Notes
|
||
|
||
* Make sure `/data/packages` exists inside the container.
|
||
* The `htpasswd` file controls user access levels.
|
||
* Test the endpoint in browser: [https://pip.aetoskia.com](https://pip.aetoskia.com)
|
||
|
||
---
|
||
|
||
## 🧬 Network Integration
|
||
|
||
Ensure `/etc/hosts` on all relevant devices (Pi nodes, local dev systems) includes:
|
||
|
||
```
|
||
192.168.1.111 git.aetoskia.com
|
||
192.168.1.111 drone.aetoskia.com
|
||
192.168.1.111 dcr.aetoskia.com
|
||
192.168.1.111 pip.aetoskia.com
|
||
```
|
||
|
||
---
|
||
|
||
## 🧠 Tips
|
||
|
||
* Always start containers with profile `private-pi` to avoid unnecessary public exposure:
|
||
|
||
```bash
|
||
docker compose --profile private-pi up -d
|
||
```
|
||
* Drone and Gitea must both trust the same SSL certificate chain under `/home/aetos/certs`.
|
||
* The `registry-ui`, `drone-runner`, and `pypi-server` depend on hostname resolution between internal services.
|
||
|
||
---
|
||
|
||
## ✅ Quick Test Checklist
|
||
|
||
| Component | URL | Expected Result |
|
||
| ---------------- |------------------------------------------------------------------------------| ------------------------------------- |
|
||
| **Gitea** | [https://git.aetoskia.com](https://git.aetoskia.com) | Gitea login page |
|
||
| **Drone** | [https://drone.aetoskia.com](https://drone.aetoskia.com) | Drone CI UI linked to Gitea |
|
||
| **Registry UI** | [https://registry.aetoskia.com](https://registry.aetoskia.com) | Authenticated Docker registry browser |
|
||
| **Registry API** | [https://dcr.aetoskia.com/v2/_catalog](https://dcr.aetoskia.com/v2/_catalog) | JSON listing of repositories |
|
||
| **PyPI Server** | [https://pip.aetoskia.com/simple](https://pip.aetoskia.com/simple) | Simple index page |
|
||
|
||
---
|
||
|
||
© Aetoskia Internal Infrastructure — All rights reserved.
|