manager-node-profiles (#1)
Reviewed-on: https://gitea.aetoskia.com/services/server-monitoring/pulls/1 Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
This commit is contained in:
163
README.md
Normal file
163
README.md
Normal file
@@ -0,0 +1,163 @@
|
|||||||
|
# 🏗️ Private Server Monitoring Stack — Aetoskia Infrastructure
|
||||||
|
|
||||||
|
This repository defines a private, self-hosted server monitoring stack running on Raspberry Pi nodes and other servers, consisting of:
|
||||||
|
|
||||||
|
* **Netdata** for system and container metrics
|
||||||
|
* **Portainer** for container management and orchestration
|
||||||
|
|
||||||
|
This setup allows a central **manager server** to monitor all nodes while providing dashboards and management UI accessible via HTTPS.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧬 Services Overview
|
||||||
|
|
||||||
|
| Service | Role | Port(s) | Depends On |
|
||||||
|
| --------------- | --------------------------------------- | ------- | ------------------- |
|
||||||
|
| netdata | System monitoring dashboard & master | 7001 | — |
|
||||||
|
| netdata_node | Node agent streaming metrics to master | 7003 | netdata (manager) |
|
||||||
|
| portainer | Container management UI & API | 7002 | — |
|
||||||
|
| portainer_agent | Node agent for remote Docker management | 9001 | portainer (manager) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Service Details
|
||||||
|
|
||||||
|
### 🐳 Netdata Manager
|
||||||
|
|
||||||
|
Central monitoring server collecting metrics from all connected nodes and exposing dashboards.
|
||||||
|
|
||||||
|
**Ports**
|
||||||
|
|
||||||
|
* 7001 → 19999 — Web dashboard (HTTPS handled via reverse proxy)
|
||||||
|
|
||||||
|
**Mounts**
|
||||||
|
|
||||||
|
* `netdata_config:/etc/netdata` — Persistent Netdata configuration
|
||||||
|
* `netdata_lib:/var/lib/netdata` — Persistent Netdata metrics database
|
||||||
|
* `netdata_cache:/var/cache/netdata` — Cache storage
|
||||||
|
|
||||||
|
**Environment**
|
||||||
|
|
||||||
|
* `NETDATA_CLAIM_TOKEN` — Optional claim token for Netdata Cloud
|
||||||
|
* `NETDATA_CLAIM_URL=https://app.netdata.cloud`
|
||||||
|
|
||||||
|
**Profiles**
|
||||||
|
|
||||||
|
* `netdata-manager` — Manager profile (always acts as manager + local node)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 🐳 Netdata Node
|
||||||
|
|
||||||
|
Agent container that streams metrics to the manager server.
|
||||||
|
|
||||||
|
**Ports**
|
||||||
|
|
||||||
|
* 7003 → 19999 — Local dashboard (optional)
|
||||||
|
|
||||||
|
**Environment**
|
||||||
|
|
||||||
|
* `NETDATA_STREAM_PARENT=https://netdata.aetoskia.com` — Manager endpoint
|
||||||
|
* `NETDATA_CLAIM_TOKEN` — Optional claim token
|
||||||
|
* `NETDATA_CLAIM_URL=https://app.netdata.cloud`
|
||||||
|
|
||||||
|
**Profiles**
|
||||||
|
|
||||||
|
* `netdata-node` — Node-only profile
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 🦾 Portainer Manager
|
||||||
|
|
||||||
|
Central container management UI and API server, visualizing Docker nodes and stacks.
|
||||||
|
|
||||||
|
**Ports**
|
||||||
|
|
||||||
|
* 7002 → 9000 — Portainer web UI
|
||||||
|
|
||||||
|
**Mounts**
|
||||||
|
|
||||||
|
* `/var/run/docker.sock:/var/run/docker.sock:ro` — Access local Docker
|
||||||
|
* `portainer_data:/data` — Persistent Portainer data
|
||||||
|
|
||||||
|
**Profiles**
|
||||||
|
|
||||||
|
* `portainer-manager` — Manager profile
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 🦾 Portainer Agent
|
||||||
|
|
||||||
|
Node agent that registers with the Portainer manager to allow remote Docker management.
|
||||||
|
|
||||||
|
**Ports**
|
||||||
|
|
||||||
|
* 9001 → 9001 — Agent API
|
||||||
|
|
||||||
|
**Mounts**
|
||||||
|
|
||||||
|
* `/var/run/docker.sock:/var/run/docker.sock` — Access local Docker
|
||||||
|
* `portainer_agent_data:/data` — Persistent agent data
|
||||||
|
|
||||||
|
**Profiles**
|
||||||
|
|
||||||
|
* `portainer-agent` — Node-only profile
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧬 Network Integration
|
||||||
|
|
||||||
|
Ensure `/etc/hosts` on all relevant devices (Pi nodes, servers) includes:
|
||||||
|
|
||||||
|
```
|
||||||
|
192.168.1.35 netdata.aetoskia.com
|
||||||
|
192.168.1.35 portainer.aetoskia.com
|
||||||
|
```
|
||||||
|
|
||||||
|
* **Netdata manager** receives metrics from `netdata_node` agents
|
||||||
|
* **Portainer manager** manages `portainer_agent` nodes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧠 Usage
|
||||||
|
|
||||||
|
### Manager Server
|
||||||
|
|
||||||
|
Start manager services (also acts as local node):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --profile netdata-manager --profile portainer-manager down --remove-orphans && docker compose --profile netdata-manager --profile portainer-manager up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Node Server
|
||||||
|
|
||||||
|
Start agent/node services:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --profile netdata-node --profile portainer-agent down --remove-orphans && docker compose --profile netdata-node --profile portainer-agent up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Quick Test Checklist
|
||||||
|
|
||||||
|
| Component | URL | Expected Result |
|
||||||
|
| --------------- | -------------------------------------------------------------------------- | ------------------------------------------------------- |
|
||||||
|
| Netdata Manager | [https://netdata.aetoskia.com:7001](https://netdata.aetoskia.com:7001) | Dashboard showing metrics for manager + connected nodes |
|
||||||
|
| Portainer | [https://portainer.aetoskia.com:7002](https://portainer.aetoskia.com:7002) | Portainer UI showing manager + registered nodes |
|
||||||
|
| Netdata Node | https://<node_ip>:7003 | Node metrics accessible locally |
|
||||||
|
| Portainer Agent | Internal agent API | Registered under manager UI |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧠 Tips
|
||||||
|
|
||||||
|
* **Manager profile** always acts as **manager + local node**.
|
||||||
|
* **Node profile** only runs **agent containers**, never acts as manager.
|
||||||
|
* Use **Docker Compose profiles** to cleanly separate roles.
|
||||||
|
* Reverse proxy (HTTPS) is recommended for dashboard access.
|
||||||
|
* Persistent mounts ensure that configuration and metrics survive container restarts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
© Aetoskia Internal Infrastructure — All rights reserved.
|
||||||
@@ -1,13 +1,21 @@
|
|||||||
services:
|
services:
|
||||||
# System Monitoring
|
# --------------------------
|
||||||
|
# Netdata Manager
|
||||||
|
# --------------------------
|
||||||
netdata:
|
netdata:
|
||||||
image: netdata/netdata:latest
|
image: netdata/netdata:latest
|
||||||
container_name: netdata
|
container_name: netdata
|
||||||
|
hostname: Private-Pi-Manager
|
||||||
ports:
|
ports:
|
||||||
- "7001:19999"
|
- "7001:19999"
|
||||||
|
cap_add:
|
||||||
|
- SYS_PTRACE
|
||||||
|
- SYS_ADMIN
|
||||||
|
security_opt:
|
||||||
|
- apparmor:unconfined
|
||||||
volumes:
|
volumes:
|
||||||
- netdata_config:/etc/netdata
|
- netdata_config:/etc/netdata
|
||||||
- netdata_lib:/var/lib/netdata
|
- netdata_lib:/var/lib/netdata
|
||||||
- netdata_cache:/var/cache/netdata
|
- netdata_cache:/var/cache/netdata
|
||||||
- /proc:/host/proc:ro
|
- /proc:/host/proc:ro
|
||||||
- /sys:/host/sys:ro
|
- /sys:/host/sys:ro
|
||||||
@@ -16,19 +24,18 @@ services:
|
|||||||
- /etc/passwd:/host/etc/passwd:ro
|
- /etc/passwd:/host/etc/passwd:ro
|
||||||
- /etc/group:/host/etc/group:ro
|
- /etc/group:/host/etc/group:ro
|
||||||
- /etc/os-release:/host/etc/os-release:ro
|
- /etc/os-release:/host/etc/os-release:ro
|
||||||
cap_add:
|
|
||||||
- SYS_PTRACE
|
|
||||||
- SYS_ADMIN
|
|
||||||
security_opt:
|
|
||||||
- apparmor:unconfined
|
|
||||||
environment:
|
environment:
|
||||||
- NETDATA_CLAIM_TOKEN=${NETDATA_CLAIM_TOKEN:-}
|
- NETDATA_CLAIM_TOKEN=baiLAOz-VoOsvVNhG1CLi6j14rL3bmYtWHIEfwfRFRX5VhnQDUxpSvKQ3WsSrH2lRw-obUX2tWxIepN9BrYHUppnXfkPndpLCeeDwH4P0ItJ7twkCL77XPnMcQnL8hqN9pBpFIs
|
||||||
- NETDATA_CLAIM_URL=https://app.netdata.cloud
|
- NETDATA_CLAIM_URL=https://app.netdata.cloud
|
||||||
networks:
|
networks:
|
||||||
- monitoring-net
|
- monitoring-net
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
profiles:
|
||||||
|
- netdata-manager
|
||||||
|
|
||||||
# Container Management
|
# --------------------------
|
||||||
|
# Portainer Manager
|
||||||
|
# --------------------------
|
||||||
portainer:
|
portainer:
|
||||||
image: portainer/portainer-ce:latest
|
image: portainer/portainer-ce:latest
|
||||||
container_name: portainer
|
container_name: portainer
|
||||||
@@ -45,42 +52,69 @@ services:
|
|||||||
- "private-pi:192.168.1.111"
|
- "private-pi:192.168.1.111"
|
||||||
security_opt:
|
security_opt:
|
||||||
- no-new-privileges:true
|
- no-new-privileges:true
|
||||||
|
profiles:
|
||||||
|
- portainer-manager
|
||||||
|
|
||||||
# # Web-based SSH Terminal
|
# --------------------------
|
||||||
# webssh:
|
# Portainer Agent
|
||||||
# image: kuaifan/webssh:arm64
|
# --------------------------
|
||||||
# container_name: webssh
|
portainer_agent:
|
||||||
# ports:
|
image: portainer/agent:latest
|
||||||
# - "7003:5032"
|
container_name: portainer_agent
|
||||||
# environment:
|
restart: unless-stopped
|
||||||
# - TZ=${TZ:-UTC}
|
ports:
|
||||||
# # Restrict to internal network for security
|
- "9001:9001"
|
||||||
# - WEBSSH_ORIGIN_LIST=*
|
volumes:
|
||||||
# - WEBSSH_POLICY=reject
|
- /var/run/docker.sock:/var/run/docker.sock
|
||||||
# networks:
|
- portainer_agent_data:/data
|
||||||
# - monitoring-net
|
networks:
|
||||||
# restart: unless-stopped
|
- monitoring-net
|
||||||
#
|
profiles:
|
||||||
# # Log Management (Optional but useful)
|
- portainer-agent
|
||||||
# dozzle:
|
|
||||||
# image: amir20/dozzle:latest
|
# --------------------------
|
||||||
# container_name: dozzle
|
# Netdata Agent
|
||||||
# ports:
|
# --------------------------
|
||||||
# - "7004:8080"
|
netdata_node:
|
||||||
# volumes:
|
image: netdata/netdata:edge
|
||||||
# - /var/run/docker.sock:/var/run/docker.sock:ro
|
container_name: netdata_agent
|
||||||
# networks:
|
hostname: Server-Pi-Node
|
||||||
# - monitoring-net
|
restart: unless-stopped
|
||||||
# restart: unless-stopped
|
ports:
|
||||||
# environment:
|
- "7003:19999"
|
||||||
# - DOZZLE_NO_ANALYTICS=true
|
cap_add:
|
||||||
|
- SYS_PTRACE
|
||||||
|
- SYS_ADMIN
|
||||||
|
security_opt:
|
||||||
|
- apparmor:unconfined
|
||||||
|
volumes:
|
||||||
|
- netdata_config:/etc/netdata
|
||||||
|
- netdata_lib:/var/lib/netdata
|
||||||
|
- netdata_cache:/var/cache/netdata
|
||||||
|
- /proc:/host/proc:ro
|
||||||
|
- /sys:/host/sys:ro
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
- /:/host/root:ro,rslave
|
||||||
|
- /etc/passwd:/host/etc/passwd:ro
|
||||||
|
- /etc/group:/host/etc/group:ro
|
||||||
|
- /etc/os-release:/host/etc/os-release:ro
|
||||||
|
environment:
|
||||||
|
- NETDATA_STREAM_PARENT=https://netdata.aetoskia.com
|
||||||
|
- NETDATA_CLAIM_TOKEN=baiLAOz-VoOsvVNhG1CLi6j14rL3bmYtWHIEfwfRFRX5VhnQDUxpSvKQ3WsSrH2lRw-obUX2tWxIepN9BrYHUppnXfkPndpLCeeDwH4P0ItJ7twkCL77XPnMcQnL8hqN9pBpFIs
|
||||||
|
- NETDATA_CLAIM_URL=https://app.netdata.cloud
|
||||||
|
- NETDATA_CLAIM_ROOMS=2fe293fe-ebff-43b9-9ad4-0b2206d29d26
|
||||||
|
networks:
|
||||||
|
- monitoring-net
|
||||||
|
profiles:
|
||||||
|
- netdata-node
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
portainer_data:
|
portainer_data:
|
||||||
|
portainer_agent_data:
|
||||||
netdata_config:
|
netdata_config:
|
||||||
netdata_lib:
|
netdata_lib:
|
||||||
netdata_cache:
|
netdata_cache:
|
||||||
|
|
||||||
networks:
|
networks:
|
||||||
monitoring-net:
|
monitoring-net:
|
||||||
driver: bridge
|
driver: bridge
|
||||||
|
|||||||
Reference in New Issue
Block a user