Maksim Totmin 727fd1195a feat: add route_fallbacks metric to Prometheus and Zabbix monitoring
- Prometheus: new pulse_route_fallbacks_total counter
- Zabbix: extend response with route_requests_total, route_fallbacks_total, uptime_seconds
- Docs: update Prometheus metrics table and Zabbix response example
2026-06-25 23:27:12 +07:00

316 lines
9.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# API Reference
## Authentication
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `POST` | `/api/auth/login` | | Login: `{"username":"admin","password":"admin"}``access_token` + `refresh_token` |
| `POST` | `/api/auth/refresh` | | Refresh access token using `refresh_token` |
All endpoints except `/api/route`, `/api/health/*`, `/api/monitoring/*` require `Authorization: Bearer <jwt>`.
Roles: `admin` (full access), `viewer` (read-only nodes/metrics).
## Core
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/api/route` | | Route a call. Query: `caller_id`, `dest`, `ingress_trunk` |
| `PUT` | `/api/route/mode` | admin | Set balancing mode: `{"mode":"best\|weighted_random"}` — см. [Routing](routing.md#balancing-modes) |
| `GET` | `/api/health` | | Full health status with connections and stats |
| `GET` | `/api/health/live` | | Liveness probe — always 200 `{"status":"alive"}` |
| `GET` | `/api/health/ready` | | Readiness — 200 `{"status":"ready"}` или 503 `{"reason":"nats_disconnected"}` |
### Health Response
```json
{
"status": "ok",
"version": "1.0.0",
"connections": {
"nats": "connected|disconnected",
"esl": {
"status": "disabled|connected|disconnected",
"host": "127.0.0.1",
"uptime_seconds": 3600,
"reconnects": 0,
"gateway_ops": 42,
"events_recv": 15000
}
},
"stats": {
"total_nodes": 3,
"healthy_nodes": 2,
"route_requests": 5000,
"route_fallbacks": 10,
"uptime_seconds": 7200
}
}
```
### Route Response (success)
```json
{ "node_id": "pbx-03", "score": 88, "sip_gateway": "sip:pbx03.lan:5060", "matched_rule": "none" }
```
`matched_rule` — имя сработавшего routing rule, либо `"none"` если применился глобальный scoring.
### Route Mode Response
`PUT /api/route/mode` возвращает текущий режим:
```json
{ "mode": "best" }
```
### Route Response (fallback — all nodes unhealthy)
```json
{
"fallback": true,
"sip_gateway": "sip:operator.lan:5060",
"reason": "all_nodes_unhealthy",
"matched_rule": "my-pool-rule",
"nodes": [
{ "id": "pbx-01", "score": -100, "reason": "disabled", "status": "disabled" },
{ "id": "pbx-02", "score": -1, "reason": "stale", "status": "stale" },
{ "id": "pbx-03", "score": 70, "reason": "", "status": "healthy" }
]
}
```
### Route Response (no nodes registered)
```json
{ "error": "no_nodes_registered", "matched_rule": "none" }
```
## Nodes
| Method | Path | Role | Description |
|--------|------|------|-------------|
| `GET` | `/api/nodes` | admin/viewer | List all nodes with scores and state |
| `GET` | `/api/nodes/{id}/metrics` | admin/viewer | Metrics history (ring buffer, ~30 min) |
| `PUT` | `/api/nodes/{id}/toggle` | admin | `{"disabled":true, "reason":"..."}` — returns updated `NodeInfo` |
### NodeInfo (GET /api/nodes)
```json
{
"node_id": "pbx-01",
"ts": 1719000000,
"status": "ok",
"active_calls": 42,
"max_calls": 250,
"idle_cpu": 35.0,
"load_avg": 0.85,
"call_failure_rate": 0.5,
"disabled": false,
"disabled_reason": "",
"score": 88.0,
"lethal_reason": "",
"is_stale": false,
"seconds_ago": 3.0
}
```
### Node Metrics (GET /api/nodes/{id}/metrics)
История метрик ноды из ring buffer (~30 мин, шаг 5 с):
```json
{
"node_id": "pbx-01",
"snapshots": [
{ "active_calls": 42, "load_avg": 0.85, "score": 88.0, ... }
],
"count": 360
}
```
## Trunks
| Method | Path | Role | Description |
|--------|------|------|-------------|
| `GET` | `/api/trunks` | admin | All trunks. Query: `type=ingress\|balance\|fallback` |
| `POST` | `/api/trunks` | admin | Create trunk |
| `PUT` | `/api/trunks/{id}` | admin | Update trunk |
| `DELETE` | `/api/trunks/{id}` | admin | Delete trunk |
Trunk model:
```json
{
"id": "trk-001",
"name": "pbx-03 balance",
"type": "ingress|balance|fallback",
"node_id": "pbx-03",
"gateway": "sip:pbx03.lan:5060",
"codecs": ["PCMU", "PCMA"],
"context": "default",
"enabled": true,
"description": "",
"created_at": "...",
"updated_at": "..."
}
```
Trunk types:
| Type | Purpose | Quantity |
|------|---------|----------|
| `ingress` | Incoming trunk (where calls come from) | 1..N |
| `balance` | Destination trunk (bound to a Node) | 1..N |
| `fallback` | Operator fallback (when all balance score < 0) | exactly 1 |
## Routing Rules
| Method | Path | Role | Description |
|--------|------|------|-------------|
| `GET` | `/api/routing/rules` | admin/viewer | Список правил с счётчиками hits |
| `POST` | `/api/routing/rules` | admin | Создать правило |
| `PUT` | `/api/routing/rules/{id}` | admin | Обновить правило |
| `DELETE` | `/api/routing/rules/{id}` | admin | Удалить правило |
| `PUT` | `/api/routing/rules/{id}/toggle` | admin | Вкл/выкл: `{"enabled":true}` |
| `PUT` | `/api/routing/rules/reorder` | admin | Новый порядок: `["rr-abc", "rr-def", ...]` |
Поля правила см. [docs/routing.md](routing.md#routing-rules). Ответы содержат полную модель правила.
## Users (admin only)
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/users` | List users (без password_hash) |
| `POST` | `/api/users` | `{"username":"...","password":"...","role":"admin\|viewer"}` |
| `PUT` | `/api/users/{id}` | Partial update: `{"username":"...","role":"viewer"}` |
| `DELETE` | `/api/users/{id}` | Удалить (204 No Content) |
Защиты:
- `user-01` нельзя удалить или понизить до viewer
- Нельзя удалить последнего администратора
- GET-ответ исключает поле `password_hash`
## Monitoring (API-key auth)
Use `X-API-Key` header (not JWT).
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/monitoring/zabbix` | JSON для Zabbix LLD |
| `GET` | `/api/monitoring/prometheus` | `text/plain` метрики для Prometheus |
### Zabbix Response
```json
{
"data": [
{
"{#NODE_ID}": "pbx-01",
"{#STATUS}": "ok",
"{#ACTIVE_CALLS}": 42,
"{#MAX_CALLS}": 250,
"{#IDLE_CPU}": 35.0,
"{#LOAD_AVG}": 0.85,
"{#CALL_FAILURE_RATE}": 0.5,
"{#SCORE}": 88.0,
"{#DISABLED}": false,
"{#IS_STALE}": false,
"{#SECONDS_AGO}": 3.0
}
],
"route_requests_total": 5000,
"route_fallbacks_total": 42,
"uptime_seconds": 7200
}
```
### Prometheus Metrics
| Метрика | Type | Labels |
|---------|------|--------|
| `pulse_route_requests_total` | counter | |
| `pulse_route_fallbacks_total` | counter | |
| `pulse_nodes_total` | gauge | |
| `pulse_nodes_healthy` | gauge | |
| `pulse_uptime_seconds` | gauge | |
| `pulse_node_active_calls` | gauge | `node` |
| `pulse_node_max_calls` | gauge | `node` |
| `pulse_node_idle_cpu` | gauge | `node` |
| `pulse_node_load_avg` | gauge | `node` |
| `pulse_node_call_failure_rate` | gauge | `node` |
| `pulse_node_score` | gauge | `node` |
| `pulse_node_seconds_ago` | gauge | `node` |
| `pulse_node_disabled` | gauge | `node` (0/1) |
| `pulse_node_stale` | gauge | `node` (0/1) |
| `pulse_node_status_ok` | gauge | `node` (0/1) |
## WebSocket
| Method | Path | Description |
|--------|------|-------------|
| `WS` | `/ws/metrics` | Real-time metrics (JWT в query `?token=`) |
Типы сообщений (поле `type`):
| Тип | Payload | Описание |
|-----|---------|----------|
| `nodes_update` | `[]NodeInfo` | Обновление метрик всех нод (каждые 5 с) |
| `balancer_update` | `BalancerInfo` | Состояние балансировщика (uptime, mode, ESL, NATS) |
| `gateway_update` | `{"gateway","trunk_id","status"}` | Статус FS-gateway (up/down) |
## Examples
```bash
# Route (no auth)
curl 'http://localhost:8080/api/route?caller_id=74951234567&dest=123&ingress_trunk=trk-001'
# → {"node_id":"pbx-03","score":88,"sip_gateway":"sip:pbx03.lan:5060"}
# Login
TOKEN=$(curl -s -X POST http://localhost:8080/api/auth/login \
-H 'Content-Type: application/json' \
-d '{"username":"admin","password":"admin"}' | jq -r '.access_token')
# Nodes with JWT
curl -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/nodes
# Toggle node (admin only)
curl -X PUT http://localhost:8080/api/nodes/pbx-01/toggle \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{"disabled":true,"reason":"maintenance"}'
# Route mode (admin)
curl -X PUT http://localhost:8080/api/route/mode \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{"mode":"weighted_random"}'
# → {"mode":"weighted_random"}
# Routing rules (admin)
curl -X POST http://localhost:8080/api/routing/rules \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{
"name":"VIP pool",
"priority":10,
"match_caller_id":"^7495",
"action_type":"pool",
"action_pool_ids":["pbx-01","pbx-02"]
}'
# Toggle rule (admin)
curl -X PUT http://localhost:8080/api/routing/rules/rr-abc123/toggle \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{"enabled":false}'
# Reorder rules (admin)
curl -X PUT http://localhost:8080/api/routing/rules/reorder \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '["rr-def456", "rr-abc123"]'
# Prometheus metrics (API-key)
curl -H "X-API-Key: $(jq -r '.monitoring_api_key' data/config.json)" \
http://localhost:8080/api/monitoring/prometheus
```