docs: restructure documentation into docs/ directory
- Create docs/ with 7 focused files: architecture, api, scoring, agent, deployment, freeswitch, development - Rewrite README.md as concise overview with links to docs/ - Remove AGENTS.md (content merged into docs/ and README)
This commit is contained in:
+178
@@ -0,0 +1,178 @@
|
||||
# Metrics Agent (pulse-lets-go-agent)
|
||||
|
||||
The agent runs on each PBX node (FreeSWITCH or Asterisk), collects metrics, and publishes them to NATS every 5 seconds.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────┐ pulse.metrics.<id> ┌─────────────────────┐
|
||||
│ pulse-lets-go-agent│──────────────────────────▶ │ NATS Server │
|
||||
│ (on each PBX node) │ every 5 seconds │ (central) │
|
||||
├─────────────────────┤ └────────┬────────────┘
|
||||
│ FreeSWITCH (ESL) │ │
|
||||
│ Asterisk (AMI) │ ▼
|
||||
│ System (/proc) │ ┌─────────────────┐
|
||||
└─────────────────────┘ │ pulse-lets-go │
|
||||
│ (scoring engine) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## NATS Message Format
|
||||
|
||||
```json
|
||||
{
|
||||
"node_id": "pbx-01",
|
||||
"ts": 1719000000,
|
||||
"status": "ok",
|
||||
"active_calls": 42,
|
||||
"max_calls": 250,
|
||||
"idle_cpu": 35,
|
||||
"load_avg": 0.85,
|
||||
"call_failure_rate": 0.5,
|
||||
"sip_gateway": "sip:pbx-01.lan:5060"
|
||||
}
|
||||
```
|
||||
|
||||
- `status`: `"ok"` or any error string (triggers lethal)
|
||||
- `call_failure_rate`: 0.0..100.0 (percentage)
|
||||
- `sip_gateway`: optional, triggers auto-trunk creation on first metric
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
make build-agent
|
||||
# → bin/pulse-lets-go-agent (static binary, ~9 MB)
|
||||
```
|
||||
|
||||
## Configuration (agent.json)
|
||||
|
||||
```json
|
||||
{
|
||||
"node_id": "uc06",
|
||||
"type": "asterisk",
|
||||
"nats_url": "nats://balancer.host:4222",
|
||||
"nats_user": "",
|
||||
"nats_password": "",
|
||||
"interval_sec": 5,
|
||||
"max_calls": 150,
|
||||
"sip_gateway": "sip:uc-pbx.lan:5060",
|
||||
"sip_gateway_auto": false,
|
||||
"failure_window": 1000,
|
||||
"esl": { "host": "127.0.0.1", "port": 8021, "password": "ClueCon" },
|
||||
"ami": { "host": "127.0.0.1", "port": 6154, "username": "admin", "password": "changeme" }
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `node_id` | string | — | Unique node ID (uc06, ses-sip) |
|
||||
| `type` | string | — | `freeswitch` or `asterisk` |
|
||||
| `nats_url` | string | — | NATS server URL |
|
||||
| `nats_user` | string | `""` | NATS username |
|
||||
| `nats_password` | string | `""` | NATS password |
|
||||
| `interval_sec` | int | `5` | Metric publish interval |
|
||||
| `max_calls` | int | `250` | Node capacity (fallback) |
|
||||
| `sip_gateway` | string | `""` | SIP address for auto-trunk creation |
|
||||
| `sip_gateway_auto` | bool | `false` | Auto-detect SIP from PBX |
|
||||
| `failure_window` | int | `1000` | Window size for call_failure_rate |
|
||||
| `esl.*` | object | — | FreeSWITCH ESL settings (`host`, `port`, `password`) |
|
||||
| `ami.*` | object | — | Asterisk AMI settings (`host`, `port`, `username`, `password`) |
|
||||
|
||||
## Collectors
|
||||
|
||||
| Collector | Source | Metrics | PBX |
|
||||
|-----------|--------|---------|-----|
|
||||
| `freeswitch` | ESL: `show channels count`, `json status`, `eval $${idle_cpu}` | active_calls, max_calls, idle_cpu | ✅ |
|
||||
| `asterisk` | AMI: `CoreShowChannels`, `CoreSettings`, Hangup events | active_calls, max_calls, call_failure_rate | ✅ |
|
||||
| `system` | `/proc/loadavg`, `/proc/stat` | load_avg, idle_cpu (fallback) | ✅ |
|
||||
|
||||
### Call Failure Rate
|
||||
|
||||
Sliding window of last N (default 1000) completed calls.
|
||||
|
||||
| Outcome | FS (ESL) | Asterisk (AMI) |
|
||||
|---------|----------|---------------|
|
||||
| Success | `CHANNEL_HANGUP` + `Hangup-Cause: NORMAL_CLEARING` | `Hangup` + `Cause: 16` |
|
||||
| Failure | Any other `Hangup-Cause` | Any other `Cause` |
|
||||
|
||||
## Collector Interface (extensibility)
|
||||
|
||||
```go
|
||||
type Collector interface {
|
||||
Type() string // "freeswitch" | "asterisk"
|
||||
Connect() error // ESL auth / AMI login
|
||||
Collect() (*PBXResult, error) // Collect metrics
|
||||
ListenHangup(onHangup func(bool)) // Subscribe to hangup events
|
||||
Close() // Close connection
|
||||
}
|
||||
```
|
||||
|
||||
New PBX type = new `collector_<type>.go` implementing 5 interface methods. No existing code changes needed.
|
||||
|
||||
## Template Configs
|
||||
|
||||
| File | Target | Type |
|
||||
|------|--------|------|
|
||||
| `contrib/agent-uc.json` | Asterisk UC nodes (05, 06, 66-69) | asterisk |
|
||||
| `contrib/agent-sessip.json` | FreeSWITCH ses-sip (ses-pbx.lan) | freeswitch |
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
# 1. Build
|
||||
make build-agent
|
||||
|
||||
# 2. Copy to node
|
||||
scp bin/pulse-lets-go-agent admin@NODE_IP:/usr/local/bin/
|
||||
|
||||
# 3. Create config
|
||||
scp contrib/agent-uc.json admin@NODE_IP:/etc/pulse-lets-go-agent/agent.json
|
||||
|
||||
# 4. Start
|
||||
ssh admin@NODE_IP "pulse-lets-go-agent -config /etc/pulse-lets-go-agent/agent.json"
|
||||
```
|
||||
|
||||
## Systemd Unit
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/pulse-lets-go-agent.service
|
||||
[Unit]
|
||||
Description=Pulse Lets Go Agent — PBX metrics collector
|
||||
After=network.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/pulse-lets-go-agent -config /etc/pulse-lets-go-agent/agent.json
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# On PBX node — check agent log:
|
||||
journalctl -u pulse-lets-go-agent -f
|
||||
# Expected:
|
||||
# [publisher] NATS connected to nats://host:4222
|
||||
# [ami] connected to 127.0.0.1:6154 (for Asterisk)
|
||||
# [agent] published metric: calls=0/150 load=0.28 cpu=99% status=ok
|
||||
|
||||
# On balancer — check node appeared:
|
||||
curl http://localhost:8080/api/nodes | jq '.[] | {node_id, score, active_calls}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| `nats: no servers available` | NATS not running or unreachable | `systemctl start nats-server` |
|
||||
| `ami auth rejected` | Wrong password or access | Check `/etc/asterisk/manager.conf` |
|
||||
| `esl connect timeout` | ESL not listening on port | `fs_cli -x "load mod_event_socket"` |
|
||||
| `calls=0/0` | max_calls not obtained | Set `max_calls` in `agent.json` |
|
||||
+138
@@ -0,0 +1,138 @@
|
||||
# API Reference
|
||||
|
||||
## Authentication
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `POST` | `/api/auth/login` | – | Login: `{"username":"admin","password":"admin"}` → `access_token` + `refresh_token` |
|
||||
| `POST` | `/api/auth/refresh` | – | Refresh access token using `refresh_token` |
|
||||
|
||||
All endpoints except `/api/route`, `/api/health/*`, `/api/monitoring/*` require `Authorization: Bearer <jwt>`.
|
||||
|
||||
Roles: `admin` (full access), `viewer` (read-only nodes/metrics).
|
||||
|
||||
## Core
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/route` | – | Route a call. Query: `caller_id`, `dest`, `ingress_trunk` |
|
||||
| `GET` | `/api/health` | – | Stats: total nodes, healthy, requests, uptime |
|
||||
| `GET` | `/api/health/live` | – | Liveness probe (always 200) |
|
||||
| `GET` | `/api/health/ready` | – | Readiness probe (NATS + metrics) |
|
||||
|
||||
### Route Response (success)
|
||||
|
||||
```json
|
||||
{ "node_id": "pbx-03", "score": 88, "sip_gateway": "sip:pbx03.lan:5060" }
|
||||
```
|
||||
|
||||
### Route Response (fallback — all nodes unhealthy)
|
||||
|
||||
```json
|
||||
{
|
||||
"fallback": true,
|
||||
"sip_gateway": "sip:operator.lan:5060",
|
||||
"reason": "all_nodes_unhealthy",
|
||||
"nodes": [...]
|
||||
}
|
||||
```
|
||||
|
||||
### Route Response (no nodes registered)
|
||||
|
||||
```json
|
||||
{ "error": "no_nodes_registered" }
|
||||
```
|
||||
|
||||
## Nodes
|
||||
|
||||
| Method | Path | Role | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/nodes` | admin/viewer | List all nodes with scores and state |
|
||||
| `GET` | `/api/nodes/{id}/metrics` | admin/viewer | Metrics history (ring buffer, ~30 min) |
|
||||
| `PUT` | `/api/nodes/{id}/toggle` | admin | `{"disabled":true, "reason":"..."}` |
|
||||
|
||||
## Trunks
|
||||
|
||||
| Method | Path | Role | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/trunks` | admin | All trunks. Query: `type=ingress\|balance\|fallback` |
|
||||
| `POST` | `/api/trunks` | admin | Create trunk |
|
||||
| `PUT` | `/api/trunks/{id}` | admin | Update trunk |
|
||||
| `DELETE` | `/api/trunks/{id}` | admin | Delete trunk |
|
||||
|
||||
Trunk model:
|
||||
```json
|
||||
{
|
||||
"id": "trk-001",
|
||||
"name": "pbx-03 balance",
|
||||
"type": "ingress|balance|fallback",
|
||||
"node_id": "pbx-03",
|
||||
"gateway": "sip:pbx03.lan:5060",
|
||||
"codecs": ["PCMU", "PCMA"],
|
||||
"context": "default",
|
||||
"enabled": true,
|
||||
"description": "",
|
||||
"created_at": "...",
|
||||
"updated_at": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Trunk types:
|
||||
|
||||
| Type | Purpose | Quantity |
|
||||
|------|---------|----------|
|
||||
| `ingress` | Incoming trunk (where calls come from) | 1..N |
|
||||
| `balance` | Destination trunk (bound to a Node) | 1..N |
|
||||
| `fallback` | Operator fallback (when all balance score < 0) | exactly 1 |
|
||||
|
||||
## Users (admin only)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/users` | List users |
|
||||
| `POST` | `/api/users` | Create user |
|
||||
| `PUT` | `/api/users/{id}` | Update user |
|
||||
| `DELETE` | `/api/users/{id}` | Delete user |
|
||||
|
||||
Note: `user-01` (primary admin) cannot be deleted or demoted to viewer.
|
||||
|
||||
## Monitoring (API-key auth)
|
||||
|
||||
Use `X-API-Key` header (not JWT).
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/monitoring/zabbix` | JSON for Zabbix LLD + items |
|
||||
| `GET` | `/api/monitoring/prometheus` | `text/plain` metrics for Prometheus |
|
||||
|
||||
## WebSocket
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `WS` | `/ws/metrics` | Real-time metrics stream for frontend (JWT in query `?token=`) |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
# Route (no auth)
|
||||
curl 'http://localhost:8080/api/route?caller_id=74951234567&dest=123&ingress_trunk=trk-001'
|
||||
# → {"node_id":"pbx-03","score":88,"sip_gateway":"sip:pbx03.lan:5060"}
|
||||
|
||||
# Login
|
||||
TOKEN=$(curl -s -X POST http://localhost:8080/api/auth/login \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"username":"admin","password":"admin"}' | jq -r '.access_token')
|
||||
|
||||
# Nodes with JWT
|
||||
curl -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/nodes
|
||||
|
||||
# Toggle node (admin only)
|
||||
curl -X PUT http://localhost:8080/api/nodes/pbx-01/toggle \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"disabled":true,"reason":"maintenance"}'
|
||||
|
||||
# Prometheus metrics (API-key)
|
||||
curl -H "X-API-Key: $(jq -r '.monitoring_api_key' data/config.json)" \
|
||||
http://localhost:8080/api/monitoring/prometheus
|
||||
```
|
||||
@@ -0,0 +1,117 @@
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
pulse-lets-go — балансировщик телефонной нагрузки на базе FreeSWITCH + NATS.
|
||||
|
||||
Агенты PBX отправляют метрики в NATS каждые 5 секунд. Балансировщик вычисляет weighted score готовности каждой ноды, кэширует лучшую и отдаёт её FreeSWITCH через `/api/route` для следующего звонка.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
┌─────────────────────┐ ESL/AMI ┌──────────────────────┐
|
||||
│ PBX-01 │──────────▶│ pulse-lets-go-agent │
|
||||
│ (FS or Asterisk) │ collect │ (on each PBX node) │
|
||||
├─────────────────────┤ ├──────────────────────┤
|
||||
│ PBX-02 │──────────▶│ pulse-lets-go-agent │
|
||||
│ (FS or Asterisk) │ ├──────────────────────┤
|
||||
├─────────────────────┤ ├──────────────────────┤
|
||||
│ PBX-N │──────────▶│ pulse-lets-go-agent │
|
||||
│ (FS or Asterisk) │ └─────────┬────────────┘
|
||||
└─────────────────────┘ │ pulse.metrics.<id>
|
||||
│ every 5s
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ NATS Server │
|
||||
│ nats://:4222 │
|
||||
└──────┬───────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ pulse-lets-go │
|
||||
│ │
|
||||
│ ┌────────────────┐ │
|
||||
│ │ NATS Sub │ │
|
||||
│ │ (pulse.metrics)│ │
|
||||
│ └────────┬───────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────┐ │
|
||||
│ │ Engine │ │
|
||||
│ │ (scoring) │──│──▶ /api/route
|
||||
│ └────────┬───────┘ │ ↓
|
||||
│ ▼ │ FreeSWITCH bridge
|
||||
│ ┌────────────────┐ │ sip:pbx-03:5060
|
||||
│ │ REST API │──│──▶ /api/nodes
|
||||
│ │ (JWT auth) │ │ /api/trunks
|
||||
│ └────────┬───────┘ │ /api/users
|
||||
│ ▼ │
|
||||
│ ┌────────────────┐ │
|
||||
│ │ WebSocket │──│──▶ SvelteKit UI
|
||||
│ └────────────────┘ │ (ws://:8080/ws/metrics)
|
||||
│ ┌────────────────┐ │
|
||||
│ │ ESL Client │──│──▶ FreeSWITCH ESL
|
||||
│ │ (gateways) │ │ :8021
|
||||
│ └────────────────┘ │
|
||||
│ ┌────────────────┐ │
|
||||
│ │ Monitoring │──│──▶ /api/monitoring/prometheus
|
||||
│ │ (API-key) │ │ /api/monitoring/zabbix
|
||||
│ └────────────────┘ │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Role |
|
||||
|-----------|------|
|
||||
| **pulse-lets-go** | Central balancer: NATS subscriber, scoring engine, REST API, WebSocket hub, ESL client |
|
||||
| **pulse-lets-go-agent** | Per-PBX metrics collector (FreeSWITCH ESL / Asterisk AMI) |
|
||||
| **NATS** | Message bus — `pulse.metrics.<node_id>` subject |
|
||||
| **FreeSWITCH** | Softswitch — uses `route.lua` + `mod_curl` to query `/api/route` |
|
||||
| **SvelteKit UI** | Dashboard with real-time metrics via WebSocket |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Component | Technology |
|
||||
|-----------|-----------|
|
||||
| Backend | Go 1.26, stdlib `net/http` + `http.ServeMux` |
|
||||
| Frontend | SvelteKit 2 + Tailwind CSS 4 + Lucide |
|
||||
| Auth | JWT (golang-jwt/v5, HS256) + bcrypt |
|
||||
| Message bus | NATS (nats.go) |
|
||||
| Storage | JSON files (`data/`) |
|
||||
| ESL | Raw TCP (no external libraries) |
|
||||
| Deployment | Single binary + systemd |
|
||||
|
||||
## Data Flow Summary
|
||||
|
||||
1. **Agent** collects metrics from PBX (ESL/AMI + /proc) every 5s
|
||||
2. **Agent** publishes `pulse.metrics.<node_id>` to NATS
|
||||
3. **Balancer** NATS subscriber receives metrics, updates in-memory node state
|
||||
4. **Scoring engine** recalculates scores, caches best node
|
||||
5. **FreeSWITCH** dialplan calls `/api/route` via `mod_curl`
|
||||
6. **Balancer** returns best node's `sip_gateway` (O(1) from cache)
|
||||
7. **FreeSWITCH** bridges the call to the returned gateway
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
pulse-lets-go/
|
||||
├── cmd/
|
||||
│ ├── pulse-lets-go/ # Main entry point
|
||||
│ ├── pulse-lets-go-agent/ # Metrics collector agent (FS / Asterisk)
|
||||
│ ├── emulator/ # Metrics emulator for testing
|
||||
│ └── siptest/ # SIP tester for e2e checks
|
||||
├── internal/
|
||||
│ ├── api/ # HTTP handlers (auth, nodes, trunks, users, monitoring, ws, route)
|
||||
│ ├── config/ # JSON config manager (atomic save, thread-safe)
|
||||
│ ├── engine/ # Scoring engine + router (sync.RWMutex, best-node cache)
|
||||
│ ├── esl/ # FreeSWITCH ESL client (raw TCP)
|
||||
│ ├── ami/ # Asterisk AMI client (raw TCP)
|
||||
│ ├── log/ # ASCII metrics logger with rotation
|
||||
│ ├── models/ # Data types (NodeMetric, NodeState, Trunk, User, ...)
|
||||
│ └── nats/ # NATS subscriber
|
||||
├── contrib/ # route.lua, agent configs
|
||||
├── deploy/ # systemd unit
|
||||
├── web/ # SvelteKit frontend
|
||||
├── data/ # Runtime JSON files (gitignored)
|
||||
└── docs/ # Documentation
|
||||
```
|
||||
@@ -0,0 +1,176 @@
|
||||
# Deployment
|
||||
|
||||
## Quick Deploy
|
||||
|
||||
```bash
|
||||
make deploy # full deploy to /opt/pulse-lets-go
|
||||
make install-systemd # install systemd unit
|
||||
```
|
||||
|
||||
## Directory Structure (after deploy)
|
||||
|
||||
```
|
||||
/opt/pulse-lets-go/
|
||||
├── bin/pulse-lets-go # Binary
|
||||
├── data/
|
||||
│ ├── config.json # Configuration
|
||||
│ ├── trunks.json # Trunks
|
||||
│ └── users.json # Users
|
||||
├── web/ # SvelteKit static files
|
||||
└── log/
|
||||
└── metrics.YYYY-MM-DD.log # ASCII metrics log
|
||||
```
|
||||
|
||||
## systemd Unit
|
||||
|
||||
```ini
|
||||
# deploy/pulse-lets-go.service
|
||||
[Unit]
|
||||
Description=Pulse Lets Go — Telephone Load Balancer
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
User=pulse
|
||||
Group=pulse
|
||||
ExecStart=/opt/pulse-lets-go/bin/pulse-lets-go
|
||||
WorkingDirectory=/opt/pulse-lets-go
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=yes
|
||||
ReadWritePaths=/opt/pulse-lets-go/data /opt/pulse-lets-go/log
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- `NoNewPrivileges=true` — privilege escalation blocked
|
||||
- `ProtectSystem=strict` — system directories read-only
|
||||
- `ProtectHome=yes` — home directory isolated
|
||||
- `PrivateTmp=true` — isolated /tmp
|
||||
- JWT secret and API-key auto-generated on first run
|
||||
|
||||
## Production Deployment (8 Steps)
|
||||
|
||||
### Step 1 — Deploy to Server
|
||||
|
||||
```bash
|
||||
make deploy
|
||||
scp -r /opt/pulse-lets-go admin@balancer.lan:/opt/
|
||||
scp contrib/route.lua admin@balancer.lan:/tmp/
|
||||
```
|
||||
|
||||
### Step 2 — NATS Server (if not installed)
|
||||
|
||||
```bash
|
||||
ssh admin@balancer.lan
|
||||
sudo mkdir -p /opt/nats
|
||||
# Install nats-server from repository or copy binary
|
||||
# Run: nats-server -p 4222 -D &
|
||||
```
|
||||
|
||||
### Step 3 — Deploy Agent on PBX Nodes
|
||||
|
||||
Repeat for each UC/Asterisk node:
|
||||
|
||||
```bash
|
||||
# Copy binary and config to each PBX:
|
||||
scp bin/pulse-lets-go-agent admin@PBX_IP:/usr/local/bin/
|
||||
scp contrib/agent-uc.json admin@PBX_IP:/etc/pulse-lets-go-agent/agent.json
|
||||
|
||||
# Start:
|
||||
ssh admin@PBX_IP "systemctl start pulse-lets-go-agent"
|
||||
|
||||
# Check log:
|
||||
# journalctl -u pulse-lets-go-agent -f
|
||||
# → [ami] connected to 127.0.0.1:6154
|
||||
# → [agent] published metric: calls=0/150 load=0.28 cpu=99% status=ok
|
||||
```
|
||||
|
||||
### Step 4 — route.lua in FS Scripts
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /etc/freeswitch/scripts
|
||||
sudo cp /tmp/route.lua /etc/freeswitch/scripts/
|
||||
```
|
||||
|
||||
### Step 5 — Dialplan
|
||||
|
||||
Add `pulse_route` extension in `/etc/freeswitch/dialplan/default.xml` before other extensions:
|
||||
|
||||
```xml
|
||||
<extension name="pulse_route">
|
||||
<condition field="destination_number" expression="^(.*)$">
|
||||
<action application="set" data="balancer_url=http://127.0.0.1:8080"/>
|
||||
<action application="lua" data="route.lua"/>
|
||||
</condition>
|
||||
</extension>
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo fs_cli -x "reloadxml"
|
||||
```
|
||||
|
||||
### Step 6 — Create Balance Trunks
|
||||
|
||||
Trunks are created automatically when an agent sends its first metric with `sip_gateway`. Manual creation:
|
||||
|
||||
```bash
|
||||
curl -X POST .../api/trunks \
|
||||
-d '{"name":"uc06","type":"balance","node_id":"uc06","gateway":"sip:uc-pbx.lan:5060","enabled":true}'
|
||||
```
|
||||
|
||||
### Step 7 — Start pulse-lets-go
|
||||
|
||||
```bash
|
||||
# config.json auto-creates on first run in /opt/pulse-lets-go/data/
|
||||
# Add esl block to config.json:
|
||||
# "esl": {"host": "127.0.0.1", "port": 8021, "password": "ClueCon"}
|
||||
/opt/pulse-lets-go/bin/pulse-lets-go
|
||||
```
|
||||
|
||||
### Step 8 — Verify
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8080/api/health | jq '.connections'
|
||||
# → {"nats":"connected","esl":"connected"}
|
||||
curl 'http://127.0.0.1:8080/api/route?caller_id=123&dest=456'
|
||||
# → {"node_id":"pbx-03","score":88,"sip_gateway":...}
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
# 1. Remove pulse_route from dialplan
|
||||
sudo sed -i '/pulse_route/,/<\/extension>/d' /etc/freeswitch/dialplan/default.xml
|
||||
sudo fs_cli -x "reloadxml"
|
||||
|
||||
# 2. Stop pulse-lets-go
|
||||
sudo systemctl stop pulse-lets-go
|
||||
|
||||
# Calls continue through original dialplan without balancer.
|
||||
```
|
||||
|
||||
## Access Permissions (production)
|
||||
|
||||
| Operation | Required Permissions |
|
||||
|-----------|---------------------|
|
||||
| Read FS configs | root or freeswitch |
|
||||
| `fs_cli -x` commands | root or freeswitch |
|
||||
| ESL (port 8021) | ACL in `event_socket.conf.xml` |
|
||||
| Install software | root |
|
||||
| Write to `/etc/freeswitch/scripts/` | root |
|
||||
| `reloadxml` | root or freeswitch |
|
||||
|
||||
## Tested Versions
|
||||
|
||||
| Component | Version |
|
||||
|-----------|---------|
|
||||
| FreeSWITCH | 1.10.12+ |
|
||||
| NATS Server | 2.10+ |
|
||||
| Go | 1.26 |
|
||||
| OS | Linux (CentOS 7+, AlmaLinux, Arch) |
|
||||
@@ -0,0 +1,119 @@
|
||||
# Development
|
||||
|
||||
## Requirements
|
||||
|
||||
- Go 1.26+
|
||||
- Node.js 22+
|
||||
- NATS Server (auto-downloaded via `make nats`)
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
git clone git@git.totmin.ru:en2zmax/pulse-lets-go.git
|
||||
cd pulse-lets-go
|
||||
make build
|
||||
```
|
||||
|
||||
## Makefile Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `make build` | Full build (Go + SvelteKit) |
|
||||
| `make dev` | Go backend + SvelteKit dev server (hot-reload) |
|
||||
| `make test` | Go tests with coverage |
|
||||
| `make test-race` | Tests with race detector |
|
||||
| `make lint` | `go vet ./...` |
|
||||
| `make fmt` | `go fmt ./...` |
|
||||
| `make build-agent` | Build pulse-lets-go-agent |
|
||||
| `make build-siptest` | Build SIP tester |
|
||||
| `make emulate-*` | Metrics emulation (normal, overload, chaos, stale, ...) |
|
||||
| `make siptest-*` | SIP testing (uas, uac, full) |
|
||||
| `make nats` | Start local NATS server |
|
||||
|
||||
## Running Locally
|
||||
|
||||
```bash
|
||||
# Terminal 1: NATS
|
||||
make nats
|
||||
|
||||
# Terminal 2: balancer
|
||||
./bin/pulse-lets-go
|
||||
|
||||
# Terminal 3: metrics emulator (3 healthy nodes)
|
||||
make emulate-normal
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
Before committing:
|
||||
|
||||
```bash
|
||||
make fmt
|
||||
make lint
|
||||
make test
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
pulse-lets-go/
|
||||
├── cmd/
|
||||
│ ├── pulse-lets-go/ # Main entry point
|
||||
│ ├── pulse-lets-go-agent/ # Metrics collector (FS / Asterisk)
|
||||
│ ├── emulator/ # Metrics emulator for testing
|
||||
│ └── siptest/ # SIP tester for e2e checks
|
||||
├── internal/
|
||||
│ ├── api/ # HTTP handlers (auth, nodes, trunks, users, monitoring, ws, route)
|
||||
│ ├── config/ # JSON config manager (atomic save, thread-safe)
|
||||
│ ├── engine/ # Scoring engine + router (sync.RWMutex, best-node cache)
|
||||
│ ├── esl/ # FreeSWITCH ESL client (raw TCP)
|
||||
│ ├── ami/ # Asterisk AMI client (raw TCP)
|
||||
│ ├── log/ # ASCII metrics logger with rotation
|
||||
│ ├── models/ # Data types (NodeMetric, NodeState, Trunk, User, ...)
|
||||
│ └── nats/ # NATS subscriber
|
||||
├── contrib/ # route.lua, agent configs
|
||||
├── deploy/ # systemd unit
|
||||
├── web/ # SvelteKit frontend
|
||||
└── data/ # Runtime JSON files (gitignored)
|
||||
```
|
||||
|
||||
## Adding a New Endpoint
|
||||
|
||||
1. Define model in `internal/models/types.go`
|
||||
2. Implement handler in `internal/api/<name>.go`
|
||||
3. Register route in `internal/api/router.go` with required middleware (auth, admin)
|
||||
4. Add tests in `internal/engine/engine_test.go` if scoring is affected
|
||||
|
||||
## Commit Messages
|
||||
|
||||
[Conventional Commits](https://www.conventionalcommits.org/) format:
|
||||
|
||||
```
|
||||
feat: add support for custom scoring weights
|
||||
fix: handle nil ESL client in route handler
|
||||
docs: add API reference table to README
|
||||
refactor: extract scoring logic into separate method
|
||||
test: add lethal condition tests for stale nodes
|
||||
chore: update Go dependencies
|
||||
```
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. `cmd/pulse-lets-go/main.go` + Go module + пустая структура `internal/`
|
||||
2. `internal/models/` — все типы данных
|
||||
3. `internal/config/` — чтение/запись JSON, атомарное сохранение
|
||||
4. `internal/nats/` — подписка, in-memory store
|
||||
5. `internal/engine/` — scorer + router
|
||||
6. `internal/api/` — все хендлеры
|
||||
7. `internal/log/` — ASCII-логгер + ring buffer
|
||||
8. Web-фронт (SvelteKit)
|
||||
9. Makefile + systemd unit
|
||||
10. Тесты + graceful shutdown
|
||||
|
||||
## PR Process
|
||||
|
||||
1. Branch from `main`, named `feature/`, `fix/`, or `docs/`
|
||||
2. One PR — one logical change
|
||||
3. All tests pass (`make test`)
|
||||
4. Code passes `make lint`
|
||||
5. If API changes — update docs
|
||||
@@ -0,0 +1,104 @@
|
||||
# FreeSWITCH Integration
|
||||
|
||||
pulse-lets-go routes calls through FreeSWITCH via `mod_curl` → Lua dialplan → `/api/route`.
|
||||
|
||||
## Dialplan (route.lua)
|
||||
|
||||
FreeSWITCH uses a Lua script in its dialplan. The script calls `/api/route` via `mod_curl` and bridges to the returned gateway.
|
||||
|
||||
```lua
|
||||
-- contrib/route.lua (simplified)
|
||||
api = freeswitch.API()
|
||||
caller_id = session:getVariable("caller_id_number") or ""
|
||||
dest = session:getVariable("destination_number") or ""
|
||||
url = "http://localhost:8080/api/route?caller_id=" .. caller_id .. "&dest=" .. dest
|
||||
|
||||
raw = api:execute("curl", url)
|
||||
body = raw:match("\r?\n\r?\n(.+)") or raw
|
||||
|
||||
local ok, route = pcall(cjson.decode, body)
|
||||
if not ok then session:hangup("NORMAL_TEMPORARY_FAILURE") return end
|
||||
|
||||
if route.fallback then
|
||||
session:bridge(route.sip_gateway) -- operator
|
||||
elseif route.sip_gateway then
|
||||
session:bridge(route.sip_gateway) -- best node
|
||||
else
|
||||
session:hangup("NORMAL_TEMPORARY_FAILURE")
|
||||
end
|
||||
```
|
||||
|
||||
## Dialplan Configuration
|
||||
|
||||
```xml
|
||||
<extension name="route_call">
|
||||
<condition field="destination_number" expression="^.*$">
|
||||
<action application="set" data="balancer_url=http://balancer.lan:8080"/>
|
||||
<action application="set" data="ingress_trunk=trk-001"/>
|
||||
<action application="lua" data="route.lua"/>
|
||||
</condition>
|
||||
</extension>
|
||||
```
|
||||
|
||||
Reload dialplan:
|
||||
```bash
|
||||
sudo fs_cli -x "reloadxml"
|
||||
```
|
||||
|
||||
## ESL (Gateway Management)
|
||||
|
||||
When `esl.host` is configured in `config.json`, the balancer:
|
||||
- Connects to FreeSWITCH ESL (`:8021`)
|
||||
- Synchronizes ingress trunks as Sofia gateways
|
||||
- Listens for gateway registration events
|
||||
- Streams gateway status to WebSocket frontend
|
||||
|
||||
### Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"esl": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8021,
|
||||
"password": "ClueCon",
|
||||
"password_env": "",
|
||||
"sofia_profile": "external",
|
||||
"gateway_prefix": "pulse"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `password_env` — alternative: name of environment variable containing password
|
||||
- `sofia_profile` — Sofia profile to manage (default: `external`)
|
||||
- `gateway_prefix` — prefix for auto-created gateways (default: `pulse`)
|
||||
|
||||
## Prerequisites (before deployment)
|
||||
|
||||
| # | Check | Command | Expected |
|
||||
|:-:|-------|---------|----------|
|
||||
| 1 | **ESL running** | `nc -z 127.0.0.1 8021 && echo OK` | `OK` |
|
||||
| 2 | **ESL password** | `echo "auth ClueCon" \| nc 127.0.0.1 8021` | `+OK accepted` |
|
||||
| 3 | **mod_curl loaded** | `fs_cli -x "show modules" \| grep mod_curl` | `<load module="mod_curl"/>` |
|
||||
| 4 | **mod_lua loaded** | `fs_cli -x "show modules" \| grep mod_lua` | `<load module="mod_lua"/>` |
|
||||
| 5 | **ESL ACL** | ACL in `event_socket.conf.xml` allows 127.0.0.1 | `localhost` or `127.0.0.1` in ACL |
|
||||
|
||||
## Loading mod_curl Without Restart
|
||||
|
||||
```bash
|
||||
fs_cli -x "load mod_curl"
|
||||
# Add to modules.conf.xml for persistence:
|
||||
# <load module="mod_curl"/>
|
||||
```
|
||||
|
||||
## Rollback (if route.lua breaks calls)
|
||||
|
||||
```bash
|
||||
# 1. Remove pulse_route from dialplan
|
||||
sudo sed -i '/pulse_route/,/<\/extension>/d' /etc/freeswitch/dialplan/default.xml
|
||||
sudo fs_cli -x "reloadxml"
|
||||
|
||||
# 2. Stop pulse-lets-go
|
||||
sudo systemctl stop pulse-lets-go
|
||||
|
||||
# Calls continue through original dialplan without balancer.
|
||||
```
|
||||
@@ -0,0 +1,71 @@
|
||||
# Scoring Engine
|
||||
|
||||
Вычисляет score (0..100) для каждой ноды. Ноды со score = -100 исключаются из роутинга.
|
||||
|
||||
## Lethal Conditions (score → -100)
|
||||
|
||||
Если хотя бы одно условие истинно — нода исключается:
|
||||
If at least one is true — node is excluded from routing.
|
||||
|
||||
| Condition | Threshold |
|
||||
|-----------|-----------|
|
||||
| `status != "ok"` | Any status other than `ok` |
|
||||
| Stale | > `stale_threshold_sec` (default 20s) |
|
||||
| `active_calls >= max_calls` | 100% capacity used |
|
||||
| `idle_cpu < idle_cpu_min` | < 5% |
|
||||
| `call_failure_rate > call_failure_rate_lethal` | > 15% |
|
||||
| `disabled == true` | Manual admin disable |
|
||||
|
||||
## Weighted Score (0..100)
|
||||
|
||||
```
|
||||
call_score = clamp(100 - (active_calls / max_calls * 100), 0, 100)
|
||||
load_score = clamp(100 - (load_avg * 50), 0, 100)
|
||||
idle_score = clamp(idle_cpu, 0, 100)
|
||||
fail_score = clamp(100 - call_failure_rate, 0, 100)
|
||||
|
||||
score = call_score × 0.40 +
|
||||
load_score × 0.30 +
|
||||
idle_score × 0.20 +
|
||||
fail_score × 0.10
|
||||
```
|
||||
|
||||
Weight defaults (configurable in `config.json`): call 40%, load 30%, idle 20%, fail 10%.
|
||||
|
||||
## Example
|
||||
|
||||
Node `pbx-01`: `active_calls=42`, `max_calls=250`, `load_avg=0.85`, `idle_cpu=35`, `fail_rate=0.5`
|
||||
|
||||
```
|
||||
call_score = clamp(100 - (42/250)*100, 0, 100) = 83.2
|
||||
load_score = clamp(100 - 0.85*50, 0, 100) = 57.5
|
||||
idle_score = clamp(35, 0, 100) = 35.0
|
||||
fail_score = clamp(100 - 0.5, 0, 100) = 99.5
|
||||
|
||||
score = 83.2×0.40 + 57.5×0.30 + 35.0×0.20 + 99.5×0.10 = 70.1
|
||||
```
|
||||
|
||||
## Caching
|
||||
|
||||
`best_node_id`, `best_score`, and `fallback_active` are recalculated on every metric update.
|
||||
`/api/route` reads them under RLock — O(1) constant time.
|
||||
|
||||
## Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"stale_threshold_sec": 20,
|
||||
"scoring": {
|
||||
"idle_cpu_min": 5,
|
||||
"call_failure_rate_lethal": 15.0,
|
||||
"weights": {
|
||||
"call_score": 0.40,
|
||||
"load_score": 0.30,
|
||||
"idle_score": 0.20,
|
||||
"fail_score": 0.10
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Weights must sum to 1.0. All thresholds configurable in `config.json`.
|
||||
Reference in New Issue
Block a user