addrkit: deterministic address resolution service (Go)

- domain: address normalization/parsing, matching with explicit statuses
- adapters: SQLite registry store, ArcGIS outages client, optional LLM rewriter, HTTP API
- seed pipeline with duplicate handling and unparsed reporting
- CLI (seed/verify/resolve), systemd unit, OpenAPI contract, tests
This commit is contained in:
Maksim Totmin
2026-10-09 21:40:46 +07:00
commit 7b152c7fb3
34 changed files with 3446 additions and 0 deletions
+14
View File
@@ -0,0 +1,14 @@
# Build artifacts
bin/
# Local config and data (never committed)
config.json
*.db
*.db-wal
*.db-shm
*.ndjson
# Editor / OS
.DS_Store
.idea/
.vscode/
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 addrkit contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+41
View File
@@ -0,0 +1,41 @@
GO ?= go
BIN_DIR ?= bin
.PHONY: all build test vet fmt tidy run clean seed
all: build
## build: compile the service and the CLI as static binaries
build:
CGO_ENABLED=0 $(GO) build -trimpath -o $(BIN_DIR)/addrkitd ./cmd/addrkitd
CGO_ENABLED=0 $(GO) build -trimpath -o $(BIN_DIR)/addrkit ./cmd/addrkit
## test: run the full test suite with the race detector
test:
$(GO) test -race ./...
## vet: static analysis
vet:
$(GO) vet ./...
## fmt: format all Go sources
fmt:
gofmt -l -w .
## tidy: sync go.mod/go.sum
tidy:
$(GO) mod tidy
## run: build and start the service with the local config
run: build
$(BIN_DIR)/addrkitd -config config.json
## seed: build registry.db from a registry dump (override IN=/OUT=)
IN ?= registry.ndjson
OUT ?= registry.db
seed:
$(BIN_DIR)/addrkit seed -in $(IN) -out $(OUT)
## clean: remove build artifacts
clean:
rm -rf $(BIN_DIR)
+117
View File
@@ -0,0 +1,117 @@
# addrkit
**Детерминированный сервис нормализации и поиска адресов** с подключаемым
внешним слоем отключений. Разбирает вольные реплики (дроби, литеры, корпуса,
строения, опечатки), сопоставляет их с каноническим реестром и отдаёт ответ
**только из данных** — LLM не участвует в пути корректности.
Проект на Go, живёт офлайн (один статический бинарь + SQLite), без MCP в
рантайме.
## Зачем
Существующая логика поиска адреса в диалоговом сценарии страдала от
нечёткого поиска по всей строке (`fuzziness: AUTO`, `size:1`), из-за чего
дробь/корпус/литера могли дать неверный дом. `addrkit` разделяет задачу на
компоненты, сопоставляет дом строго и вместо угадывания возвращает явный
статус (`partial`/`ambiguous`).
## Архитектура
Гексагональная: домен чистый и без I/O, всё внешнее — в адаптерах.
```
cmd/addrkitd сервис (HTTP)
cmd/addrkit CLI: seed | verify | resolve
internal/
domain/address нормализация и разбор адреса (чистые функции)
domain/match сопоставление с индексом, статусы
domain/outages выборка отключений по дому (чистая логика)
adapters/registrystore SQLite (modernc, без CGO)
adapters/arcgis клиент портала (timeout/retry)
adapters/llm опциональный rewriter (OpenAI-совместимый)
adapters/httpapi версионированный JSON API
config конфиг + ${VAR}
seed сборка индекса из дампа реестра
```
Поток запроса:
```
"2-я Станционная улица 1/1 стр."
-> address.Parse -> {ordinal:2, name:станционная, house:1, frac:1}
-> match.Resolve -> exact street -> exact house -> FOUND
-> (опц.) outages.Source -> фильтр по дому -> ответ
```
## Быстрый старт
```bash
make build
# 1. Собрать индекс из дампа реестра (Bulk NDJSON или JSONL).
./bin/addrkit seed -in registry.ndjson -out registry.db -version 2026-10-01
# 2. Запустить сервис.
cp config.example.json config.json
ADDRKIT_PORTAL_URL='https://map.example/.../101/query' ./bin/addrkitd -config config.json
# 3. Проверить.
curl -s localhost:6171/v1/resolve -d '{"address":"2-я Станционная улица 1/1 стр."}'
curl -s localhost:6171/v1/outages -d '{"address":"станционная 30а к9/1"}'
```
CLI без сервиса:
```bash
./bin/addrkit resolve -db registry.db "кошурникова 11/1"
./bin/addrkit verify -db registry.db
```
## API v1
| Метод | Путь | Назначение |
|---|---|---|
| POST | `/v1/resolve` | адрес → `{status, address, district, confidence, candidates[]}` |
| POST | `/v1/outages` | адрес → канонический адрес + отключения по дому |
| GET | `/v1/healthz` | живость |
| GET | `/v1/readyz` | готовность (индекс загружен) |
| GET | `/v1/version` | версия данных |
Статусы: `found`, `partial`, `ambiguous`, `not_found`, `portal_timeout`,
`infra_error`.
## Конфигурация
`config.json` (см. `config.example.json`). Значения поддерживают `${VAR}`;
незаданная переменная = ошибка запуска (fail closed). Секреты — только из
окружения.
## Принятые решения
- **Детерминизм.** Один вход — один выход, порядок кандидатов стабилен.
`resolve` не зависит от сети.
- **Дом сравнивается строго.** Литера/дробь/корпус должны совпасть; иначе
`partial`, а не «похожий» дом.
- **LLM — опция, выключена по умолчанию.** `adapters/llm` лишь предлагает
варианты записи адреса; результат всегда берётся из реестра, поэтому модель
не может «выдумать» адрес.
- **Данные.** Сервис владеет своим индексом, построенным из дампа реестра.
Источник истины и регламент обновления — предмет отдельного согласования.
- **Часовой пояс.** Даты отключений трактуются в `Asia/Novosibirsk` (UTC+7).
## Тесты
```bash
make test # go test -race ./...
```
Проверка парсера на реальном реестре (вне репозитория):
```bash
ADDRKIT_REGISTRY=/path/to/address_registry.ndjson \
go test ./internal/seed -run TestRealRegistryParseRate -v
```
На дампе из 73 936 строк парсер распознаёт **99.98 %** (14 строк — заведомо
битые форматы; они не угадываются, а попадают в отчёт `stats.unparsed_sample`).
+146
View File
@@ -0,0 +1,146 @@
// Command addrkit is the operator CLI: it seeds the canonical registry from a
// registry dump, verifies a built database and resolves a single address for
// quick checks.
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"os"
"git.totmin.ru/en2zmax/addrkit/internal/adapters/registrystore"
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
"git.totmin.ru/en2zmax/addrkit/internal/seed"
)
func main() {
if len(os.Args) < 2 {
usage()
os.Exit(2)
}
switch os.Args[1] {
case "seed":
os.Exit(runSeed(os.Args[2:]))
case "verify":
os.Exit(runVerify(os.Args[2:]))
case "resolve":
os.Exit(runResolve(os.Args[2:]))
case "help", "-h", "--help":
usage()
os.Exit(0)
default:
fmt.Fprintf(os.Stderr, "unknown command %q\n", os.Args[1])
usage()
os.Exit(2)
}
}
func usage() {
fmt.Fprint(os.Stderr, `addrkit — address registry toolkit
Usage:
addrkit seed -in registry.ndjson -out registry.db [-version 2026-10-09]
addrkit verify -db registry.db
addrkit resolve -db registry.db "2-я Станционная улица 1/1 стр."
`)
}
func runSeed(args []string) int {
fs := flag.NewFlagSet("seed", flag.ExitOnError)
in := fs.String("in", "", "input registry dump (Bulk NDJSON or JSONL)")
out := fs.String("out", "", "output SQLite database")
version := fs.String("version", "", "data version (defaults to input file mtime)")
_ = fs.Parse(args)
if *in == "" || *out == "" {
fmt.Fprintln(os.Stderr, "seed: -in and -out are required")
return 2
}
if *version == "" {
if fi, err := os.Stat(*in); err == nil {
*version = fi.ModTime().UTC().Format("20060102T150405Z")
}
}
f, err := os.Open(*in)
if err != nil {
fmt.Fprintf(os.Stderr, "seed: open input: %v\n", err)
return 1
}
defer f.Close()
sources, err := seed.ReadBulkNDJSON(f)
if err != nil {
fmt.Fprintf(os.Stderr, "seed: read: %v\n", err)
return 1
}
entries, stats := seed.Build(sources)
if stats.Parsed == 0 {
fmt.Fprintln(os.Stderr, "seed: no parseable entries; refusing to write empty index")
return 1
}
if err := registrystore.Save(*out, *version, entries); err != nil {
fmt.Fprintf(os.Stderr, "seed: save: %v\n", err)
return 1
}
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
_ = enc.Encode(map[string]any{
"output": *out,
"version": *version,
"entries": len(entries),
"stats": stats,
})
return 0
}
func runVerify(args []string) int {
fs := flag.NewFlagSet("verify", flag.ExitOnError)
db := fs.String("db", "", "SQLite database")
_ = fs.Parse(args)
if *db == "" {
fmt.Fprintln(os.Stderr, "verify: -db is required")
return 2
}
version, entries, err := registrystore.Load(*db)
if err != nil {
fmt.Fprintf(os.Stderr, "verify: %v\n", err)
return 1
}
streets := make(map[string]struct{})
for _, e := range entries {
streets[e.StreetKey] = struct{}{}
}
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
_ = enc.Encode(map[string]any{
"version": version,
"entries": len(entries),
"distinct_streets": len(streets),
})
return 0
}
func runResolve(args []string) int {
fs := flag.NewFlagSet("resolve", flag.ExitOnError)
db := fs.String("db", "", "SQLite database")
_ = fs.Parse(args)
rest := fs.Args()
if *db == "" || len(rest) == 0 {
fmt.Fprintln(os.Stderr, `resolve: -db and an address are required`)
return 2
}
version, entries, err := registrystore.Load(*db)
if err != nil {
fmt.Fprintf(os.Stderr, "resolve: %v\n", err)
return 1
}
resolver := match.NewResolver(match.NewIndex(version, entries, match.DefaultOptions()), nil)
res := resolver.Resolve(context.Background(), rest[0])
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
_ = enc.Encode(res)
return 0
}
+120
View File
@@ -0,0 +1,120 @@
// Command addrkitd runs the address resolution service.
//
// It loads the canonical registry into memory, wires the optional adapters
// (ArcGIS portal, LLM rewriter) and serves the versioned JSON API. The process
// fails closed: if the registry cannot be loaded it exits rather than serving
// an empty index.
package main
import (
"context"
"errors"
"flag"
"log/slog"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/adapters/arcgis"
"git.totmin.ru/en2zmax/addrkit/internal/adapters/httpapi"
"git.totmin.ru/en2zmax/addrkit/internal/adapters/llm"
"git.totmin.ru/en2zmax/addrkit/internal/adapters/registrystore"
"git.totmin.ru/en2zmax/addrkit/internal/config"
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
)
func main() {
cfgPath := flag.String("config", "config.json", "path to the JSON config file")
flag.Parse()
cfg, err := config.Load(*cfgPath)
if err != nil {
slog.Error("config", "error", err)
os.Exit(1)
}
logger := newLogger(cfg.LogLevel)
version, entries, err := registrystore.Load(cfg.DataPath)
if err != nil {
// Fail closed: no registry, no service.
logger.Error("load registry", "error", err, "path", cfg.DataPath)
os.Exit(1)
}
logger.Info("registry loaded", "version", version, "entries", len(entries))
index := match.NewIndex(version, entries, match.Options{
FuzzyThreshold: cfg.Resolve.FuzzyThreshold,
AmbiguityMargin: cfg.Resolve.AmbiguityMargin,
MaxCandidates: cfg.Resolve.MaxCandidates,
})
var rewriter match.Rewriter
if cfg.LLM.Enabled {
rewriter = llm.New(cfg.LLM)
logger.Info("llm rewriter enabled", "url", cfg.LLM.URL, "model", cfg.LLM.Model)
}
resolver := match.NewResolver(index, rewriter)
portal := arcgis.New(
cfg.Portal.URL,
time.Duration(cfg.Portal.TimeoutSeconds)*time.Second,
cfg.Portal.Retries,
)
loc := operationalLocation()
srv := httpapi.New(resolver, portal, logger, loc)
httpServer := &http.Server{
Addr: cfg.Addr,
Handler: srv.Handler(),
ReadHeaderTimeout: 5 * time.Second,
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
go func() {
logger.Info("listening", "addr", cfg.Addr, "tz", loc.String())
if err := httpServer.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
logger.Error("serve", "error", err)
stop()
}
}()
<-ctx.Done()
logger.Info("shutting down")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := httpServer.Shutdown(shutdownCtx); err != nil {
logger.Error("shutdown", "error", err)
}
}
func newLogger(level string) *slog.Logger {
var lvl slog.Level
switch level {
case "debug":
lvl = slog.LevelDebug
case "warn":
lvl = slog.LevelWarn
case "error":
lvl = slog.LevelError
default:
lvl = slog.LevelInfo
}
return slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: lvl}))
}
// operationalLocation returns Asia/Novosibirsk when the tz database is
// available and falls back to a fixed UTC+7 offset otherwise (containers
// without tzdata).
func operationalLocation() *time.Location {
if loc, err := time.LoadLocation("Asia/Novosibirsk"); err == nil {
return loc
}
return time.FixedZone("Asia/Novosibirsk", 7*3600)
}
+22
View File
@@ -0,0 +1,22 @@
{
"addr": ":6171",
"data_path": "registry.db",
"log_level": "info",
"resolve": {
"fuzzy_threshold": 0.82,
"ambiguity_margin": 0.03,
"max_candidates": 5
},
"portal": {
"url": "${ADDRKIT_PORTAL_URL}",
"timeout_seconds": 2,
"retries": 1
},
"llm": {
"enabled": false,
"url": "${ADDRKIT_LLM_URL}",
"token": "${ADDRKIT_LLM_TOKEN}",
"model": "qwen3-14b-awq",
"timeout_seconds": 3
}
}
+28
View File
@@ -0,0 +1,28 @@
[Unit]
Description=addrkit address resolution service
Documentation=file:///opt/addrkit/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=lexicom
Group=lexicom
WorkingDirectory=/opt/addrkit
# Environment carries the ${VAR} secrets referenced by config.json.
EnvironmentFile=-/etc/addrkit/env
ExecStart=/opt/addrkit/bin/addrkitd -config /opt/addrkit/config.json
Restart=always
RestartSec=5
# Defence in depth: the service only reads its own directory.
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadOnlyPaths=/opt/addrkit
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
+20
View File
@@ -0,0 +1,20 @@
module git.totmin.ru/en2zmax/addrkit
go 1.27.1
require (
golang.org/x/text v0.42.0
modernc.org/sqlite v1.60.1
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.48.0 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
+52
View File
@@ -0,0 +1,52 @@
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+226
View File
@@ -0,0 +1,226 @@
// Package arcgis is the adapter for the external outages portal exposed as an
// ArcGIS MapServer /query endpoint.
//
// The portal only supports substring (LIKE) filtering, so precision is
// guaranteed locally: whatever the portal returns is re-filtered by the
// outages domain using the parsed house unit. The client adds the reliability
// the portal lacks: a hard timeout, bounded retries with backoff and context
// cancellation.
package arcgis
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
)
// Client is a bounded HTTP client for the portal.
type Client struct {
baseURL string
http *http.Client
retries int
}
// New builds a client. timeout applies to a single attempt; retries is the
// number of *additional* attempts after the first.
func New(baseURL string, timeout time.Duration, retries int) *Client {
return &Client{
baseURL: baseURL,
http: &http.Client{Timeout: timeout},
retries: retries,
}
}
// Query asks the portal for outages matching the canonical address. The
// returned records are raw portal rows; callers must still filter by house.
func (c *Client) Query(ctx context.Context, canonical string) ([]outages.Outage, error) {
params := url.Values{}
params.Set("where", "address like '"+likePattern(canonical)+"'")
params.Set("outFields", "")
params.Set("returnGeometry", "false")
params.Set("f", "json")
var lastErr error
for attempt := 0; attempt <= c.retries; attempt++ {
if attempt > 0 {
backoff := time.Duration(attempt) * 150 * time.Millisecond
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(backoff):
}
}
rows, err := c.do(ctx, params)
if err == nil {
return rows, nil
}
lastErr = err
}
return nil, fmt.Errorf("portal query failed after %d attempts: %w", c.retries+1, lastErr)
}
func (c *Client) do(ctx context.Context, params url.Values) ([]outages.Outage, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"?"+params.Encode(), nil)
if err != nil {
return nil, fmt.Errorf("build request: %w", err)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("portal status %d", resp.StatusCode)
}
// Bound the response size to protect the service from a misbehaving portal.
body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if err != nil {
return nil, err
}
return parseResponse(body)
}
// portalResponse mirrors the subset of the ArcGIS /query envelope we use.
type portalResponse struct {
Features []struct {
Value string `json:"value"`
Attributes map[string]any `json:"attributes"`
} `json:"features"`
Fields []struct {
Name string `json:"name"`
Domain struct {
CodedValues []struct {
Code any `json:"code"`
Name string `json:"name"`
} `json:"codedValues"`
} `json:"domain"`
} `json:"fields"`
}
func parseResponse(body []byte) ([]outages.Outage, error) {
var pr portalResponse
if err := json.Unmarshal(body, &pr); err != nil {
return nil, fmt.Errorf("decode portal response: %w", err)
}
// Build code -> name maps from the coded-value domains.
maps := map[string]map[string]string{}
for _, f := range pr.Fields {
if len(f.Domain.CodedValues) == 0 {
continue
}
m := make(map[string]string, len(f.Domain.CodedValues))
for _, cv := range f.Domain.CodedValues {
m[codeKey(cv.Code)] = cv.Name
}
maps[f.Name] = m
}
out := make([]outages.Outage, 0, len(pr.Features))
for _, feat := range pr.Features {
o := outages.Outage{
Address: addressOf(feat.Attributes, feat.Value),
Type: lookup(maps, "type_id", feat.Attributes),
System: lookup(maps, "system_id", feat.Attributes),
Time: lookup(maps, "time_category", feat.Attributes),
StartDate: stringOf(feat.Attributes["start_date"]),
EndDate: stringOf(feat.Attributes["end_date"]),
}
o.Action = actionFor(o.System)
out = append(out, o)
}
return out, nil
}
func addressOf(attrs map[string]any, fallback string) string {
if v, ok := attrs["address"]; ok {
if s := stringOf(v); s != "" {
return s
}
}
return fallback
}
func lookup(maps map[string]map[string]string, field string, attrs map[string]any) string {
m, ok := maps[field]
if !ok {
return stringOf(attrs[field])
}
return m[codeKey(attrs[field])]
}
// codeKey normalises a coded value coming from JSON (float64) or a string.
func codeKey(v any) string {
switch t := v.(type) {
case nil:
return ""
case string:
return t
case float64:
return strconv.FormatInt(int64(t), 10)
case json.Number:
return t.String()
default:
return fmt.Sprintf("%v", v)
}
}
func stringOf(v any) string {
if s, ok := v.(string); ok {
return s
}
if v == nil {
return ""
}
return fmt.Sprintf("%v", v)
}
// actionFor maps a portal system name to the canonical action code used by the
// scenario (kept for parity with the existing integration).
func actionFor(system string) string {
switch system {
case "Горячее водоснабжение":
return "waterWarm"
case "Холодное водоснабжение и канализация", "Холодное водоснабжение":
return "waterCold"
case "Электроснабжение":
return "electricitySupply"
case "Газоснабжение":
return "gasSupply"
case "Теплоснабжение":
return "heating"
default:
return ""
}
}
// likePattern builds the substring pattern the portal supports. Numeric tokens
// are padded with spaces, matching the historical integration, so that "1" does
// not incidentally match "10".
func likePattern(canonical string) string {
parts := strings.Fields(canonical)
for i, part := range parts {
if containsDigit(part) {
parts[i] = " " + part + " "
}
}
return "%" + strings.Join(parts, "%") + "%"
}
func containsDigit(s string) bool {
for _, r := range s {
if r >= '0' && r <= '9' {
return true
}
}
return false
}
+68
View File
@@ -0,0 +1,68 @@
package arcgis
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
)
const envelope = `{
"features": [
{"value": "Станционная улица 1/1", "attributes": {
"address": "Станционная улица 1/1",
"type_id": 1, "system_id": 5, "time_category": 2,
"start_date": "2026-10-01T00:00:00", "end_date": "2026-10-31T00:00:00"
}}
],
"fields": [
{"name": "type_id", "domain": {"codedValues": [{"code": 1, "name": "Плановое"}]}},
{"name": "system_id", "domain": {"codedValues": [{"code": 5, "name": "Теплоснабжение"}]}},
{"name": "time_category", "domain": {"codedValues": [{"code": 2, "name": "Текущие"}]}}
]
}`
func TestQueryParsesEnvelope(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.URL.Query().Get("f"); got != "json" {
t.Errorf("missing f=json, got %q", got)
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(envelope))
}))
defer ts.Close()
c := New(ts.URL, time.Second, 0)
rows, err := c.Query(context.Background(), "Станционная улица 1/1")
if err != nil {
t.Fatalf("Query: %v", err)
}
if len(rows) != 1 {
t.Fatalf("rows = %d, want 1", len(rows))
}
got := rows[0]
if got.Type != "Плановое" || got.System != "Теплоснабжение" || got.Time != "Текущие" {
t.Errorf("coded values not resolved: %+v", got)
}
if got.Action != "heating" {
t.Errorf("Action = %q, want heating", got.Action)
}
}
func TestQueryRetriesThenFails(t *testing.T) {
var calls int
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
http.Error(w, "boom", http.StatusInternalServerError)
}))
defer ts.Close()
c := New(ts.URL, time.Second, 2)
if _, err := c.Query(context.Background(), "Станционная улица 1/1"); err == nil {
t.Fatal("expected error")
}
if calls != 3 {
t.Errorf("calls = %d, want 3 (1 + 2 retries)", calls)
}
}
+101
View File
@@ -0,0 +1,101 @@
package httpapi
import (
"context"
"crypto/rand"
"encoding/hex"
"log/slog"
"net/http"
"time"
)
type ctxKey int
const requestIDKey ctxKey = 0
// requestIDMW assigns a request id (for correlation in logs) and echoes it.
func requestIDMW(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
id := r.Header.Get("X-Request-Id")
if id == "" {
id = newRequestID()
}
w.Header().Set("X-Request-Id", id)
ctx := context.WithValue(r.Context(), requestIDKey, id)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// loggingMW logs one line per request with method, path, status and duration.
// It records no addresses (PII) — only routing metadata.
func loggingMW(log *slog.Logger) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
next.ServeHTTP(rec, r)
log.Info("http",
"method", r.Method,
"path", r.URL.Path,
"status", rec.status,
"duration_ms", time.Since(start).Milliseconds(),
"request_id", requestIDFrom(r.Context()),
)
})
}
}
// timeoutMW bounds the whole request, so a slow portal cannot tie up a
// goroutine indefinitely.
func timeoutMW(d time.Duration) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), d)
defer cancel()
next.ServeHTTP(w, r.WithContext(ctx))
})
}
}
// recoverMW turns a panic into a 500 instead of crashing the process.
func recoverMW(log *slog.Logger) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if rec := recover(); rec != nil {
log.Error("panic", "value", rec, "request_id", requestIDFrom(r.Context()))
writeError(w, http.StatusInternalServerError, "internal error")
}
}()
next.ServeHTTP(w, r)
})
}
}
func requestIDFrom(ctx context.Context) string {
if v, ok := ctx.Value(requestIDKey).(string); ok {
return v
}
return ""
}
type statusRecorder struct {
http.ResponseWriter
status int
}
func (s *statusRecorder) WriteHeader(code int) {
s.status = code
s.ResponseWriter.WriteHeader(code)
}
// newRequestID returns a short random hex id. crypto/rand never fails in
// practice; on the off chance it does we fall back to a constant so the
// request still completes.
func newRequestID() string {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
return "unknown"
}
return hex.EncodeToString(b[:])
}
+165
View File
@@ -0,0 +1,165 @@
// Package httpapi exposes the addrkit domain over a small, versioned JSON API.
//
// It contains no business logic: handlers decode requests, call the resolver
// and the outages source, and encode responses. Cross-cutting concerns
// (timeouts, recovery, request ids, logging) live in middleware.
package httpapi
import (
"context"
"encoding/json"
"errors"
"log/slog"
"net/http"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
)
// Resolver is the read side of the domain (implemented by *match.Resolver).
type Resolver interface {
Resolve(ctx context.Context, raw string) match.Result
Version() string
Ready() bool
}
// Server wires the domain to HTTP.
type Server struct {
resolver Resolver
outages outages.Source
log *slog.Logger
now func() time.Time
loc *time.Location
timeout time.Duration
}
// New builds a server. loc controls how outage dates are interpreted; it
// should be the operational timezone (Asia/Novosibirsk).
func New(resolver Resolver, src outages.Source, log *slog.Logger, loc *time.Location) *Server {
if loc == nil {
loc = time.UTC
}
return &Server{
resolver: resolver,
outages: src,
log: log,
now: time.Now,
loc: loc,
timeout: 10 * time.Second,
}
}
// Handler returns the fully wrapped HTTP handler.
func (s *Server) Handler() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("POST /v1/resolve", s.handleResolve)
mux.HandleFunc("POST /v1/outages", s.handleOutages)
mux.HandleFunc("GET /v1/healthz", s.handleHealthz)
mux.HandleFunc("GET /v1/readyz", s.handleReadyz)
mux.HandleFunc("GET /v1/version", s.handleVersion)
return recoverMW(s.log)(requestIDMW(loggingMW(s.log)(timeoutMW(s.timeout)(mux))))
}
type resolveRequest struct {
Address string `json:"address"`
}
func (s *Server) handleResolve(w http.ResponseWriter, r *http.Request) {
req, ok := s.decode(w, r)
if !ok {
return
}
writeJSON(w, http.StatusOK, s.resolver.Resolve(r.Context(), req.Address))
}
type outagesResponse struct {
Status address.Status `json:"status"`
Address string `json:"address,omitempty"`
District string `json:"district,omitempty"`
Outages []outages.Outage `json:"outages,omitempty"`
AsOf string `json:"as_of,omitempty"`
DataVersion string `json:"data_version,omitempty"`
}
func (s *Server) handleOutages(w http.ResponseWriter, r *http.Request) {
req, ok := s.decode(w, r)
if !ok {
return
}
ctx := r.Context()
res := s.resolver.Resolve(ctx, req.Address)
out := outagesResponse{
Status: res.Status,
Address: res.Address,
District: res.District,
DataVersion: res.DataVersion,
}
if res.Status != address.StatusFound {
// No canonical address => nothing to query. Return the resolver's
// verdict so the caller can ask for clarification.
writeJSON(w, http.StatusOK, out)
return
}
raw, err := s.outages.Query(ctx, res.Address)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) {
out.Status = address.StatusPortalTimeout
} else {
out.Status = address.StatusInfra
}
writeJSON(w, http.StatusOK, out)
return
}
parsed := address.Parse(res.Address)
current := outages.Current(raw, s.now(), s.loc)
out.Outages = outages.MatchingHouse(current, parsed)
out.AsOf = s.now().In(s.loc).Format(time.RFC3339)
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handleHealthz(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handleReadyz(w http.ResponseWriter, _ *http.Request) {
if !s.resolver.Ready() {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"status": "not_ready"})
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "ready"})
}
func (s *Server) handleVersion(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"data_version": s.resolver.Version()})
}
// decode reads the JSON body into a resolveRequest, writing a 400 on failure.
func (s *Server) decode(w http.ResponseWriter, r *http.Request) (resolveRequest, bool) {
var req resolveRequest
r.Body = http.MaxBytesReader(w, r.Body, 64*1024)
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid json body")
return resolveRequest{}, false
}
if req.Address == "" {
writeError(w, http.StatusBadRequest, "address is required")
return resolveRequest{}, false
}
return req, true
}
func writeJSON(w http.ResponseWriter, status int, v any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(v)
}
func writeError(w http.ResponseWriter, status int, msg string) {
writeJSON(w, status, map[string]any{"error": 1, "message": msg})
}
+128
View File
@@ -0,0 +1,128 @@
package httpapi
import (
"bytes"
"context"
"encoding/json"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"testing"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
)
func testResolver(t *testing.T) *match.Resolver {
t.Helper()
var entries []address.Entry
for _, a := range []string{"Станционная улица 1/1", "Искитимская улица 5"} {
p := address.Parse(a)
entries = append(entries, address.Entry{
ID: a, Address: a, District: "Тестовый",
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
House: p.House, HouseFrac: p.HouseFrac, HouseLetter: p.HouseLetter,
})
}
return match.NewResolver(match.NewIndex("test-version", entries, match.DefaultOptions()), nil)
}
type fakeSource struct {
list []outages.Outage
err error
}
func (f fakeSource) Query(context.Context, string) ([]outages.Outage, error) {
return f.list, f.err
}
func newServer(t *testing.T, src fakeSource) *httptest.Server {
t.Helper()
srv := New(testResolver(t), src, slog.New(slog.NewTextHandler(io.Discard, nil)), time.UTC)
ts := httptest.NewServer(srv.Handler())
t.Cleanup(ts.Close)
return ts
}
func postJSON(t *testing.T, url, body string) *http.Response {
t.Helper()
resp, err := http.Post(url, "application/json", bytes.NewBufferString(body))
if err != nil {
t.Fatalf("post: %v", err)
}
return resp
}
func TestResolveEndpoint(t *testing.T) {
ts := newServer(t, fakeSource{})
resp := postJSON(t, ts.URL+"/v1/resolve", `{"address":"2-я Станционная 1/1"}`)
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d", resp.StatusCode)
}
var res match.Result
if err := json.NewDecoder(resp.Body).Decode(&res); err != nil {
t.Fatalf("decode: %v", err)
}
if res.Status != address.StatusFound || res.Address != "Станционная улица 1/1" {
t.Fatalf("unexpected result: %+v", res)
}
}
func TestOutagesEndpointFiltersHouse(t *testing.T) {
src := fakeSource{list: []outages.Outage{
{Address: "Станционная улица 1/1", System: "Теплоснабжение"},
{Address: "Станционная улица 1/2", System: "Теплоснабжение"},
}}
ts := newServer(t, src)
resp := postJSON(t, ts.URL+"/v1/outages", `{"address":"станционная 1/1"}`)
defer resp.Body.Close()
var out outagesResponse
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode: %v", err)
}
if out.Status != address.StatusFound {
t.Fatalf("status = %q, want found", out.Status)
}
if len(out.Outages) != 1 || out.Outages[0].Address != "Станционная улица 1/1" {
t.Fatalf("house filter failed: %+v", out.Outages)
}
}
func TestResolveValidation(t *testing.T) {
ts := newServer(t, fakeSource{})
resp := postJSON(t, ts.URL+"/v1/resolve", `{}`)
defer resp.Body.Close()
if resp.StatusCode != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", resp.StatusCode)
}
}
func TestReadyzAndVersion(t *testing.T) {
ts := newServer(t, fakeSource{})
resp, err := http.Get(ts.URL + "/v1/readyz")
if err != nil {
t.Fatalf("readyz: %v", err)
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("readyz status = %d", resp.StatusCode)
}
resp, err = http.Get(ts.URL + "/v1/version")
if err != nil {
t.Fatalf("version: %v", err)
}
defer resp.Body.Close()
var v map[string]string
_ = json.NewDecoder(resp.Body).Decode(&v)
if v["data_version"] != "test-version" {
t.Fatalf("version = %q", v["data_version"])
}
}
+137
View File
@@ -0,0 +1,137 @@
// Package llm implements the optional, disabled-by-default query rewriter.
//
// It talks to any OpenAI-compatible chat completions endpoint (for example the
// on-prem vLLM already used by AMS). Its contract is deliberately narrow: it
// may only propose alternative spellings of the user's address. It never
// returns the answer, and its output is validated by the deterministic
// resolver, so a misbehaving model can at worst waste a request — it cannot
// make the service answer with a wrong address.
package llm
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/config"
)
// Rewriter asks a model for normalised variants of an address.
type Rewriter struct {
endpoint string
token string
model string
http *http.Client
}
// New builds a rewriter from configuration. The URL is treated as a base; the
// OpenAI path is appended if it is not already present.
func New(cfg config.LLMConfig) *Rewriter {
base := strings.TrimRight(cfg.URL, "/")
if !strings.HasSuffix(base, "/chat/completions") {
base += "/v1/chat/completions"
}
timeout := time.Duration(cfg.TimeoutSeconds) * time.Second
if timeout <= 0 {
timeout = 3 * time.Second
}
return &Rewriter{
endpoint: base,
token: cfg.Token,
model: cfg.Model,
http: &http.Client{Timeout: timeout},
}
}
const systemPrompt = `Ты — нормализатор адресов. Тебе дают адрес, написанный как угодно.
Верни ТОЛЬКО JSON вида {"variants":["...","..."]} — до 3 вариантов записи того же адреса
в формате "<название улицы> <дом>[/<дробь>][ <литера>][ к<корпус>][ стр.<строение>]".
Не выдумывай улицы и дома, которых нет во входе. Если не уверен — верни пустой список.`
type chatRequest struct {
Model string `json:"model"`
Messages []chatMessage `json:"messages"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens"`
}
type chatMessage struct {
Role string `json:"role"`
Content string `json:"content"`
}
type chatResponse struct {
Choices []struct {
Message chatMessage `json:"message"`
} `json:"choices"`
}
// Rewrite returns up to three normalised variants of raw.
func (r *Rewriter) Rewrite(ctx context.Context, raw string) ([]string, error) {
payload, err := json.Marshal(chatRequest{
Model: r.model,
Messages: []chatMessage{
{Role: "system", Content: systemPrompt},
{Role: "user", Content: raw},
},
Temperature: 0,
MaxTokens: 256,
})
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, r.endpoint, bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
if r.token != "" && r.token != "EMPTY" {
req.Header.Set("Authorization", "Bearer "+r.token)
}
resp, err := r.http.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("llm status %d", resp.StatusCode)
}
var cr chatResponse
if err := json.NewDecoder(resp.Body).Decode(&cr); err != nil {
return nil, fmt.Errorf("decode llm response: %w", err)
}
if len(cr.Choices) == 0 {
return nil, nil
}
return parseVariants(cr.Choices[0].Message.Content), nil
}
// parseVariants extracts the {"variants":[...]} object from a model reply that
// may be wrapped in prose or code fences.
func parseVariants(content string) []string {
start := strings.Index(content, "{")
end := strings.LastIndex(content, "}")
if start < 0 || end <= start {
return nil
}
var parsed struct {
Variants []string `json:"variants"`
}
if err := json.Unmarshal([]byte(content[start:end+1]), &parsed); err != nil {
return nil
}
out := make([]string, 0, len(parsed.Variants))
for _, v := range parsed.Variants {
if s := strings.TrimSpace(v); s != "" {
out = append(out, s)
}
}
return out
}
+169
View File
@@ -0,0 +1,169 @@
// Package registrystore persists the canonical registry as a single,
// self-contained SQLite file. It is intentionally boring: one file, no server,
// built once by the seed pipeline and opened read-only by the service.
package registrystore
import (
"database/sql"
"fmt"
"os"
_ "modernc.org/sqlite" // registers the "sqlite" driver
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
const schema = `
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS address (
id TEXT PRIMARY KEY,
address TEXT NOT NULL,
district TEXT NOT NULL DEFAULT '',
ordinal TEXT NOT NULL DEFAULT '',
name_key TEXT NOT NULL DEFAULT '',
name_display TEXT NOT NULL DEFAULT '',
street_type TEXT NOT NULL DEFAULT '',
street_key TEXT NOT NULL,
house TEXT NOT NULL DEFAULT '',
house_frac TEXT NOT NULL DEFAULT '',
house_letter TEXT NOT NULL DEFAULT '',
building TEXT NOT NULL DEFAULT '',
building_frac TEXT NOT NULL DEFAULT '',
building_letter TEXT NOT NULL DEFAULT '',
structure TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS idx_address_street ON address(street_key);
`
// Save writes entries to path atomically: it builds a temporary file and
// renames it into place, so readers never observe a half-written index.
//
// The build uses the rollback journal (not WAL) on purpose, so that the result
// is a single file with no companion -wal/-shm that a rename would orphan.
func Save(path, version string, entries []address.Entry) error {
tmp := path + ".tmp"
removeDBFiles(tmp)
db, err := sql.Open("sqlite", "file:"+tmp+
"?_pragma=journal_mode(DELETE)"+
"&_pragma=synchronous(FULL)"+
"&_pragma=busy_timeout(5000)")
if err != nil {
return fmt.Errorf("open temp db: %w", err)
}
if _, err := db.Exec(schema); err != nil {
db.Close()
return fmt.Errorf("create schema: %w", err)
}
if err := insertAll(db, version, entries); err != nil {
db.Close()
return err
}
if err := db.Close(); err != nil {
return fmt.Errorf("close temp db: %w", err)
}
if err := os.Rename(tmp, path); err != nil {
return fmt.Errorf("publish db: %w", err)
}
// Drop only the temp file's journal siblings; the published base file must
// survive.
removeSidecars(tmp)
return nil
}
func insertAll(db *sql.DB, version string, entries []address.Entry) error {
tx, err := db.Begin()
if err != nil {
return fmt.Errorf("begin: %w", err)
}
defer tx.Rollback()
if _, err := tx.Exec(`INSERT INTO meta(key, value) VALUES ('version', ?)`, version); err != nil {
return fmt.Errorf("insert meta: %w", err)
}
stmt, err := tx.Prepare(`INSERT INTO address (
id, address, district, ordinal, name_key, name_display, street_type, street_key,
house, house_frac, house_letter, building, building_frac, building_letter, structure
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`)
if err != nil {
return fmt.Errorf("prepare insert: %w", err)
}
defer stmt.Close()
for i, e := range entries {
id := e.ID
if id == "" {
// Guarantee a primary key even for sources without ids.
id = fmt.Sprintf("row-%d", i)
}
if _, err := stmt.Exec(
id, e.Address, e.District, e.Ordinal, e.NameKey, e.NameDisplay, e.StreetType, e.StreetKey,
e.House, e.HouseFrac, e.HouseLetter, e.Building, e.BuildingFrac, e.BuildingLetter, e.Structure,
); err != nil {
return fmt.Errorf("insert %q: %w", e.Address, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("commit: %w", err)
}
return nil
}
// Load opens path read-only and returns the data version and all entries.
func Load(path string) (version string, entries []address.Entry, err error) {
if _, err := os.Stat(path); err != nil {
return "", nil, fmt.Errorf("stat db: %w", err)
}
db, err := sql.Open("sqlite", "file:"+path+
"?mode=ro"+
"&_pragma=query_only(1)"+
"&_pragma=busy_timeout(5000)")
if err != nil {
return "", nil, fmt.Errorf("open db: %w", err)
}
defer db.Close()
if err := db.QueryRow(`SELECT value FROM meta WHERE key='version'`).Scan(&version); err != nil {
return "", nil, fmt.Errorf("read version: %w", err)
}
rows, err := db.Query(`SELECT id, address, district, ordinal, name_key, name_display,
street_type, street_key, house, house_frac, house_letter, building, building_frac,
building_letter, structure FROM address`)
if err != nil {
return "", nil, fmt.Errorf("query: %w", err)
}
defer rows.Close()
for rows.Next() {
var e address.Entry
if err := rows.Scan(
&e.ID, &e.Address, &e.District, &e.Ordinal, &e.NameKey, &e.NameDisplay,
&e.StreetType, &e.StreetKey, &e.House, &e.HouseFrac, &e.HouseLetter,
&e.Building, &e.BuildingFrac, &e.BuildingLetter, &e.Structure,
); err != nil {
return "", nil, fmt.Errorf("scan: %w", err)
}
entries = append(entries, e)
}
if err := rows.Err(); err != nil {
return "", nil, fmt.Errorf("rows: %w", err)
}
return version, entries, nil
}
// removeDBFiles deletes a database file and its journal siblings.
func removeDBFiles(base string) {
_ = os.Remove(base)
removeSidecars(base)
}
// removeSidecars deletes only the -wal/-shm companions of a database file.
func removeSidecars(base string) {
_ = os.Remove(base + "-wal")
_ = os.Remove(base + "-shm")
}
@@ -0,0 +1,59 @@
package registrystore
import (
"path/filepath"
"testing"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
func TestSaveLoadRoundTrip(t *testing.T) {
path := filepath.Join(t.TempDir(), "registry.db")
addr := "Станционная улица 1/1"
p := address.Parse(addr)
entries := []address.Entry{{
ID: "a1", Address: addr, District: "Ленинский",
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
House: p.House, HouseFrac: p.HouseFrac,
}}
if err := Save(path, "2026-10-09", entries); err != nil {
t.Fatalf("Save: %v", err)
}
version, got, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if version != "2026-10-09" {
t.Errorf("version = %q", version)
}
if len(got) != 1 {
t.Fatalf("entries = %d, want 1", len(got))
}
if got[0].Address != addr || got[0].District != "Ленинский" || got[0].StreetKey != p.StreetKey {
t.Errorf("round-tripped entry mismatch: %+v", got[0])
}
}
// TestSaveIsAtomicOverwrite verifies that re-seeding replaces the previous data
// rather than merging with it.
func TestSaveIsAtomicOverwrite(t *testing.T) {
path := filepath.Join(t.TempDir(), "registry.db")
if err := Save(path, "v1", []address.Entry{{ID: "x", Address: "Первая улица 1"}}); err != nil {
t.Fatalf("Save v1: %v", err)
}
if err := Save(path, "v2", []address.Entry{{ID: "y", Address: "Вторая улица 2"}}); err != nil {
t.Fatalf("Save v2: %v", err)
}
version, got, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if version != "v2" || len(got) != 1 || got[0].Address != "Вторая улица 2" {
t.Fatalf("unexpected state after overwrite: version=%q entries=%+v", version, got)
}
}
+119
View File
@@ -0,0 +1,119 @@
// Package config loads the addrkit configuration from a JSON file, expanding
// ${VAR} placeholders from the environment. It fails closed: missing required
// values are an error, never a silent default.
package config
import (
"encoding/json"
"fmt"
"os"
"regexp"
"strings"
)
// Config is the whole service configuration.
type Config struct {
// Addr is the listen address of the HTTP server, e.g. ":6171".
Addr string `json:"addr"`
// DataPath is the SQLite file produced by the seed pipeline.
DataPath string `json:"data_path"`
// LogLevel is one of debug|info|warn|error.
LogLevel string `json:"log_level"`
Resolve ResolveConfig `json:"resolve"`
Portal PortalConfig `json:"portal"`
LLM LLMConfig `json:"llm"`
}
// ResolveConfig tunes the matching engine.
type ResolveConfig struct {
FuzzyThreshold float64 `json:"fuzzy_threshold"`
AmbiguityMargin float64 `json:"ambiguity_margin"`
MaxCandidates int `json:"max_candidates"`
}
// PortalConfig configures the external outages portal (ArcGIS).
type PortalConfig struct {
URL string `json:"url"`
TimeoutSeconds int `json:"timeout_seconds"`
Retries int `json:"retries"`
}
// LLMConfig configures the optional, disabled-by-default query rewriter.
type LLMConfig struct {
Enabled bool `json:"enabled"`
URL string `json:"url"`
Token string `json:"token"`
Model string `json:"model"`
TimeoutSeconds int `json:"timeout_seconds"`
}
// Default returns a config with sane defaults for the non-required fields.
func Default() Config {
return Config{
Addr: ":6171",
LogLevel: "info",
Resolve: ResolveConfig{
FuzzyThreshold: 0.82,
AmbiguityMargin: 0.03,
MaxCandidates: 5,
},
Portal: PortalConfig{
TimeoutSeconds: 2,
Retries: 1,
},
LLM: LLMConfig{
TimeoutSeconds: 3,
},
}
}
// Load reads, expands and validates a config file.
func Load(path string) (Config, error) {
raw, err := os.ReadFile(path)
if err != nil {
return Config{}, fmt.Errorf("read config: %w", err)
}
cfg := Default()
dec := json.NewDecoder(strings.NewReader(expandEnv(string(raw))))
if err := dec.Decode(&cfg); err != nil {
return Config{}, fmt.Errorf("parse config: %w", err)
}
if err := cfg.validate(); err != nil {
return Config{}, err
}
return cfg, nil
}
func (c Config) validate() error {
if c.Addr == "" {
return fmt.Errorf("config: addr is required")
}
if c.DataPath == "" {
return fmt.Errorf("config: data_path is required")
}
if c.LLM.Enabled && c.LLM.URL == "" {
return fmt.Errorf("config: llm.url is required when llm.enabled is true")
}
if c.Portal.URL == "" {
return fmt.Errorf("config: portal.url is required")
}
return nil
}
// envRef matches ${VAR} and $VAR placeholders.
var envRef = regexp.MustCompile(`\$\{([A-Za-z_][A-Za-z0-9_]*)\}|\$([A-Za-z_][A-Za-z0-9_]*)`)
// expandEnv replaces ${VAR} / $VAR with the environment value. An unset
// variable expands to the empty string; validation then rejects configs that
// relied on it (fail closed).
func expandEnv(s string) string {
return envRef.ReplaceAllStringFunc(s, func(m string) string {
sub := envRef.FindStringSubmatch(m)
name := sub[1]
if name == "" {
name = sub[2]
}
return os.Getenv(name)
})
}
+102
View File
@@ -0,0 +1,102 @@
package address
import (
"strings"
"unicode"
"golang.org/x/text/cases"
"golang.org/x/text/unicode/norm"
)
// fold is a Unicode-aware case folder. It lower-cases Cyrillic and Latin
// alike, which is what we want for matching.
var fold = cases.Fold()
// NormalizeText folds a free-form string into a canonical form suitable for
// parsing and matching.
//
// The transformation is intentionally lossy:
// - Unicode NFKC (so lookalike characters collapse);
// - case folding (so "СТАНИЦИОННАЯ" == "станционная");
// - "ё" -> "е" (users and ASR disagree about the diaeresis);
// - dashes and various Unicode spaces are unified;
// - punctuation that only separates tokens becomes a space;
// - runs of whitespace collapse to a single space.
//
// NormalizeText is idempotent: NormalizeText(NormalizeText(s)) == NormalizeText(s).
func NormalizeText(s string) string {
s = norm.NFKC.String(s)
s = fold.String(s)
s = strings.ReplaceAll(s, "ё", "е")
s = mapSeparators(s)
s = strings.Join(strings.Fields(s), " ")
return strings.TrimSpace(s)
}
// mapSeparators unifies characters that carry no semantic meaning for address
// matching. It never removes digits, letters or the '/' house separator.
func mapSeparators(s string) string {
var b strings.Builder
b.Grow(len(s))
for _, r := range s {
switch {
case r == '\u00a0' || unicode.IsSpace(r):
b.WriteByte(' ')
case isDash(r):
b.WriteByte('-')
case isSeparatorPunct(r):
b.WriteByte(' ')
default:
b.WriteRune(r)
}
}
return b.String()
}
// isDash reports whether r is one of the many dash-like code points that users
// and copy-paste bring in: "2–я" with an en dash is still an ordinal.
func isDash(r rune) bool {
switch r {
case '-', '\u2010', '\u2011', '\u2012', '\u2013', '\u2014', '\u2212':
return true
default:
return false
}
}
// isSeparatorPunct reports whether r separates tokens. Dots are included so
// that abbreviations like "ул." normalize to their stem ("ул").
func isSeparatorPunct(r rune) bool {
switch r {
case '.', ',', ';', '(', ')', '"', '\'', '«', '»', ':', '!', '?':
return true
default:
return false
}
}
// isDigits reports whether s is a non-empty run of ASCII digits.
func isDigits(s string) bool {
if s == "" {
return false
}
for _, r := range s {
if r < '0' || r > '9' {
return false
}
}
return true
}
// trimLeadingZeros removes redundant leading zeros from a numeric token:
// "007" -> "7". It leaves non-numeric strings untouched.
func trimLeadingZeros(s string) string {
if !isDigits(s) {
return s
}
t := strings.TrimLeft(s, "0")
if t == "" {
return "0"
}
return t
}
+143
View File
@@ -0,0 +1,143 @@
package address
import (
"regexp"
"sort"
"strings"
)
// The house block is parsed from the end of the string, component by
// component, from the outermost (structure) to the innermost (house). Working
// right-to-left is what lets us tell a genuine house number apart from a digit
// that is part of the street name (e.g. "2-я Станционная улица 1/1 стр.").
var (
// "... стр." / "... строение 3" / "... сооружение" / "... с 2"
reStructure = regexp.MustCompile(`(?:^|\s)(?:стр|строение|сооружение|с)\.?\s*(\d+)?$`)
// "... к9/1" / "... корпус 2а" / "... кД" / "... к 1/4"
reBuilding = regexp.MustCompile(`(?:^|\s)(?:к|корп|корпус)\.?\s*(\d*)\s*([а-я])?(?:/(\d+)\s*([а-я])?)?$`)
// "... 1/1" / "... 33а" / "... 12 б" / "... 45а/4" / "... 1/1а"
reHouse = regexp.MustCompile(`(?:^|\s)(\d+)\s*([а-я])?(?:/(\d+)\s*([а-я])?)?$`)
// "2-я", "10-й", "3-е"
reOrdinalToken = regexp.MustCompile(`^(\d+)-[а-я]{1,3}$`)
)
// Parse decomposes a free-form address into its structured components. It is
// total: unparseable input still yields a non-nil ParsedAddress whose street
// part is best-effort and whose HasHouse is false.
func Parse(raw string) ParsedAddress {
normalized := NormalizeText(raw)
p := ParsedAddress{Raw: raw, Normalized: normalized}
rest := normalized
if g, start, ok := matchSuffix(reStructure, rest); ok {
p.Structure = trimLeadingZeros(g[1])
rest = strings.TrimSpace(rest[:start])
}
if g, start, ok := matchSuffix(reBuilding, rest); ok {
// The regex also matches a bare "к"; only accept it when a digit or a
// letter actually follows, otherwise it is not a corpus marker.
if g[1] != "" || g[2] != "" {
p.Building = trimLeadingZeros(g[1])
p.BuildingLetter = firstNonEmpty(g[2], g[4])
p.BuildingFrac = trimLeadingZeros(g[3])
rest = strings.TrimSpace(rest[:start])
}
}
if g, start, ok := matchSuffix(reHouse, rest); ok {
p.House = trimLeadingZeros(g[1])
p.HouseLetter = firstNonEmpty(g[2], g[4])
p.HouseFrac = trimLeadingZeros(g[3])
p.HasHouse = true
rest = strings.TrimSpace(rest[:start])
}
parseStreet(&p, rest)
return p
}
// firstNonEmpty returns the first non-empty argument. House/building letters
// can appear either before or after the fraction ("45а/4" vs "1/1а").
func firstNonEmpty(values ...string) string {
for _, v := range values {
if v != "" {
return v
}
}
return ""
}
// parseStreet fills the street-related fields of p from the leftover string
// (the address with its house block removed).
func parseStreet(p *ParsedAddress, s string) {
tokens := strings.Fields(s)
if len(tokens) == 0 {
p.StreetKey = streetKey("", "")
return
}
name := make([]string, 0, len(tokens))
typeFound := false
for _, tok := range tokens {
// Only the first recognised type token is treated as the type; a
// second occurrence is part of the name.
if !typeFound {
if t, ok := streetTypes[tok]; ok {
p.StreetType = t
typeFound = true
continue
}
}
name = append(name, tok)
}
name = extractOrdinal(p, name)
p.NameDisplay = strings.Join(name, " ")
sorted := append([]string(nil), name...)
sort.Strings(sorted)
p.NameKey = strings.Join(sorted, " ")
p.StreetKey = streetKey(p.Ordinal, p.NameKey)
}
// extractOrdinal recognises a leading ordinal ("2-я", "2 я", "вторая") and
// removes it from the name tokens.
func extractOrdinal(p *ParsedAddress, tokens []string) []string {
if len(tokens) == 0 {
return tokens
}
if m := reOrdinalToken.FindStringSubmatch(tokens[0]); m != nil {
p.Ordinal = normalizeOrdinal(m[1])
return tokens[1:]
}
if isDigits(tokens[0]) && len(tokens) >= 2 && ordinalSuffixes[tokens[1]] {
p.Ordinal = normalizeOrdinal(tokens[0])
return tokens[2:]
}
if d, ok := ordinalWords[tokens[0]]; ok {
p.Ordinal = normalizeOrdinal(d)
return tokens[1:]
}
return tokens
}
// matchSuffix returns the capture groups of the match of re that ends exactly
// at the end of s, together with the byte offset where it starts. Go's regexp
// is leftmost-first, so we scan all matches and pick the right-anchored one.
func matchSuffix(re *regexp.Regexp, s string) (groups []string, start int, ok bool) {
for _, loc := range re.FindAllStringSubmatchIndex(s, -1) {
if loc[1] != len(s) {
continue
}
n := len(loc) / 2
groups = make([]string, n)
for i := 0; i < n; i++ {
if loc[2*i] >= 0 {
groups[i] = s[loc[2*i]:loc[2*i+1]]
}
}
return groups, loc[0], true
}
return nil, 0, false
}
+148
View File
@@ -0,0 +1,148 @@
package address
import "testing"
func TestParseHouseBlock(t *testing.T) {
tests := []struct {
name string
raw string
wantOrdinal string
wantName string
wantType string
wantHouse string
wantFrac string
wantLetter string
wantBuild string
wantBFrac string
wantBLetter string
wantStruct string
wantHas bool
}{
{
name: "ordinal, fraction and structure",
raw: "2-я Станционная улица 1/1 стр.",
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
wantHouse: "1", wantFrac: "1", wantStruct: "", wantHas: true,
},
{
name: "building with fraction and structure",
raw: "Станционная улица 30а к9/1 стр.",
wantName: "станционная", wantType: "улица",
wantHouse: "30", wantLetter: "а", wantBuild: "9", wantBFrac: "1", wantHas: true,
},
{
name: "letter glued to house",
raw: "10-й Бронный переулок 33а",
wantOrdinal: "10", wantName: "бронный", wantType: "переулок",
wantHouse: "33", wantLetter: "а", wantHas: true,
},
{
name: "microdistrict, two-word name",
raw: "Зеленый бор микрорайон 13",
wantName: "бор зеленый", wantType: "микрорайон",
wantHouse: "13", wantHas: true,
},
{
name: "spelled-out ordinal",
raw: "вторая Станционная улица 1",
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
wantHouse: "1", wantHas: true,
},
{
name: "en dash variant",
raw: "2–я Станционная улица 1",
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
wantHouse: "1", wantHas: true,
},
{
name: "no house",
raw: "Станционная улица",
wantName: "станционная", wantType: "улица", wantHas: false,
},
{
name: "structure with number only",
raw: "Станционная улица 3 стр.",
wantName: "станционная", wantType: "улица",
wantHouse: "3", wantStruct: "", wantHas: true,
},
{
name: "letter before fraction",
raw: "Выборная улица 45а/4",
wantName: "выборная", wantType: "улица",
wantHouse: "45", wantLetter: "а", wantFrac: "4", wantHas: true,
},
{
name: "corpus letter without digit",
raw: "1-я Ельцовка улица 1 кД",
wantOrdinal: "1", wantName: "ельцовка", wantType: "улица",
wantHouse: "1", wantBLetter: "д", wantHas: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := Parse(tt.raw)
if got.Ordinal != tt.wantOrdinal {
t.Errorf("Ordinal = %q, want %q", got.Ordinal, tt.wantOrdinal)
}
if got.NameKey != tt.wantName {
t.Errorf("NameKey = %q, want %q", got.NameKey, tt.wantName)
}
if got.StreetType != tt.wantType {
t.Errorf("StreetType = %q, want %q", got.StreetType, tt.wantType)
}
if got.House != tt.wantHouse {
t.Errorf("House = %q, want %q", got.House, tt.wantHouse)
}
if got.HouseFrac != tt.wantFrac {
t.Errorf("HouseFrac = %q, want %q", got.HouseFrac, tt.wantFrac)
}
if got.HouseLetter != tt.wantLetter {
t.Errorf("HouseLetter = %q, want %q", got.HouseLetter, tt.wantLetter)
}
if got.Building != tt.wantBuild {
t.Errorf("Building = %q, want %q", got.Building, tt.wantBuild)
}
if got.BuildingFrac != tt.wantBFrac {
t.Errorf("BuildingFrac = %q, want %q", got.BuildingFrac, tt.wantBFrac)
}
if got.BuildingLetter != tt.wantBLetter {
t.Errorf("BuildingLetter = %q, want %q", got.BuildingLetter, tt.wantBLetter)
}
if got.Structure != tt.wantStruct {
t.Errorf("Structure = %q, want %q", got.Structure, tt.wantStruct)
}
if got.HasHouse != tt.wantHas {
t.Errorf("HasHouse = %v, want %v", got.HasHouse, tt.wantHas)
}
})
}
}
// TestParseDeterministic guards the core invariant: parsing the same input
// always yields the same structured result and the same street key.
func TestParseDeterministic(t *testing.T) {
input := "2-я Станционная улица 1/1 стр."
first := Parse(input)
for i := 0; i < 100; i++ {
if got := Parse(input); got.StreetKey != first.StreetKey || got.House != first.House {
t.Fatalf("non-deterministic parse: %+v vs %+v", got, first)
}
}
}
func TestNormalizeIdempotent(t *testing.T) {
inputs := []string{
" 2-Я Станционная\u00a0улица 1/1 стр. ",
"ЁЛКИ д. 5, кв. 3",
"СТАНИЦИОННАЯ — улица, 30а",
"",
}
for _, in := range inputs {
once := NormalizeText(in)
twice := NormalizeText(once)
if once != twice {
t.Errorf("NormalizeText not idempotent for %q: %q != %q", in, once, twice)
}
}
}
+66
View File
@@ -0,0 +1,66 @@
package address
import "strings"
// streetTypes maps a surface form (including common abbreviations) to a single
// canonical street type. The type is informational for matching (we strip it
// from the key) but useful for disambiguation and display.
var streetTypes = map[string]string{
"ул": "улица", "улица": "улица",
"пер": "переулок", "пер-к": "переулок", "переулок": "переулок",
"пр": "проезд", "проезд": "проезд",
"пр-т": "проспект", "просп": "проспект", "проспект": "проспект",
"ш": "шоссе", "шоссе": "шоссе",
"мкр": "микрорайон", "микрорайон": "микрорайон",
"туп": "тупик", "тупик": "тупик",
"спуск": "спуск",
"б-р": "бульвар", "бул": "бульвар", "бульвар": "бульвар",
"пл": "площадь", "площадь": "площадь",
"пос": "поселок", "поселок": "поселок",
"тракт": "тракт",
"аллея": "аллея",
"наб": "набережная", "набережная": "набережная",
"кольцо": "кольцо",
"просека": "просека",
"квартал": "квартал",
"территория": "территория",
"дорога": "дорога",
"разъезд": "разъезд",
}
// ordinalWords maps spelled-out ordinals to their digit form. Only the forms
// that actually occur in practice are listed; unknown words are treated as
// part of the street name.
var ordinalWords = map[string]string{
"первый": "1", "первая": "1", "первое": "1",
"второй": "2", "вторая": "2", "второе": "2",
"третий": "3", "третья": "3", "третье": "3",
"четвертый": "4", "четвертая": "4", "четвертое": "4",
"пятый": "5", "пятая": "5", "пятое": "5",
"шестой": "6", "шестая": "6", "шестое": "6",
"седьмой": "7", "седьмая": "7", "седьмое": "7",
"восьмой": "8", "восьмая": "8", "восьмое": "8",
"девятый": "9", "девятая": "9", "девятое": "9",
"десятый": "10", "десятая": "10", "десятое": "10",
}
// ordinalSuffixes are the gender/number endings that may follow an ordinal
// digit when it is written as a separate token ("2 я", "10 й").
var ordinalSuffixes = map[string]bool{
"я": true, "й": true, "е": true,
"ая": true, "ый": true, "ой": true, "ое": true,
"ий": true, "ья": true, "ье": true, "ьи": true,
}
// streetKey builds the canonical, type-independent street key:
// "<ordinal>|<sorted name tokens>". The type word is dropped so that a user
// who omits it ("станционная 1" instead of "Станционная улица 1") still
// matches; type confusion is surfaced later as ambiguity.
func streetKey(ordinal, nameKey string) string {
return ordinal + "|" + nameKey
}
// normalizeOrdinal trims leading zeros so "02-я" and "2-я" agree.
func normalizeOrdinal(s string) string {
return trimLeadingZeros(strings.TrimSpace(s))
}
+89
View File
@@ -0,0 +1,89 @@
// Package address contains the deterministic address domain: text
// normalization, structural parsing and the canonical data types shared by the
// rest of addrkit.
//
// The package is deliberately free of any I/O (no HTTP, no database, no
// filesystem). Everything here is a pure function of its inputs, which makes
// the core trivially testable and reusable by the service, the CLI and tests.
package address
// Status is the machine-readable outcome of an address resolution.
//
// It is intentionally explicit: callers must never have to guess whether an
// empty result means "not found" or "the upstream failed".
type Status string
const (
// StatusFound means the street and the exact house unit were resolved.
StatusFound Status = "found"
// StatusPartial means the street was resolved but the house unit was not
// confirmed. The answer is safe but incomplete; the caller may ask the
// user to clarify the house number.
StatusPartial Status = "partial"
// StatusAmbiguous means more than one plausible match exists and the
// resolver refuses to guess.
StatusAmbiguous Status = "ambiguous"
// StatusNotFound means nothing in the registry matched.
StatusNotFound Status = "not_found"
// StatusPortalTimeout means the external outages portal did not answer in
// time. It is an infrastructure outcome, not "no outages".
StatusPortalTimeout Status = "portal_timeout"
// StatusInfra is a generic terminal infrastructure failure.
StatusInfra Status = "infra_error"
)
// ParsedAddress is the structured form of a free-form address string.
//
// The parser is lossy on purpose: punctuation, case and spacing are folded
// away so that different surface forms of the same address converge on the
// same StreetKey.
type ParsedAddress struct {
Raw string `json:"raw"`
Normalized string `json:"normalized"`
// Street components.
//
// NameKey is the order-insensitive canonical name (tokens sorted), so
// "Станционная 2-я" and "2-я Станционная" produce the same key.
Ordinal string `json:"ordinal,omitempty"`
NameKey string `json:"name_key,omitempty"`
NameDisplay string `json:"name_display,omitempty"`
StreetType string `json:"street_type,omitempty"`
StreetKey string `json:"street_key"`
// House unit components. A house "1/1 к2 стр.3" decomposes into
// House=1, HouseFrac=1, Building=2, Structure=3.
HasHouse bool `json:"has_house"`
House string `json:"house,omitempty"`
HouseFrac string `json:"house_frac,omitempty"`
HouseLetter string `json:"house_letter,omitempty"`
Building string `json:"building,omitempty"`
BuildingFrac string `json:"building_frac,omitempty"`
BuildingLetter string `json:"building_letter,omitempty"`
Structure string `json:"structure,omitempty"`
}
// Entry is a single canonical row from the address registry.
//
// Entries are produced at seed time by parsing the registry's display address
// with the very same parser used at query time, so index and query always
// agree on how an address is decomposed.
type Entry struct {
ID string `json:"id"`
Address string `json:"address"`
District string `json:"district,omitempty"`
Ordinal string `json:"ordinal,omitempty"`
NameKey string `json:"name_key,omitempty"`
NameDisplay string `json:"name_display,omitempty"`
StreetType string `json:"street_type,omitempty"`
StreetKey string `json:"street_key"`
House string `json:"house,omitempty"`
HouseFrac string `json:"house_frac,omitempty"`
HouseLetter string `json:"house_letter,omitempty"`
Building string `json:"building,omitempty"`
BuildingFrac string `json:"building_frac,omitempty"`
BuildingLetter string `json:"building_letter,omitempty"`
Structure string `json:"structure,omitempty"`
}
+299
View File
@@ -0,0 +1,299 @@
// Package match resolves a ParsedAddress against a canonical registry index.
//
// It is the deterministic heart of addrkit. It knows nothing about storage,
// HTTP or LLMs: it operates on an in-memory index built from []address.Entry
// and always answers from that data. A wrong house number is never invented —
// when the exact house unit is not present the result is "partial", and when
// several streets are plausible the result is "ambiguous".
package match
import (
"sort"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
// Candidate is one plausible match, returned alongside a non-"found" result so
// the caller (or an operator) can see what the resolver was choosing between.
type Candidate struct {
Address string `json:"address"`
District string `json:"district,omitempty"`
House string `json:"house,omitempty"`
Score float64 `json:"score"`
}
// Result is the outcome of a resolution.
type Result struct {
Status address.Status `json:"status"`
Address string `json:"address,omitempty"`
District string `json:"district,omitempty"`
Confidence float64 `json:"confidence"`
Candidates []Candidate `json:"candidates,omitempty"`
DataVersion string `json:"data_version,omitempty"`
}
// Options tunes matching behaviour.
type Options struct {
// FuzzyThreshold is the minimum similarity (0..1) for a street to be
// accepted as a typo-tolerant match. Below it the result is not_found.
FuzzyThreshold float64
// AmbiguityMargin: streets within this similarity of the best match make
// the result ambiguous instead of silently picking one.
AmbiguityMargin float64
// MaxCandidates caps the candidates returned to the caller.
MaxCandidates int
}
// DefaultOptions returns conservative defaults.
func DefaultOptions() Options {
return Options{
FuzzyThreshold: 0.82,
AmbiguityMargin: 0.03,
MaxCandidates: 5,
}
}
type streetRef struct {
key string
ordinal string
nameKey string
streetType string
}
// Index is an immutable, in-memory registry index. It is safe for concurrent
// reads once constructed.
type Index struct {
version string
opts Options
byKey map[string][]address.Entry
streets []streetRef
}
// NewIndex builds an index from canonical entries. Entries sharing a StreetKey
// are grouped; distinct streets are collected for typo-tolerant matching.
func NewIndex(version string, entries []address.Entry, opts Options) *Index {
ix := &Index{
version: version,
opts: opts,
byKey: make(map[string][]address.Entry, len(entries)),
}
seen := make(map[string]struct{}, len(entries))
for _, e := range entries {
ix.byKey[e.StreetKey] = append(ix.byKey[e.StreetKey], e)
if _, ok := seen[e.StreetKey]; !ok {
seen[e.StreetKey] = struct{}{}
ix.streets = append(ix.streets, streetRef{
key: e.StreetKey,
ordinal: e.Ordinal,
nameKey: e.NameKey,
streetType: e.StreetType,
})
}
}
sort.Slice(ix.streets, func(i, j int) bool { return ix.streets[i].key < ix.streets[j].key })
return ix
}
// Version returns the data version the index was built from.
func (ix *Index) Version() string { return ix.version }
// Size returns the number of indexed entries.
func (ix *Index) Size() int {
n := 0
for _, v := range ix.byKey {
n += len(v)
}
return n
}
// Resolve resolves a parsed address. It never returns an address that is not
// present in the index.
func (ix *Index) Resolve(p address.ParsedAddress) Result {
res := Result{DataVersion: ix.version}
entries := ix.byKey[p.StreetKey]
similarity := 1.0
if len(entries) == 0 {
var ambiguousKeys []string
entries, similarity, ambiguousKeys = ix.fuzzyStreets(p)
if len(ambiguousKeys) > 0 {
res.Status = address.StatusAmbiguous
res.Confidence = similarity
res.Candidates = ix.representatives(ambiguousKeys, similarity)
return res
}
}
if len(entries) == 0 {
res.Status = address.StatusNotFound
return res
}
if !p.HasHouse {
// Street is known but the caller did not supply a house: safe but
// incomplete.
res.Status = address.StatusPartial
res.Confidence = clamp01(similarity * 0.8)
res.Candidates = candidates(entries, similarity, ix.opts.MaxCandidates)
return res
}
matched := matchHouseUnit(entries, p)
switch len(matched) {
case 1:
res.Status = address.StatusFound
res.Address = matched[0].Address
res.District = matched[0].District
res.Confidence = clamp01(similarity)
return res
case 0:
// Street recognised, exact house unit absent, or several units share
// the same number (ambiguous at the unit level).
if sameHouseSeveral(entries, p) {
res.Status = address.StatusAmbiguous
res.Confidence = clamp01(similarity * 0.9)
res.Candidates = candidates(matchHouseUnitAll(entries, p), similarity, ix.opts.MaxCandidates)
return res
}
res.Status = address.StatusPartial
res.Confidence = clamp01(similarity * 0.9)
res.Candidates = candidates(entries, similarity, ix.opts.MaxCandidates)
return res
default:
res.Status = address.StatusAmbiguous
res.Confidence = clamp01(similarity * 0.9)
res.Candidates = candidates(matched, similarity, ix.opts.MaxCandidates)
return res
}
}
// fuzzyStreets finds streets similar to the query. It returns the entries of
// the best street when the match is unambiguous, or the list of competing
// street keys when several are within the ambiguity margin.
func (ix *Index) fuzzyStreets(p address.ParsedAddress) (entries []address.Entry, similarity float64, ambiguousKeys []string) {
if p.NameKey == "" {
return nil, 0, nil
}
best := 0.0
bestKey := ""
for _, s := range ix.streets {
// If both sides carry an ordinal and they differ, the streets are
// different even if the names rhyme.
if p.Ordinal != "" && s.ordinal != "" && p.Ordinal != s.ordinal {
continue
}
if sim := ratio(p.NameKey, s.nameKey); sim > best {
best = sim
bestKey = s.key
}
}
if best < ix.opts.FuzzyThreshold {
return nil, best, nil
}
for _, s := range ix.streets {
if p.Ordinal != "" && s.ordinal != "" && p.Ordinal != s.ordinal {
continue
}
if ratio(p.NameKey, s.nameKey) >= best-ix.opts.AmbiguityMargin {
ambiguousKeys = append(ambiguousKeys, s.key)
}
}
if len(ambiguousKeys) > 1 {
return nil, best, ambiguousKeys
}
return ix.byKey[bestKey], best, nil
}
// representatives returns one candidate per competing street, which is far
// more useful to a user than five houses of a single street.
func (ix *Index) representatives(keys []string, score float64) []Candidate {
sort.Strings(keys)
out := make([]Candidate, 0, len(keys))
for _, k := range keys {
es := ix.byKey[k]
if len(es) == 0 {
continue
}
e := es[0]
out = append(out, Candidate{
Address: e.Address,
District: e.District,
House: e.House,
Score: clamp01(score),
})
if len(out) == ix.opts.MaxCandidates {
break
}
}
return out
}
// matchHouseUnit returns entries whose house unit equals p's, exactly.
func matchHouseUnit(entries []address.Entry, p address.ParsedAddress) []address.Entry {
var out []address.Entry
for _, e := range entries {
if houseUnitEqual(e, p) {
out = append(out, e)
}
}
return out
}
// matchHouseUnitAll returns all entries for entries that share the same house
// number as p (used to surface ambiguity).
func matchHouseUnitAll(entries []address.Entry, p address.ParsedAddress) []address.Entry {
var out []address.Entry
for _, e := range entries {
if e.House == p.House && e.HouseFrac == p.HouseFrac && e.HouseLetter == p.HouseLetter {
out = append(out, e)
}
}
return out
}
// sameHouseSeveral reports whether more than one entry matches the house
// number but with differing buildings/letters, i.e. the unit is ambiguous.
func sameHouseSeveral(entries []address.Entry, p address.ParsedAddress) bool {
return len(matchHouseUnitAll(entries, p)) > 1
}
// houseUnitEqual compares the parts of a house unit. Unspecified (empty) parts
// must also be empty on the entry side, so "60/1" never matches "60/1 к148".
func houseUnitEqual(e address.Entry, p address.ParsedAddress) bool {
return e.House == p.House &&
e.HouseFrac == p.HouseFrac &&
e.HouseLetter == p.HouseLetter &&
e.Building == p.Building &&
e.BuildingFrac == p.BuildingFrac &&
e.BuildingLetter == p.BuildingLetter
}
func candidates(entries []address.Entry, score float64, limit int) []Candidate {
if limit <= 0 || len(entries) == 0 {
return nil
}
out := make([]Candidate, 0, len(entries))
for _, e := range entries {
out = append(out, Candidate{
Address: e.Address,
District: e.District,
House: e.House,
Score: clamp01(score),
})
if len(out) == limit {
break
}
}
return out
}
func clamp01(v float64) float64 {
if v < 0 {
return 0
}
if v > 1 {
return 1
}
return v
}
+137
View File
@@ -0,0 +1,137 @@
package match
import (
"testing"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
// buildIndex parses canonical addresses into entries, mirroring what the seed
// pipeline does, so tests exercise the real code path end to end.
func buildIndex(t *testing.T, addresses ...string) *Index {
t.Helper()
entries := make([]address.Entry, 0, len(addresses))
for i, a := range addresses {
p := address.Parse(a)
entries = append(entries, address.Entry{
ID: a, Address: a, District: "Тестовый",
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
House: p.House, HouseFrac: p.HouseFrac, HouseLetter: p.HouseLetter,
Building: p.Building, BuildingFrac: p.BuildingFrac, BuildingLetter: p.BuildingLetter,
Structure: p.Structure,
})
_ = i
}
return NewIndex("test", entries, DefaultOptions())
}
func TestResolveStatuses(t *testing.T) {
ix := buildIndex(t,
"Станционная улица 1/1",
"Станционная улица 2/1",
"Станционная улица 3/1 к1",
"Станционная улица 3/1 к2",
"Искитимская улица 5",
)
tests := []struct {
name string
query string
wantStatus address.Status
wantAddr string
}{
{
name: "found with ordinal, fraction and structure noise",
query: "2-я Станционная улица 1/1 стр.",
wantStatus: address.StatusFound,
wantAddr: "Станционная улица 1/1",
},
{
name: "found exact house",
query: "станционная 2/1",
wantStatus: address.StatusFound,
wantAddr: "Станционная улица 2/1",
},
{
name: "found with corpus",
query: "Станционная улица 3/1 корпус 1",
wantStatus: address.StatusFound,
wantAddr: "Станционная улица 3/1 к1",
},
{
name: "typo in street name tolerated",
query: "станционая 1/1",
wantStatus: address.StatusFound,
wantAddr: "Станционная улица 1/1",
},
{
name: "missing house is partial",
query: "станционная улица",
wantStatus: address.StatusPartial,
},
{
name: "wrong house fraction is partial",
query: "станционная 1/2",
wantStatus: address.StatusPartial,
},
{
name: "ambiguous building",
query: "станционная 3/1",
wantStatus: address.StatusAmbiguous,
},
{
name: "unknown street",
query: "ленина 5",
wantStatus: address.StatusNotFound,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
res := ix.Resolve(address.Parse(tt.query))
if res.Status != tt.wantStatus {
t.Fatalf("status = %q, want %q (candidates: %+v)", res.Status, tt.wantStatus, res.Candidates)
}
if tt.wantAddr != "" && res.Address != tt.wantAddr {
t.Errorf("address = %q, want %q", res.Address, tt.wantAddr)
}
if res.DataVersion != "test" {
t.Errorf("data version = %q, want test", res.DataVersion)
}
})
}
}
// TestResolveNeverInventsAddress is the central safety property: whenever the
// status is "found", the returned address must exist in the index.
func TestResolveNeverInventsAddress(t *testing.T) {
addresses := []string{
"Станционная улица 1/1",
"Станционная улица 2/1",
"Искитимская улица 5",
}
ix := buildIndex(t, addresses...)
known := make(map[string]bool, len(addresses))
for _, a := range addresses {
known[a] = true
}
queries := []string{
"2-я Станционная 1/1", "станционная 2/1 к9", "искитимская 5", "искитимская 5/1",
"несуществующая 99", "станционная 1/1 стр.3", " ",
}
for _, q := range queries {
res := ix.Resolve(address.Parse(q))
if res.Status == address.StatusFound && !known[res.Address] {
t.Fatalf("invented address %q for query %q", res.Address, q)
}
}
}
func TestRatio(t *testing.T) {
if got := ratio("станционная", "станционная"); got != 1 {
t.Errorf("identical strings ratio = %v, want 1", got)
}
if got := ratio("станционная", "искитимская"); got > 0.6 {
t.Errorf("unrelated strings ratio = %v, want < 0.6", got)
}
}
+62
View File
@@ -0,0 +1,62 @@
package match
// levenshtein returns the edit distance between two strings, measured in
// runes. It uses the rolling-row formulation: O(min(n,m)) memory.
//
// The resolver uses it only to tolerate typos in the *street name*; house
// numbers are always compared exactly, because a wrong house is worse than a
// missing one.
func levenshtein(a, b string) int {
ra, rb := []rune(a), []rune(b)
if len(ra) == 0 {
return len(rb)
}
if len(rb) == 0 {
return len(ra)
}
prev := make([]int, len(rb)+1)
curr := make([]int, len(rb)+1)
for j := range prev {
prev[j] = j
}
for i := 1; i <= len(ra); i++ {
curr[0] = i
for j := 1; j <= len(rb); j++ {
cost := 1
if ra[i-1] == rb[j-1] {
cost = 0
}
curr[j] = min3(prev[j]+1, curr[j-1]+1, prev[j-1]+cost)
}
prev, curr = curr, prev
}
return prev[len(rb)]
}
// ratio returns a 0..1 similarity where 1 means identical. It is
// 1 - distance/max(len), which is stable and easy to reason about.
func ratio(a, b string) float64 {
if a == b {
return 1
}
la, lb := len([]rune(a)), len([]rune(b))
m := la
if lb > m {
m = lb
}
if m == 0 {
return 1
}
return 1 - float64(levenshtein(a, b))/float64(m)
}
func min3(a, b, c int) int {
m := a
if b < m {
m = b
}
if c < m {
m = c
}
return m
}
+89
View File
@@ -0,0 +1,89 @@
package match
import (
"context"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
// Rewriter is an optional, pluggable query rewriter. Its only job is to
// propose alternative surface forms of a user address (for example an LLM
// normalising "кошурникова одиннадцать дробь один" to "Кошевого 11/1"). The
// rewriter never decides the answer: every proposal is parsed and resolved
// deterministically, and only entries that exist in the index can be returned.
//
// It is defined here (consumer side) so the domain stays free of any transport
// or vendor dependency.
type Rewriter interface {
Rewrite(ctx context.Context, raw string) ([]string, error)
}
// Resolver wraps an Index with an optional Rewriter.
//
// The deterministic path is always tried first. The rewriter is consulted only
// when the deterministic result is not a confident "found", and its
// suggestions can only ever turn a miss into a match against real data.
type Resolver struct {
index *Index
rewriter Rewriter
}
// NewResolver builds a resolver. rewriter may be nil to disable rewriting.
func NewResolver(index *Index, rewriter Rewriter) *Resolver {
return &Resolver{index: index, rewriter: rewriter}
}
// Version returns the current data version.
func (r *Resolver) Version() string { return r.index.Version() }
// Ready reports whether the resolver has a usable index.
func (r *Resolver) Ready() bool { return r.index != nil && r.index.Size() > 0 }
// Resolve parses raw and resolves it, consulting the rewriter when enabled.
func (r *Resolver) Resolve(ctx context.Context, raw string) Result {
parsed := address.Parse(raw)
res := r.index.Resolve(parsed)
if res.Status == address.StatusFound || r.rewriter == nil {
return res
}
if res.Status != address.StatusNotFound && res.Status != address.StatusAmbiguous {
return res
}
alternatives, err := r.rewriter.Rewrite(ctx, raw)
if err != nil || len(alternatives) == 0 {
return res
}
best := res
for _, alt := range alternatives {
candidate := r.index.Resolve(address.Parse(alt))
if better(candidate, best) {
best = candidate
}
}
return best
}
// better ranks results: a stronger status wins, then higher confidence.
func better(a, b Result) bool {
ra, rb := statusRank(a.Status), statusRank(b.Status)
if ra != rb {
return ra > rb
}
return a.Confidence > b.Confidence
}
func statusRank(s address.Status) int {
switch s {
case address.StatusFound:
return 3
case address.StatusAmbiguous:
return 2
case address.StatusPartial:
return 1
default:
return 0
}
}
+109
View File
@@ -0,0 +1,109 @@
// Package outages contains the deterministic, pure logic for selecting outages
// that belong to a concrete house unit. It does not talk to the network: the
// portal client lives in an adapter and feeds this package.
package outages
import (
"context"
"strconv"
"strings"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
// Outage is a single outage record as returned by the external portal.
type Outage struct {
Type string `json:"type,omitempty"`
System string `json:"system,omitempty"`
Address string `json:"address,omitempty"`
StartDate string `json:"start_date,omitempty"`
EndDate string `json:"end_date,omitempty"`
Time string `json:"time,omitempty"`
Action string `json:"action,omitempty"`
}
// Source is anything that can answer "what outages are there for this
// canonical address". It is implemented by the ArcGIS portal adapter and by
// fakes in tests.
type Source interface {
Query(ctx context.Context, canonical string) ([]Outage, error)
}
// MatchingHouse keeps only the outages that address the same house unit as the
// target. This is the fix for the classic bug where a substring match returned
// a neighbouring house's outage: matching is done on the parsed structure, not
// on the raw text.
func MatchingHouse(list []Outage, target address.ParsedAddress) []Outage {
out := make([]Outage, 0, len(list))
for _, o := range list {
op := address.Parse(o.Address)
if houseUnitEqual(target, op) {
out = append(out, o)
}
}
return out
}
// houseUnitEqual compares two parsed house units. House number, fraction,
// letter, building (corpus) and its fraction/letter must all match. The
// structure (строение) is deliberately ignored: it is frequently omitted by
// users and does not change which house an outage refers to.
func houseUnitEqual(a, b address.ParsedAddress) bool {
return a.House == b.House &&
a.HouseFrac == b.HouseFrac &&
a.HouseLetter == b.HouseLetter &&
a.Building == b.Building &&
a.BuildingFrac == b.BuildingFrac &&
a.BuildingLetter == b.BuildingLetter
}
// Current keeps outages whose [start, end] range covers now. The portal
// already filters by date, but we re-check locally because the portal's notion
// of "today" and ours must agree (Asia/Novosibirsk, UTC+7).
func Current(list []Outage, now time.Time, loc *time.Location) []Outage {
out := make([]Outage, 0, len(list))
for _, o := range list {
start, okStart := parsePortalTime(o.StartDate, loc)
end, okEnd := parsePortalTime(o.EndDate, loc)
if !okStart || !okEnd {
// Undated records are kept: better to report a possibly-current
// outage than to silently drop it.
out = append(out, o)
continue
}
if (now.Equal(start) || now.After(start)) && (now.Equal(end) || now.Before(end)) {
out = append(out, o)
}
}
return out
}
// parsePortalTime accepts the shapes the portal may produce: epoch
// milliseconds, epoch seconds, RFC3339 and a few common date formats. All
// results are interpreted in loc.
func parsePortalTime(s string, loc *time.Location) (time.Time, bool) {
s = strings.TrimSpace(s)
if s == "" {
return time.Time{}, false
}
if n, err := strconv.ParseInt(s, 10, 64); err == nil {
// Heuristic: 13+ digits is milliseconds, otherwise seconds.
if n > 1e12 {
return time.UnixMilli(n).In(loc), true
}
return time.Unix(n, 0).In(loc), true
}
layouts := []string{
time.RFC3339,
"2006-01-02T15:04:05",
"2006-01-02 15:04:05",
"2006-01-02",
}
for _, layout := range layouts {
if t, err := time.ParseInLocation(layout, s, loc); err == nil {
return t, true
}
}
return time.Time{}, false
}
+57
View File
@@ -0,0 +1,57 @@
package outages
import (
"strconv"
"testing"
"time"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
func TestMatchingHouse(t *testing.T) {
list := []Outage{
{Address: "Станционная улица 1/1", System: "Теплоснабжение"},
{Address: "Станционная улица 1/2", System: "Теплоснабжение"},
{Address: "Станционная улица 1/1 к1", System: "Теплоснабжение"},
{Address: "Станционная улица 10", System: "Теплоснабжение"},
}
target := address.Parse("2-я Станционная улица 1/1")
got := MatchingHouse(list, target)
if len(got) != 1 {
t.Fatalf("expected exactly 1 matching outage, got %d: %+v", len(got), got)
}
if got[0].Address != "Станционная улица 1/1" {
t.Errorf("matched wrong outage: %q", got[0].Address)
}
}
func TestMatchingHouseIgnoresStructure(t *testing.T) {
list := []Outage{{Address: "Станционная улица 1/1 стр.2"}}
target := address.Parse("станционная 1/1 стр.3")
if got := MatchingHouse(list, target); len(got) != 1 {
t.Fatalf("structure must not affect house matching; got %d", len(got))
}
}
func TestCurrent(t *testing.T) {
loc := time.FixedZone("NSK", 7*3600)
now := time.Date(2026, 10, 9, 12, 0, 0, 0, loc)
within := now.Add(-time.Hour).UnixMilli()
after := now.Add(time.Hour).UnixMilli()
before := now.Add(-48 * time.Hour).UnixMilli()
list := []Outage{
{Address: "A", StartDate: itoa(within), EndDate: itoa(after)}, // current
{Address: "B", StartDate: itoa(before), EndDate: itoa(before)}, // past
{Address: "C", StartDate: "", EndDate: ""}, // undated -> kept
}
got := Current(list, now, loc)
if len(got) != 2 {
t.Fatalf("expected 2 current outages, got %d: %+v", len(got), got)
}
}
func itoa(v int64) string {
return strconv.FormatInt(v, 10)
}
+52
View File
@@ -0,0 +1,52 @@
package seed
import (
"math"
"os"
"testing"
)
// TestRealRegistryParseRate validates the parser against a real registry dump.
// It is skipped unless ADDRKIT_REGISTRY points at one, so it never runs in a
// plain `go test ./...` (no large fixture in the repo).
//
// Run it explicitly:
//
// ADDRKIT_REGISTRY=/path/to/address_registry.ndjson \
// go test ./internal/seed -run TestRealRegistryParseRate -v
func TestRealRegistryParseRate(t *testing.T) {
path := os.Getenv("ADDRKIT_REGISTRY")
if path == "" {
t.Skip("ADDRKIT_REGISTRY not set")
}
f, err := os.Open(path)
if err != nil {
t.Fatalf("open registry: %v", err)
}
defer f.Close()
sources, err := ReadBulkNDJSON(f)
if err != nil {
t.Fatalf("read registry: %v", err)
}
entries, stats := Build(sources)
rate := 0.0
if stats.Total > 0 {
rate = float64(stats.Parsed) / float64(stats.Total) * 100
}
t.Logf("total=%d parsed=%d unparsed=%d distinct_streets=%d parse_rate=%.2f%%",
stats.Total, stats.Parsed, stats.Unparsed, stats.DistinctStreets, rate)
if len(stats.UnparsedSample) > 0 {
t.Logf("unparsed sample: %v", stats.UnparsedSample)
}
// A well-formed registry should parse almost perfectly. If this drops we
// want a loud failure, not a silently degraded index.
if stats.Total > 0 && rate < 98.0 {
t.Errorf("parse rate %.2f%% below 98%%", math.Round(rate*100)/100)
}
if len(entries) == 0 {
t.Error("no entries produced")
}
}
+165
View File
@@ -0,0 +1,165 @@
// Package seed turns a registry dump into canonical address entries. It is the
// only place that knows how the source data is shaped.
//
// The same parser used at query time is used here, so the index and the query
// never disagree about how an address decomposes.
package seed
import (
"bufio"
"encoding/json"
"io"
"sort"
"strings"
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
)
// Source is one raw registry row.
type Source struct {
ID string
Address string
District string
}
// Stats summarises a build and, crucially, surfaces rows the parser could not
// structure. A non-empty UnparsedCount is a data-quality signal worth alerting
// on, not an error.
type Stats struct {
Total int `json:"total"`
Parsed int `json:"parsed"`
Unparsed int `json:"unparsed"`
Duplicates int `json:"duplicates"`
DistinctStreets int `json:"distinct_streets"`
UnparsedSample []string `json:"unparsed_sample,omitempty"`
}
// ReadBulkNDJSON reads an Elasticsearch Bulk NDJSON dump: pairs of an action
// line ({"index":{"_id":...}}) followed by a source line
// ({"address":...,"district_name":...}). Bare source lines (no action line)
// are also accepted, which makes plain JSONL usable too.
func ReadBulkNDJSON(r io.Reader) ([]Source, error) {
sc := bufio.NewScanner(r)
// 1 MiB per line is far beyond any realistic address row.
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
var (
sources []Source
pendingID string
)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" {
continue
}
var obj map[string]json.RawMessage
if err := json.Unmarshal([]byte(line), &obj); err != nil {
continue // skip malformed lines rather than aborting the whole seed
}
if action, ok := obj["index"]; ok {
var a struct {
ID string `json:"_id"`
}
_ = json.Unmarshal(action, &a)
pendingID = a.ID
continue
}
if addrRaw, ok := obj["address"]; ok {
var addr string
if err := json.Unmarshal(addrRaw, &addr); err != nil || addr == "" {
continue
}
src := Source{ID: pendingID, Address: addr}
if d, ok := obj["district_name"]; ok {
_ = json.Unmarshal(d, &src.District)
}
sources = append(sources, src)
pendingID = ""
}
}
if err := sc.Err(); err != nil {
return nil, err
}
return sources, nil
}
// Build parses every source row into a canonical Entry and reports statistics.
// Rows without a house number are still indexed for street lookups but counted
// as unparsed, since a registry row without a house is suspicious.
func Build(sources []Source) ([]address.Entry, Stats) {
entries := make([]address.Entry, 0, len(sources))
streets := make(map[string]struct{})
seen := make(map[string]struct{})
stats := Stats{Total: len(sources)}
for _, src := range sources {
p := address.Parse(src.Address)
if p.NameKey == "" {
stats.Unparsed++
appendSample(&stats, src.Address)
continue
}
entry := address.Entry{
ID: src.ID,
Address: src.Address,
District: src.District,
Ordinal: p.Ordinal,
NameKey: p.NameKey,
NameDisplay: p.NameDisplay,
StreetType: p.StreetType,
StreetKey: p.StreetKey,
House: p.House,
HouseFrac: p.HouseFrac,
HouseLetter: p.HouseLetter,
Building: p.Building,
BuildingFrac: p.BuildingFrac,
BuildingLetter: p.BuildingLetter,
Structure: p.Structure,
}
// The source registry contains rows that differ only by "стр."
// (structure). Since structure does not affect which house an outage
// belongs to, drop such duplicates rather than surfacing them as
// spurious ambiguity at query time.
if key := dedupKey(entry); key != "" {
if _, ok := seen[key]; ok {
stats.Duplicates++
continue
}
seen[key] = struct{}{}
}
entries = append(entries, entry)
streets[p.StreetKey] = struct{}{}
if p.HasHouse {
stats.Parsed++
} else {
stats.Unparsed++
appendSample(&stats, src.Address)
}
}
stats.DistinctStreets = len(streets)
sort.Strings(stats.UnparsedSample)
return entries, stats
}
// dedupKey identifies the same house unit within the same street, ignoring the
// structure component.
func dedupKey(e address.Entry) string {
if e.House == "" {
return ""
}
return strings.Join([]string{
e.StreetKey, e.House, e.HouseFrac, e.HouseLetter,
e.Building, e.BuildingFrac, e.BuildingLetter,
}, "|")
}
func appendSample(s *Stats, address string) {
const maxSamples = 20
if len(s.UnparsedSample) < maxSamples {
s.UnparsedSample = append(s.UnparsedSample, address)
}
}
+55
View File
@@ -0,0 +1,55 @@
package seed
import (
"strings"
"testing"
)
func TestReadBulkNDJSON(t *testing.T) {
input := `{"index": {"_index": "address_registry", "_id": "a1"}}
{"address": "Станционная улица 1/1", "district_name": "Ленинский"}
{"index": {"_id": "a2"}}
{"address": "Искитимская улица 5", "district_name": "Кировский"}
{"address": "Третья улица 7"}
`
src, err := ReadBulkNDJSON(strings.NewReader(input))
if err != nil {
t.Fatalf("ReadBulkNDJSON: %v", err)
}
if len(src) != 3 {
t.Fatalf("got %d sources, want 3", len(src))
}
if src[0].ID != "a1" || src[0].District != "Ленинский" {
t.Errorf("first source = %+v", src[0])
}
if src[2].Address != "Третья улица 7" {
t.Errorf("bare source not read: %+v", src[2])
}
}
func TestBuild(t *testing.T) {
sources := []Source{
{ID: "1", Address: "Станционная улица 1/1"},
{ID: "2", Address: "Искитимская улица 5"},
{ID: "3", Address: "Станционная улица"}, // no house -> unparsed
}
entries, stats := Build(sources)
if stats.Total != 3 {
t.Errorf("Total = %d, want 3", stats.Total)
}
if stats.Parsed != 2 {
t.Errorf("Parsed = %d, want 2", stats.Parsed)
}
if stats.Unparsed != 1 {
t.Errorf("Unparsed = %d, want 1", stats.Unparsed)
}
if len(entries) != 3 {
t.Errorf("entries = %d, want 3", len(entries))
}
// Two distinct streets: "станционная" and "искитимская".
if stats.DistinctStreets != 2 {
t.Errorf("DistinctStreets = %d, want 2", stats.DistinctStreets)
}
}
+120
View File
@@ -0,0 +1,120 @@
openapi: 3.1.0
info:
title: addrkit
version: "1.0.0"
description: >
Детерминированный сервис нормализации и поиска адресов. Ответ всегда
берётся из реестра; LLM не участвует в пути корректности.
servers:
- url: http://localhost:6171
paths:
/v1/resolve:
post:
summary: Разобрать и сопоставить адрес
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [address]
properties:
address:
type: string
example: "2-я Станционная улица 1/1 стр."
responses:
"200":
description: Результат сопоставления
content:
application/json:
schema:
$ref: "#/components/schemas/ResolveResult"
"400":
description: Некорректный запрос
/v1/outages:
post:
summary: Отключения по адресу (после строгого сопоставления дома)
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [address]
properties:
address:
type: string
responses:
"200":
description: Канонический адрес и отключения
content:
application/json:
schema:
$ref: "#/components/schemas/OutagesResult"
/v1/healthz:
get:
summary: Живость
responses:
"200": { description: ok }
/v1/readyz:
get:
summary: Готовность (индекс загружен)
responses:
"200": { description: ready }
"503": { description: not ready }
/v1/version:
get:
summary: Версия данных
responses:
"200":
description: ok
content:
application/json:
schema:
type: object
properties:
data_version: { type: string }
components:
schemas:
Status:
type: string
enum: [found, partial, ambiguous, not_found, portal_timeout, infra_error]
Candidate:
type: object
properties:
address: { type: string }
district: { type: string }
house: { type: string }
score: { type: number, format: double }
ResolveResult:
type: object
required: [status, confidence]
properties:
status: { $ref: "#/components/schemas/Status" }
address: { type: string }
district: { type: string }
confidence: { type: number, format: double }
candidates:
type: array
items: { $ref: "#/components/schemas/Candidate" }
data_version: { type: string }
Outage:
type: object
properties:
type: { type: string }
system: { type: string }
address: { type: string }
start_date: { type: string }
end_date: { type: string }
action: { type: string }
OutagesResult:
type: object
properties:
status: { $ref: "#/components/schemas/Status" }
address: { type: string }
district: { type: string }
outages:
type: array
items: { $ref: "#/components/schemas/Outage" }
as_of: { type: string }
data_version: { type: string }