addrkit: deterministic address resolution service (Go)
- domain: address normalization/parsing, matching with explicit statuses - adapters: SQLite registry store, ArcGIS outages client, optional LLM rewriter, HTTP API - seed pipeline with duplicate handling and unparsed reporting - CLI (seed/verify/resolve), systemd unit, OpenAPI contract, tests
This commit is contained in:
+14
@@ -0,0 +1,14 @@
|
|||||||
|
# Build artifacts
|
||||||
|
bin/
|
||||||
|
|
||||||
|
# Local config and data (never committed)
|
||||||
|
config.json
|
||||||
|
*.db
|
||||||
|
*.db-wal
|
||||||
|
*.db-shm
|
||||||
|
*.ndjson
|
||||||
|
|
||||||
|
# Editor / OS
|
||||||
|
.DS_Store
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 addrkit contributors
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
GO ?= go
|
||||||
|
BIN_DIR ?= bin
|
||||||
|
|
||||||
|
.PHONY: all build test vet fmt tidy run clean seed
|
||||||
|
|
||||||
|
all: build
|
||||||
|
|
||||||
|
## build: compile the service and the CLI as static binaries
|
||||||
|
build:
|
||||||
|
CGO_ENABLED=0 $(GO) build -trimpath -o $(BIN_DIR)/addrkitd ./cmd/addrkitd
|
||||||
|
CGO_ENABLED=0 $(GO) build -trimpath -o $(BIN_DIR)/addrkit ./cmd/addrkit
|
||||||
|
|
||||||
|
## test: run the full test suite with the race detector
|
||||||
|
test:
|
||||||
|
$(GO) test -race ./...
|
||||||
|
|
||||||
|
## vet: static analysis
|
||||||
|
vet:
|
||||||
|
$(GO) vet ./...
|
||||||
|
|
||||||
|
## fmt: format all Go sources
|
||||||
|
fmt:
|
||||||
|
gofmt -l -w .
|
||||||
|
|
||||||
|
## tidy: sync go.mod/go.sum
|
||||||
|
tidy:
|
||||||
|
$(GO) mod tidy
|
||||||
|
|
||||||
|
## run: build and start the service with the local config
|
||||||
|
run: build
|
||||||
|
$(BIN_DIR)/addrkitd -config config.json
|
||||||
|
|
||||||
|
## seed: build registry.db from a registry dump (override IN=/OUT=)
|
||||||
|
IN ?= registry.ndjson
|
||||||
|
OUT ?= registry.db
|
||||||
|
seed:
|
||||||
|
$(BIN_DIR)/addrkit seed -in $(IN) -out $(OUT)
|
||||||
|
|
||||||
|
## clean: remove build artifacts
|
||||||
|
clean:
|
||||||
|
rm -rf $(BIN_DIR)
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
# addrkit
|
||||||
|
|
||||||
|
**Детерминированный сервис нормализации и поиска адресов** с подключаемым
|
||||||
|
внешним слоем отключений. Разбирает вольные реплики (дроби, литеры, корпуса,
|
||||||
|
строения, опечатки), сопоставляет их с каноническим реестром и отдаёт ответ
|
||||||
|
**только из данных** — LLM не участвует в пути корректности.
|
||||||
|
|
||||||
|
Проект на Go, живёт офлайн (один статический бинарь + SQLite), без MCP в
|
||||||
|
рантайме.
|
||||||
|
|
||||||
|
## Зачем
|
||||||
|
|
||||||
|
Существующая логика поиска адреса в диалоговом сценарии страдала от
|
||||||
|
нечёткого поиска по всей строке (`fuzziness: AUTO`, `size:1`), из-за чего
|
||||||
|
дробь/корпус/литера могли дать неверный дом. `addrkit` разделяет задачу на
|
||||||
|
компоненты, сопоставляет дом строго и вместо угадывания возвращает явный
|
||||||
|
статус (`partial`/`ambiguous`).
|
||||||
|
|
||||||
|
## Архитектура
|
||||||
|
|
||||||
|
Гексагональная: домен чистый и без I/O, всё внешнее — в адаптерах.
|
||||||
|
|
||||||
|
```
|
||||||
|
cmd/addrkitd сервис (HTTP)
|
||||||
|
cmd/addrkit CLI: seed | verify | resolve
|
||||||
|
internal/
|
||||||
|
domain/address нормализация и разбор адреса (чистые функции)
|
||||||
|
domain/match сопоставление с индексом, статусы
|
||||||
|
domain/outages выборка отключений по дому (чистая логика)
|
||||||
|
adapters/registrystore SQLite (modernc, без CGO)
|
||||||
|
adapters/arcgis клиент портала (timeout/retry)
|
||||||
|
adapters/llm опциональный rewriter (OpenAI-совместимый)
|
||||||
|
adapters/httpapi версионированный JSON API
|
||||||
|
config конфиг + ${VAR}
|
||||||
|
seed сборка индекса из дампа реестра
|
||||||
|
```
|
||||||
|
|
||||||
|
Поток запроса:
|
||||||
|
|
||||||
|
```
|
||||||
|
"2-я Станционная улица 1/1 стр."
|
||||||
|
-> address.Parse -> {ordinal:2, name:станционная, house:1, frac:1}
|
||||||
|
-> match.Resolve -> exact street -> exact house -> FOUND
|
||||||
|
-> (опц.) outages.Source -> фильтр по дому -> ответ
|
||||||
|
```
|
||||||
|
|
||||||
|
## Быстрый старт
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make build
|
||||||
|
|
||||||
|
# 1. Собрать индекс из дампа реестра (Bulk NDJSON или JSONL).
|
||||||
|
./bin/addrkit seed -in registry.ndjson -out registry.db -version 2026-10-01
|
||||||
|
|
||||||
|
# 2. Запустить сервис.
|
||||||
|
cp config.example.json config.json
|
||||||
|
ADDRKIT_PORTAL_URL='https://map.example/.../101/query' ./bin/addrkitd -config config.json
|
||||||
|
|
||||||
|
# 3. Проверить.
|
||||||
|
curl -s localhost:6171/v1/resolve -d '{"address":"2-я Станционная улица 1/1 стр."}'
|
||||||
|
curl -s localhost:6171/v1/outages -d '{"address":"станционная 30а к9/1"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
CLI без сервиса:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/addrkit resolve -db registry.db "кошурникова 11/1"
|
||||||
|
./bin/addrkit verify -db registry.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## API v1
|
||||||
|
|
||||||
|
| Метод | Путь | Назначение |
|
||||||
|
|---|---|---|
|
||||||
|
| POST | `/v1/resolve` | адрес → `{status, address, district, confidence, candidates[]}` |
|
||||||
|
| POST | `/v1/outages` | адрес → канонический адрес + отключения по дому |
|
||||||
|
| GET | `/v1/healthz` | живость |
|
||||||
|
| GET | `/v1/readyz` | готовность (индекс загружен) |
|
||||||
|
| GET | `/v1/version` | версия данных |
|
||||||
|
|
||||||
|
Статусы: `found`, `partial`, `ambiguous`, `not_found`, `portal_timeout`,
|
||||||
|
`infra_error`.
|
||||||
|
|
||||||
|
## Конфигурация
|
||||||
|
|
||||||
|
`config.json` (см. `config.example.json`). Значения поддерживают `${VAR}`;
|
||||||
|
незаданная переменная = ошибка запуска (fail closed). Секреты — только из
|
||||||
|
окружения.
|
||||||
|
|
||||||
|
## Принятые решения
|
||||||
|
|
||||||
|
- **Детерминизм.** Один вход — один выход, порядок кандидатов стабилен.
|
||||||
|
`resolve` не зависит от сети.
|
||||||
|
- **Дом сравнивается строго.** Литера/дробь/корпус должны совпасть; иначе
|
||||||
|
`partial`, а не «похожий» дом.
|
||||||
|
- **LLM — опция, выключена по умолчанию.** `adapters/llm` лишь предлагает
|
||||||
|
варианты записи адреса; результат всегда берётся из реестра, поэтому модель
|
||||||
|
не может «выдумать» адрес.
|
||||||
|
- **Данные.** Сервис владеет своим индексом, построенным из дампа реестра.
|
||||||
|
Источник истины и регламент обновления — предмет отдельного согласования.
|
||||||
|
- **Часовой пояс.** Даты отключений трактуются в `Asia/Novosibirsk` (UTC+7).
|
||||||
|
|
||||||
|
## Тесты
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # go test -race ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверка парсера на реальном реестре (вне репозитория):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ADDRKIT_REGISTRY=/path/to/address_registry.ndjson \
|
||||||
|
go test ./internal/seed -run TestRealRegistryParseRate -v
|
||||||
|
```
|
||||||
|
|
||||||
|
На дампе из 73 936 строк парсер распознаёт **99.98 %** (14 строк — заведомо
|
||||||
|
битые форматы; они не угадываются, а попадают в отчёт `stats.unparsed_sample`).
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
// Command addrkit is the operator CLI: it seeds the canonical registry from a
|
||||||
|
// registry dump, verifies a built database and resolves a single address for
|
||||||
|
// quick checks.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/adapters/registrystore"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/seed"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
if len(os.Args) < 2 {
|
||||||
|
usage()
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
switch os.Args[1] {
|
||||||
|
case "seed":
|
||||||
|
os.Exit(runSeed(os.Args[2:]))
|
||||||
|
case "verify":
|
||||||
|
os.Exit(runVerify(os.Args[2:]))
|
||||||
|
case "resolve":
|
||||||
|
os.Exit(runResolve(os.Args[2:]))
|
||||||
|
case "help", "-h", "--help":
|
||||||
|
usage()
|
||||||
|
os.Exit(0)
|
||||||
|
default:
|
||||||
|
fmt.Fprintf(os.Stderr, "unknown command %q\n", os.Args[1])
|
||||||
|
usage()
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func usage() {
|
||||||
|
fmt.Fprint(os.Stderr, `addrkit — address registry toolkit
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
addrkit seed -in registry.ndjson -out registry.db [-version 2026-10-09]
|
||||||
|
addrkit verify -db registry.db
|
||||||
|
addrkit resolve -db registry.db "2-я Станционная улица 1/1 стр."
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func runSeed(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("seed", flag.ExitOnError)
|
||||||
|
in := fs.String("in", "", "input registry dump (Bulk NDJSON or JSONL)")
|
||||||
|
out := fs.String("out", "", "output SQLite database")
|
||||||
|
version := fs.String("version", "", "data version (defaults to input file mtime)")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
|
||||||
|
if *in == "" || *out == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "seed: -in and -out are required")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
if *version == "" {
|
||||||
|
if fi, err := os.Stat(*in); err == nil {
|
||||||
|
*version = fi.ModTime().UTC().Format("20060102T150405Z")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
f, err := os.Open(*in)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "seed: open input: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
|
||||||
|
sources, err := seed.ReadBulkNDJSON(f)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "seed: read: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
entries, stats := seed.Build(sources)
|
||||||
|
if stats.Parsed == 0 {
|
||||||
|
fmt.Fprintln(os.Stderr, "seed: no parseable entries; refusing to write empty index")
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := registrystore.Save(*out, *version, entries); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "seed: save: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
enc := json.NewEncoder(os.Stdout)
|
||||||
|
enc.SetIndent("", " ")
|
||||||
|
_ = enc.Encode(map[string]any{
|
||||||
|
"output": *out,
|
||||||
|
"version": *version,
|
||||||
|
"entries": len(entries),
|
||||||
|
"stats": stats,
|
||||||
|
})
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func runVerify(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("verify", flag.ExitOnError)
|
||||||
|
db := fs.String("db", "", "SQLite database")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
if *db == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "verify: -db is required")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
version, entries, err := registrystore.Load(*db)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "verify: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
streets := make(map[string]struct{})
|
||||||
|
for _, e := range entries {
|
||||||
|
streets[e.StreetKey] = struct{}{}
|
||||||
|
}
|
||||||
|
enc := json.NewEncoder(os.Stdout)
|
||||||
|
enc.SetIndent("", " ")
|
||||||
|
_ = enc.Encode(map[string]any{
|
||||||
|
"version": version,
|
||||||
|
"entries": len(entries),
|
||||||
|
"distinct_streets": len(streets),
|
||||||
|
})
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func runResolve(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("resolve", flag.ExitOnError)
|
||||||
|
db := fs.String("db", "", "SQLite database")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
rest := fs.Args()
|
||||||
|
if *db == "" || len(rest) == 0 {
|
||||||
|
fmt.Fprintln(os.Stderr, `resolve: -db and an address are required`)
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
version, entries, err := registrystore.Load(*db)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "resolve: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
resolver := match.NewResolver(match.NewIndex(version, entries, match.DefaultOptions()), nil)
|
||||||
|
res := resolver.Resolve(context.Background(), rest[0])
|
||||||
|
enc := json.NewEncoder(os.Stdout)
|
||||||
|
enc.SetIndent("", " ")
|
||||||
|
_ = enc.Encode(res)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
// Command addrkitd runs the address resolution service.
|
||||||
|
//
|
||||||
|
// It loads the canonical registry into memory, wires the optional adapters
|
||||||
|
// (ArcGIS portal, LLM rewriter) and serves the versioned JSON API. The process
|
||||||
|
// fails closed: if the registry cannot be loaded it exits rather than serving
|
||||||
|
// an empty index.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/adapters/arcgis"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/adapters/httpapi"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/adapters/llm"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/adapters/registrystore"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/config"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
cfgPath := flag.String("config", "config.json", "path to the JSON config file")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
cfg, err := config.Load(*cfgPath)
|
||||||
|
if err != nil {
|
||||||
|
slog.Error("config", "error", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
logger := newLogger(cfg.LogLevel)
|
||||||
|
|
||||||
|
version, entries, err := registrystore.Load(cfg.DataPath)
|
||||||
|
if err != nil {
|
||||||
|
// Fail closed: no registry, no service.
|
||||||
|
logger.Error("load registry", "error", err, "path", cfg.DataPath)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
logger.Info("registry loaded", "version", version, "entries", len(entries))
|
||||||
|
|
||||||
|
index := match.NewIndex(version, entries, match.Options{
|
||||||
|
FuzzyThreshold: cfg.Resolve.FuzzyThreshold,
|
||||||
|
AmbiguityMargin: cfg.Resolve.AmbiguityMargin,
|
||||||
|
MaxCandidates: cfg.Resolve.MaxCandidates,
|
||||||
|
})
|
||||||
|
|
||||||
|
var rewriter match.Rewriter
|
||||||
|
if cfg.LLM.Enabled {
|
||||||
|
rewriter = llm.New(cfg.LLM)
|
||||||
|
logger.Info("llm rewriter enabled", "url", cfg.LLM.URL, "model", cfg.LLM.Model)
|
||||||
|
}
|
||||||
|
resolver := match.NewResolver(index, rewriter)
|
||||||
|
|
||||||
|
portal := arcgis.New(
|
||||||
|
cfg.Portal.URL,
|
||||||
|
time.Duration(cfg.Portal.TimeoutSeconds)*time.Second,
|
||||||
|
cfg.Portal.Retries,
|
||||||
|
)
|
||||||
|
|
||||||
|
loc := operationalLocation()
|
||||||
|
srv := httpapi.New(resolver, portal, logger, loc)
|
||||||
|
|
||||||
|
httpServer := &http.Server{
|
||||||
|
Addr: cfg.Addr,
|
||||||
|
Handler: srv.Handler(),
|
||||||
|
ReadHeaderTimeout: 5 * time.Second,
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
logger.Info("listening", "addr", cfg.Addr, "tz", loc.String())
|
||||||
|
if err := httpServer.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||||
|
logger.Error("serve", "error", err)
|
||||||
|
stop()
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
<-ctx.Done()
|
||||||
|
logger.Info("shutting down")
|
||||||
|
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if err := httpServer.Shutdown(shutdownCtx); err != nil {
|
||||||
|
logger.Error("shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newLogger(level string) *slog.Logger {
|
||||||
|
var lvl slog.Level
|
||||||
|
switch level {
|
||||||
|
case "debug":
|
||||||
|
lvl = slog.LevelDebug
|
||||||
|
case "warn":
|
||||||
|
lvl = slog.LevelWarn
|
||||||
|
case "error":
|
||||||
|
lvl = slog.LevelError
|
||||||
|
default:
|
||||||
|
lvl = slog.LevelInfo
|
||||||
|
}
|
||||||
|
return slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: lvl}))
|
||||||
|
}
|
||||||
|
|
||||||
|
// operationalLocation returns Asia/Novosibirsk when the tz database is
|
||||||
|
// available and falls back to a fixed UTC+7 offset otherwise (containers
|
||||||
|
// without tzdata).
|
||||||
|
func operationalLocation() *time.Location {
|
||||||
|
if loc, err := time.LoadLocation("Asia/Novosibirsk"); err == nil {
|
||||||
|
return loc
|
||||||
|
}
|
||||||
|
return time.FixedZone("Asia/Novosibirsk", 7*3600)
|
||||||
|
}
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"addr": ":6171",
|
||||||
|
"data_path": "registry.db",
|
||||||
|
"log_level": "info",
|
||||||
|
"resolve": {
|
||||||
|
"fuzzy_threshold": 0.82,
|
||||||
|
"ambiguity_margin": 0.03,
|
||||||
|
"max_candidates": 5
|
||||||
|
},
|
||||||
|
"portal": {
|
||||||
|
"url": "${ADDRKIT_PORTAL_URL}",
|
||||||
|
"timeout_seconds": 2,
|
||||||
|
"retries": 1
|
||||||
|
},
|
||||||
|
"llm": {
|
||||||
|
"enabled": false,
|
||||||
|
"url": "${ADDRKIT_LLM_URL}",
|
||||||
|
"token": "${ADDRKIT_LLM_TOKEN}",
|
||||||
|
"model": "qwen3-14b-awq",
|
||||||
|
"timeout_seconds": 3
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=addrkit address resolution service
|
||||||
|
Documentation=file:///opt/addrkit/README.md
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=lexicom
|
||||||
|
Group=lexicom
|
||||||
|
WorkingDirectory=/opt/addrkit
|
||||||
|
# Environment carries the ${VAR} secrets referenced by config.json.
|
||||||
|
EnvironmentFile=-/etc/addrkit/env
|
||||||
|
ExecStart=/opt/addrkit/bin/addrkitd -config /opt/addrkit/config.json
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
# Defence in depth: the service only reads its own directory.
|
||||||
|
NoNewPrivileges=true
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=true
|
||||||
|
PrivateTmp=true
|
||||||
|
ReadOnlyPaths=/opt/addrkit
|
||||||
|
StandardOutput=journal
|
||||||
|
StandardError=journal
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
module git.totmin.ru/en2zmax/addrkit
|
||||||
|
|
||||||
|
go 1.27.1
|
||||||
|
|
||||||
|
require (
|
||||||
|
golang.org/x/text v0.42.0
|
||||||
|
modernc.org/sqlite v1.60.1
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||||
|
github.com/google/uuid v1.6.0 // indirect
|
||||||
|
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||||
|
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||||
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||||
|
golang.org/x/sys v0.48.0 // indirect
|
||||||
|
modernc.org/libc v1.77.1 // indirect
|
||||||
|
modernc.org/mathutil v1.7.1 // indirect
|
||||||
|
modernc.org/memory v1.12.1 // indirect
|
||||||
|
)
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||||
|
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||||
|
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
|
||||||
|
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
|
||||||
|
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||||
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
|
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||||
|
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
||||||
|
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
|
||||||
|
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||||
|
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||||
|
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||||
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||||
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||||
|
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
|
||||||
|
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
|
||||||
|
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
|
||||||
|
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
|
||||||
|
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
||||||
|
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||||
|
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
|
||||||
|
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
|
||||||
|
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
|
||||||
|
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
|
||||||
|
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
|
||||||
|
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
|
||||||
|
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
|
||||||
|
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
|
||||||
|
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
|
||||||
|
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
|
||||||
|
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
|
||||||
|
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
|
||||||
|
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
|
||||||
|
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
|
||||||
|
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
|
||||||
|
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
|
||||||
|
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
|
||||||
|
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
|
||||||
|
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
|
||||||
|
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
|
||||||
|
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
|
||||||
|
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||||
|
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
|
||||||
|
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
|
||||||
|
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
|
||||||
|
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
|
||||||
|
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
|
||||||
|
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
|
||||||
|
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
|
||||||
|
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
|
||||||
|
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
|
||||||
|
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
|
||||||
@@ -0,0 +1,226 @@
|
|||||||
|
// Package arcgis is the adapter for the external outages portal exposed as an
|
||||||
|
// ArcGIS MapServer /query endpoint.
|
||||||
|
//
|
||||||
|
// The portal only supports substring (LIKE) filtering, so precision is
|
||||||
|
// guaranteed locally: whatever the portal returns is re-filtered by the
|
||||||
|
// outages domain using the parsed house unit. The client adds the reliability
|
||||||
|
// the portal lacks: a hard timeout, bounded retries with backoff and context
|
||||||
|
// cancellation.
|
||||||
|
package arcgis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Client is a bounded HTTP client for the portal.
|
||||||
|
type Client struct {
|
||||||
|
baseURL string
|
||||||
|
http *http.Client
|
||||||
|
retries int
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a client. timeout applies to a single attempt; retries is the
|
||||||
|
// number of *additional* attempts after the first.
|
||||||
|
func New(baseURL string, timeout time.Duration, retries int) *Client {
|
||||||
|
return &Client{
|
||||||
|
baseURL: baseURL,
|
||||||
|
http: &http.Client{Timeout: timeout},
|
||||||
|
retries: retries,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Query asks the portal for outages matching the canonical address. The
|
||||||
|
// returned records are raw portal rows; callers must still filter by house.
|
||||||
|
func (c *Client) Query(ctx context.Context, canonical string) ([]outages.Outage, error) {
|
||||||
|
params := url.Values{}
|
||||||
|
params.Set("where", "address like '"+likePattern(canonical)+"'")
|
||||||
|
params.Set("outFields", "")
|
||||||
|
params.Set("returnGeometry", "false")
|
||||||
|
params.Set("f", "json")
|
||||||
|
|
||||||
|
var lastErr error
|
||||||
|
for attempt := 0; attempt <= c.retries; attempt++ {
|
||||||
|
if attempt > 0 {
|
||||||
|
backoff := time.Duration(attempt) * 150 * time.Millisecond
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return nil, ctx.Err()
|
||||||
|
case <-time.After(backoff):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rows, err := c.do(ctx, params)
|
||||||
|
if err == nil {
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
|
lastErr = err
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("portal query failed after %d attempts: %w", c.retries+1, lastErr)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) do(ctx context.Context, params url.Values) ([]outages.Outage, error) {
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"?"+params.Encode(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("build request: %w", err)
|
||||||
|
}
|
||||||
|
resp, err := c.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("portal status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
// Bound the response size to protect the service from a misbehaving portal.
|
||||||
|
body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return parseResponse(body)
|
||||||
|
}
|
||||||
|
|
||||||
|
// portalResponse mirrors the subset of the ArcGIS /query envelope we use.
|
||||||
|
type portalResponse struct {
|
||||||
|
Features []struct {
|
||||||
|
Value string `json:"value"`
|
||||||
|
Attributes map[string]any `json:"attributes"`
|
||||||
|
} `json:"features"`
|
||||||
|
Fields []struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Domain struct {
|
||||||
|
CodedValues []struct {
|
||||||
|
Code any `json:"code"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
} `json:"codedValues"`
|
||||||
|
} `json:"domain"`
|
||||||
|
} `json:"fields"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseResponse(body []byte) ([]outages.Outage, error) {
|
||||||
|
var pr portalResponse
|
||||||
|
if err := json.Unmarshal(body, &pr); err != nil {
|
||||||
|
return nil, fmt.Errorf("decode portal response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build code -> name maps from the coded-value domains.
|
||||||
|
maps := map[string]map[string]string{}
|
||||||
|
for _, f := range pr.Fields {
|
||||||
|
if len(f.Domain.CodedValues) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
m := make(map[string]string, len(f.Domain.CodedValues))
|
||||||
|
for _, cv := range f.Domain.CodedValues {
|
||||||
|
m[codeKey(cv.Code)] = cv.Name
|
||||||
|
}
|
||||||
|
maps[f.Name] = m
|
||||||
|
}
|
||||||
|
|
||||||
|
out := make([]outages.Outage, 0, len(pr.Features))
|
||||||
|
for _, feat := range pr.Features {
|
||||||
|
o := outages.Outage{
|
||||||
|
Address: addressOf(feat.Attributes, feat.Value),
|
||||||
|
Type: lookup(maps, "type_id", feat.Attributes),
|
||||||
|
System: lookup(maps, "system_id", feat.Attributes),
|
||||||
|
Time: lookup(maps, "time_category", feat.Attributes),
|
||||||
|
StartDate: stringOf(feat.Attributes["start_date"]),
|
||||||
|
EndDate: stringOf(feat.Attributes["end_date"]),
|
||||||
|
}
|
||||||
|
o.Action = actionFor(o.System)
|
||||||
|
out = append(out, o)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func addressOf(attrs map[string]any, fallback string) string {
|
||||||
|
if v, ok := attrs["address"]; ok {
|
||||||
|
if s := stringOf(v); s != "" {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
func lookup(maps map[string]map[string]string, field string, attrs map[string]any) string {
|
||||||
|
m, ok := maps[field]
|
||||||
|
if !ok {
|
||||||
|
return stringOf(attrs[field])
|
||||||
|
}
|
||||||
|
return m[codeKey(attrs[field])]
|
||||||
|
}
|
||||||
|
|
||||||
|
// codeKey normalises a coded value coming from JSON (float64) or a string.
|
||||||
|
func codeKey(v any) string {
|
||||||
|
switch t := v.(type) {
|
||||||
|
case nil:
|
||||||
|
return ""
|
||||||
|
case string:
|
||||||
|
return t
|
||||||
|
case float64:
|
||||||
|
return strconv.FormatInt(int64(t), 10)
|
||||||
|
case json.Number:
|
||||||
|
return t.String()
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("%v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func stringOf(v any) string {
|
||||||
|
if s, ok := v.(string); ok {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
if v == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%v", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// actionFor maps a portal system name to the canonical action code used by the
|
||||||
|
// scenario (kept for parity with the existing integration).
|
||||||
|
func actionFor(system string) string {
|
||||||
|
switch system {
|
||||||
|
case "Горячее водоснабжение":
|
||||||
|
return "waterWarm"
|
||||||
|
case "Холодное водоснабжение и канализация", "Холодное водоснабжение":
|
||||||
|
return "waterCold"
|
||||||
|
case "Электроснабжение":
|
||||||
|
return "electricitySupply"
|
||||||
|
case "Газоснабжение":
|
||||||
|
return "gasSupply"
|
||||||
|
case "Теплоснабжение":
|
||||||
|
return "heating"
|
||||||
|
default:
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// likePattern builds the substring pattern the portal supports. Numeric tokens
|
||||||
|
// are padded with spaces, matching the historical integration, so that "1" does
|
||||||
|
// not incidentally match "10".
|
||||||
|
func likePattern(canonical string) string {
|
||||||
|
parts := strings.Fields(canonical)
|
||||||
|
for i, part := range parts {
|
||||||
|
if containsDigit(part) {
|
||||||
|
parts[i] = " " + part + " "
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "%" + strings.Join(parts, "%") + "%"
|
||||||
|
}
|
||||||
|
|
||||||
|
func containsDigit(s string) bool {
|
||||||
|
for _, r := range s {
|
||||||
|
if r >= '0' && r <= '9' {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
package arcgis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const envelope = `{
|
||||||
|
"features": [
|
||||||
|
{"value": "Станционная улица 1/1", "attributes": {
|
||||||
|
"address": "Станционная улица 1/1",
|
||||||
|
"type_id": 1, "system_id": 5, "time_category": 2,
|
||||||
|
"start_date": "2026-10-01T00:00:00", "end_date": "2026-10-31T00:00:00"
|
||||||
|
}}
|
||||||
|
],
|
||||||
|
"fields": [
|
||||||
|
{"name": "type_id", "domain": {"codedValues": [{"code": 1, "name": "Плановое"}]}},
|
||||||
|
{"name": "system_id", "domain": {"codedValues": [{"code": 5, "name": "Теплоснабжение"}]}},
|
||||||
|
{"name": "time_category", "domain": {"codedValues": [{"code": 2, "name": "Текущие"}]}}
|
||||||
|
]
|
||||||
|
}`
|
||||||
|
|
||||||
|
func TestQueryParsesEnvelope(t *testing.T) {
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if got := r.URL.Query().Get("f"); got != "json" {
|
||||||
|
t.Errorf("missing f=json, got %q", got)
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
_, _ = w.Write([]byte(envelope))
|
||||||
|
}))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
c := New(ts.URL, time.Second, 0)
|
||||||
|
rows, err := c.Query(context.Background(), "Станционная улица 1/1")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Query: %v", err)
|
||||||
|
}
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatalf("rows = %d, want 1", len(rows))
|
||||||
|
}
|
||||||
|
got := rows[0]
|
||||||
|
if got.Type != "Плановое" || got.System != "Теплоснабжение" || got.Time != "Текущие" {
|
||||||
|
t.Errorf("coded values not resolved: %+v", got)
|
||||||
|
}
|
||||||
|
if got.Action != "heating" {
|
||||||
|
t.Errorf("Action = %q, want heating", got.Action)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueryRetriesThenFails(t *testing.T) {
|
||||||
|
var calls int
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
http.Error(w, "boom", http.StatusInternalServerError)
|
||||||
|
}))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
c := New(ts.URL, time.Second, 2)
|
||||||
|
if _, err := c.Query(context.Background(), "Станционная улица 1/1"); err == nil {
|
||||||
|
t.Fatal("expected error")
|
||||||
|
}
|
||||||
|
if calls != 3 {
|
||||||
|
t.Errorf("calls = %d, want 3 (1 + 2 retries)", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type ctxKey int
|
||||||
|
|
||||||
|
const requestIDKey ctxKey = 0
|
||||||
|
|
||||||
|
// requestIDMW assigns a request id (for correlation in logs) and echoes it.
|
||||||
|
func requestIDMW(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
id := r.Header.Get("X-Request-Id")
|
||||||
|
if id == "" {
|
||||||
|
id = newRequestID()
|
||||||
|
}
|
||||||
|
w.Header().Set("X-Request-Id", id)
|
||||||
|
ctx := context.WithValue(r.Context(), requestIDKey, id)
|
||||||
|
next.ServeHTTP(w, r.WithContext(ctx))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// loggingMW logs one line per request with method, path, status and duration.
|
||||||
|
// It records no addresses (PII) — only routing metadata.
|
||||||
|
func loggingMW(log *slog.Logger) func(http.Handler) http.Handler {
|
||||||
|
return func(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
start := time.Now()
|
||||||
|
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
|
||||||
|
next.ServeHTTP(rec, r)
|
||||||
|
log.Info("http",
|
||||||
|
"method", r.Method,
|
||||||
|
"path", r.URL.Path,
|
||||||
|
"status", rec.status,
|
||||||
|
"duration_ms", time.Since(start).Milliseconds(),
|
||||||
|
"request_id", requestIDFrom(r.Context()),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// timeoutMW bounds the whole request, so a slow portal cannot tie up a
|
||||||
|
// goroutine indefinitely.
|
||||||
|
func timeoutMW(d time.Duration) func(http.Handler) http.Handler {
|
||||||
|
return func(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), d)
|
||||||
|
defer cancel()
|
||||||
|
next.ServeHTTP(w, r.WithContext(ctx))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// recoverMW turns a panic into a 500 instead of crashing the process.
|
||||||
|
func recoverMW(log *slog.Logger) func(http.Handler) http.Handler {
|
||||||
|
return func(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
defer func() {
|
||||||
|
if rec := recover(); rec != nil {
|
||||||
|
log.Error("panic", "value", rec, "request_id", requestIDFrom(r.Context()))
|
||||||
|
writeError(w, http.StatusInternalServerError, "internal error")
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func requestIDFrom(ctx context.Context) string {
|
||||||
|
if v, ok := ctx.Value(requestIDKey).(string); ok {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
type statusRecorder struct {
|
||||||
|
http.ResponseWriter
|
||||||
|
status int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *statusRecorder) WriteHeader(code int) {
|
||||||
|
s.status = code
|
||||||
|
s.ResponseWriter.WriteHeader(code)
|
||||||
|
}
|
||||||
|
|
||||||
|
// newRequestID returns a short random hex id. crypto/rand never fails in
|
||||||
|
// practice; on the off chance it does we fall back to a constant so the
|
||||||
|
// request still completes.
|
||||||
|
func newRequestID() string {
|
||||||
|
var b [16]byte
|
||||||
|
if _, err := rand.Read(b[:]); err != nil {
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(b[:])
|
||||||
|
}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
// Package httpapi exposes the addrkit domain over a small, versioned JSON API.
|
||||||
|
//
|
||||||
|
// It contains no business logic: handlers decode requests, call the resolver
|
||||||
|
// and the outages source, and encode responses. Cross-cutting concerns
|
||||||
|
// (timeouts, recovery, request ids, logging) live in middleware.
|
||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Resolver is the read side of the domain (implemented by *match.Resolver).
|
||||||
|
type Resolver interface {
|
||||||
|
Resolve(ctx context.Context, raw string) match.Result
|
||||||
|
Version() string
|
||||||
|
Ready() bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// Server wires the domain to HTTP.
|
||||||
|
type Server struct {
|
||||||
|
resolver Resolver
|
||||||
|
outages outages.Source
|
||||||
|
log *slog.Logger
|
||||||
|
now func() time.Time
|
||||||
|
loc *time.Location
|
||||||
|
timeout time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a server. loc controls how outage dates are interpreted; it
|
||||||
|
// should be the operational timezone (Asia/Novosibirsk).
|
||||||
|
func New(resolver Resolver, src outages.Source, log *slog.Logger, loc *time.Location) *Server {
|
||||||
|
if loc == nil {
|
||||||
|
loc = time.UTC
|
||||||
|
}
|
||||||
|
return &Server{
|
||||||
|
resolver: resolver,
|
||||||
|
outages: src,
|
||||||
|
log: log,
|
||||||
|
now: time.Now,
|
||||||
|
loc: loc,
|
||||||
|
timeout: 10 * time.Second,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handler returns the fully wrapped HTTP handler.
|
||||||
|
func (s *Server) Handler() http.Handler {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("POST /v1/resolve", s.handleResolve)
|
||||||
|
mux.HandleFunc("POST /v1/outages", s.handleOutages)
|
||||||
|
mux.HandleFunc("GET /v1/healthz", s.handleHealthz)
|
||||||
|
mux.HandleFunc("GET /v1/readyz", s.handleReadyz)
|
||||||
|
mux.HandleFunc("GET /v1/version", s.handleVersion)
|
||||||
|
|
||||||
|
return recoverMW(s.log)(requestIDMW(loggingMW(s.log)(timeoutMW(s.timeout)(mux))))
|
||||||
|
}
|
||||||
|
|
||||||
|
type resolveRequest struct {
|
||||||
|
Address string `json:"address"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleResolve(w http.ResponseWriter, r *http.Request) {
|
||||||
|
req, ok := s.decode(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, s.resolver.Resolve(r.Context(), req.Address))
|
||||||
|
}
|
||||||
|
|
||||||
|
type outagesResponse struct {
|
||||||
|
Status address.Status `json:"status"`
|
||||||
|
Address string `json:"address,omitempty"`
|
||||||
|
District string `json:"district,omitempty"`
|
||||||
|
Outages []outages.Outage `json:"outages,omitempty"`
|
||||||
|
AsOf string `json:"as_of,omitempty"`
|
||||||
|
DataVersion string `json:"data_version,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleOutages(w http.ResponseWriter, r *http.Request) {
|
||||||
|
req, ok := s.decode(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx := r.Context()
|
||||||
|
res := s.resolver.Resolve(ctx, req.Address)
|
||||||
|
|
||||||
|
out := outagesResponse{
|
||||||
|
Status: res.Status,
|
||||||
|
Address: res.Address,
|
||||||
|
District: res.District,
|
||||||
|
DataVersion: res.DataVersion,
|
||||||
|
}
|
||||||
|
if res.Status != address.StatusFound {
|
||||||
|
// No canonical address => nothing to query. Return the resolver's
|
||||||
|
// verdict so the caller can ask for clarification.
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
raw, err := s.outages.Query(ctx, res.Address)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) {
|
||||||
|
out.Status = address.StatusPortalTimeout
|
||||||
|
} else {
|
||||||
|
out.Status = address.StatusInfra
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
parsed := address.Parse(res.Address)
|
||||||
|
current := outages.Current(raw, s.now(), s.loc)
|
||||||
|
out.Outages = outages.MatchingHouse(current, parsed)
|
||||||
|
out.AsOf = s.now().In(s.loc).Format(time.RFC3339)
|
||||||
|
writeJSON(w, http.StatusOK, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleHealthz(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleReadyz(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
if !s.resolver.Ready() {
|
||||||
|
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"status": "not_ready"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, map[string]string{"status": "ready"})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleVersion(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
writeJSON(w, http.StatusOK, map[string]string{"data_version": s.resolver.Version()})
|
||||||
|
}
|
||||||
|
|
||||||
|
// decode reads the JSON body into a resolveRequest, writing a 400 on failure.
|
||||||
|
func (s *Server) decode(w http.ResponseWriter, r *http.Request) (resolveRequest, bool) {
|
||||||
|
var req resolveRequest
|
||||||
|
r.Body = http.MaxBytesReader(w, r.Body, 64*1024)
|
||||||
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||||
|
writeError(w, http.StatusBadRequest, "invalid json body")
|
||||||
|
return resolveRequest{}, false
|
||||||
|
}
|
||||||
|
if req.Address == "" {
|
||||||
|
writeError(w, http.StatusBadRequest, "address is required")
|
||||||
|
return resolveRequest{}, false
|
||||||
|
}
|
||||||
|
return req, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeJSON(w http.ResponseWriter, status int, v any) {
|
||||||
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
w.WriteHeader(status)
|
||||||
|
_ = json.NewEncoder(w).Encode(v)
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeError(w http.ResponseWriter, status int, msg string) {
|
||||||
|
writeJSON(w, status, map[string]any{"error": 1, "message": msg})
|
||||||
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/match"
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/outages"
|
||||||
|
)
|
||||||
|
|
||||||
|
func testResolver(t *testing.T) *match.Resolver {
|
||||||
|
t.Helper()
|
||||||
|
var entries []address.Entry
|
||||||
|
for _, a := range []string{"Станционная улица 1/1", "Искитимская улица 5"} {
|
||||||
|
p := address.Parse(a)
|
||||||
|
entries = append(entries, address.Entry{
|
||||||
|
ID: a, Address: a, District: "Тестовый",
|
||||||
|
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
|
||||||
|
House: p.House, HouseFrac: p.HouseFrac, HouseLetter: p.HouseLetter,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return match.NewResolver(match.NewIndex("test-version", entries, match.DefaultOptions()), nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeSource struct {
|
||||||
|
list []outages.Outage
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeSource) Query(context.Context, string) ([]outages.Outage, error) {
|
||||||
|
return f.list, f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func newServer(t *testing.T, src fakeSource) *httptest.Server {
|
||||||
|
t.Helper()
|
||||||
|
srv := New(testResolver(t), src, slog.New(slog.NewTextHandler(io.Discard, nil)), time.UTC)
|
||||||
|
ts := httptest.NewServer(srv.Handler())
|
||||||
|
t.Cleanup(ts.Close)
|
||||||
|
return ts
|
||||||
|
}
|
||||||
|
|
||||||
|
func postJSON(t *testing.T, url, body string) *http.Response {
|
||||||
|
t.Helper()
|
||||||
|
resp, err := http.Post(url, "application/json", bytes.NewBufferString(body))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("post: %v", err)
|
||||||
|
}
|
||||||
|
return resp
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveEndpoint(t *testing.T) {
|
||||||
|
ts := newServer(t, fakeSource{})
|
||||||
|
resp := postJSON(t, ts.URL+"/v1/resolve", `{"address":"2-я Станционная 1/1"}`)
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
var res match.Result
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&res); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
if res.Status != address.StatusFound || res.Address != "Станционная улица 1/1" {
|
||||||
|
t.Fatalf("unexpected result: %+v", res)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOutagesEndpointFiltersHouse(t *testing.T) {
|
||||||
|
src := fakeSource{list: []outages.Outage{
|
||||||
|
{Address: "Станционная улица 1/1", System: "Теплоснабжение"},
|
||||||
|
{Address: "Станционная улица 1/2", System: "Теплоснабжение"},
|
||||||
|
}}
|
||||||
|
ts := newServer(t, src)
|
||||||
|
resp := postJSON(t, ts.URL+"/v1/outages", `{"address":"станционная 1/1"}`)
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
var out outagesResponse
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
if out.Status != address.StatusFound {
|
||||||
|
t.Fatalf("status = %q, want found", out.Status)
|
||||||
|
}
|
||||||
|
if len(out.Outages) != 1 || out.Outages[0].Address != "Станционная улица 1/1" {
|
||||||
|
t.Fatalf("house filter failed: %+v", out.Outages)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveValidation(t *testing.T) {
|
||||||
|
ts := newServer(t, fakeSource{})
|
||||||
|
resp := postJSON(t, ts.URL+"/v1/resolve", `{}`)
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusBadRequest {
|
||||||
|
t.Fatalf("status = %d, want 400", resp.StatusCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReadyzAndVersion(t *testing.T) {
|
||||||
|
ts := newServer(t, fakeSource{})
|
||||||
|
|
||||||
|
resp, err := http.Get(ts.URL + "/v1/readyz")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("readyz: %v", err)
|
||||||
|
}
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
t.Fatalf("readyz status = %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err = http.Get(ts.URL + "/v1/version")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("version: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
var v map[string]string
|
||||||
|
_ = json.NewDecoder(resp.Body).Decode(&v)
|
||||||
|
if v["data_version"] != "test-version" {
|
||||||
|
t.Fatalf("version = %q", v["data_version"])
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
// Package llm implements the optional, disabled-by-default query rewriter.
|
||||||
|
//
|
||||||
|
// It talks to any OpenAI-compatible chat completions endpoint (for example the
|
||||||
|
// on-prem vLLM already used by AMS). Its contract is deliberately narrow: it
|
||||||
|
// may only propose alternative spellings of the user's address. It never
|
||||||
|
// returns the answer, and its output is validated by the deterministic
|
||||||
|
// resolver, so a misbehaving model can at worst waste a request — it cannot
|
||||||
|
// make the service answer with a wrong address.
|
||||||
|
package llm
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Rewriter asks a model for normalised variants of an address.
|
||||||
|
type Rewriter struct {
|
||||||
|
endpoint string
|
||||||
|
token string
|
||||||
|
model string
|
||||||
|
http *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a rewriter from configuration. The URL is treated as a base; the
|
||||||
|
// OpenAI path is appended if it is not already present.
|
||||||
|
func New(cfg config.LLMConfig) *Rewriter {
|
||||||
|
base := strings.TrimRight(cfg.URL, "/")
|
||||||
|
if !strings.HasSuffix(base, "/chat/completions") {
|
||||||
|
base += "/v1/chat/completions"
|
||||||
|
}
|
||||||
|
timeout := time.Duration(cfg.TimeoutSeconds) * time.Second
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = 3 * time.Second
|
||||||
|
}
|
||||||
|
return &Rewriter{
|
||||||
|
endpoint: base,
|
||||||
|
token: cfg.Token,
|
||||||
|
model: cfg.Model,
|
||||||
|
http: &http.Client{Timeout: timeout},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const systemPrompt = `Ты — нормализатор адресов. Тебе дают адрес, написанный как угодно.
|
||||||
|
Верни ТОЛЬКО JSON вида {"variants":["...","..."]} — до 3 вариантов записи того же адреса
|
||||||
|
в формате "<название улицы> <дом>[/<дробь>][ <литера>][ к<корпус>][ стр.<строение>]".
|
||||||
|
Не выдумывай улицы и дома, которых нет во входе. Если не уверен — верни пустой список.`
|
||||||
|
|
||||||
|
type chatRequest struct {
|
||||||
|
Model string `json:"model"`
|
||||||
|
Messages []chatMessage `json:"messages"`
|
||||||
|
Temperature float64 `json:"temperature"`
|
||||||
|
MaxTokens int `json:"max_tokens"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type chatMessage struct {
|
||||||
|
Role string `json:"role"`
|
||||||
|
Content string `json:"content"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type chatResponse struct {
|
||||||
|
Choices []struct {
|
||||||
|
Message chatMessage `json:"message"`
|
||||||
|
} `json:"choices"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Rewrite returns up to three normalised variants of raw.
|
||||||
|
func (r *Rewriter) Rewrite(ctx context.Context, raw string) ([]string, error) {
|
||||||
|
payload, err := json.Marshal(chatRequest{
|
||||||
|
Model: r.model,
|
||||||
|
Messages: []chatMessage{
|
||||||
|
{Role: "system", Content: systemPrompt},
|
||||||
|
{Role: "user", Content: raw},
|
||||||
|
},
|
||||||
|
Temperature: 0,
|
||||||
|
MaxTokens: 256,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, r.endpoint, bytes.NewReader(payload))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if r.token != "" && r.token != "EMPTY" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+r.token)
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := r.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("llm status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
var cr chatResponse
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&cr); err != nil {
|
||||||
|
return nil, fmt.Errorf("decode llm response: %w", err)
|
||||||
|
}
|
||||||
|
if len(cr.Choices) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return parseVariants(cr.Choices[0].Message.Content), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseVariants extracts the {"variants":[...]} object from a model reply that
|
||||||
|
// may be wrapped in prose or code fences.
|
||||||
|
func parseVariants(content string) []string {
|
||||||
|
start := strings.Index(content, "{")
|
||||||
|
end := strings.LastIndex(content, "}")
|
||||||
|
if start < 0 || end <= start {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var parsed struct {
|
||||||
|
Variants []string `json:"variants"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(content[start:end+1]), &parsed); err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(parsed.Variants))
|
||||||
|
for _, v := range parsed.Variants {
|
||||||
|
if s := strings.TrimSpace(v); s != "" {
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
// Package registrystore persists the canonical registry as a single,
|
||||||
|
// self-contained SQLite file. It is intentionally boring: one file, no server,
|
||||||
|
// built once by the seed pipeline and opened read-only by the service.
|
||||||
|
package registrystore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
_ "modernc.org/sqlite" // registers the "sqlite" driver
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
const schema = `
|
||||||
|
CREATE TABLE IF NOT EXISTS meta (
|
||||||
|
key TEXT PRIMARY KEY,
|
||||||
|
value TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS address (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
address TEXT NOT NULL,
|
||||||
|
district TEXT NOT NULL DEFAULT '',
|
||||||
|
ordinal TEXT NOT NULL DEFAULT '',
|
||||||
|
name_key TEXT NOT NULL DEFAULT '',
|
||||||
|
name_display TEXT NOT NULL DEFAULT '',
|
||||||
|
street_type TEXT NOT NULL DEFAULT '',
|
||||||
|
street_key TEXT NOT NULL,
|
||||||
|
house TEXT NOT NULL DEFAULT '',
|
||||||
|
house_frac TEXT NOT NULL DEFAULT '',
|
||||||
|
house_letter TEXT NOT NULL DEFAULT '',
|
||||||
|
building TEXT NOT NULL DEFAULT '',
|
||||||
|
building_frac TEXT NOT NULL DEFAULT '',
|
||||||
|
building_letter TEXT NOT NULL DEFAULT '',
|
||||||
|
structure TEXT NOT NULL DEFAULT ''
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_address_street ON address(street_key);
|
||||||
|
`
|
||||||
|
|
||||||
|
// Save writes entries to path atomically: it builds a temporary file and
|
||||||
|
// renames it into place, so readers never observe a half-written index.
|
||||||
|
//
|
||||||
|
// The build uses the rollback journal (not WAL) on purpose, so that the result
|
||||||
|
// is a single file with no companion -wal/-shm that a rename would orphan.
|
||||||
|
func Save(path, version string, entries []address.Entry) error {
|
||||||
|
tmp := path + ".tmp"
|
||||||
|
removeDBFiles(tmp)
|
||||||
|
|
||||||
|
db, err := sql.Open("sqlite", "file:"+tmp+
|
||||||
|
"?_pragma=journal_mode(DELETE)"+
|
||||||
|
"&_pragma=synchronous(FULL)"+
|
||||||
|
"&_pragma=busy_timeout(5000)")
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("open temp db: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := db.Exec(schema); err != nil {
|
||||||
|
db.Close()
|
||||||
|
return fmt.Errorf("create schema: %w", err)
|
||||||
|
}
|
||||||
|
if err := insertAll(db, version, entries); err != nil {
|
||||||
|
db.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := db.Close(); err != nil {
|
||||||
|
return fmt.Errorf("close temp db: %w", err)
|
||||||
|
}
|
||||||
|
if err := os.Rename(tmp, path); err != nil {
|
||||||
|
return fmt.Errorf("publish db: %w", err)
|
||||||
|
}
|
||||||
|
// Drop only the temp file's journal siblings; the published base file must
|
||||||
|
// survive.
|
||||||
|
removeSidecars(tmp)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func insertAll(db *sql.DB, version string, entries []address.Entry) error {
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("begin: %w", err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
|
||||||
|
if _, err := tx.Exec(`INSERT INTO meta(key, value) VALUES ('version', ?)`, version); err != nil {
|
||||||
|
return fmt.Errorf("insert meta: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
stmt, err := tx.Prepare(`INSERT INTO address (
|
||||||
|
id, address, district, ordinal, name_key, name_display, street_type, street_key,
|
||||||
|
house, house_frac, house_letter, building, building_frac, building_letter, structure
|
||||||
|
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("prepare insert: %w", err)
|
||||||
|
}
|
||||||
|
defer stmt.Close()
|
||||||
|
|
||||||
|
for i, e := range entries {
|
||||||
|
id := e.ID
|
||||||
|
if id == "" {
|
||||||
|
// Guarantee a primary key even for sources without ids.
|
||||||
|
id = fmt.Sprintf("row-%d", i)
|
||||||
|
}
|
||||||
|
if _, err := stmt.Exec(
|
||||||
|
id, e.Address, e.District, e.Ordinal, e.NameKey, e.NameDisplay, e.StreetType, e.StreetKey,
|
||||||
|
e.House, e.HouseFrac, e.HouseLetter, e.Building, e.BuildingFrac, e.BuildingLetter, e.Structure,
|
||||||
|
); err != nil {
|
||||||
|
return fmt.Errorf("insert %q: %w", e.Address, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
return fmt.Errorf("commit: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load opens path read-only and returns the data version and all entries.
|
||||||
|
func Load(path string) (version string, entries []address.Entry, err error) {
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
return "", nil, fmt.Errorf("stat db: %w", err)
|
||||||
|
}
|
||||||
|
db, err := sql.Open("sqlite", "file:"+path+
|
||||||
|
"?mode=ro"+
|
||||||
|
"&_pragma=query_only(1)"+
|
||||||
|
"&_pragma=busy_timeout(5000)")
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, fmt.Errorf("open db: %w", err)
|
||||||
|
}
|
||||||
|
defer db.Close()
|
||||||
|
|
||||||
|
if err := db.QueryRow(`SELECT value FROM meta WHERE key='version'`).Scan(&version); err != nil {
|
||||||
|
return "", nil, fmt.Errorf("read version: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := db.Query(`SELECT id, address, district, ordinal, name_key, name_display,
|
||||||
|
street_type, street_key, house, house_frac, house_letter, building, building_frac,
|
||||||
|
building_letter, structure FROM address`)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, fmt.Errorf("query: %w", err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var e address.Entry
|
||||||
|
if err := rows.Scan(
|
||||||
|
&e.ID, &e.Address, &e.District, &e.Ordinal, &e.NameKey, &e.NameDisplay,
|
||||||
|
&e.StreetType, &e.StreetKey, &e.House, &e.HouseFrac, &e.HouseLetter,
|
||||||
|
&e.Building, &e.BuildingFrac, &e.BuildingLetter, &e.Structure,
|
||||||
|
); err != nil {
|
||||||
|
return "", nil, fmt.Errorf("scan: %w", err)
|
||||||
|
}
|
||||||
|
entries = append(entries, e)
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return "", nil, fmt.Errorf("rows: %w", err)
|
||||||
|
}
|
||||||
|
return version, entries, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// removeDBFiles deletes a database file and its journal siblings.
|
||||||
|
func removeDBFiles(base string) {
|
||||||
|
_ = os.Remove(base)
|
||||||
|
removeSidecars(base)
|
||||||
|
}
|
||||||
|
|
||||||
|
// removeSidecars deletes only the -wal/-shm companions of a database file.
|
||||||
|
func removeSidecars(base string) {
|
||||||
|
_ = os.Remove(base + "-wal")
|
||||||
|
_ = os.Remove(base + "-shm")
|
||||||
|
}
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
package registrystore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSaveLoadRoundTrip(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "registry.db")
|
||||||
|
|
||||||
|
addr := "Станционная улица 1/1"
|
||||||
|
p := address.Parse(addr)
|
||||||
|
entries := []address.Entry{{
|
||||||
|
ID: "a1", Address: addr, District: "Ленинский",
|
||||||
|
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
|
||||||
|
House: p.House, HouseFrac: p.HouseFrac,
|
||||||
|
}}
|
||||||
|
|
||||||
|
if err := Save(path, "2026-10-09", entries); err != nil {
|
||||||
|
t.Fatalf("Save: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
version, got, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
if version != "2026-10-09" {
|
||||||
|
t.Errorf("version = %q", version)
|
||||||
|
}
|
||||||
|
if len(got) != 1 {
|
||||||
|
t.Fatalf("entries = %d, want 1", len(got))
|
||||||
|
}
|
||||||
|
if got[0].Address != addr || got[0].District != "Ленинский" || got[0].StreetKey != p.StreetKey {
|
||||||
|
t.Errorf("round-tripped entry mismatch: %+v", got[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSaveIsAtomicOverwrite verifies that re-seeding replaces the previous data
|
||||||
|
// rather than merging with it.
|
||||||
|
func TestSaveIsAtomicOverwrite(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "registry.db")
|
||||||
|
|
||||||
|
if err := Save(path, "v1", []address.Entry{{ID: "x", Address: "Первая улица 1"}}); err != nil {
|
||||||
|
t.Fatalf("Save v1: %v", err)
|
||||||
|
}
|
||||||
|
if err := Save(path, "v2", []address.Entry{{ID: "y", Address: "Вторая улица 2"}}); err != nil {
|
||||||
|
t.Fatalf("Save v2: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
version, got, err := Load(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
if version != "v2" || len(got) != 1 || got[0].Address != "Вторая улица 2" {
|
||||||
|
t.Fatalf("unexpected state after overwrite: version=%q entries=%+v", version, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
// Package config loads the addrkit configuration from a JSON file, expanding
|
||||||
|
// ${VAR} placeholders from the environment. It fails closed: missing required
|
||||||
|
// values are an error, never a silent default.
|
||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config is the whole service configuration.
|
||||||
|
type Config struct {
|
||||||
|
// Addr is the listen address of the HTTP server, e.g. ":6171".
|
||||||
|
Addr string `json:"addr"`
|
||||||
|
// DataPath is the SQLite file produced by the seed pipeline.
|
||||||
|
DataPath string `json:"data_path"`
|
||||||
|
// LogLevel is one of debug|info|warn|error.
|
||||||
|
LogLevel string `json:"log_level"`
|
||||||
|
|
||||||
|
Resolve ResolveConfig `json:"resolve"`
|
||||||
|
Portal PortalConfig `json:"portal"`
|
||||||
|
LLM LLMConfig `json:"llm"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveConfig tunes the matching engine.
|
||||||
|
type ResolveConfig struct {
|
||||||
|
FuzzyThreshold float64 `json:"fuzzy_threshold"`
|
||||||
|
AmbiguityMargin float64 `json:"ambiguity_margin"`
|
||||||
|
MaxCandidates int `json:"max_candidates"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// PortalConfig configures the external outages portal (ArcGIS).
|
||||||
|
type PortalConfig struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
TimeoutSeconds int `json:"timeout_seconds"`
|
||||||
|
Retries int `json:"retries"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// LLMConfig configures the optional, disabled-by-default query rewriter.
|
||||||
|
type LLMConfig struct {
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
URL string `json:"url"`
|
||||||
|
Token string `json:"token"`
|
||||||
|
Model string `json:"model"`
|
||||||
|
TimeoutSeconds int `json:"timeout_seconds"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Default returns a config with sane defaults for the non-required fields.
|
||||||
|
func Default() Config {
|
||||||
|
return Config{
|
||||||
|
Addr: ":6171",
|
||||||
|
LogLevel: "info",
|
||||||
|
Resolve: ResolveConfig{
|
||||||
|
FuzzyThreshold: 0.82,
|
||||||
|
AmbiguityMargin: 0.03,
|
||||||
|
MaxCandidates: 5,
|
||||||
|
},
|
||||||
|
Portal: PortalConfig{
|
||||||
|
TimeoutSeconds: 2,
|
||||||
|
Retries: 1,
|
||||||
|
},
|
||||||
|
LLM: LLMConfig{
|
||||||
|
TimeoutSeconds: 3,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load reads, expands and validates a config file.
|
||||||
|
func Load(path string) (Config, error) {
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, fmt.Errorf("read config: %w", err)
|
||||||
|
}
|
||||||
|
cfg := Default()
|
||||||
|
dec := json.NewDecoder(strings.NewReader(expandEnv(string(raw))))
|
||||||
|
if err := dec.Decode(&cfg); err != nil {
|
||||||
|
return Config{}, fmt.Errorf("parse config: %w", err)
|
||||||
|
}
|
||||||
|
if err := cfg.validate(); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c Config) validate() error {
|
||||||
|
if c.Addr == "" {
|
||||||
|
return fmt.Errorf("config: addr is required")
|
||||||
|
}
|
||||||
|
if c.DataPath == "" {
|
||||||
|
return fmt.Errorf("config: data_path is required")
|
||||||
|
}
|
||||||
|
if c.LLM.Enabled && c.LLM.URL == "" {
|
||||||
|
return fmt.Errorf("config: llm.url is required when llm.enabled is true")
|
||||||
|
}
|
||||||
|
if c.Portal.URL == "" {
|
||||||
|
return fmt.Errorf("config: portal.url is required")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// envRef matches ${VAR} and $VAR placeholders.
|
||||||
|
var envRef = regexp.MustCompile(`\$\{([A-Za-z_][A-Za-z0-9_]*)\}|\$([A-Za-z_][A-Za-z0-9_]*)`)
|
||||||
|
|
||||||
|
// expandEnv replaces ${VAR} / $VAR with the environment value. An unset
|
||||||
|
// variable expands to the empty string; validation then rejects configs that
|
||||||
|
// relied on it (fail closed).
|
||||||
|
func expandEnv(s string) string {
|
||||||
|
return envRef.ReplaceAllStringFunc(s, func(m string) string {
|
||||||
|
sub := envRef.FindStringSubmatch(m)
|
||||||
|
name := sub[1]
|
||||||
|
if name == "" {
|
||||||
|
name = sub[2]
|
||||||
|
}
|
||||||
|
return os.Getenv(name)
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
package address
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
"golang.org/x/text/cases"
|
||||||
|
"golang.org/x/text/unicode/norm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fold is a Unicode-aware case folder. It lower-cases Cyrillic and Latin
|
||||||
|
// alike, which is what we want for matching.
|
||||||
|
var fold = cases.Fold()
|
||||||
|
|
||||||
|
// NormalizeText folds a free-form string into a canonical form suitable for
|
||||||
|
// parsing and matching.
|
||||||
|
//
|
||||||
|
// The transformation is intentionally lossy:
|
||||||
|
// - Unicode NFKC (so lookalike characters collapse);
|
||||||
|
// - case folding (so "СТАНИЦИОННАЯ" == "станционная");
|
||||||
|
// - "ё" -> "е" (users and ASR disagree about the diaeresis);
|
||||||
|
// - dashes and various Unicode spaces are unified;
|
||||||
|
// - punctuation that only separates tokens becomes a space;
|
||||||
|
// - runs of whitespace collapse to a single space.
|
||||||
|
//
|
||||||
|
// NormalizeText is idempotent: NormalizeText(NormalizeText(s)) == NormalizeText(s).
|
||||||
|
func NormalizeText(s string) string {
|
||||||
|
s = norm.NFKC.String(s)
|
||||||
|
s = fold.String(s)
|
||||||
|
s = strings.ReplaceAll(s, "ё", "е")
|
||||||
|
s = mapSeparators(s)
|
||||||
|
s = strings.Join(strings.Fields(s), " ")
|
||||||
|
return strings.TrimSpace(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
// mapSeparators unifies characters that carry no semantic meaning for address
|
||||||
|
// matching. It never removes digits, letters or the '/' house separator.
|
||||||
|
func mapSeparators(s string) string {
|
||||||
|
var b strings.Builder
|
||||||
|
b.Grow(len(s))
|
||||||
|
for _, r := range s {
|
||||||
|
switch {
|
||||||
|
case r == '\u00a0' || unicode.IsSpace(r):
|
||||||
|
b.WriteByte(' ')
|
||||||
|
case isDash(r):
|
||||||
|
b.WriteByte('-')
|
||||||
|
case isSeparatorPunct(r):
|
||||||
|
b.WriteByte(' ')
|
||||||
|
default:
|
||||||
|
b.WriteRune(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// isDash reports whether r is one of the many dash-like code points that users
|
||||||
|
// and copy-paste bring in: "2–я" with an en dash is still an ordinal.
|
||||||
|
func isDash(r rune) bool {
|
||||||
|
switch r {
|
||||||
|
case '-', '\u2010', '\u2011', '\u2012', '\u2013', '\u2014', '\u2212':
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// isSeparatorPunct reports whether r separates tokens. Dots are included so
|
||||||
|
// that abbreviations like "ул." normalize to their stem ("ул").
|
||||||
|
func isSeparatorPunct(r rune) bool {
|
||||||
|
switch r {
|
||||||
|
case '.', ',', ';', '(', ')', '"', '\'', '«', '»', ':', '!', '?':
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// isDigits reports whether s is a non-empty run of ASCII digits.
|
||||||
|
func isDigits(s string) bool {
|
||||||
|
if s == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, r := range s {
|
||||||
|
if r < '0' || r > '9' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// trimLeadingZeros removes redundant leading zeros from a numeric token:
|
||||||
|
// "007" -> "7". It leaves non-numeric strings untouched.
|
||||||
|
func trimLeadingZeros(s string) string {
|
||||||
|
if !isDigits(s) {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
t := strings.TrimLeft(s, "0")
|
||||||
|
if t == "" {
|
||||||
|
return "0"
|
||||||
|
}
|
||||||
|
return t
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
package address
|
||||||
|
|
||||||
|
import (
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The house block is parsed from the end of the string, component by
|
||||||
|
// component, from the outermost (structure) to the innermost (house). Working
|
||||||
|
// right-to-left is what lets us tell a genuine house number apart from a digit
|
||||||
|
// that is part of the street name (e.g. "2-я Станционная улица 1/1 стр.").
|
||||||
|
var (
|
||||||
|
// "... стр." / "... строение 3" / "... сооружение" / "... с 2"
|
||||||
|
reStructure = regexp.MustCompile(`(?:^|\s)(?:стр|строение|сооружение|с)\.?\s*(\d+)?$`)
|
||||||
|
// "... к9/1" / "... корпус 2а" / "... кД" / "... к 1/4"
|
||||||
|
reBuilding = regexp.MustCompile(`(?:^|\s)(?:к|корп|корпус)\.?\s*(\d*)\s*([а-я])?(?:/(\d+)\s*([а-я])?)?$`)
|
||||||
|
// "... 1/1" / "... 33а" / "... 12 б" / "... 45а/4" / "... 1/1а"
|
||||||
|
reHouse = regexp.MustCompile(`(?:^|\s)(\d+)\s*([а-я])?(?:/(\d+)\s*([а-я])?)?$`)
|
||||||
|
|
||||||
|
// "2-я", "10-й", "3-е"
|
||||||
|
reOrdinalToken = regexp.MustCompile(`^(\d+)-[а-я]{1,3}$`)
|
||||||
|
)
|
||||||
|
|
||||||
|
// Parse decomposes a free-form address into its structured components. It is
|
||||||
|
// total: unparseable input still yields a non-nil ParsedAddress whose street
|
||||||
|
// part is best-effort and whose HasHouse is false.
|
||||||
|
func Parse(raw string) ParsedAddress {
|
||||||
|
normalized := NormalizeText(raw)
|
||||||
|
p := ParsedAddress{Raw: raw, Normalized: normalized}
|
||||||
|
|
||||||
|
rest := normalized
|
||||||
|
if g, start, ok := matchSuffix(reStructure, rest); ok {
|
||||||
|
p.Structure = trimLeadingZeros(g[1])
|
||||||
|
rest = strings.TrimSpace(rest[:start])
|
||||||
|
}
|
||||||
|
if g, start, ok := matchSuffix(reBuilding, rest); ok {
|
||||||
|
// The regex also matches a bare "к"; only accept it when a digit or a
|
||||||
|
// letter actually follows, otherwise it is not a corpus marker.
|
||||||
|
if g[1] != "" || g[2] != "" {
|
||||||
|
p.Building = trimLeadingZeros(g[1])
|
||||||
|
p.BuildingLetter = firstNonEmpty(g[2], g[4])
|
||||||
|
p.BuildingFrac = trimLeadingZeros(g[3])
|
||||||
|
rest = strings.TrimSpace(rest[:start])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if g, start, ok := matchSuffix(reHouse, rest); ok {
|
||||||
|
p.House = trimLeadingZeros(g[1])
|
||||||
|
p.HouseLetter = firstNonEmpty(g[2], g[4])
|
||||||
|
p.HouseFrac = trimLeadingZeros(g[3])
|
||||||
|
p.HasHouse = true
|
||||||
|
rest = strings.TrimSpace(rest[:start])
|
||||||
|
}
|
||||||
|
|
||||||
|
parseStreet(&p, rest)
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// firstNonEmpty returns the first non-empty argument. House/building letters
|
||||||
|
// can appear either before or after the fraction ("45а/4" vs "1/1а").
|
||||||
|
func firstNonEmpty(values ...string) string {
|
||||||
|
for _, v := range values {
|
||||||
|
if v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseStreet fills the street-related fields of p from the leftover string
|
||||||
|
// (the address with its house block removed).
|
||||||
|
func parseStreet(p *ParsedAddress, s string) {
|
||||||
|
tokens := strings.Fields(s)
|
||||||
|
if len(tokens) == 0 {
|
||||||
|
p.StreetKey = streetKey("", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
name := make([]string, 0, len(tokens))
|
||||||
|
typeFound := false
|
||||||
|
for _, tok := range tokens {
|
||||||
|
// Only the first recognised type token is treated as the type; a
|
||||||
|
// second occurrence is part of the name.
|
||||||
|
if !typeFound {
|
||||||
|
if t, ok := streetTypes[tok]; ok {
|
||||||
|
p.StreetType = t
|
||||||
|
typeFound = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
name = append(name, tok)
|
||||||
|
}
|
||||||
|
|
||||||
|
name = extractOrdinal(p, name)
|
||||||
|
p.NameDisplay = strings.Join(name, " ")
|
||||||
|
|
||||||
|
sorted := append([]string(nil), name...)
|
||||||
|
sort.Strings(sorted)
|
||||||
|
p.NameKey = strings.Join(sorted, " ")
|
||||||
|
p.StreetKey = streetKey(p.Ordinal, p.NameKey)
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractOrdinal recognises a leading ordinal ("2-я", "2 я", "вторая") and
|
||||||
|
// removes it from the name tokens.
|
||||||
|
func extractOrdinal(p *ParsedAddress, tokens []string) []string {
|
||||||
|
if len(tokens) == 0 {
|
||||||
|
return tokens
|
||||||
|
}
|
||||||
|
|
||||||
|
if m := reOrdinalToken.FindStringSubmatch(tokens[0]); m != nil {
|
||||||
|
p.Ordinal = normalizeOrdinal(m[1])
|
||||||
|
return tokens[1:]
|
||||||
|
}
|
||||||
|
if isDigits(tokens[0]) && len(tokens) >= 2 && ordinalSuffixes[tokens[1]] {
|
||||||
|
p.Ordinal = normalizeOrdinal(tokens[0])
|
||||||
|
return tokens[2:]
|
||||||
|
}
|
||||||
|
if d, ok := ordinalWords[tokens[0]]; ok {
|
||||||
|
p.Ordinal = normalizeOrdinal(d)
|
||||||
|
return tokens[1:]
|
||||||
|
}
|
||||||
|
return tokens
|
||||||
|
}
|
||||||
|
|
||||||
|
// matchSuffix returns the capture groups of the match of re that ends exactly
|
||||||
|
// at the end of s, together with the byte offset where it starts. Go's regexp
|
||||||
|
// is leftmost-first, so we scan all matches and pick the right-anchored one.
|
||||||
|
func matchSuffix(re *regexp.Regexp, s string) (groups []string, start int, ok bool) {
|
||||||
|
for _, loc := range re.FindAllStringSubmatchIndex(s, -1) {
|
||||||
|
if loc[1] != len(s) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
n := len(loc) / 2
|
||||||
|
groups = make([]string, n)
|
||||||
|
for i := 0; i < n; i++ {
|
||||||
|
if loc[2*i] >= 0 {
|
||||||
|
groups[i] = s[loc[2*i]:loc[2*i+1]]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return groups, loc[0], true
|
||||||
|
}
|
||||||
|
return nil, 0, false
|
||||||
|
}
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
package address
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestParseHouseBlock(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
raw string
|
||||||
|
wantOrdinal string
|
||||||
|
wantName string
|
||||||
|
wantType string
|
||||||
|
wantHouse string
|
||||||
|
wantFrac string
|
||||||
|
wantLetter string
|
||||||
|
wantBuild string
|
||||||
|
wantBFrac string
|
||||||
|
wantBLetter string
|
||||||
|
wantStruct string
|
||||||
|
wantHas bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "ordinal, fraction and structure",
|
||||||
|
raw: "2-я Станционная улица 1/1 стр.",
|
||||||
|
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
|
||||||
|
wantHouse: "1", wantFrac: "1", wantStruct: "", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "building with fraction and structure",
|
||||||
|
raw: "Станционная улица 30а к9/1 стр.",
|
||||||
|
wantName: "станционная", wantType: "улица",
|
||||||
|
wantHouse: "30", wantLetter: "а", wantBuild: "9", wantBFrac: "1", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "letter glued to house",
|
||||||
|
raw: "10-й Бронный переулок 33а",
|
||||||
|
wantOrdinal: "10", wantName: "бронный", wantType: "переулок",
|
||||||
|
wantHouse: "33", wantLetter: "а", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "microdistrict, two-word name",
|
||||||
|
raw: "Зеленый бор микрорайон 13",
|
||||||
|
wantName: "бор зеленый", wantType: "микрорайон",
|
||||||
|
wantHouse: "13", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "spelled-out ordinal",
|
||||||
|
raw: "вторая Станционная улица 1",
|
||||||
|
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
|
||||||
|
wantHouse: "1", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "en dash variant",
|
||||||
|
raw: "2–я Станционная улица 1",
|
||||||
|
wantOrdinal: "2", wantName: "станционная", wantType: "улица",
|
||||||
|
wantHouse: "1", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "no house",
|
||||||
|
raw: "Станционная улица",
|
||||||
|
wantName: "станционная", wantType: "улица", wantHas: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "structure with number only",
|
||||||
|
raw: "Станционная улица 3 стр.",
|
||||||
|
wantName: "станционная", wantType: "улица",
|
||||||
|
wantHouse: "3", wantStruct: "", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "letter before fraction",
|
||||||
|
raw: "Выборная улица 45а/4",
|
||||||
|
wantName: "выборная", wantType: "улица",
|
||||||
|
wantHouse: "45", wantLetter: "а", wantFrac: "4", wantHas: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "corpus letter without digit",
|
||||||
|
raw: "1-я Ельцовка улица 1 кД",
|
||||||
|
wantOrdinal: "1", wantName: "ельцовка", wantType: "улица",
|
||||||
|
wantHouse: "1", wantBLetter: "д", wantHas: true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got := Parse(tt.raw)
|
||||||
|
if got.Ordinal != tt.wantOrdinal {
|
||||||
|
t.Errorf("Ordinal = %q, want %q", got.Ordinal, tt.wantOrdinal)
|
||||||
|
}
|
||||||
|
if got.NameKey != tt.wantName {
|
||||||
|
t.Errorf("NameKey = %q, want %q", got.NameKey, tt.wantName)
|
||||||
|
}
|
||||||
|
if got.StreetType != tt.wantType {
|
||||||
|
t.Errorf("StreetType = %q, want %q", got.StreetType, tt.wantType)
|
||||||
|
}
|
||||||
|
if got.House != tt.wantHouse {
|
||||||
|
t.Errorf("House = %q, want %q", got.House, tt.wantHouse)
|
||||||
|
}
|
||||||
|
if got.HouseFrac != tt.wantFrac {
|
||||||
|
t.Errorf("HouseFrac = %q, want %q", got.HouseFrac, tt.wantFrac)
|
||||||
|
}
|
||||||
|
if got.HouseLetter != tt.wantLetter {
|
||||||
|
t.Errorf("HouseLetter = %q, want %q", got.HouseLetter, tt.wantLetter)
|
||||||
|
}
|
||||||
|
if got.Building != tt.wantBuild {
|
||||||
|
t.Errorf("Building = %q, want %q", got.Building, tt.wantBuild)
|
||||||
|
}
|
||||||
|
if got.BuildingFrac != tt.wantBFrac {
|
||||||
|
t.Errorf("BuildingFrac = %q, want %q", got.BuildingFrac, tt.wantBFrac)
|
||||||
|
}
|
||||||
|
if got.BuildingLetter != tt.wantBLetter {
|
||||||
|
t.Errorf("BuildingLetter = %q, want %q", got.BuildingLetter, tt.wantBLetter)
|
||||||
|
}
|
||||||
|
if got.Structure != tt.wantStruct {
|
||||||
|
t.Errorf("Structure = %q, want %q", got.Structure, tt.wantStruct)
|
||||||
|
}
|
||||||
|
if got.HasHouse != tt.wantHas {
|
||||||
|
t.Errorf("HasHouse = %v, want %v", got.HasHouse, tt.wantHas)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseDeterministic guards the core invariant: parsing the same input
|
||||||
|
// always yields the same structured result and the same street key.
|
||||||
|
func TestParseDeterministic(t *testing.T) {
|
||||||
|
input := "2-я Станционная улица 1/1 стр."
|
||||||
|
first := Parse(input)
|
||||||
|
for i := 0; i < 100; i++ {
|
||||||
|
if got := Parse(input); got.StreetKey != first.StreetKey || got.House != first.House {
|
||||||
|
t.Fatalf("non-deterministic parse: %+v vs %+v", got, first)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNormalizeIdempotent(t *testing.T) {
|
||||||
|
inputs := []string{
|
||||||
|
" 2-Я Станционная\u00a0улица 1/1 стр. ",
|
||||||
|
"ЁЛКИ д. 5, кв. 3",
|
||||||
|
"СТАНИЦИОННАЯ — улица, 30а",
|
||||||
|
"",
|
||||||
|
}
|
||||||
|
for _, in := range inputs {
|
||||||
|
once := NormalizeText(in)
|
||||||
|
twice := NormalizeText(once)
|
||||||
|
if once != twice {
|
||||||
|
t.Errorf("NormalizeText not idempotent for %q: %q != %q", in, once, twice)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
package address
|
||||||
|
|
||||||
|
import "strings"
|
||||||
|
|
||||||
|
// streetTypes maps a surface form (including common abbreviations) to a single
|
||||||
|
// canonical street type. The type is informational for matching (we strip it
|
||||||
|
// from the key) but useful for disambiguation and display.
|
||||||
|
var streetTypes = map[string]string{
|
||||||
|
"ул": "улица", "улица": "улица",
|
||||||
|
"пер": "переулок", "пер-к": "переулок", "переулок": "переулок",
|
||||||
|
"пр": "проезд", "проезд": "проезд",
|
||||||
|
"пр-т": "проспект", "просп": "проспект", "проспект": "проспект",
|
||||||
|
"ш": "шоссе", "шоссе": "шоссе",
|
||||||
|
"мкр": "микрорайон", "микрорайон": "микрорайон",
|
||||||
|
"туп": "тупик", "тупик": "тупик",
|
||||||
|
"спуск": "спуск",
|
||||||
|
"б-р": "бульвар", "бул": "бульвар", "бульвар": "бульвар",
|
||||||
|
"пл": "площадь", "площадь": "площадь",
|
||||||
|
"пос": "поселок", "поселок": "поселок",
|
||||||
|
"тракт": "тракт",
|
||||||
|
"аллея": "аллея",
|
||||||
|
"наб": "набережная", "набережная": "набережная",
|
||||||
|
"кольцо": "кольцо",
|
||||||
|
"просека": "просека",
|
||||||
|
"квартал": "квартал",
|
||||||
|
"территория": "территория",
|
||||||
|
"дорога": "дорога",
|
||||||
|
"разъезд": "разъезд",
|
||||||
|
}
|
||||||
|
|
||||||
|
// ordinalWords maps spelled-out ordinals to their digit form. Only the forms
|
||||||
|
// that actually occur in practice are listed; unknown words are treated as
|
||||||
|
// part of the street name.
|
||||||
|
var ordinalWords = map[string]string{
|
||||||
|
"первый": "1", "первая": "1", "первое": "1",
|
||||||
|
"второй": "2", "вторая": "2", "второе": "2",
|
||||||
|
"третий": "3", "третья": "3", "третье": "3",
|
||||||
|
"четвертый": "4", "четвертая": "4", "четвертое": "4",
|
||||||
|
"пятый": "5", "пятая": "5", "пятое": "5",
|
||||||
|
"шестой": "6", "шестая": "6", "шестое": "6",
|
||||||
|
"седьмой": "7", "седьмая": "7", "седьмое": "7",
|
||||||
|
"восьмой": "8", "восьмая": "8", "восьмое": "8",
|
||||||
|
"девятый": "9", "девятая": "9", "девятое": "9",
|
||||||
|
"десятый": "10", "десятая": "10", "десятое": "10",
|
||||||
|
}
|
||||||
|
|
||||||
|
// ordinalSuffixes are the gender/number endings that may follow an ordinal
|
||||||
|
// digit when it is written as a separate token ("2 я", "10 й").
|
||||||
|
var ordinalSuffixes = map[string]bool{
|
||||||
|
"я": true, "й": true, "е": true,
|
||||||
|
"ая": true, "ый": true, "ой": true, "ое": true,
|
||||||
|
"ий": true, "ья": true, "ье": true, "ьи": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// streetKey builds the canonical, type-independent street key:
|
||||||
|
// "<ordinal>|<sorted name tokens>". The type word is dropped so that a user
|
||||||
|
// who omits it ("станционная 1" instead of "Станционная улица 1") still
|
||||||
|
// matches; type confusion is surfaced later as ambiguity.
|
||||||
|
func streetKey(ordinal, nameKey string) string {
|
||||||
|
return ordinal + "|" + nameKey
|
||||||
|
}
|
||||||
|
|
||||||
|
// normalizeOrdinal trims leading zeros so "02-я" and "2-я" agree.
|
||||||
|
func normalizeOrdinal(s string) string {
|
||||||
|
return trimLeadingZeros(strings.TrimSpace(s))
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
// Package address contains the deterministic address domain: text
|
||||||
|
// normalization, structural parsing and the canonical data types shared by the
|
||||||
|
// rest of addrkit.
|
||||||
|
//
|
||||||
|
// The package is deliberately free of any I/O (no HTTP, no database, no
|
||||||
|
// filesystem). Everything here is a pure function of its inputs, which makes
|
||||||
|
// the core trivially testable and reusable by the service, the CLI and tests.
|
||||||
|
package address
|
||||||
|
|
||||||
|
// Status is the machine-readable outcome of an address resolution.
|
||||||
|
//
|
||||||
|
// It is intentionally explicit: callers must never have to guess whether an
|
||||||
|
// empty result means "not found" or "the upstream failed".
|
||||||
|
type Status string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// StatusFound means the street and the exact house unit were resolved.
|
||||||
|
StatusFound Status = "found"
|
||||||
|
// StatusPartial means the street was resolved but the house unit was not
|
||||||
|
// confirmed. The answer is safe but incomplete; the caller may ask the
|
||||||
|
// user to clarify the house number.
|
||||||
|
StatusPartial Status = "partial"
|
||||||
|
// StatusAmbiguous means more than one plausible match exists and the
|
||||||
|
// resolver refuses to guess.
|
||||||
|
StatusAmbiguous Status = "ambiguous"
|
||||||
|
// StatusNotFound means nothing in the registry matched.
|
||||||
|
StatusNotFound Status = "not_found"
|
||||||
|
// StatusPortalTimeout means the external outages portal did not answer in
|
||||||
|
// time. It is an infrastructure outcome, not "no outages".
|
||||||
|
StatusPortalTimeout Status = "portal_timeout"
|
||||||
|
// StatusInfra is a generic terminal infrastructure failure.
|
||||||
|
StatusInfra Status = "infra_error"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ParsedAddress is the structured form of a free-form address string.
|
||||||
|
//
|
||||||
|
// The parser is lossy on purpose: punctuation, case and spacing are folded
|
||||||
|
// away so that different surface forms of the same address converge on the
|
||||||
|
// same StreetKey.
|
||||||
|
type ParsedAddress struct {
|
||||||
|
Raw string `json:"raw"`
|
||||||
|
Normalized string `json:"normalized"`
|
||||||
|
|
||||||
|
// Street components.
|
||||||
|
//
|
||||||
|
// NameKey is the order-insensitive canonical name (tokens sorted), so
|
||||||
|
// "Станционная 2-я" and "2-я Станционная" produce the same key.
|
||||||
|
Ordinal string `json:"ordinal,omitempty"`
|
||||||
|
NameKey string `json:"name_key,omitempty"`
|
||||||
|
NameDisplay string `json:"name_display,omitempty"`
|
||||||
|
StreetType string `json:"street_type,omitempty"`
|
||||||
|
StreetKey string `json:"street_key"`
|
||||||
|
|
||||||
|
// House unit components. A house "1/1 к2 стр.3" decomposes into
|
||||||
|
// House=1, HouseFrac=1, Building=2, Structure=3.
|
||||||
|
HasHouse bool `json:"has_house"`
|
||||||
|
House string `json:"house,omitempty"`
|
||||||
|
HouseFrac string `json:"house_frac,omitempty"`
|
||||||
|
HouseLetter string `json:"house_letter,omitempty"`
|
||||||
|
Building string `json:"building,omitempty"`
|
||||||
|
BuildingFrac string `json:"building_frac,omitempty"`
|
||||||
|
BuildingLetter string `json:"building_letter,omitempty"`
|
||||||
|
Structure string `json:"structure,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Entry is a single canonical row from the address registry.
|
||||||
|
//
|
||||||
|
// Entries are produced at seed time by parsing the registry's display address
|
||||||
|
// with the very same parser used at query time, so index and query always
|
||||||
|
// agree on how an address is decomposed.
|
||||||
|
type Entry struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Address string `json:"address"`
|
||||||
|
District string `json:"district,omitempty"`
|
||||||
|
|
||||||
|
Ordinal string `json:"ordinal,omitempty"`
|
||||||
|
NameKey string `json:"name_key,omitempty"`
|
||||||
|
NameDisplay string `json:"name_display,omitempty"`
|
||||||
|
StreetType string `json:"street_type,omitempty"`
|
||||||
|
StreetKey string `json:"street_key"`
|
||||||
|
|
||||||
|
House string `json:"house,omitempty"`
|
||||||
|
HouseFrac string `json:"house_frac,omitempty"`
|
||||||
|
HouseLetter string `json:"house_letter,omitempty"`
|
||||||
|
Building string `json:"building,omitempty"`
|
||||||
|
BuildingFrac string `json:"building_frac,omitempty"`
|
||||||
|
BuildingLetter string `json:"building_letter,omitempty"`
|
||||||
|
Structure string `json:"structure,omitempty"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,299 @@
|
|||||||
|
// Package match resolves a ParsedAddress against a canonical registry index.
|
||||||
|
//
|
||||||
|
// It is the deterministic heart of addrkit. It knows nothing about storage,
|
||||||
|
// HTTP or LLMs: it operates on an in-memory index built from []address.Entry
|
||||||
|
// and always answers from that data. A wrong house number is never invented —
|
||||||
|
// when the exact house unit is not present the result is "partial", and when
|
||||||
|
// several streets are plausible the result is "ambiguous".
|
||||||
|
package match
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Candidate is one plausible match, returned alongside a non-"found" result so
|
||||||
|
// the caller (or an operator) can see what the resolver was choosing between.
|
||||||
|
type Candidate struct {
|
||||||
|
Address string `json:"address"`
|
||||||
|
District string `json:"district,omitempty"`
|
||||||
|
House string `json:"house,omitempty"`
|
||||||
|
Score float64 `json:"score"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Result is the outcome of a resolution.
|
||||||
|
type Result struct {
|
||||||
|
Status address.Status `json:"status"`
|
||||||
|
Address string `json:"address,omitempty"`
|
||||||
|
District string `json:"district,omitempty"`
|
||||||
|
Confidence float64 `json:"confidence"`
|
||||||
|
Candidates []Candidate `json:"candidates,omitempty"`
|
||||||
|
DataVersion string `json:"data_version,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Options tunes matching behaviour.
|
||||||
|
type Options struct {
|
||||||
|
// FuzzyThreshold is the minimum similarity (0..1) for a street to be
|
||||||
|
// accepted as a typo-tolerant match. Below it the result is not_found.
|
||||||
|
FuzzyThreshold float64
|
||||||
|
// AmbiguityMargin: streets within this similarity of the best match make
|
||||||
|
// the result ambiguous instead of silently picking one.
|
||||||
|
AmbiguityMargin float64
|
||||||
|
// MaxCandidates caps the candidates returned to the caller.
|
||||||
|
MaxCandidates int
|
||||||
|
}
|
||||||
|
|
||||||
|
// DefaultOptions returns conservative defaults.
|
||||||
|
func DefaultOptions() Options {
|
||||||
|
return Options{
|
||||||
|
FuzzyThreshold: 0.82,
|
||||||
|
AmbiguityMargin: 0.03,
|
||||||
|
MaxCandidates: 5,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type streetRef struct {
|
||||||
|
key string
|
||||||
|
ordinal string
|
||||||
|
nameKey string
|
||||||
|
streetType string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Index is an immutable, in-memory registry index. It is safe for concurrent
|
||||||
|
// reads once constructed.
|
||||||
|
type Index struct {
|
||||||
|
version string
|
||||||
|
opts Options
|
||||||
|
byKey map[string][]address.Entry
|
||||||
|
streets []streetRef
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewIndex builds an index from canonical entries. Entries sharing a StreetKey
|
||||||
|
// are grouped; distinct streets are collected for typo-tolerant matching.
|
||||||
|
func NewIndex(version string, entries []address.Entry, opts Options) *Index {
|
||||||
|
ix := &Index{
|
||||||
|
version: version,
|
||||||
|
opts: opts,
|
||||||
|
byKey: make(map[string][]address.Entry, len(entries)),
|
||||||
|
}
|
||||||
|
seen := make(map[string]struct{}, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
ix.byKey[e.StreetKey] = append(ix.byKey[e.StreetKey], e)
|
||||||
|
if _, ok := seen[e.StreetKey]; !ok {
|
||||||
|
seen[e.StreetKey] = struct{}{}
|
||||||
|
ix.streets = append(ix.streets, streetRef{
|
||||||
|
key: e.StreetKey,
|
||||||
|
ordinal: e.Ordinal,
|
||||||
|
nameKey: e.NameKey,
|
||||||
|
streetType: e.StreetType,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(ix.streets, func(i, j int) bool { return ix.streets[i].key < ix.streets[j].key })
|
||||||
|
return ix
|
||||||
|
}
|
||||||
|
|
||||||
|
// Version returns the data version the index was built from.
|
||||||
|
func (ix *Index) Version() string { return ix.version }
|
||||||
|
|
||||||
|
// Size returns the number of indexed entries.
|
||||||
|
func (ix *Index) Size() int {
|
||||||
|
n := 0
|
||||||
|
for _, v := range ix.byKey {
|
||||||
|
n += len(v)
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve resolves a parsed address. It never returns an address that is not
|
||||||
|
// present in the index.
|
||||||
|
func (ix *Index) Resolve(p address.ParsedAddress) Result {
|
||||||
|
res := Result{DataVersion: ix.version}
|
||||||
|
|
||||||
|
entries := ix.byKey[p.StreetKey]
|
||||||
|
similarity := 1.0
|
||||||
|
|
||||||
|
if len(entries) == 0 {
|
||||||
|
var ambiguousKeys []string
|
||||||
|
entries, similarity, ambiguousKeys = ix.fuzzyStreets(p)
|
||||||
|
if len(ambiguousKeys) > 0 {
|
||||||
|
res.Status = address.StatusAmbiguous
|
||||||
|
res.Confidence = similarity
|
||||||
|
res.Candidates = ix.representatives(ambiguousKeys, similarity)
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(entries) == 0 {
|
||||||
|
res.Status = address.StatusNotFound
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
|
||||||
|
if !p.HasHouse {
|
||||||
|
// Street is known but the caller did not supply a house: safe but
|
||||||
|
// incomplete.
|
||||||
|
res.Status = address.StatusPartial
|
||||||
|
res.Confidence = clamp01(similarity * 0.8)
|
||||||
|
res.Candidates = candidates(entries, similarity, ix.opts.MaxCandidates)
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
|
||||||
|
matched := matchHouseUnit(entries, p)
|
||||||
|
switch len(matched) {
|
||||||
|
case 1:
|
||||||
|
res.Status = address.StatusFound
|
||||||
|
res.Address = matched[0].Address
|
||||||
|
res.District = matched[0].District
|
||||||
|
res.Confidence = clamp01(similarity)
|
||||||
|
return res
|
||||||
|
case 0:
|
||||||
|
// Street recognised, exact house unit absent, or several units share
|
||||||
|
// the same number (ambiguous at the unit level).
|
||||||
|
if sameHouseSeveral(entries, p) {
|
||||||
|
res.Status = address.StatusAmbiguous
|
||||||
|
res.Confidence = clamp01(similarity * 0.9)
|
||||||
|
res.Candidates = candidates(matchHouseUnitAll(entries, p), similarity, ix.opts.MaxCandidates)
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
res.Status = address.StatusPartial
|
||||||
|
res.Confidence = clamp01(similarity * 0.9)
|
||||||
|
res.Candidates = candidates(entries, similarity, ix.opts.MaxCandidates)
|
||||||
|
return res
|
||||||
|
default:
|
||||||
|
res.Status = address.StatusAmbiguous
|
||||||
|
res.Confidence = clamp01(similarity * 0.9)
|
||||||
|
res.Candidates = candidates(matched, similarity, ix.opts.MaxCandidates)
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// fuzzyStreets finds streets similar to the query. It returns the entries of
|
||||||
|
// the best street when the match is unambiguous, or the list of competing
|
||||||
|
// street keys when several are within the ambiguity margin.
|
||||||
|
func (ix *Index) fuzzyStreets(p address.ParsedAddress) (entries []address.Entry, similarity float64, ambiguousKeys []string) {
|
||||||
|
if p.NameKey == "" {
|
||||||
|
return nil, 0, nil
|
||||||
|
}
|
||||||
|
best := 0.0
|
||||||
|
bestKey := ""
|
||||||
|
for _, s := range ix.streets {
|
||||||
|
// If both sides carry an ordinal and they differ, the streets are
|
||||||
|
// different even if the names rhyme.
|
||||||
|
if p.Ordinal != "" && s.ordinal != "" && p.Ordinal != s.ordinal {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if sim := ratio(p.NameKey, s.nameKey); sim > best {
|
||||||
|
best = sim
|
||||||
|
bestKey = s.key
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if best < ix.opts.FuzzyThreshold {
|
||||||
|
return nil, best, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, s := range ix.streets {
|
||||||
|
if p.Ordinal != "" && s.ordinal != "" && p.Ordinal != s.ordinal {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if ratio(p.NameKey, s.nameKey) >= best-ix.opts.AmbiguityMargin {
|
||||||
|
ambiguousKeys = append(ambiguousKeys, s.key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(ambiguousKeys) > 1 {
|
||||||
|
return nil, best, ambiguousKeys
|
||||||
|
}
|
||||||
|
return ix.byKey[bestKey], best, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// representatives returns one candidate per competing street, which is far
|
||||||
|
// more useful to a user than five houses of a single street.
|
||||||
|
func (ix *Index) representatives(keys []string, score float64) []Candidate {
|
||||||
|
sort.Strings(keys)
|
||||||
|
out := make([]Candidate, 0, len(keys))
|
||||||
|
for _, k := range keys {
|
||||||
|
es := ix.byKey[k]
|
||||||
|
if len(es) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
e := es[0]
|
||||||
|
out = append(out, Candidate{
|
||||||
|
Address: e.Address,
|
||||||
|
District: e.District,
|
||||||
|
House: e.House,
|
||||||
|
Score: clamp01(score),
|
||||||
|
})
|
||||||
|
if len(out) == ix.opts.MaxCandidates {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// matchHouseUnit returns entries whose house unit equals p's, exactly.
|
||||||
|
func matchHouseUnit(entries []address.Entry, p address.ParsedAddress) []address.Entry {
|
||||||
|
var out []address.Entry
|
||||||
|
for _, e := range entries {
|
||||||
|
if houseUnitEqual(e, p) {
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// matchHouseUnitAll returns all entries for entries that share the same house
|
||||||
|
// number as p (used to surface ambiguity).
|
||||||
|
func matchHouseUnitAll(entries []address.Entry, p address.ParsedAddress) []address.Entry {
|
||||||
|
var out []address.Entry
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.House == p.House && e.HouseFrac == p.HouseFrac && e.HouseLetter == p.HouseLetter {
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// sameHouseSeveral reports whether more than one entry matches the house
|
||||||
|
// number but with differing buildings/letters, i.e. the unit is ambiguous.
|
||||||
|
func sameHouseSeveral(entries []address.Entry, p address.ParsedAddress) bool {
|
||||||
|
return len(matchHouseUnitAll(entries, p)) > 1
|
||||||
|
}
|
||||||
|
|
||||||
|
// houseUnitEqual compares the parts of a house unit. Unspecified (empty) parts
|
||||||
|
// must also be empty on the entry side, so "60/1" never matches "60/1 к148".
|
||||||
|
func houseUnitEqual(e address.Entry, p address.ParsedAddress) bool {
|
||||||
|
return e.House == p.House &&
|
||||||
|
e.HouseFrac == p.HouseFrac &&
|
||||||
|
e.HouseLetter == p.HouseLetter &&
|
||||||
|
e.Building == p.Building &&
|
||||||
|
e.BuildingFrac == p.BuildingFrac &&
|
||||||
|
e.BuildingLetter == p.BuildingLetter
|
||||||
|
}
|
||||||
|
|
||||||
|
func candidates(entries []address.Entry, score float64, limit int) []Candidate {
|
||||||
|
if limit <= 0 || len(entries) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make([]Candidate, 0, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
out = append(out, Candidate{
|
||||||
|
Address: e.Address,
|
||||||
|
District: e.District,
|
||||||
|
House: e.House,
|
||||||
|
Score: clamp01(score),
|
||||||
|
})
|
||||||
|
if len(out) == limit {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func clamp01(v float64) float64 {
|
||||||
|
if v < 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
if v > 1 {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
package match
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
// buildIndex parses canonical addresses into entries, mirroring what the seed
|
||||||
|
// pipeline does, so tests exercise the real code path end to end.
|
||||||
|
func buildIndex(t *testing.T, addresses ...string) *Index {
|
||||||
|
t.Helper()
|
||||||
|
entries := make([]address.Entry, 0, len(addresses))
|
||||||
|
for i, a := range addresses {
|
||||||
|
p := address.Parse(a)
|
||||||
|
entries = append(entries, address.Entry{
|
||||||
|
ID: a, Address: a, District: "Тестовый",
|
||||||
|
Ordinal: p.Ordinal, NameKey: p.NameKey, StreetType: p.StreetType, StreetKey: p.StreetKey,
|
||||||
|
House: p.House, HouseFrac: p.HouseFrac, HouseLetter: p.HouseLetter,
|
||||||
|
Building: p.Building, BuildingFrac: p.BuildingFrac, BuildingLetter: p.BuildingLetter,
|
||||||
|
Structure: p.Structure,
|
||||||
|
})
|
||||||
|
_ = i
|
||||||
|
}
|
||||||
|
return NewIndex("test", entries, DefaultOptions())
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveStatuses(t *testing.T) {
|
||||||
|
ix := buildIndex(t,
|
||||||
|
"Станционная улица 1/1",
|
||||||
|
"Станционная улица 2/1",
|
||||||
|
"Станционная улица 3/1 к1",
|
||||||
|
"Станционная улица 3/1 к2",
|
||||||
|
"Искитимская улица 5",
|
||||||
|
)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
query string
|
||||||
|
wantStatus address.Status
|
||||||
|
wantAddr string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "found with ordinal, fraction and structure noise",
|
||||||
|
query: "2-я Станционная улица 1/1 стр.",
|
||||||
|
wantStatus: address.StatusFound,
|
||||||
|
wantAddr: "Станционная улица 1/1",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "found exact house",
|
||||||
|
query: "станционная 2/1",
|
||||||
|
wantStatus: address.StatusFound,
|
||||||
|
wantAddr: "Станционная улица 2/1",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "found with corpus",
|
||||||
|
query: "Станционная улица 3/1 корпус 1",
|
||||||
|
wantStatus: address.StatusFound,
|
||||||
|
wantAddr: "Станционная улица 3/1 к1",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "typo in street name tolerated",
|
||||||
|
query: "станционая 1/1",
|
||||||
|
wantStatus: address.StatusFound,
|
||||||
|
wantAddr: "Станционная улица 1/1",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "missing house is partial",
|
||||||
|
query: "станционная улица",
|
||||||
|
wantStatus: address.StatusPartial,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong house fraction is partial",
|
||||||
|
query: "станционная 1/2",
|
||||||
|
wantStatus: address.StatusPartial,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "ambiguous building",
|
||||||
|
query: "станционная 3/1",
|
||||||
|
wantStatus: address.StatusAmbiguous,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unknown street",
|
||||||
|
query: "ленина 5",
|
||||||
|
wantStatus: address.StatusNotFound,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
res := ix.Resolve(address.Parse(tt.query))
|
||||||
|
if res.Status != tt.wantStatus {
|
||||||
|
t.Fatalf("status = %q, want %q (candidates: %+v)", res.Status, tt.wantStatus, res.Candidates)
|
||||||
|
}
|
||||||
|
if tt.wantAddr != "" && res.Address != tt.wantAddr {
|
||||||
|
t.Errorf("address = %q, want %q", res.Address, tt.wantAddr)
|
||||||
|
}
|
||||||
|
if res.DataVersion != "test" {
|
||||||
|
t.Errorf("data version = %q, want test", res.DataVersion)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestResolveNeverInventsAddress is the central safety property: whenever the
|
||||||
|
// status is "found", the returned address must exist in the index.
|
||||||
|
func TestResolveNeverInventsAddress(t *testing.T) {
|
||||||
|
addresses := []string{
|
||||||
|
"Станционная улица 1/1",
|
||||||
|
"Станционная улица 2/1",
|
||||||
|
"Искитимская улица 5",
|
||||||
|
}
|
||||||
|
ix := buildIndex(t, addresses...)
|
||||||
|
known := make(map[string]bool, len(addresses))
|
||||||
|
for _, a := range addresses {
|
||||||
|
known[a] = true
|
||||||
|
}
|
||||||
|
queries := []string{
|
||||||
|
"2-я Станционная 1/1", "станционная 2/1 к9", "искитимская 5", "искитимская 5/1",
|
||||||
|
"несуществующая 99", "станционная 1/1 стр.3", " ",
|
||||||
|
}
|
||||||
|
for _, q := range queries {
|
||||||
|
res := ix.Resolve(address.Parse(q))
|
||||||
|
if res.Status == address.StatusFound && !known[res.Address] {
|
||||||
|
t.Fatalf("invented address %q for query %q", res.Address, q)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRatio(t *testing.T) {
|
||||||
|
if got := ratio("станционная", "станционная"); got != 1 {
|
||||||
|
t.Errorf("identical strings ratio = %v, want 1", got)
|
||||||
|
}
|
||||||
|
if got := ratio("станционная", "искитимская"); got > 0.6 {
|
||||||
|
t.Errorf("unrelated strings ratio = %v, want < 0.6", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
package match
|
||||||
|
|
||||||
|
// levenshtein returns the edit distance between two strings, measured in
|
||||||
|
// runes. It uses the rolling-row formulation: O(min(n,m)) memory.
|
||||||
|
//
|
||||||
|
// The resolver uses it only to tolerate typos in the *street name*; house
|
||||||
|
// numbers are always compared exactly, because a wrong house is worse than a
|
||||||
|
// missing one.
|
||||||
|
func levenshtein(a, b string) int {
|
||||||
|
ra, rb := []rune(a), []rune(b)
|
||||||
|
if len(ra) == 0 {
|
||||||
|
return len(rb)
|
||||||
|
}
|
||||||
|
if len(rb) == 0 {
|
||||||
|
return len(ra)
|
||||||
|
}
|
||||||
|
prev := make([]int, len(rb)+1)
|
||||||
|
curr := make([]int, len(rb)+1)
|
||||||
|
for j := range prev {
|
||||||
|
prev[j] = j
|
||||||
|
}
|
||||||
|
for i := 1; i <= len(ra); i++ {
|
||||||
|
curr[0] = i
|
||||||
|
for j := 1; j <= len(rb); j++ {
|
||||||
|
cost := 1
|
||||||
|
if ra[i-1] == rb[j-1] {
|
||||||
|
cost = 0
|
||||||
|
}
|
||||||
|
curr[j] = min3(prev[j]+1, curr[j-1]+1, prev[j-1]+cost)
|
||||||
|
}
|
||||||
|
prev, curr = curr, prev
|
||||||
|
}
|
||||||
|
return prev[len(rb)]
|
||||||
|
}
|
||||||
|
|
||||||
|
// ratio returns a 0..1 similarity where 1 means identical. It is
|
||||||
|
// 1 - distance/max(len), which is stable and easy to reason about.
|
||||||
|
func ratio(a, b string) float64 {
|
||||||
|
if a == b {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
la, lb := len([]rune(a)), len([]rune(b))
|
||||||
|
m := la
|
||||||
|
if lb > m {
|
||||||
|
m = lb
|
||||||
|
}
|
||||||
|
if m == 0 {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 1 - float64(levenshtein(a, b))/float64(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
func min3(a, b, c int) int {
|
||||||
|
m := a
|
||||||
|
if b < m {
|
||||||
|
m = b
|
||||||
|
}
|
||||||
|
if c < m {
|
||||||
|
m = c
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
package match
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Rewriter is an optional, pluggable query rewriter. Its only job is to
|
||||||
|
// propose alternative surface forms of a user address (for example an LLM
|
||||||
|
// normalising "кошурникова одиннадцать дробь один" to "Кошевого 11/1"). The
|
||||||
|
// rewriter never decides the answer: every proposal is parsed and resolved
|
||||||
|
// deterministically, and only entries that exist in the index can be returned.
|
||||||
|
//
|
||||||
|
// It is defined here (consumer side) so the domain stays free of any transport
|
||||||
|
// or vendor dependency.
|
||||||
|
type Rewriter interface {
|
||||||
|
Rewrite(ctx context.Context, raw string) ([]string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolver wraps an Index with an optional Rewriter.
|
||||||
|
//
|
||||||
|
// The deterministic path is always tried first. The rewriter is consulted only
|
||||||
|
// when the deterministic result is not a confident "found", and its
|
||||||
|
// suggestions can only ever turn a miss into a match against real data.
|
||||||
|
type Resolver struct {
|
||||||
|
index *Index
|
||||||
|
rewriter Rewriter
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewResolver builds a resolver. rewriter may be nil to disable rewriting.
|
||||||
|
func NewResolver(index *Index, rewriter Rewriter) *Resolver {
|
||||||
|
return &Resolver{index: index, rewriter: rewriter}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Version returns the current data version.
|
||||||
|
func (r *Resolver) Version() string { return r.index.Version() }
|
||||||
|
|
||||||
|
// Ready reports whether the resolver has a usable index.
|
||||||
|
func (r *Resolver) Ready() bool { return r.index != nil && r.index.Size() > 0 }
|
||||||
|
|
||||||
|
// Resolve parses raw and resolves it, consulting the rewriter when enabled.
|
||||||
|
func (r *Resolver) Resolve(ctx context.Context, raw string) Result {
|
||||||
|
parsed := address.Parse(raw)
|
||||||
|
res := r.index.Resolve(parsed)
|
||||||
|
|
||||||
|
if res.Status == address.StatusFound || r.rewriter == nil {
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
if res.Status != address.StatusNotFound && res.Status != address.StatusAmbiguous {
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
|
||||||
|
alternatives, err := r.rewriter.Rewrite(ctx, raw)
|
||||||
|
if err != nil || len(alternatives) == 0 {
|
||||||
|
return res
|
||||||
|
}
|
||||||
|
|
||||||
|
best := res
|
||||||
|
for _, alt := range alternatives {
|
||||||
|
candidate := r.index.Resolve(address.Parse(alt))
|
||||||
|
if better(candidate, best) {
|
||||||
|
best = candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return best
|
||||||
|
}
|
||||||
|
|
||||||
|
// better ranks results: a stronger status wins, then higher confidence.
|
||||||
|
func better(a, b Result) bool {
|
||||||
|
ra, rb := statusRank(a.Status), statusRank(b.Status)
|
||||||
|
if ra != rb {
|
||||||
|
return ra > rb
|
||||||
|
}
|
||||||
|
return a.Confidence > b.Confidence
|
||||||
|
}
|
||||||
|
|
||||||
|
func statusRank(s address.Status) int {
|
||||||
|
switch s {
|
||||||
|
case address.StatusFound:
|
||||||
|
return 3
|
||||||
|
case address.StatusAmbiguous:
|
||||||
|
return 2
|
||||||
|
case address.StatusPartial:
|
||||||
|
return 1
|
||||||
|
default:
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
// Package outages contains the deterministic, pure logic for selecting outages
|
||||||
|
// that belong to a concrete house unit. It does not talk to the network: the
|
||||||
|
// portal client lives in an adapter and feeds this package.
|
||||||
|
package outages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Outage is a single outage record as returned by the external portal.
|
||||||
|
type Outage struct {
|
||||||
|
Type string `json:"type,omitempty"`
|
||||||
|
System string `json:"system,omitempty"`
|
||||||
|
Address string `json:"address,omitempty"`
|
||||||
|
StartDate string `json:"start_date,omitempty"`
|
||||||
|
EndDate string `json:"end_date,omitempty"`
|
||||||
|
Time string `json:"time,omitempty"`
|
||||||
|
Action string `json:"action,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Source is anything that can answer "what outages are there for this
|
||||||
|
// canonical address". It is implemented by the ArcGIS portal adapter and by
|
||||||
|
// fakes in tests.
|
||||||
|
type Source interface {
|
||||||
|
Query(ctx context.Context, canonical string) ([]Outage, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MatchingHouse keeps only the outages that address the same house unit as the
|
||||||
|
// target. This is the fix for the classic bug where a substring match returned
|
||||||
|
// a neighbouring house's outage: matching is done on the parsed structure, not
|
||||||
|
// on the raw text.
|
||||||
|
func MatchingHouse(list []Outage, target address.ParsedAddress) []Outage {
|
||||||
|
out := make([]Outage, 0, len(list))
|
||||||
|
for _, o := range list {
|
||||||
|
op := address.Parse(o.Address)
|
||||||
|
if houseUnitEqual(target, op) {
|
||||||
|
out = append(out, o)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// houseUnitEqual compares two parsed house units. House number, fraction,
|
||||||
|
// letter, building (corpus) and its fraction/letter must all match. The
|
||||||
|
// structure (строение) is deliberately ignored: it is frequently omitted by
|
||||||
|
// users and does not change which house an outage refers to.
|
||||||
|
func houseUnitEqual(a, b address.ParsedAddress) bool {
|
||||||
|
return a.House == b.House &&
|
||||||
|
a.HouseFrac == b.HouseFrac &&
|
||||||
|
a.HouseLetter == b.HouseLetter &&
|
||||||
|
a.Building == b.Building &&
|
||||||
|
a.BuildingFrac == b.BuildingFrac &&
|
||||||
|
a.BuildingLetter == b.BuildingLetter
|
||||||
|
}
|
||||||
|
|
||||||
|
// Current keeps outages whose [start, end] range covers now. The portal
|
||||||
|
// already filters by date, but we re-check locally because the portal's notion
|
||||||
|
// of "today" and ours must agree (Asia/Novosibirsk, UTC+7).
|
||||||
|
func Current(list []Outage, now time.Time, loc *time.Location) []Outage {
|
||||||
|
out := make([]Outage, 0, len(list))
|
||||||
|
for _, o := range list {
|
||||||
|
start, okStart := parsePortalTime(o.StartDate, loc)
|
||||||
|
end, okEnd := parsePortalTime(o.EndDate, loc)
|
||||||
|
if !okStart || !okEnd {
|
||||||
|
// Undated records are kept: better to report a possibly-current
|
||||||
|
// outage than to silently drop it.
|
||||||
|
out = append(out, o)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if (now.Equal(start) || now.After(start)) && (now.Equal(end) || now.Before(end)) {
|
||||||
|
out = append(out, o)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// parsePortalTime accepts the shapes the portal may produce: epoch
|
||||||
|
// milliseconds, epoch seconds, RFC3339 and a few common date formats. All
|
||||||
|
// results are interpreted in loc.
|
||||||
|
func parsePortalTime(s string, loc *time.Location) (time.Time, bool) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if s == "" {
|
||||||
|
return time.Time{}, false
|
||||||
|
}
|
||||||
|
if n, err := strconv.ParseInt(s, 10, 64); err == nil {
|
||||||
|
// Heuristic: 13+ digits is milliseconds, otherwise seconds.
|
||||||
|
if n > 1e12 {
|
||||||
|
return time.UnixMilli(n).In(loc), true
|
||||||
|
}
|
||||||
|
return time.Unix(n, 0).In(loc), true
|
||||||
|
}
|
||||||
|
layouts := []string{
|
||||||
|
time.RFC3339,
|
||||||
|
"2006-01-02T15:04:05",
|
||||||
|
"2006-01-02 15:04:05",
|
||||||
|
"2006-01-02",
|
||||||
|
}
|
||||||
|
for _, layout := range layouts {
|
||||||
|
if t, err := time.ParseInLocation(layout, s, loc); err == nil {
|
||||||
|
return t, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return time.Time{}, false
|
||||||
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package outages
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strconv"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestMatchingHouse(t *testing.T) {
|
||||||
|
list := []Outage{
|
||||||
|
{Address: "Станционная улица 1/1", System: "Теплоснабжение"},
|
||||||
|
{Address: "Станционная улица 1/2", System: "Теплоснабжение"},
|
||||||
|
{Address: "Станционная улица 1/1 к1", System: "Теплоснабжение"},
|
||||||
|
{Address: "Станционная улица 10", System: "Теплоснабжение"},
|
||||||
|
}
|
||||||
|
target := address.Parse("2-я Станционная улица 1/1")
|
||||||
|
|
||||||
|
got := MatchingHouse(list, target)
|
||||||
|
if len(got) != 1 {
|
||||||
|
t.Fatalf("expected exactly 1 matching outage, got %d: %+v", len(got), got)
|
||||||
|
}
|
||||||
|
if got[0].Address != "Станционная улица 1/1" {
|
||||||
|
t.Errorf("matched wrong outage: %q", got[0].Address)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMatchingHouseIgnoresStructure(t *testing.T) {
|
||||||
|
list := []Outage{{Address: "Станционная улица 1/1 стр.2"}}
|
||||||
|
target := address.Parse("станционная 1/1 стр.3")
|
||||||
|
if got := MatchingHouse(list, target); len(got) != 1 {
|
||||||
|
t.Fatalf("structure must not affect house matching; got %d", len(got))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCurrent(t *testing.T) {
|
||||||
|
loc := time.FixedZone("NSK", 7*3600)
|
||||||
|
now := time.Date(2026, 10, 9, 12, 0, 0, 0, loc)
|
||||||
|
within := now.Add(-time.Hour).UnixMilli()
|
||||||
|
after := now.Add(time.Hour).UnixMilli()
|
||||||
|
before := now.Add(-48 * time.Hour).UnixMilli()
|
||||||
|
|
||||||
|
list := []Outage{
|
||||||
|
{Address: "A", StartDate: itoa(within), EndDate: itoa(after)}, // current
|
||||||
|
{Address: "B", StartDate: itoa(before), EndDate: itoa(before)}, // past
|
||||||
|
{Address: "C", StartDate: "", EndDate: ""}, // undated -> kept
|
||||||
|
}
|
||||||
|
got := Current(list, now, loc)
|
||||||
|
if len(got) != 2 {
|
||||||
|
t.Fatalf("expected 2 current outages, got %d: %+v", len(got), got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func itoa(v int64) string {
|
||||||
|
return strconv.FormatInt(v, 10)
|
||||||
|
}
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
package seed
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestRealRegistryParseRate validates the parser against a real registry dump.
|
||||||
|
// It is skipped unless ADDRKIT_REGISTRY points at one, so it never runs in a
|
||||||
|
// plain `go test ./...` (no large fixture in the repo).
|
||||||
|
//
|
||||||
|
// Run it explicitly:
|
||||||
|
//
|
||||||
|
// ADDRKIT_REGISTRY=/path/to/address_registry.ndjson \
|
||||||
|
// go test ./internal/seed -run TestRealRegistryParseRate -v
|
||||||
|
func TestRealRegistryParseRate(t *testing.T) {
|
||||||
|
path := os.Getenv("ADDRKIT_REGISTRY")
|
||||||
|
if path == "" {
|
||||||
|
t.Skip("ADDRKIT_REGISTRY not set")
|
||||||
|
}
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open registry: %v", err)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
|
||||||
|
sources, err := ReadBulkNDJSON(f)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read registry: %v", err)
|
||||||
|
}
|
||||||
|
entries, stats := Build(sources)
|
||||||
|
|
||||||
|
rate := 0.0
|
||||||
|
if stats.Total > 0 {
|
||||||
|
rate = float64(stats.Parsed) / float64(stats.Total) * 100
|
||||||
|
}
|
||||||
|
t.Logf("total=%d parsed=%d unparsed=%d distinct_streets=%d parse_rate=%.2f%%",
|
||||||
|
stats.Total, stats.Parsed, stats.Unparsed, stats.DistinctStreets, rate)
|
||||||
|
if len(stats.UnparsedSample) > 0 {
|
||||||
|
t.Logf("unparsed sample: %v", stats.UnparsedSample)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A well-formed registry should parse almost perfectly. If this drops we
|
||||||
|
// want a loud failure, not a silently degraded index.
|
||||||
|
if stats.Total > 0 && rate < 98.0 {
|
||||||
|
t.Errorf("parse rate %.2f%% below 98%%", math.Round(rate*100)/100)
|
||||||
|
}
|
||||||
|
if len(entries) == 0 {
|
||||||
|
t.Error("no entries produced")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
// Package seed turns a registry dump into canonical address entries. It is the
|
||||||
|
// only place that knows how the source data is shaped.
|
||||||
|
//
|
||||||
|
// The same parser used at query time is used here, so the index and the query
|
||||||
|
// never disagree about how an address decomposes.
|
||||||
|
package seed
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.totmin.ru/en2zmax/addrkit/internal/domain/address"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Source is one raw registry row.
|
||||||
|
type Source struct {
|
||||||
|
ID string
|
||||||
|
Address string
|
||||||
|
District string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stats summarises a build and, crucially, surfaces rows the parser could not
|
||||||
|
// structure. A non-empty UnparsedCount is a data-quality signal worth alerting
|
||||||
|
// on, not an error.
|
||||||
|
type Stats struct {
|
||||||
|
Total int `json:"total"`
|
||||||
|
Parsed int `json:"parsed"`
|
||||||
|
Unparsed int `json:"unparsed"`
|
||||||
|
Duplicates int `json:"duplicates"`
|
||||||
|
DistinctStreets int `json:"distinct_streets"`
|
||||||
|
UnparsedSample []string `json:"unparsed_sample,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReadBulkNDJSON reads an Elasticsearch Bulk NDJSON dump: pairs of an action
|
||||||
|
// line ({"index":{"_id":...}}) followed by a source line
|
||||||
|
// ({"address":...,"district_name":...}). Bare source lines (no action line)
|
||||||
|
// are also accepted, which makes plain JSONL usable too.
|
||||||
|
func ReadBulkNDJSON(r io.Reader) ([]Source, error) {
|
||||||
|
sc := bufio.NewScanner(r)
|
||||||
|
// 1 MiB per line is far beyond any realistic address row.
|
||||||
|
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
||||||
|
|
||||||
|
var (
|
||||||
|
sources []Source
|
||||||
|
pendingID string
|
||||||
|
)
|
||||||
|
for sc.Scan() {
|
||||||
|
line := strings.TrimSpace(sc.Text())
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var obj map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal([]byte(line), &obj); err != nil {
|
||||||
|
continue // skip malformed lines rather than aborting the whole seed
|
||||||
|
}
|
||||||
|
|
||||||
|
if action, ok := obj["index"]; ok {
|
||||||
|
var a struct {
|
||||||
|
ID string `json:"_id"`
|
||||||
|
}
|
||||||
|
_ = json.Unmarshal(action, &a)
|
||||||
|
pendingID = a.ID
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if addrRaw, ok := obj["address"]; ok {
|
||||||
|
var addr string
|
||||||
|
if err := json.Unmarshal(addrRaw, &addr); err != nil || addr == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
src := Source{ID: pendingID, Address: addr}
|
||||||
|
if d, ok := obj["district_name"]; ok {
|
||||||
|
_ = json.Unmarshal(d, &src.District)
|
||||||
|
}
|
||||||
|
sources = append(sources, src)
|
||||||
|
pendingID = ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := sc.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return sources, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build parses every source row into a canonical Entry and reports statistics.
|
||||||
|
// Rows without a house number are still indexed for street lookups but counted
|
||||||
|
// as unparsed, since a registry row without a house is suspicious.
|
||||||
|
func Build(sources []Source) ([]address.Entry, Stats) {
|
||||||
|
entries := make([]address.Entry, 0, len(sources))
|
||||||
|
streets := make(map[string]struct{})
|
||||||
|
seen := make(map[string]struct{})
|
||||||
|
stats := Stats{Total: len(sources)}
|
||||||
|
|
||||||
|
for _, src := range sources {
|
||||||
|
p := address.Parse(src.Address)
|
||||||
|
if p.NameKey == "" {
|
||||||
|
stats.Unparsed++
|
||||||
|
appendSample(&stats, src.Address)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
entry := address.Entry{
|
||||||
|
ID: src.ID,
|
||||||
|
Address: src.Address,
|
||||||
|
District: src.District,
|
||||||
|
Ordinal: p.Ordinal,
|
||||||
|
NameKey: p.NameKey,
|
||||||
|
NameDisplay: p.NameDisplay,
|
||||||
|
StreetType: p.StreetType,
|
||||||
|
StreetKey: p.StreetKey,
|
||||||
|
House: p.House,
|
||||||
|
HouseFrac: p.HouseFrac,
|
||||||
|
HouseLetter: p.HouseLetter,
|
||||||
|
Building: p.Building,
|
||||||
|
BuildingFrac: p.BuildingFrac,
|
||||||
|
BuildingLetter: p.BuildingLetter,
|
||||||
|
Structure: p.Structure,
|
||||||
|
}
|
||||||
|
|
||||||
|
// The source registry contains rows that differ only by "стр."
|
||||||
|
// (structure). Since structure does not affect which house an outage
|
||||||
|
// belongs to, drop such duplicates rather than surfacing them as
|
||||||
|
// spurious ambiguity at query time.
|
||||||
|
if key := dedupKey(entry); key != "" {
|
||||||
|
if _, ok := seen[key]; ok {
|
||||||
|
stats.Duplicates++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[key] = struct{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
entries = append(entries, entry)
|
||||||
|
streets[p.StreetKey] = struct{}{}
|
||||||
|
if p.HasHouse {
|
||||||
|
stats.Parsed++
|
||||||
|
} else {
|
||||||
|
stats.Unparsed++
|
||||||
|
appendSample(&stats, src.Address)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
stats.DistinctStreets = len(streets)
|
||||||
|
sort.Strings(stats.UnparsedSample)
|
||||||
|
return entries, stats
|
||||||
|
}
|
||||||
|
|
||||||
|
// dedupKey identifies the same house unit within the same street, ignoring the
|
||||||
|
// structure component.
|
||||||
|
func dedupKey(e address.Entry) string {
|
||||||
|
if e.House == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return strings.Join([]string{
|
||||||
|
e.StreetKey, e.House, e.HouseFrac, e.HouseLetter,
|
||||||
|
e.Building, e.BuildingFrac, e.BuildingLetter,
|
||||||
|
}, "|")
|
||||||
|
}
|
||||||
|
|
||||||
|
func appendSample(s *Stats, address string) {
|
||||||
|
const maxSamples = 20
|
||||||
|
if len(s.UnparsedSample) < maxSamples {
|
||||||
|
s.UnparsedSample = append(s.UnparsedSample, address)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package seed
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReadBulkNDJSON(t *testing.T) {
|
||||||
|
input := `{"index": {"_index": "address_registry", "_id": "a1"}}
|
||||||
|
{"address": "Станционная улица 1/1", "district_name": "Ленинский"}
|
||||||
|
{"index": {"_id": "a2"}}
|
||||||
|
{"address": "Искитимская улица 5", "district_name": "Кировский"}
|
||||||
|
|
||||||
|
{"address": "Третья улица 7"}
|
||||||
|
`
|
||||||
|
src, err := ReadBulkNDJSON(strings.NewReader(input))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadBulkNDJSON: %v", err)
|
||||||
|
}
|
||||||
|
if len(src) != 3 {
|
||||||
|
t.Fatalf("got %d sources, want 3", len(src))
|
||||||
|
}
|
||||||
|
if src[0].ID != "a1" || src[0].District != "Ленинский" {
|
||||||
|
t.Errorf("first source = %+v", src[0])
|
||||||
|
}
|
||||||
|
if src[2].Address != "Третья улица 7" {
|
||||||
|
t.Errorf("bare source not read: %+v", src[2])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuild(t *testing.T) {
|
||||||
|
sources := []Source{
|
||||||
|
{ID: "1", Address: "Станционная улица 1/1"},
|
||||||
|
{ID: "2", Address: "Искитимская улица 5"},
|
||||||
|
{ID: "3", Address: "Станционная улица"}, // no house -> unparsed
|
||||||
|
}
|
||||||
|
entries, stats := Build(sources)
|
||||||
|
|
||||||
|
if stats.Total != 3 {
|
||||||
|
t.Errorf("Total = %d, want 3", stats.Total)
|
||||||
|
}
|
||||||
|
if stats.Parsed != 2 {
|
||||||
|
t.Errorf("Parsed = %d, want 2", stats.Parsed)
|
||||||
|
}
|
||||||
|
if stats.Unparsed != 1 {
|
||||||
|
t.Errorf("Unparsed = %d, want 1", stats.Unparsed)
|
||||||
|
}
|
||||||
|
if len(entries) != 3 {
|
||||||
|
t.Errorf("entries = %d, want 3", len(entries))
|
||||||
|
}
|
||||||
|
// Two distinct streets: "станционная" and "искитимская".
|
||||||
|
if stats.DistinctStreets != 2 {
|
||||||
|
t.Errorf("DistinctStreets = %d, want 2", stats.DistinctStreets)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
openapi: 3.1.0
|
||||||
|
info:
|
||||||
|
title: addrkit
|
||||||
|
version: "1.0.0"
|
||||||
|
description: >
|
||||||
|
Детерминированный сервис нормализации и поиска адресов. Ответ всегда
|
||||||
|
берётся из реестра; LLM не участвует в пути корректности.
|
||||||
|
servers:
|
||||||
|
- url: http://localhost:6171
|
||||||
|
paths:
|
||||||
|
/v1/resolve:
|
||||||
|
post:
|
||||||
|
summary: Разобрать и сопоставить адрес
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [address]
|
||||||
|
properties:
|
||||||
|
address:
|
||||||
|
type: string
|
||||||
|
example: "2-я Станционная улица 1/1 стр."
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Результат сопоставления
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: "#/components/schemas/ResolveResult"
|
||||||
|
"400":
|
||||||
|
description: Некорректный запрос
|
||||||
|
/v1/outages:
|
||||||
|
post:
|
||||||
|
summary: Отключения по адресу (после строгого сопоставления дома)
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [address]
|
||||||
|
properties:
|
||||||
|
address:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Канонический адрес и отключения
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: "#/components/schemas/OutagesResult"
|
||||||
|
/v1/healthz:
|
||||||
|
get:
|
||||||
|
summary: Живость
|
||||||
|
responses:
|
||||||
|
"200": { description: ok }
|
||||||
|
/v1/readyz:
|
||||||
|
get:
|
||||||
|
summary: Готовность (индекс загружен)
|
||||||
|
responses:
|
||||||
|
"200": { description: ready }
|
||||||
|
"503": { description: not ready }
|
||||||
|
/v1/version:
|
||||||
|
get:
|
||||||
|
summary: Версия данных
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: ok
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
data_version: { type: string }
|
||||||
|
components:
|
||||||
|
schemas:
|
||||||
|
Status:
|
||||||
|
type: string
|
||||||
|
enum: [found, partial, ambiguous, not_found, portal_timeout, infra_error]
|
||||||
|
Candidate:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
address: { type: string }
|
||||||
|
district: { type: string }
|
||||||
|
house: { type: string }
|
||||||
|
score: { type: number, format: double }
|
||||||
|
ResolveResult:
|
||||||
|
type: object
|
||||||
|
required: [status, confidence]
|
||||||
|
properties:
|
||||||
|
status: { $ref: "#/components/schemas/Status" }
|
||||||
|
address: { type: string }
|
||||||
|
district: { type: string }
|
||||||
|
confidence: { type: number, format: double }
|
||||||
|
candidates:
|
||||||
|
type: array
|
||||||
|
items: { $ref: "#/components/schemas/Candidate" }
|
||||||
|
data_version: { type: string }
|
||||||
|
Outage:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
type: { type: string }
|
||||||
|
system: { type: string }
|
||||||
|
address: { type: string }
|
||||||
|
start_date: { type: string }
|
||||||
|
end_date: { type: string }
|
||||||
|
action: { type: string }
|
||||||
|
OutagesResult:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
status: { $ref: "#/components/schemas/Status" }
|
||||||
|
address: { type: string }
|
||||||
|
district: { type: string }
|
||||||
|
outages:
|
||||||
|
type: array
|
||||||
|
items: { $ref: "#/components/schemas/Outage" }
|
||||||
|
as_of: { type: string }
|
||||||
|
data_version: { type: string }
|
||||||
Reference in New Issue
Block a user