🔄 Auto-sync: from Discussion #1582 every hour.
Proposal #82: External Test Lab & Community DevNet — M1 Report
Автор: @paranjko · Категория: 
Governance Proposal Reports · Создано: 2026-08-11 23:12 UTC · Обновлено: 2026-08-11 23:12 UTC
📝 Описание
Period: 2026-07-10 — 2026-08-09 · Proposal: #82 (discussion) · Next unlock: 2026-08-13
Workstream A — DevNet infrastructure (M1: Setup)
| M1 requirement | Status |
|---|---|
| DevNet design agreed | Done — architecture note |
| ≥5 nodes online | Done — node0–node4 deployed, one provider each, chain gonka-devnet-community; live state on the status site |
| Infrastructure blockers documented | Done — see below |
Delivered beyond the M1 bar:
- Deployment runbook
1.0.0-alpha.1(net-deployment-runbook/) — reproducible baseline ontestnet-0.2.14(phased CLI: prepare → identities → qualify-ml → genesis → join → verify), governed0.2.14 → 0.2.15upgrade rehearsal, single-node stop/start/verify/reset, network reset contract. The M2 artifact "runbook sufficient to reproduce a node" (ROLE-JOIN.md) is already published. - Operator handoff flow — an external operator can qualify, deploy, and register a node without receiving coordinator secrets; the coordinator confirms on-chain registration and grants funding/ML permission.
- Public observability — status site (per-node cards from committed chain state, honest OFFLINE on reset) and Grafana dashboards: 24h network view, 7-day inference view, triage overview.
- Authenticated access — api.gonka-dev.net/v1: OpenAI-compatible, chain-accounted DevShard v4 gateway (v3/v4 approved via on-chain governance), automatic escrow rotation across epochs, API keys self-issued through a Telegram bot from a finite pool, no artificial per-key quotas.
All images and binaries are pinned by digest; DevNet keys are fully isolated from mainnet.
Blockers encountered (documented and resolved)
- Blackwell GPU support. The RTX Pro 2000 ML host (compute capability 12.0) is not supported by the MLNode images pinned to the
0.2.14baseline. Resolved with a pinned hardware-runtime exception: a newer upstream MLNode runtime for that host only, while the chain release and model overlay stay on the baseline (documented in the runbook release profile). - PoC validation timing on DAPI 0.2.14. The stock gateway could submit its first MLNode distribution in the same block as a newer store commit, missing the validation snapshot. Resolved with a ten-block PoC validation delay as an explicit test-lab genesis parameter (documented in the runbook).
- Provider port limitations (Novita.ai). Evaluated as pay-as-you-go GPU capacity: usable for ML hosts, but does not expose the ports a full network node requires. Kept as reserve ML capacity (remaining balance to be attached to a network node's ML host later).
No hard procurement blockers were encountered during Month 1. One recurring friction: some datacenters and GPU neoclouds require KYC before activating servers, which adds delay to provisioning and slows down multi-provider deployment.
Workstream B — Testing lab (M1: Setup and QA onboarding)
| M1 requirement | Status |
|---|---|
| Initial test strategy | Done — testing/test-strategy.md |
| QA search and selection | Done — first engineer hired and active |
| Operating model | Done — task board, issue templates (defect / validation request / DevNet access), SECURITY.md private disclosure path |
Hiring. 60+ CVs screened, 8 candidates interviewed, one QA / infrastructure engineer hired, started on July 28. As part of onboarding he took on the DevNet deployment itself: the runbook, observability, and gateway work above; onboarding continues in Month 2. The search for the second engineer restarts in Month 2, likely with paid listing promotion; one candidate is in the pipeline and receives a test assignment once the DevNet stabilizes.
Incidents
The network spent most of the period in initial deployment and stabilization; instability during bootstrap was expected and was resolved as part of deployment work rather than tracked as incidents. Public monitoring is live (status site with per-node state and validator map, dashboards).
Spending (budget lines per proposal §14)
| Line | Monthly cap | Spent (Month 1) | Notes |
|---|---|---|---|
| DevNet machines | $5,000 | ≈ $1,550 ($1,065 + €420) | 5 nodes + dedicated ML host + domain, itemized below |
| Burst GPU rental | $6,500 | $100 | Novita.ai pay-as-you-go GPU credit (evaluated for ML hosts; balance reserved) |
| Tooling and reporting | $250 | $0 | |
| External Testing Engineers | $8,000 | $2,000 | first QA engineer salary (paid by Aug 11) |
| Contingency | $2,250 | $0 | |
| Total | $22,000 | ≈ $3,650 |
Infrastructure breakdown (paid invoices; receipts available for review on request):
| Date | Provider | Configuration | Role | Billing period | Amount |
|---|---|---|---|---|---|
| 07-19 | DatabaseMart | GPU VPS, RTX Pro 2000 16 GB | node4 ML host | 1 month | $119.00 |
| 07-19 | OneProvider | dedicated CPU (EPYC/64 GB), Portland US | node4 network node + public edge | 1 month | $77.50 |
| 07-24 | DatabaseMart | dedicated, RTX A5000 24 GB | node0 (genesis) | 1 month | $254.77 |
| 07-27 | OneProvider | dedicated + Tesla T4 16 GB, Helsinki FI | node1 | 1 month | $331.63 |
| 07-27 | Novita.ai | pay-as-you-go GPU credit | burst ML capacity (evaluated; balance reserved) | — | $100.00 |
| 07-28 | GIGAGPU | dedicated, RTX 3090 24 GB, UK | node3 | 1 month | $270.03 |
| 07-29 | NameSilo | gonka-dev.net registration |
domain | 1 year | $11.85 |
| 07-29 + 08-05 | LeaderGPU (LeaderTelecom) | dedicated, RTX 4090 24 GB, NL | node2 | 5 weeks | €420.00 |
Month 2 plan (M2 targets)
- Scale to 9+ MLNodes across target regions; publish the regional layout summary (#3, #4).
- Publish smoke and regression checklists (#5).
- Run the second QA engineer hiring round; we aim to close the hire, prioritizing fit over speed (#6).
- Begin validation work as testable artifacts are handed off (validation requests).