🐛 Bug: Incorrect Governance Model Matching Causes Registration Failures #438

Closed @Asplana92 opened 2025-11-15 23:53 UTC 2 comments Updated 2026-04-28 20:48 UTC

🐛 Bug Report: Incorrect Governance Model Matching in API

Severity: Medium
Category: Bug Discovery + Improvement Proposal
Bounty Program: Yes (Discord Announcement)
Reporter: @Asplana92
Date Discovered: November 15, 2025
Block Height: ~1,304,000 - 1,305,300

🎯 TL;DR / Summary

The API uses prefix-matching instead of exact-matching when searching for governance models, causing name collisions and blocking hardware registration.

Impact:
- ❌ Prevents hardware registration when model names overlap
- ⏱️ ~2 hours debugging time per affected operator
- 🔄 100% reproducible
- 📊 Affects any host using models with similar prefixes

🔍 Problem Description

Current Behavior (Incorrect)

When registering hardware, the API searches for governance models using substring/prefix matching instead of exact model ID comparison.

Example collision:

User's node-config.json:
{
  "models": {
    "Qwen/Qwen2.5-7B-Instruct": {"args": []}
  }
}

API logic:
1. Searches for "Qwen/Qwen2.5-7B-Instruct" in governance models
2. Finds "RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16" 
   (contains substring "Qwen2.5-7B-Instruct")
3. Uses WRONG model ID: "RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16"
4. Tries to register hardware with this model
5. ❌ ERROR: "Failed to get governance models: model not found"

Expected Behavior

The API should use exact model ID matching:

// CORRECT approach
function findGovernanceModel(modelId, governanceModels) {
  const exactMatch = governanceModels.find(m => m.id === modelId)

  if (!exactMatch) {
    const available = governanceModels.map(m => m.id).join('\n  - ')
    throw new Error(
      `Model '${modelId}' not found in governance.\n` +
      `Available models:\n  - ${available}`
    )
  }

  return exactMatch
}

📜 Error Logs

2025/11/15 16:21:34 INFO RegisterNode. Governance model 
  model_id=RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16

2025/11/15 16:22:50 ERROR Failed to get governance models: 
  model not found for RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16

What happened: 1. ✅ User configured Qwen/Qwen2.5-7B-Instruct 2. ❌ API found RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16 (prefix match) 3. ❌ Tried to register with wrong model ID 4. ❌ Registration failed

🔁 Reproduction Steps

Prerequisites

Fresh Gonka node setup
Governance models containing similar names:
Qwen/Qwen2.5-7B-Instruct
RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16

Steps to Reproduce

Configure node with specific model:

cat > node-config.json << 'EOF'
[{
  "inference": {
    "type": "inference",
    "host": "inference",
    "port": "8085",
    "models": {
      "Qwen/Qwen2.5-7B-Instruct": {
        "args": []
      }
    }
  }
}]
EOF

Start API:
```
docker compose up api -d
```

Observe error:

docker logs api | grep "governance model"
# ERROR: model not found for RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16

Expected: API should register Qwen/Qwen2.5-7B-Instruct
Actual: API tries to register RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16

💥 Impact Assessment

Who is Affected

✅ Any host using models with similar name prefixes
✅ New operators setting up nodes for the first time
✅ Production deployments with specific model requirements

Common Collision Examples

Base Model              Collides With
─────────────────────   ──────────────────────────────────────────
Qwen/Qwen2.5-7B        → RedHatAI/Qwen2.5-7B-Instruct-quantized
Llama-3.1-8B           → Llama-3.1-8B-Instruct-Turbo
Mistral-7B             → Mistral-7B-Instruct-v0.3

Business Impact

📉 Prevents new hosts from joining (reduces network decentralization)
⏱️ 2-4 hours debugging time per affected operator
🔄 Poor onboarding experience for new participants
❌ Blocks hardware registration until workaround is found

Severity Justification

Medium Severity because:
✅ Workaround exists (use exact model ID)
❌ Not documented anywhere
❌ Blocks critical functionality (hardware registration)
✅ 100% reproducible

💡 Proposed Solution

Option 1: Exact Matching (Recommended) ⭐

/**
 * Find governance model by exact ID match
 * @param {string} modelId - Model ID from node-config.json
 * @param {Array} governanceModels - Models from blockchain
 * @returns {Object} Matched governance model
 * @throws {Error} If model not found or multiple matches
 */
function findGovernanceModel(modelId, governanceModels) {
  // EXACT match only
  const exactMatch = governanceModels.find(m => m.id === modelId)

  if (exactMatch) {
    return exactMatch
  }

  // Helpful error message with available models
  const available = governanceModels
    .map(m => `  - ${m.id}`)
    .join('\n')

  throw new Error(
    `Governance model '${modelId}' not found.\n\n` +
    `Available models:\n${available}\n\n` +
    `Please use exact model ID from the list above.`
  )
}

Benefits: - ✅ Prevents name collisions - ✅ Clear error messages - ✅ Lists available models for easy copy-paste - ✅ Backward compatible (exact matches still work)

Option 2: Ambiguity Detection

/**
 * Find governance model with collision detection
 */
function findGovernanceModelSafe(modelId, governanceModels) {
  // Check for exact match first
  const exactMatch = governanceModels.find(m => m.id === modelId)
  if (exactMatch) {
    return exactMatch
  }

  // Check for prefix collisions
  const prefixMatches = governanceModels.filter(m => 
    m.id.includes(modelId) || modelId.includes(m.id)
  )

  if (prefixMatches.length > 1) {
    throw new Error(
      `Ambiguous model ID '${modelId}'. Multiple matches found:\n` +
      prefixMatches.map(m => `  - ${m.id}`).join('\n') +
      `\n\nPlease use exact model ID.`
    )
  }

  if (prefixMatches.length === 1) {
    console.warn(
      `WARNING: Using prefix match for '${modelId}' → '${prefixMatches[0].id}'. ` +
      `Consider using exact model ID.`
    )
    return prefixMatches[0]
  }

  throw new Error(`Model '${modelId}' not found in governance.`)
}

Benefits: - ✅ Detects collisions explicitly - ✅ Backward compatible with fuzzy matching - ⚠️ More complex logic

✅ Workaround (Current)

Until fixed, operators can work around this issue:

Query available governance models:
```
inferenced query inference models-all
```

Use EXACT model ID in config:

{
  "models": {
    "Qwen/Qwen2.5-7B-Instruct": {"args": []}
  }
}

Verify no similar names exist:

inferenced query inference models-all | grep "Qwen"

🧪 Testing Environment

Setup: - Server: Hetzner Cloud, Ubuntu 22.04 LTS - Docker: 27.3.1 - Network: gonka-mainnet - Block Height: ~1,304,000 - 1,305,300 - API Version: Latest (from docker-compose.yml)

Governance Models Present:

- Qwen/Qwen2.5-7B-Instruct
- RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a16
- (others)

💰 Bounty Program Alignment

Per Gonka Bounty Program Discord Announcement:

Vulnerability Bounty Program:
- Describing an unknown vulnerability: 1,000-5,000 gonka coins
- Proposal describing additional improvement: 1,000-5,000 gonka coins

This Submission Includes:

✅ Bug discovery and description
✅ Root cause analysis (prefix vs exact matching)
✅ Proposed solution with code examples
✅ Reproduction steps (100% reproducible)
✅ Impact assessment (affects all similar model names)
✅ Workaround documentation

Optional PR

Happy to implement Option 1 (Exact Matching) if the team approves this approach! 🚀

📊 Discovery Timeline

Time	Event
Nov 15, 16:26	Started hardware registration
Nov 15, 16:45	Encountered error (wrong model ID)
Nov 15, 17:20	Identified root cause (prefix matching)
Nov 15, 17:40	Implemented workaround (exact model ID)
Nov 15, 18:00	Hardware successfully registered ✅

Total debugging time: ~2 hours
Workaround difficulty: Medium (requires blockchain query knowledge)

🤝 Additional Notes

Anonymous submission: No
Reporter: Asplana92 (Discord: @tolik_iarik)
Contact: Available on Discord for questions
Willing to implement fix: Yes ✅
Testing availability: Can test patched builds

None found (first report of this issue)

🙏 Acknowledgments

Thank you to the Gonka team for: - Creating the Bounty Program - Maintaining responsive Discord support - Building an open-source decentralized AI network

Looking forward to contributing to improved operator experience! 🚀

Submitted by: @Asplana92
Date: November 15, 2025
Bounty Category: Bug Discovery + Improvement Proposal

💬 Comments (2)

@AlexeySamosadov commented 2026-02-08 14:14 UTC

PR created: https://github.com/gonka-ai/gonka/pull/680

Improves error messages for invalid governance models.

@0xgonka commented 2026-04-28 20:48 UTC

already fixed

🔄 Auto-synced from Issue #438 every hour.