Skip to main content

Implementation Plan: Intelligent Skill & Integration Discovery + MCP

ARCHIVE — This document is historical reference only. It may contain outdated information. See docs/status.md for current project state.

Feature: Skill Finding via Semantic Search + Model Context Protocol (MCP) Integration Date: February 15, 2026 Duration: 5-7 weeks (16 steps) Owner: Backend + LLM Team

Overview​

This document breaks the feature down into 16 implementable steps across 6 phases.

Phases:

  1. Foundation (Steps 1-3): Core services (catalog, discovery, selector)
  2. MCP Integration (Steps 4-6): MCPIntegration, schema extraction, tool routing
  3. LLM Integration (Steps 7-9): Orchestrator, system prompt, tool bridge
  4. Testing & Validation (Steps 10-13): Unit, integration, E2E tests
  5. Marketplace (Step 14): Auto-embed on install
  6. Documentation (Steps 15-16): Update docs + guides

Step 1: Create ToolCatalogService​

Effort: M (3-5 days)

Build the core service that manages tool embeddings, caching, and invalidation.

Files to Create​

  • backend/chat/tool_catalog.py

Key Classes​

  • ToolCatalogEntry — tool with embedding vector
  • ToolCatalogService — manages catalog (embed, search, invalidate)

Acceptance Criteria​

  • ✅ Service initializes with all tools from InterfaceManager
  • ✅ Vector search returns tools ordered by similarity
  • ✅ Text search works via embedding-based lookup
  • ✅ Cache invalidation rebuilds catalog
  • ✅ Supports both LanceDB (local) and pgvector (server)

Step 2: Implement ToolDiscoveryIntegration​

Effort: M (3-5 days)

New integration that lets LLM and users discover available tools.

Files to Create​

  • backend/interfaces/integrations/tool_discovery.py

Actions​

  • find(query) — search for matching tools
  • list_all(category?) — list all tools
  • describe(tool_name) — details for specific tool

Acceptance Criteria​

  • ✅ Integration registers in InterfaceManager
  • ✅ find() returns relevant tools
  • ✅ list_all() groups tools by category
  • ✅ describe() shows tool details

Step 3: Build SmartToolSelector​

Effort: M (3-5 days)

Algorithm that selects relevant tools per request, filters by ACL, ensures core tools always present.

Files to Create​

  • backend/chat/tool_selector.py

Key Classes​

  • SmartToolSelector — select K tools based on message + ACL
  • SelectedTools — result model with metadata

Core Tools Always Included​

memory__search, memory__store, memory__recall, memory__forget
tasks__list, tasks__create, tasks__update_status
frontend__show_card, frontend__show_form, frontend__show_choices
notifications__send
tool_discovery__find

Acceptance Criteria​

  • ✅ Returns 8-12 tools per request
  • ✅ Core tools always included
  • ✅ ACL filtering blocks unauthorized tools
  • ✅ Tools ordered by relevance
  • ✅ Fallback: returns core tools when no matches

Step 4: Implement MCPIntegration (NEW — MCP Support)​

Effort: M (3-5 days)

Manage and expose MCP (Model Context Protocol) servers, allowing external tools to be registered.

Files to Create​

  • backend/interfaces/integrations/mcp.py

Actions​

  • register_server(name, endpoint, api_key?) — register new MCP server
  • list_servers() — list registered servers
  • call_tool(server_name, tool_name, params) — execute tool on MCP server

Key Features​

  • Validates MCP server endpoint reachable
  • Fetches server schema (list of tools)
  • Converts MCP tool schema to ActionDefinition format
  • Registers tools with ToolCatalogService
  • Routes tool calls to correct MCP server

Acceptance Criteria​

  • ✅ MCPIntegration registers as interface
  • ✅ register_server() validates endpoint, fetches schema
  • ✅ MCP tools converted to ActionDefinition correctly
  • ✅ New tools registered with ToolCatalogService
  • ✅ call_tool() routes to correct server
  • ✅ list_servers() returns server details

Step 5: Extract & Embed MCP Tool Schemas​

Effort: M (3-5 days)

When MCP servers are registered, extract their tool definitions and embed them.

What Happens​

  1. Schema Extraction: Fetch MCP server schema (list of tools)
  2. Tool Conversion: Convert each MCP tool to ActionDefinition format
  3. Embedding & Registration: Each tool embedded and added to catalog
    • Tools immediately appear in tool_discovery + SmartToolSelector

Files to Modify​

  • backend/interfaces/integrations/mcp.py (schema extraction logic)
  • backend/chat/tool_catalog.py (handle dynamic registration from MCP)

Acceptance Criteria​

  • ✅ MCP tool schemas extracted correctly
  • ✅ Tools converted to ActionDefinition with proper types
  • ✅ Each tool embedded (vector stored in LanceDB + pgvector)
  • ✅ Tools appear in tool_discovery results immediately
  • ✅ Handles schema parsing errors gracefully

Step 6: Route MCP Tool Execution​

Effort: S (2-3 days)

When LLM calls an MCP tool, route it to the correct MCP server and return result.

Execution Flow​

LLM: → mcp_web_research__search_web(query="...")
    ↓
Orchestrator recognizes "mcp_*" tool name
    ↓
MCPIntegration._call_tool():
    1. Look up MCP server config
    2. Get API key from VaultProvider
    3. Format params per server schema
    4. Call MCP server endpoint
    5. Return result to LLM

Files to Modify​

  • backend/chat/orchestrator.py (add MCP tool routing)
  • backend/interfaces/integrations/mcp.py (_call_tool implementation)

Acceptance Criteria​

  • ✅ MCP tool calls routed correctly
  • ✅ API keys retrieved from vault securely
  • ✅ Result formatting matches LLM expectations
  • ✅ Errors handled gracefully
  • ✅ ACL filtering applies to MCP tools

Step 7: Update Orchestrator​

Effort: M (3-5 days)

Integrate SmartToolSelector into agent loop.

Changes in orchestrator.py​

# Before: tools = actions_to_anthropic_tools(interface_manager)
# After:
selected = await tool_selector.select(user_message, user_id, group_id, space_id)
tools = actions_to_anthropic_tools(
    interface_manager,
    tools_to_include=[t.full_name for t in selected.selected]
)

Changes in tools.py​

  • Add tools_to_include parameter to actions_to_anthropic_tools()
  • Filter tools to only selected ones

Acceptance Criteria​

  • ✅ Only selected tools passed to LLM
  • ✅ Tool execution works
  • ✅ Approval workflow still functions
  • ✅ Tool results returned correctly

Step 8: Revise System Prompt Builder​

Effort: S (1-2 days)

Update system prompt for transparency and reduced size.

Changes to prompts.py​

  • New section: "Tools Available for This Request" (list selected tools)
  • New section: "Discover More Tools" (explain tool_discovery escape hatch)
  • Tool usage guidelines only for selected tools

Result​

  • System prompt reduced from 3-5 KB → 1-1.5 KB (60-70% savings)
  • Transparent: LLM knows tools are filtered
  • Discovery guided: LLM knows it can ask for more

Acceptance Criteria​

  • ✅ System prompt shows selected tools clearly
  • ✅ Size is 60-70% smaller (measure: token count)
  • ✅ Transparency message included
  • ✅ Discovery guidance present

Step 9: Update Tool Bridge​

Effort: S (1-2 days)

Ensure tool bridge handles variable-length tool lists.

Acceptance Criteria​

  • ✅ Filtering doesn't break tool execution
  • ✅ Tool descriptions accurate for selected tools
  • ✅ Dynamic tool lists work correctly

Step 10: Unit Tests — ToolCatalogService​

Effort: M (3 days)

Tests​

  • test_initialize()
  • test_search_vector()
  • test_search_text()
  • test_add_tool()
  • test_remove_tool()
  • test_cache_invalidation()

Step 11: Unit Tests — SmartToolSelector + MCP​

Effort: M (3 days)

Tests​

  • test_select_basic()
  • test_core_tools_included()
  • test_mcp_tools_included()
  • test_acl_filtering()
  • test_fallback()
  • test_mcp_tool_registration()

Step 12: Integration Tests​

Effort: M (3 days)

Tests​

  • test_full_chat_flow_with_tool_selection()
  • test_llm_calls_tool_discovery()
  • test_mcp_tool_execution()
  • test_mcp_server_registration()
  • test_tool_execution_after_selection()

Step 13: E2E Validation​

Effort: M (3 days)

Tests​

  • test_token_savings_60_percent()
  • test_embedding_latency()
  • test_tool_selection_accuracy()
  • test_mcp_end_to_end()

Acceptance Criteria (all steps 10-13)​

  • ✅ All tests pass
  • ✅ System prompt reduced 60-70%
  • ✅ Tool selection accuracy >85%
  • ✅ MCP tools work end-to-end
  • ✅ Embedding latency <200ms
  • ✅ No regressions in existing tests

Step 14: Marketplace Integration​

Effort: M (3 days)

When users install skills/integrations/MCP servers, automatically embed them.

Changes​

Update install services to register new tools in catalog.

Acceptance Criteria​

  • ✅ Newly installed skills appear in tool discovery
  • ✅ Newly installed MCP servers appear in discovery
  • ✅ Embedding is asynchronous (doesn't block install)
  • ✅ Tool catalog updated immediately
  • ✅ SmartToolSelector can select new tools

Step 15: Update Documentation​

Effort: S (1-2 days)

Files to Update​

  1. docs/interfaces.md — add MCP section + tool discovery
  2. docs/architecture.md — add MCPIntegration + ToolCatalogService components
  3. docs/api.md — document tool_discovery + mcp actions
  4. Docstrings in all new files

Acceptance Criteria​

  • ✅ All components documented
  • ✅ MCP architecture explained
  • ✅ Examples provided

Step 16: Feature Documentation​

Effort: S (1 day)

Update/create user-friendly guides.

Files​

  • Update QUICK_REFERENCE_Tool_Discovery.md to include MCP
  • Create docs/features/MCP_Integration_Guide.md for developers

Acceptance Criteria​

  • ✅ User guide is clear
  • ✅ Developer guide explains MCP registration flow
  • ✅ Examples provided

Timeline Summary​

WeekStepsFocus
1-21-3Build core services (catalog, discovery, selector)
24-6MCP Integration (register, extract, execute)
2-37-9LLM Integration (orchestrator, prompt, bridge)
3-410-13Testing + validation
414Marketplace integration
4+15-16Documentation

Total Effort: 5-7 weeks Team: Backend lead (1-9, 15), MCP specialist (4-6), QA (10-13), Marketplace lead (14), Tech writer (15-16)

Last Updated: February 15, 2026 Status: Ready to Implement