The Knowledge Graph Architecture: How AiFiler Connects Data Without Bottlenecks

You've got 500 documents across three client projects. One is a contract. Another references that contract. A third contains financial data tied to both. Most document tools treat these as isolated files. AiFiler treats them as a connected system.

That's the knowledge graph. And the architecture behind it is what separates a document storage tool from a document intelligence tool.

The Problem We Solved

Traditional document management systems store files in silos. You search by filename, tags, or metadata. When you need relationships—"show me all documents connected to this contract"—you're either doing manual linking or running expensive queries across your entire database.

We needed something different: a way to automatically extract relationships between documents, store them efficiently, and query them without the latency that kills user experience.

The challenge wasn't just technical. It was architectural. We needed to:

1. Extract relationships automatically using AI without blocking document ingestion
2. Store 8 different edge types (document-to-document, document-to-entity, workspace-to-document, etc.) in a queryable format
3. Query across millions of relationships in under 100ms
4. Keep the system resilient when AI extraction fails or returns incomplete data

The Architecture: Four Layers

User Query (Universal Command or Search)
         ↓
Query Router (lib/intelligence/universalRouter.ts)
         ↓
Knowledge Hooks (lib/knowledge/) + Matrix Hooks (lib/tables/hooks/)
         ↓
Supabase (Browser + Admin clients)
         ↓
PostgreSQL + Vector Store (Relationships + Embeddings)

Let's walk through each layer.

Layer 1: The Query Router

When you type a question into Universal Command (Ctrl+Shift+A), it doesn't go straight to the database. It goes through universalRouter.ts, which decides: Is this a search? A filter? A relationship query? An action?

The router examines the intent using heuristics in lib/intentHeuristics.ts. If you ask "show me all documents linked to the Q3 budget," the router recognizes this as a relationship traversal query, not a simple text search.

This matters because relationship queries need different data than keyword searches. They need edge metadata, traversal depth, and filtered node types.

Layer 2: Knowledge Hooks and State Management

Once the router identifies a relationship query, it delegates to lib/knowledge/, which manages the knowledge graph state.

The key hook is useKnowledgeGraph(), which maintains:

• Nodes: Documents, entities, workspaces, people
• Edges: The 8 relationship types connecting them
• Metadata: Creation date, confidence score, source (AI-extracted vs. user-created)

We use SWR with localStorage prefixing for offline-first data fetching. This means:

• The graph is cached locally
• Queries against cached data return instantly
• Background sync updates the cache without blocking the UI
• If the network is slow, you still see results

// Simplified pattern from lib/knowledge/
const useKnowledgeGraph = () => {
  const { data, isLoading, mutate } = useSWR(
    `/api/knowledge/graph?workspaceId=${workspaceId}`,
    fetcher,
    {
      dedupingInterval: 60000, // 1 minute
      focusThrottleInterval: 30000,
      revalidateOnFocus: false,
    }
  );
  
  return { nodes: data?.nodes, edges: data?.edges, isLoading, mutate };
};

Layer 3: The Database Schema

Here's where the architecture gets interesting. We don't store relationships as a separate table. Instead, we embed them in the document metadata and use Supabase's full-text search + vector capabilities.

Each document has:

• Document ID: Unique identifier
• Workspace ID: Scoping for multi-tenant queries
• Embeddings: Vector representation of content
• Relationships: JSON array of edges
• Extracted Entities: Named entities (people, companies, dates) extracted during ingestion

The relationships JSON looks like:

{
  "edges": [
    {
      "type": "references",
      "targetId": "doc_456",
      "confidence": 0.92,
      "extractedAt": "2025-04-15T10:23:00Z",
      "source": "ai"
    },
    {
      "type": "mentions_entity",
      "targetId": "entity_acme_corp",
      "confidence": 0.87,
      "extractedAt": "2025-04-15T10:23:00Z",
      "source": "ai"
    }
  ]
}

This design has three benefits:

1. No join overhead: All relationship data lives with the document, so a single row fetch gives you everything
2. Flexible edge types: Adding a new relationship type doesn't require schema migration
3. Confidence scoring: AI-extracted relationships include confidence scores, so you can filter by reliability

Layer 4: The Query Execution

When you search for "documents linked to the Q3 budget," here's what happens:

1. Find the root document: Query for "Q3 budget" using full-text search
2. Traverse the edges: Extract the edges array from that document
3. Fetch connected documents: Query for documents where id IN (targetIds from edges)
4. Filter by edge type: If you specified "references only," filter the results
5. Return with metadata: Include relationship confidence, extraction date, and source

The entire operation uses indexed queries on workspace_id and document_id, so even with 100k documents, it completes in 40–80ms.

Why We Didn't Use a Graph Database

You might expect us to use Neo4j or another dedicated graph database. We considered it. Here's why we didn't:

Cost and operational complexity: Graph databases excel at deep traversals (finding paths 5+ hops away). Most document relationships are 1–2 hops. The overhead of running a separate database didn't justify the benefit.

Embedding relationships in document metadata gave us 95% of the benefit with 10% of the operational burden.

Multi-tenancy: Supabase's row-level security (RLS) policies work seamlessly with our schema. Isolating data by workspace is built into the query layer, not an afterthought.

Vector search: We needed semantic search alongside relationship search. Supabase's pgvector integration meant we could do both in one query.

The Intelligence Layer: Automatic Relationship Extraction

This is where the AI happens. When you upload a document, lib/ingest/parseFile.ts extracts text and sends it to Claude via the Anthropic Files API (lib/ai/fileStore.ts).

Claude extracts:

• Document references: "This contract references the SLA from Q3_SLA.docx"
• Entity mentions: People, companies, dates, amounts
• Implicit relationships: "The budget assumes 30% growth" → linked to growth forecasts
• Document type: Contract, memo, analysis, etc.

The extraction runs asynchronously. The document is searchable immediately; relationships populate as extraction completes.

If extraction fails (malformed PDF, OCR error), the document is still usable—you just won't have AI-extracted relationships until you retry.

Lessons Learned

Confidence scores matter more than perfection. We initially tried to extract only high-confidence relationships. This meant missing 40% of useful connections. Now we extract everything with confidence scores. Users can filter by confidence threshold.

Offline-first design prevents cascading failures. When the knowledge graph API is slow, the UI doesn't freeze. Cached data renders instantly. Users see stale relationships rather than spinners.

Edge metadata is as important as the edges themselves. Knowing why two documents are connected (AI-extracted vs. user-created, confidence score, extraction date) helps users trust the graph.

Why This Matters for You

If you're managing 100+ documents across multiple projects, the knowledge graph saves you hours every week. Instead of manually linking related files, AiFiler finds them automatically.

Use Universal Command (Ctrl+Shift+A) to ask "show me all documents linked to this contract" or "find all mentions of Acme Corp." The query executes in under 100ms because the architecture is built for speed, not just capability.

The knowledge graph also powers Batch Operations. When you select 50 documents and ask "move these to the Q4 folder," AiFiler can show you related documents you might have missed—all because it understands the relationships between your files.

The architecture is invisible until you need it. But when you're managing complexity—multiple clients, overlapping projects, regulatory requirements—it's the difference between chaos and clarity.