The Knowledge Graph Architecture: How AiFiler Connects Data Without Bottlenecks

You've got 50,000 documents. A contract mentions a client. That client appears in emails, proposals, and meeting notes. A team member wrote an analysis that references three of those documents. Someone else tagged half of them with "Q4-2024." Now you need to find everything related to that client—not just by name, but by relationship.

This is where most document tools fail. They treat documents as isolated objects. Search returns a flat list. You manually connect the dots.

AiFiler's knowledge graph does the opposite. It stores documents as nodes in a graph, with relationships (edges) that capture how they connect. When you search, the graph doesn't just find documents—it traverses relationships to surface context you didn't explicitly ask for.

Here's how we built it, why it matters, and what it means for how you work.

The Problem We Solved

Traditional document management uses inverted indices. You search for "client-name," and the system returns every document containing those words. Fast, but dumb. It doesn't know that the client appears in a contract, an email thread, and a folder—or that those are related.

Relational databases (SQL) can model relationships, but they're designed for transactional consistency, not graph traversal. Querying "show me everything connected to this client" requires multiple joins and gets slower as your graph grows.

We needed something different: a system that could store complex relationships, traverse them quickly, and scale without bottlenecks.

The Architecture: Nodes, Edges, and Real-Time Indexing

AiFiler's knowledge graph has three layers:

Layer 1: Node Storage (Supabase PostgreSQL) Every document is a node. So is every tag, folder, client, person, and topic extracted from your documents. Nodes have properties: document type, creation date, owner, extracted entities, etc. We use Supabase's PostgreSQL for this because we need ACID compliance and complex queries, and PostgreSQL's JSONB columns let us store flexible metadata without schema migrations.

Layer 2: Edge Types (8 Relationship Patterns) Nodes connect via edges. We defined eight core edge types:

• CONTAINS: A folder contains a document
• MENTIONS: A document mentions an entity (person, client, topic)
• REFERENCES: A document cites another document
• AUTHORED_BY: A document was created by a person
• TAGGED_WITH: A document has a tag
• RELATED_TO: A document is semantically similar to another (computed by Claude)
• BELONGS_TO: A document belongs to a project or workspace
• DERIVED_FROM: A document was created from another (e.g., a proposal from a template)

Each edge is directional and can have properties (confidence score, timestamp, source of the relationship).

Layer 3: Real-Time Indexing When you upload a document, AiFiler:

1. Parses the file (via lib/ingest/parseFile.ts): Extracts text, metadata, and structure
2. Sends to Claude (via lib/ai/client.ts): Uses Claude's API to identify entities, extract relationships, and compute semantic similarity
3. Creates nodes and edges (via lib/intelligence/actionExecutor.ts): Writes to PostgreSQL with immediate indexing
4. Updates the graph cache (via SWR with localStorage prefixing): Makes data available to the UI in milliseconds

The entire pipeline is asynchronous. You can search immediately; Claude's analysis runs in the background and updates results as it completes.

How Queries Work: Graph Traversal Without N+1

When you use Universal Command (Ctrl+Shift+A) to search, AiFiler doesn't just match keywords. It:

1. Finds seed nodes matching your query (documents, tags, people)
2. Traverses edges up to a configurable depth (default: 3 hops)
3. Ranks results by relevance, relationship strength, and recency
4. Returns context: Not just matching documents, but why they matched (which relationship connected them)

Example: You search "Q4 budget for Acme Corp."

• AiFiler finds documents tagged "Q4" and "budget"
• It finds documents mentioning "Acme Corp"
• It traverses MENTIONS edges to find documents written by people who work on Acme
• It traverses REFERENCES edges to find documents cited by those documents
• It returns a ranked list with context: "This proposal mentions Acme (direct match), this budget was created by someone on the Acme team (1 hop), this analysis references the budget (2 hops)"

The query engine uses PostgreSQL's recursive CTEs (common table expressions) to traverse the graph. This is much faster than application-level traversal because the database does the heavy lifting.

WITH RECURSIVE graph_traversal AS (
  SELECT node_id, edge_type, 1 as depth
  FROM edges
  WHERE source_node_id = $1
  
  UNION ALL
  
  SELECT e.node_id, e.edge_type, gt.depth + 1
  FROM edges e
  JOIN graph_traversal gt ON e.source_node_id = gt.node_id
  WHERE gt.depth < $2
)
SELECT DISTINCT node_id FROM graph_traversal
ORDER BY depth, relevance_score DESC;

This query starts from a single node and expands outward, stopping at a depth limit. PostgreSQL evaluates it in a single pass, not N+1 queries.

Why This Matters for You

Faster discovery: You don't need to remember exact filenames or keywords. Search by relationship: "show me everything this client is involved in" returns documents you forgot existed.

Reduced manual work: No more manually tagging related documents or building folders by hand. The graph finds connections automatically.

Context in search results: AiFiler tells you why a document matched, not just that it did. This saves time when you're evaluating results.

Scales with your data: As your document count grows, graph traversal gets smarter (more connections to explore) but doesn't get slower. PostgreSQL's query planner optimizes recursive CTEs based on statistics, so performance stays consistent.

Enables batch operations: When you batch-move 500 documents to a new folder, the graph updates all relationship indices in a single transaction. No orphaned edges, no inconsistent state.

The Tradeoffs We Made

We could have used a dedicated graph database like Neo4j. It would give us faster traversal for very deep queries (10+ hops). But it would add operational complexity (another service to run, sync, and monitor) and cost.

Instead, we optimized PostgreSQL for graph queries. We index edges aggressively. We precompute common traversals (like "all documents related to this client") and cache them. For the 99th percentile of queries (deep, complex traversals), we accept slightly higher latency in exchange for simplicity and reliability.

This is a deliberate choice: PostgreSQL's maturity and our team's expertise with it outweigh the marginal performance gain from a specialized graph database.

What's Next

We're working on two improvements:

Semantic clustering: Instead of just storing explicit relationships, we're using Claude to group semantically similar documents (even if they don't mention each other). This will surface relevant documents that have no direct connection.

Relationship confidence scores: Not all edges are equal. A MENTIONS edge based on explicit text is more reliable than one based on semantic similarity. We're adding confidence scores so queries can weight results accordingly.

The knowledge graph is the foundation of everything AiFiler does. Every search, every batch operation, every AI-assisted workflow depends on it. Understanding how it works helps you use AiFiler more effectively—and it shows why we built it this way instead of taking shortcuts.