Documentation

YAML config

The recommended way to define your schema. Write a lattice.config.yml file and Lattice creates your tables, wires up rendering, and can generate TypeScript types for you.since v0.4

# lattice.config.yml
db: ./data/app.db

entities:
  user:
    fields:
      id: { type: uuid, primaryKey: true }
      name: { type: text, required: true }
      email: { type: text }
      role: { type: text, default: member }
    render: default-table
    outputFile: context/USERS.md

  task:
    fields:
      id: { type: uuid, primaryKey: true }
      title: { type: text, required: true }
      status: { type: text, default: open }
      priority: { type: integer, default: 1 }
      assignee_id: { type: uuid, ref: user }
    render:
      template: default-list
      formatRow: '{{title}} ({{status}}) — {{assignee.name}}'
    outputFile: context/TASKS.md

Type mappings

Composite primary keys

For junction tables or any table with a multi-column primary key, use primaryKey as an array. Lattice auto-generates the composite PRIMARY KEY constraint.since v0.17

  agent_skill:
    fields:
      agent_id: { type: uuid }
      skill_id: { type: uuid }
    primaryKey: [agent_id, skill_id]

This creates the table with PRIMARY KEY (agent_id, skill_id) instead of requiring a single-column primary key field.

Run npx lattice generate to create TypeScript types and a SQL migration file from your YAML config. This gives you type-safe access to your tables without writing any boilerplate.

Relations

The per-field ref: <table> shorthand above declares a foreign key. As of 4.0 the going-forward form is an explicit relations: block — the ref: shorthand is still accepted (it is parsed into a belongsTo relation), and when the GUI opens a config it rewrites ref: on disk to the explicit relations: form. Existing configs keep working untouched.since v4.0

TypeScript define()

If you prefer defining tables in code instead of YAML, use define(). This gives you full control over column types and render logic. Call it before init().

db.define('agents', {
  columns: {
    id: 'TEXT PRIMARY KEY',
    name: 'TEXT NOT NULL',
    persona: 'TEXT',
    active: 'INTEGER DEFAULT 1',
  },
  render(rows) {
    return rows
      .filter((r) => r.active)
      .map((r) => `## ${r.name}\n\n${r.persona ?? ''}`)
      .join('\n\n---\n\n');
  },
  outputFile: 'AGENTS.md',
});

The render field accepts a function (like above), a built-in template name ('default-list', 'default-table', 'default-detail', 'default-json'), or a template spec with hooks.

Schema-only tables

If you only need a table for data storage (no rendered context file), omit both render and outputFile. Lattice creates the table and gives you the full CRUD API without generating any files during render.since v0.17

// Schema-only — no rendered output
db.define('session', {
  columns: {
    id: 'TEXT PRIMARY KEY',
    agent_id: 'TEXT NOT NULL',
    started_at: 'TEXT',
    token_count: 'INTEGER DEFAULT 0',
  },
});

// Full CRUD works as normal
const id = await db.insert('session', { agent_id: 'a1', started_at: new Date().toISOString() });
const row = await db.get('session', id);

Constructor

Three ways to create a Lattice instance:

// From YAML config (recommended)
const db = new Lattice({ config: './lattice.config.yml' });

// From a database path (define tables in code)
const db = new Lattice('./app.db');

// In-memory for tests
const db = new Lattice(':memory:');

// With options
const db = new Lattice('./app.db', {
  wal: true,              // WAL journal mode (default: true)
  busyTimeout: 10_000,    // ms to wait on locked DB (default: 5000)
  renderSkipsEmpty: true, // skip read+write for spec-less tables on render() (default: false)
  security: {
    sanitize: true,
    auditTables: ['users', 'credentials'],
    fieldLimits: { notes: 50_000, bio: 2_000 },
  },
});

init() / close()

init() opens the database, creates any tables you defined, and runs migrations. Call it once when your process starts. Call close() when you're done.

await db.init({
  migrations: [
    { version: 1, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
    { version: 2, sql: 'ALTER TABLE tasks ADD COLUMN priority INTEGER DEFAULT 0' },
  ],
});

// Migrations run once each — safe to call init() on every process start.

db.close(); // call on shutdown

You can also apply migrations after init using migrate(). This is useful when migrations are defined separately from your init call, or when plugins add their own tables.Migration.version accepts either a number or a string (e.g. a semver tag or date-based identifier).since v0.17

await db.init();

// Apply migrations after init
await db.migrate([
  { version: 3, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
  { version: '2026-04-01', sql: 'ALTER TABLE tasks ADD COLUMN priority INTEGER DEFAULT 0' },
]);

CLI commands

The CLI is bundled with the package. Run commands with npx lattice.

npx lattice generate --config ./lattice.config.yml --out ./generated --scaffold

npx lattice render --config ./lattice.config.yml --output ./context

npx lattice reconcile --output ./context --protected SESSION.md

npx lattice status --output ./context

npx lattice watch --output ./context --interval 3000 --cleanup --protected SESSION.md

npx lattice gui --config ./lattice.config.yml

Reading & writing records

Standard operations for creating, reading, updating, and deleting records. All methods are async and return Promises.

// Create a record — returns the generated id
const id = await db.insert('task', { title: 'Write docs', status: 'open' });

// Update or create by primary key
await db.upsert('task', { id: 'task-001', title: 'Updated title', status: 'done' });

// Update or create by any column
await db.upsertBy('user', 'email', 'alice@example.com', { name: 'Alice' });

// Update specific fields on an existing record
await db.update('task', 'task-001', { status: 'done' });

// Fetch a single record by primary key
const task = await db.get('task', 'task-001');

// Delete a record
await db.delete('task', 'task-001');

// Insert and return the full row
const newTask = await db.insertReturning('task', { title: 'Write docs', status: 'open' });
// newTask = { id: 'generated-uuid', title: 'Write docs', status: 'open', ... }

// Update and return the updated row
const updated = await db.updateReturning('task', 'task-001', { status: 'done' });
// updated = { id: 'task-001', title: 'Write docs', status: 'done', ... }

insertReturning() and updateReturning() combine a write with a read in a single call, returning the full row including any auto-generated fields (UUIDs, defaults, timestamps).since v0.17

Queries & filters

Query records with filters, sorting, and pagination. Filters support these operators: eq, ne, gt, gte, lt, lte, like, in, isNull, isNotNull.

// Find open tasks with priority >= 3, newest first
const urgent = await db.query('task', {
  where: { status: 'open' },
  filters: [{ col: 'priority', op: 'gte', val: 3 }],
  orderBy: 'created_at',
  orderDir: 'desc',
  limit: 20,
});

// Count matching records
const openCount = await db.count('task', { where: { status: 'open' } });

Natural-key operations

When you identify records by a name or slug instead of a UUID, these methods handle the lookup automatically.since v0.11 They work on any table, including tables not registered with define().

// Create or update a record by its name
await db.upsertByNaturalKey('agent', 'name', 'Alice', {
  role: 'engineer', status: 'active',
});

// Update only the fields you pass (leaves others untouched)
await db.enrichByNaturalKey('agent', 'name', 'Alice', { title: 'Senior Engineer' });

// Look up a record by name
const alice = await db.getByNaturalKey('agent', 'name', 'Alice');

// Get all non-deleted records
const agents = await db.getActive('agent', 'name');
const count = await db.countActive('agent');

// Soft-delete records that are no longer in a source file
await db.softDeleteMissing('agent', 'name', 'agents.yaml', ['Alice', 'Bob']);

// Link / unlink records in a junction table
await db.link('agent_skill', { agent_id: 'a1', skill_id: 's1' });
await db.unlink('agent_skill', { agent_id: 'a1', skill_id: 's1' });

Seeding from files

Load records in bulk from YAML or JSON files. Lattice upserts by natural key, links junction table entries, and soft-deletes anything that's no longer in the source data.since v0.13

import { parse } from 'yaml';
import { readFileSync } from 'fs';

const rules = parse(readFileSync('rules.yaml', 'utf8'));

await db.seed({
  data: rules,
  table: 'rule',
  naturalKey: 'title',
  sourceFile: 'rules.yaml',
  linkTo: {
    targetAgents: {
      junction: 'rule_agent',
      foreignKey: 'agent_id',
      resolveBy: 'name',
      resolveTable: 'agent',
    },
  },
  softDeleteMissing: true,
});

Rendering basics

Rendering is how Lattice turns database rows into text files. There are four methods depending on what you need:

// One-shot render
const result = await db.render('./context');
// { filesWritten: ['context/AGENTS.md'], filesSkipped: 2, durationMs: 12 }

// Render + process agent output
await db.sync('./context');

// Watch with auto-cleanup
const stop = await db.watch('./context', {
  interval: 5_000,
  cleanup: { removeOrphanedDirectories: true, protectedFiles: ['SESSION.md'] },
});

// Render + clean up orphaned directories
await db.reconcile('./context', {
  removeOrphanedDirectories: true,
  protectedFiles: ['SESSION.md'],
  dryRun: false,   // set to true to preview without deleting
});

Files are written atomically (write to temp, rename). If a file's content hasn't changed, Lattice skips it — so re-rendering is cheap.

Entity directories

defineEntityContext() creates a directory for each row in a table.since v0.5 You declare which files each directory should contain and where the data comes from. Lattice handles querying, directory creation, and cleanup.

db.defineEntityContext('agent', {
  // How to name each directory (e.g. agents/alice/)
  slug: (row) => row.slug as string,

  // Apply to all relationship sources
  sourceDefaults: { softDelete: true },

  // Global index file
  index: {
    outputFile: 'agents/AGENTS.md',
    render: (rows) => rows.map((r) => `- ${r.name}`).join('\n'),
  },

  // Files inside each agent's directory
  files: {
    'AGENT.md': {
      source: { type: 'self' },
      render: ([r]) => `# ${r.name}\n\n${r.persona ?? ''}`,
    },
    'TASKS.md': {
      source: { type: 'hasMany', table: 'task', foreignKey: 'agent_id',
                orderBy: 'created_at', orderDir: 'desc', limit: 20 },
      render: (rows) => rows.map((r) => `- ${r.title}`).join('\n'),
      omitIfEmpty: true,
      budget: 4000,
    },
    'SKILLS.md': {
      source: {
        type: 'manyToMany',
        junctionTable: 'agent_skill',
        localKey: 'agent_id',
        remoteKey: 'skill_id',
        remoteTable: 'skill',
      },
      render: (rows) => rows.map((r) => `- ${r.name}`).join('\n'),
      omitIfEmpty: true,
    },
  },

  // Merge all files into one combined file
  combined: { outputFile: 'CONTEXT.md', exclude: [] },
  // These files (written by agents) are never deleted during cleanup
  protectedFiles: ['SESSION.md'],
});

Source types

Each file in an entity directory gets its data from a "source". There are six types:

The hasMany, manyToMany, and belongsTo sources accept optional query options: softDelete, filters, orderBy, orderDir, and limit.since v0.6 Set sourceDefaults on the entity context to apply the same options to every source.

Render templates

Instead of writing a custom render function for every file, you can use a built-in template.since v0.9

render: {
  template: 'entity-table',
  heading: 'Skills',
  columns: [
    { key: 'name', header: 'Name' },
    { key: 'level', header: 'Level', format: (v) => String(v || '—') },
  ],
}

render: {
  template: 'entity-profile',
  heading: (r) => r.name as string,
  fields: [
    { key: 'status', label: 'Status' },
    { key: 'role', label: 'Role' },
  ],
  sections: [
    { key: 'skills', heading: 'Skills', render: 'list',
      formatItem: (s) => s.name },
  ],
}

render: {
  template: 'entity-sections',
  heading: 'Rules',
  perRow: {
    heading: (r) => r.title as string,
    metadata: [{ key: 'scope', label: 'Scope' }],
    body: (r) => r.rule_text as string,
  },
}

Cleanup & reconciliation

When you delete an entity from the database, its directory becomes an orphan. Use reconcile() to remove it. Lattice tracks which directories it created using a .lattice/manifest.json file, so it only touches directories it owns.

Files listed in protectedFiles are never deleted. If an entity is removed but its directory still has protected files, Lattice removes only its own files and leaves the directory intact with a warning.

Reverse-sync (v0.16+)

AI agents frequently edit rendered context files directly. Without reverse-sync, those edits are destroyed on the next render. Add a reverseSync function to any EntityFileSpec to parse external edits back into the database before re-rendering.

// In your entity context definition:
'AGENT.md': {
  source: { type: 'self' },
  render: ([r]) => `# ${r.name}\n**Role:** ${r.role}\n`,
  reverseSync: (content, entityRow) => {
    const match = content.match(/^# (.+)$/m);
    if (match && match[1] !== entityRow.name) {
      return [{ table: 'agent', pk: { id: entityRow.id }, set: { name: match[1] } }];
    }
    return [];
  },
}

// Control via reconcile options:
await db.reconcile(dir);                             // reverse-sync enabled (default)
await db.reconcile(dir, { reverseSync: 'dry-run' }); // detect but don't modify DB
await db.reconcile(dir, { reverseSync: false });      // skip entirely

Lattice stores SHA-256 hashes of rendered content in the manifest. On the next reconcile, it compares hashes to detect modifications. Only files with a reverseSync function are checked — files without it are overwritten as before.

File loopback (v3.4+)

When the GUI is serving a workspace, editing a rendered .md context file on disk is captured back into the database through the normal write path — so the change lands in the changelog (versioned and undoable) and shows up live, exactly as if it had been made in the GUI. Frontmatter and body key: value fields round-trip automatically; an edit that can't be safely parsed (free-form or custom render) is surfaced as a notice rather than guessed at, so a lossy render can't corrupt a row. Render echoes are suppressed via the manifest, so there is no write loop.since v3.4

// Round-trip frontmatter + body `key: value` edits from the rendered
// tree back into the DB. Changelog-aware: pass `apply` to route each
// update through a versioned write, `useDefault` to round-trip without a
// hand-written reverseSync. Render-written files are recognized as echoes.
const result = await db.reverseSyncFromFiles('./context', { useDefault: true });
console.log(result.filesChanged, result.updatesApplied);

Seamless GUI auto-update (v3.4+)

Launched from an npm install, lattice gui runs as a small supervisor that silently installs the latest published version before opening, and keeps checking in the background. When a new version lands it relaunches the server on the same port; the open tab reconnects, notices the version changed, and reloads onto the new build — no manual refresh or reinstall. A git checkout or npx copy is left untouched.since v3.4

New GET /api/version and GET /api/update/status report the running version and the update state.

Writeback pipeline

The writeback pipeline watches files that agents write to and parses their output back into the database. You define which files to watch, how to parse them, and where to store the results.

db.defineWriteback({
  // Watch all SESSION.md files across agent directories
  file: './context/agents/*/SESSION.md',

  // Parse new content since last offset
  parse(content, fromOffset) {
    const newContent = content.slice(fromOffset);
    const entries = parseMarkdownItems(newContent);
    return { entries, nextOffset: content.length };
  },

  // Save each entry to the database
  async persist(entry, filePath) {
    await db.insert('event', {
      source_file: filePath,
      ...(entry as Record<string, unknown>),
    });
  },

  // Prevent duplicate processing
  dedupeKey: (entry) => (entry as { id: string }).id,
});

// db.sync() renders context AND processes writeback files
await db.sync('./context');

By default, writeback offsets are held in memory and lost on restart. For persistence across restarts, plug in a SQLite state store:since v0.12

import { createSQLiteStateStore } from 'latticesql';

// Offsets and dedup keys survive process restarts
const stateStore = createSQLiteStateStore('./state.db');

db.defineWriteback({
  file: './context/agents/*/SESSION.md',
  stateStore,
  parse(content, fromOffset) { /* ... */ },
  async persist(entry) { /* ... */ },
});

SESSION.md pattern

SESSION.md is a convention for agent-written output. Context files generated by Lattice are read-only. SESSION.md is the one file per entity directory where agents can write structured entries that get ingested back into the database.

Agents write structured entries in this format:

---
type: write
timestamp: 2026-03-25T10:30:00Z
op: update
table: agent
target: agent-id-here
reason: Completed deployment task.
---
status: idle
last_task: deploy-api
===

Lattice ships parsers for this format:

import { parseSessionWrites, parseSessionMD, applyWriteEntry } from 'latticesql';

// Parse write entries only
const result = parseSessionWrites(sessionContent);
for (const entry of result.entries) {
  // entry.op, entry.table, entry.target, entry.fields, entry.reason
  const applied = applyWriteEntry(db.db, entry);  // pass raw SQLite connection
}

// Parse all entry types (events, learnings, writes, etc.)
const all = parseSessionMD(content, startOffset);
// all.entries: SessionEntry[], all.lastOffset: number

Write hooks

Run code after a record is inserted, updated, or deleted. Useful for keeping denormalized fields in sync, fan-out updates, or computed columns.since v0.10

db.defineWriteHook({
  table: 'agent',
  on: ['insert', 'update'],
  watchColumns: ['team_id', 'division'],  // only fire when these change
  handler: (ctx) => {
    // ctx.table, ctx.op, ctx.row, ctx.pk, ctx.changedColumns
    updateTeamCounts(ctx.row.team_id);
  },
});

Hook errors are caught and routed to error handlers — they never crash the caller. Multiple hooks per table are supported.

Token budgets

Limit the rendered output of a table to a token budget. When content exceeds the budget, rows are pruned by priority and a truncation footer is appended.since v1.3

db.define('tickets', {
  columns: { id: 'TEXT PRIMARY KEY', title: 'TEXT', updated_at: 'TEXT' },
  render: (rows) => rows.map((r) => `- ${r.title}`).join('\n'),
  outputFile: 'TICKETS.md',
  tokenBudget: 4000,            // max estimated tokens (~4 chars/token)
  prioritizeBy: 'updated_at',   // keep most recent rows when pruning
});
// Output: "- Fix auth bug\n- Deploy v2\n\n[truncated: 47 of 123 rows rendered, ~3800 tokens]"

prioritizeBy accepts a column name (sorted descending) or a (a, b) => number comparator. When omitted, rows at the end of the query result are dropped first.

Relevance filtering

Dynamically filter rows based on the current task context. Only relevant rows are rendered.since v1.3

db.define('knowledge', {
  columns: { id: 'TEXT PRIMARY KEY', topic: 'TEXT', body: 'TEXT' },
  render: (rows) => rows.map((r) => `## ${r.topic}\n${r.body}`).join('\n\n'),
  outputFile: 'KNOWLEDGE.md',
  relevanceFilter: (row, ctx) =>
    ctx ? String(row.body).toLowerCase().includes(ctx.toLowerCase()) : true,
});

// Set the current task — only matching rows are rendered
db.setTaskContext('deployment');
await db.render('./context');

// Clear context — all rows rendered again
db.setTaskContext('');

Enrichment pipeline

Transform rows between filtering and rendering. Add computed fields, cluster by category, or summarize large datasets — without modifying the underlying data.since v1.3

db.define('incidents', {
  columns: { id: 'TEXT PRIMARY KEY', severity: 'TEXT', title: 'TEXT', created_at: 'TEXT' },
  render: (rows) => JSON.stringify(rows, null, 2),
  outputFile: 'incidents.json',
  enrich: [
    // Add computed age field
    (rows) => rows.map((r) => ({
      ...r,
      _age_hours: Math.round((Date.now() - new Date(r.created_at).getTime()) / 3600000),
    })),
    // Summarize if too many rows
    (rows) => rows.length > 100
      ? [{ _summary: `${rows.length} incidents, ${rows.filter(r => r.severity === 'P0').length} critical` }]
      : rows,
  ],
});

Reward memory

Track which data is useful to your agents. High-reward rows are prioritized in rendering; low-scoring rows can be auto-pruned via soft-delete.since v1.3

db.define('tips', {
  columns: { id: 'TEXT PRIMARY KEY', tip: 'TEXT', deleted_at: 'TEXT' },
  render: (rows) => rows.map((r) => `- ${r.tip}`).join('\n'),
  outputFile: 'TIPS.md',
  rewardTracking: true,   // auto-adds _reward_total, _reward_count columns
  pruneBelow: 0.3,        // soft-delete rows with reward < 0.3
});

await db.init();
const id = await db.insert('tips', { tip: 'Use batch inserts for bulk data' });

// After the agent confirms this tip was useful
await db.reward('tips', id, { relevance: 0.9, accuracy: 1.0 });
// _reward_total = 0.95, _reward_count = 1

// Second signal — running average
await db.reward('tips', id, { relevance: 0.5 });
// _reward_total = 0.7, _reward_count = 2

Semantic search

Enable embedding-based semantic search on any table. Bring your own embedding function — Lattice stores vectors in a companion SQLite table and computes cosine similarity in JS. No external vector database required.since v1.3

import { Lattice } from 'latticesql';

db.define('docs', {
  columns: { id: 'TEXT PRIMARY KEY', title: 'TEXT', body: 'TEXT' },
  render: (rows) => rows.map((r) => `## ${r.title}\n${r.body}`).join('\n\n---\n\n'),
  outputFile: 'DOCS.md',
  embeddings: {
    fields: ['title', 'body'],
    embed: async (text) => {
      const res = await openai.embeddings.create({
        input: text, model: 'text-embedding-3-small',
      });
      return res.data[0].embedding;
    },
  },
});

await db.init();
await db.insert('docs', { title: 'Deploy guide', body: 'How to deploy to production...' });

// Search by meaning, not keywords
const results = await db.search('docs', 'ship to prod', { topK: 5, minScore: 0.7 });
for (const { row, score } of results) {
  console.log(`${score.toFixed(2)} — ${row.title}`);
}

Writeback validation

Validate agent-written data before persisting. Reject low-quality or inconsistent entries with scoring and threshold-based gating.since v1.3

db.defineWriteback({
  file: './agent-output/*.md',
  parse: (content, offset) => ({
    entries: [content.slice(offset)],
    nextOffset: content.length,
  }),
  persist: async (entry) => { /* save to DB */ },
  validate: async (entry) => {
    const text = entry as string;
    const hasFields = text.includes('## Title') && text.includes('## Body');
    return {
      pass: hasFields,
      score: hasFields ? 0.9 : 0.1,
      reason: hasFields ? undefined : 'Missing required sections',
    };
  },
  rejectBelow: 0.5,
  onReject: (entry, result) => {
    console.warn(`Rejected: ${result.reason} (score: ${result.score})`);
  },
});

Connect Claude

Open Settings → User → Assistant. The primary action (v3.3+) is Connect with Claude — an Authorization-Code + PKCE flow that links your Claude Pro / Max / Enterprise subscription, so the assistant runs on your own account with no API key to paste. It works out of the box (the public OAuth client is built in) and uses a loopback callback derived from the GUI's own origin. Prefer a raw key? Expand Advanced — use an API key instead and paste an Anthropic API key (or set ANTHROPIC_API_KEY); keys are stored encrypted in the native secrets entity. Every endpoint/client value is overridable via ANTHROPIC_OAUTH_*.

Chat + cross-turn memory

The rail runs a Claude tool-calling loop streamed over SSE. The model can list, read, create, update, link, delete, and revert rows in the active database — deletes are guarded and reversible, restored from version history just like a manual delete. Every edit goes through the same audited, undoable mutation path as a manual edit — it lands in the activity feed and the version history and can be reverted. Conversations persist in the native chat_threads / chat_messages entities.

Replies render inline object-link pills: when the assistant references a record it retrieved, the [label](lattice://<table>/<id>) link becomes a clickable pill that opens that row via the mode-aware navigator (it only links ids it actually saw, preferring the user-facing record). Each data change the assistant makes surfaces as an activity card in the live feed — an operation icon, a human summary, and a duration — collapsed by type across objects (“Deleted 19 tables”, “Removed 49 rows across 9 tables”). Read-only calls produce no card, and the feed is per-conversation: each turn persists its data-change events and replays them when you reopen the thread.

The assistant remembers what it read across turns: earlier tool calls and their results (row ids included) are replayed into the model context, so a follow-up such as “now update that row” reuses the id it just listed instead of guessing. Replay is bounded to recent turns within a size budget and is secret-redacted; set LATTICE_CHAT_REHYDRATE=false to disable it. Reads are deterministically ordered, so listing a table twice returns the same rows.

The assistant knows the record you're viewing: when a file or row detail is open, the chat passes it as context, so “delete this file” or “summarize this” act on it directly. It is a hint only — every action still flows through the same permission-gated tools. It can also answer questions about Lattice itself (e.g. “what is private mode?”) via a lattice_help tool that searches Lattice's own documentation rather than guessing or querying your data.

The Context Constructor (file & text ingest)

Drag files onto the rail, click the upload button, or paste text or a URL. Each source is referenced, not copied (a native files row), extracted (text directly; PDF, Word, PowerPoint, Excel, OpenDocument, EPUB, and RTF parsed natively in-process — no external CLI; images by Claude vision; a pasted URL is crawled for readable text), summarized with Claude Haiku, and organized — classified against your existing records and linked, auto-creating the files_<entity> junction when none exists. A source that fits nothing (at higher aggressiveness) becomes a new notes object. New objects, enrichment, links, and junctions are all reversible via the version history.

Library API

The same intelligence is a first-class, GUI-independent API (inert without an LLM client), importable from latticesql:

import { organizeSource, describeImage, crawlUrl, enrichKnowledge } from 'latticesql';

// Sort a source into your own schema: summarize + classify + link,
// creating a new object only when nothing fits. Inert without an LLM client.
const result = await organizeSource(db, {
  text: 'Acme Consulting — signed MSA, net-30, effective 2026-01-01',
  client, // your LlmClient
});

Plus the summarizeText / classifyLinks primitives. sharp + file-type (images) and jsdom + @mozilla/readability (crawler) are optional, lazily-loaded deps.

Inference Aggressiveness

A single Conservative ↔ Aggressive slider (Settings → Assistant) tunes how much the assistant extrapolates: the model sampling temperature, how liberally the ingest classifier proposes links, and whether ingest auto-creates a missing junction (gated at ≥ 0.25) versus only suggesting it. Default 0.5; settable via PUT /api/assistant/aggressiveness.

Voice (optional)

Set OPENAI_API_KEY (Whisper) or ELEVENLABS_API_KEY to enable the composer mic. When no microphone is available the button is shown disabled with a tooltip rather than erroring.

New tools (v3.4+) — get_row_context & add_column

Two new assistant tools reduce round-trips and allow schema edits from chat.since v3.4

Prepared statement cache

since v1.4

Lattice automatically caches compiled SQLite prepared statements. Repeated calls with the same SQL reuse the compiled statement instead of recompiling on every invocation. DDL statements (CREATE, ALTER, DROP, PRAGMA) bypass the cache. The cache clears automatically on close() and after schema or migration changes. No API changes required.

Batch entity query resolution

since v1.4

Entity context rendering pre-fetches related rows for all entities in a single WHERE IN (...) query per source, replacing the previous per-entity query pattern. hasMany, manyToMany, and belongsTo sources are batched automatically. custom and enriched sources fall back to per-entity resolution. IN clauses are chunked at 500 parameters to stay under SQLite's limit. No API changes required.

Render change detection

since v1.4

Lattice tracks per-table write version counters. Use isDirty() to check if any table has been written to since the last render, and markDirty(table?) after escape-hatch writes.

// Custom polling loop that skips redundant renders
setInterval(async () => {
  if (db.isDirty()) {
    await db.render(outputDir);
  }
}, 5000);

// After direct DB writes, mark dirty
db.db.prepare('UPDATE tasks SET status = ?').run('done');
db.markDirty('tasks');

Migration validation

since v1.4

Pass a validateMigrationSQL function in InitOptions to validate all pending migration SQL before any migrations execute. If validation fails, no migrations run and an error is thrown. Multi-statement migration SQL is fully supported.

await db.init({
  migrations: [
    { version: 1, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
  ],
  validateMigrationSQL: (sql) => {
    if (sql.trim().length === 0) return { valid: false, errors: ['Empty SQL'] };
    return { valid: true };
  },
});

Reports

Build time-windowed reports from your data. Useful for daily summaries, activity digests, or status dashboards.since v0.14

const report = await db.buildReport({
  since: '24h',   // or '8h', '7d', or an ISO timestamp
  sections: [
    { name: 'tasks', query: { table: 'task' }, format: 'count_and_list' },
    { name: 'events', query: { table: 'event', groupBy: 'type' }, format: 'counts' },
    { name: 'alerts', query: { table: 'event',
      filters: [{ col: 'severity', op: 'lte', val: 2 }] }, format: 'list' },
  ],
});

report.sections.forEach(s => console.log(`${s.name}: ${s.count}`));

Markdown helpers

Helper functions you can use inside render functions to reduce boilerplate.since v0.6

import { frontmatter, markdownTable, slugify, truncate } from 'latticesql';

// YAML frontmatter with auto timestamp
frontmatter({ agent: 'Alice', skill_count: 5 });

// Markdown table from rows
markdownTable(rows, [
  { key: 'name', header: 'Name' },
  { key: 'status', header: 'Status', format: (v) => String(v || '—') },
]);

// URL-safe slug
slugify('Alice Smith');   // 'alice-smith'

// Truncate to character budget
truncate(longContent, 4000);

Auto-update

For applications that manage their own updates at runtime, autoUpdate() checks npm for a newer version and installs it automatically. Call it once at startup, before initializing Lattice.since v1.1

import { autoUpdate } from 'latticesql';

// Call at app startup — checks npm, installs if outdated
const result = await autoUpdate();
if (result.restartRequired) {
  process.exit(0); // Let process manager restart
}

Safe to call on every startup — skips if already on the latest version. Pass { quiet: true } to suppress console output.

interface AutoUpdateResult {
  updated: boolean;
  packages: Array<{ name: string; from: string; to: string }>;
  restartRequired: boolean;
}

Events

Subscribe to lifecycle events for monitoring and audit logging.

db.on('audit',     ({ table, operation, id, timestamp }) => { /* ... */ });
db.on('render',    ({ filesWritten, filesSkipped, durationMs }) => { /* ... */ });
db.on('writeback', ({ filePath, entriesProcessed }) => { /* ... */ });
db.on('error',     (err: Error) => { /* ... */ });

audit events fire on every insert/update/delete for tables listed in security.auditTables.

The three flows: migrate, join, invite

There are exactly three things you do with a cloud: migrate into one, join an existing one, or invite someone to yours.

• Migrate — point a local Lattice at a fresh, empty Postgres. Lattice copies your data in, installs RLS, and stamps you as the owner of every migrated row.
• Join — redeem the email-bound invite token the owner sent you (your email + the token). The token decrypts locally to your scoped credentials and connects directly; the member UI never handles a connection string, and there is no server to sign into.
• Invite — an owner (whose role holds CREATEROLE) provisions a scoped, NOSUPERUSER member role and mints a single, email-bound token carrying its credential to hand the new member.

All the cloud helpers import from latticesql. Migrate a local Lattice into a fresh cloud, then back-fill ownership before forcing RLS on each table:

import {
  Lattice,
  openTargetLatticeForMigration,
  migrateLatticeData,
  installCloudRls,
  backfillOwnership,
  enableRlsForTable,
  archiveLocalSqlite,
} from 'latticesql';

const encryptionKey = process.env.LATTICE_ENCRYPTION_KEY;
const cloudUrl = 'postgres://alice:secret@cloud.example.com:5432/app';

const source = new Lattice({ config: './lattice.config.yml' }, { encryptionKey });
await source.init();

const target = await openTargetLatticeForMigration('./lattice.config.yml', cloudUrl, encryptionKey);
await migrateLatticeData(source, target); // → { tablesCopied, rowsCopied }

// Owner-side RLS install. Backfill ownership BEFORE forcing RLS on each table.
await installCloudRls(target);
for (const table of target.getRegisteredTableNames()) {
  if (table.startsWith('__lattice_')) continue;
  const pk = target.getPrimaryKey(table);
  if (pk.length === 0) continue; // unkeyable table — no per-row RLS
  await backfillOwnership(target, table, pk);
  await enableRlsForTable(target, table, pk);
}
target.close();

archiveLocalSqlite('./data/app.db'); // renames to .db.local-bak

A member joins by connecting directly as their scoped role — that is the whole credential. probeCloud confirms the target is reachable and is already a Lattice cloud (RLS installed):

import { Lattice, probeCloud } from 'latticesql';

const url = 'postgres://lm_bob_a91c:the-password@cloud.example.com:5432/app';

const probe = await probeCloud(url);
// → { reachable: true, dialect: 'postgres', isCloud: true }
if (!probe.reachable) throw new Error(probe.error);
if (!probe.isCloud) throw new Error('Not a Lattice cloud yet — ask the owner to migrate into it.');

const db = new Lattice(url);
await db.init();
const visibleItems = await db.query('items'); // RLS-filtered to what this role may see

An owner invites a member by provisioning a scoped role and handing over its credentials. Member roles are created NOSUPERUSER NOCREATEDB NOCREATEROLE and added to the lattice_members group; the generated password is shown once:

import { Lattice, memberRoleName, generateMemberPassword, provisionMemberRole, revokeMemberRole } from 'latticesql';

// owner connection — must hold CREATEROLE
const db = new Lattice('postgres://alice:secret@cloud.example.com:5432/app');
await db.init();

const role = memberRoleName('bob');        // e.g. 'lm_bob_a91c' — collision-safe, ≤63 bytes
const password = generateMemberPassword(); // 48 hex chars
await provisionMemberRole(db, role, password);
// Hand off:  host / port / dbname  +  user=role  +  password — that blob IS the invite.

// Removing a member drops the role (their rows persist but become unreachable):
await revokeMemberRole(db, 'lm_bob_a91c');

since v4.0 The members group is now per-cloud, derived from the database and schema, so unrelated clouds that share one Postgres cluster no longer share a members group. Library consumers resolve it with memberGroupFor(db); this replaces the previous exported MEMBER_GROUP constant.

Sharing: private by default

Every row is private to its owner the moment it's written — the per-table trigger stamps visibility = 'private'. The owner opts a row into wider visibility: everyone (every member sees it) or custom (the owner plus an explicit grant list). Sharing runs through the owner-only SECURITY DEFINER function — Postgres raises for anyone who isn't the row's owner:

-- Make one row visible to every member of the cloud:
SELECT lattice_set_row_visibility('items', 'item-42', 'everyone');

-- Take it private again:
SELECT lattice_set_row_visibility('items', 'item-42', 'private');

-- Or grant just one member (sets visibility = 'custom'):
SELECT lattice_grant_row('items', 'item-42', 'lm_bob_a91c');
SELECT lattice_revoke_row('items', 'item-42', 'lm_bob_a91c');

From the library, setRowVisibility validates private | everyone before calling the function. The primary key is the row's canonical key string (composite keys are TAB-joined). Because sharing lives in out-of-band bookkeeping, opting a row in or out never touches your table's columns:

import { Lattice, setRowVisibility } from 'latticesql';

const db = new Lattice('postgres://alice:secret@cloud.example.com:5432/app');
await db.init();

// Only the row's owner may call this; Postgres raises otherwise.
await setRowVisibility(db, 'items', 'item-42', 'everyone');

Per-column audiences (experimental)

RLS is whole-row. A layered primitive takes it to the cell: declare an audience on a column and Lattice generates a cell-masking view <table>_v beside the base table. The audience is a +-joined (OR) set of clauses — role:<name>, subject:<col>, source:<col>, or everyone — each compiling to a session_user-keyed SECURITY DEFINER predicate.

person:
  fields:
    id: { type: uuid, primaryKey: true }
    name: { type: text }
    comp: { type: text, audience: 'subject:subject_role+role:hr' }
    subject_role: { type: text }

Members SELECT the view (base SELECT is revoked); a masked cell reads as NULL. App roles are owner-managed via lattice_assign_role(member, role) — members can't self-promote. A column with no audience behaves exactly as before. Off by default.

S3-backed file bytes (opt-in)

A files row is shared by RLS like any other row, but by default a file's bytes live only on the uploader's local disk — so another member can SELECT the row yet can't fetch the content. Enabling S3 for the cloud closes that gap: uploaded bytes also go to an S3 bucket under a content-addressed (<prefix>/<sha256>) key, and any member who can see the row pulls them down in the viewer.

Access rides entirely on the files-row RLS — the serve route does db.get('files', id) as the member's own scoped role, so a row RLS won't let them read returns 404 before S3 is ever touched. The bucket credential is least-privilege — GetObject + PutObject, no ListBucket, no Delete — and is per-member, machine-local, and encrypted (it never lives in the shared database). This is app-mediated access control; see the full caveats in docs/cloud.md.

Chat system prompt (owner-set)

A cloud owner can set a chat system prompt that's bundled into every member's assistant chat for that workspace — house style, domain facts, a fiscal calendar, whatever the team should always have in context. It's stored in the shared DB (__lattice_cloud_settings) and reached through owner-gated SECURITY DEFINER helpers; members never see the control or the prompt value through the UI or API. On a local SQLite workspace there are no members and nothing to keep secret, so the editor is hidden.

Realtime + offline editing

A per-row AFTER INSERT trigger fires pg_notify('lattice_changes', …) carrying only metadata (table, pk, op) — never row content — so a connected GUI can refetch the affected row through RLS. SQLite databases are unchanged; LISTEN/NOTIFY is Postgres-only.

Offline editing is preserved as a client-side local edit queue: edits made while disconnected are held locally and replayed when you reconnect. It is a client behavior only — not tied to any replica or sync server (there is no server). On reconnect the queued writes go to the cloud as your role and land under the same RLS rules as any other write.

Per-viewer rendered context (v3.4+)

On a cloud, the background render reads every table through the member's row-level-security connection and per-column masking view. The rendered markdown a member's assistant reads off disk contains only the rows they may see, with owner-only columns blanked and any per-viewer enrichment folded in; it re-renders promptly when sharing changes. Owners and local single-user workspaces render the full tree unchanged.since v3.4

Cloud resilience & search (v3.4+)

since v3.4

Full-text search on migrated clouds: when you migrate a local Lattice into a cloud, the full-text index is now built automatically, so search works immediately. Call db.rebuildFtsIndexes() on any cloud to rebuild from scratch.

Open-time convergence is per-table fault-isolated: when opening a cloud, if a table is unmanageable (owned by a different Postgres role), it is skipped with an actionable reason instead of failing the entire workspace. Warnings are surfaced via GET /api/dbconfig as convergeWarnings. Use POST /api/workspaces/reload to re-register entities without restarting.

Raw Postgres URL healing: a raw postgres://user:password@host/db string in a workspace config is healed on open into an encrypted ${LATTICE_DB:label} credential reference, so the secret no longer lingers in cleartext on disk.

GUI cloud endpoints

lattice gui drives all three flows from the browser. The relevant endpoints are localhost-only, same model as the rest of the GUI:

v3.4 additions:

The full architecture — the RLS / role model, the three flows in detail, the S3 and system-prompt designs, per-column audiences, and the sharing API — lives in docs/cloud.md.

Machine-local config at ~/.lattice/

Files outside any Lattice DB, so switching projects doesn't cost your identity or your scoped credentials:

Native secrets + files entities

Every Lattice opened by lattice gui automatically registers two framework-shipped tables. Available to any Lattice via the public helper:

import { Lattice } from 'latticesql';
import { registerNativeEntities } from 'latticesql/framework/native-entities';

const db = new Lattice(
  { config: './lattice.config.yml' },
  { encryptionKey: process.env.LATTICE_ENCRYPTION_KEY },
);
registerNativeEntities(db);
await db.init();

// 'secrets.value' is encrypted at rest:
const id = await db.insert('secrets', {
  name: 'OPENAI_API_KEY',
  kind: 'api-key',
  value: 'sk-...',           // stored as 'enc:<base64>' in SQLite
});
const row = await db.get('secrets', id);
console.log(row.value);      // → 'sk-...' (decrypted)

Encryption uses a new TableDefinition.encrypted field — the same shape as the existing entity-context option, now extended to plain define() tables. attachBlob(srcPath, latticeRoot) writes any file into a content-addressed store at <root>/data/blobs/<sha256> and returns metadata suitable for a files row. On a cloud with S3 enabled, those bytes also go to the bucket.