Documentation

YAML config

The recommended way to define your schema. Write a lattice.config.yml file and Lattice creates your tables, wires up rendering, and can generate TypeScript types for you.since v0.4

# lattice.config.yml
db: ./data/app.db

entities:
  user:
    fields:
      id: { type: uuid, primaryKey: true }
      name: { type: text, required: true }
      email: { type: text }
      role: { type: text, default: member }
    render: default-table
    outputFile: context/USERS.md

  task:
    fields:
      id: { type: uuid, primaryKey: true }
      title: { type: text, required: true }
      status: { type: text, default: open }
      priority: { type: integer, default: 1 }
      assignee_id: { type: uuid, ref: user }
    render:
      template: default-list
      formatRow: '{{title}} ({{status}}) — {{assignee.name}}'
    outputFile: context/TASKS.md

Type mappings

Composite primary keys

For junction tables or any table with a multi-column primary key, use primaryKey as an array. Lattice auto-generates the composite PRIMARY KEY constraint.since v0.17

  agent_skill:
    fields:
      agent_id: { type: uuid }
      skill_id: { type: uuid }
    primaryKey: [agent_id, skill_id]

This creates the table with PRIMARY KEY (agent_id, skill_id) instead of requiring a single-column primary key field.

Run npx lattice generate to create TypeScript types and a SQL migration file from your YAML config. This gives you type-safe access to your tables without writing any boilerplate.

Relations

The per-field ref: <table> shorthand above declares a foreign key. As of 4.0 the going-forward form is an explicit relations: block — the ref: shorthand is still accepted (it is parsed into a belongsTo relation), and when the GUI opens a config it rewrites ref: on disk to the explicit relations: form. Existing configs keep working untouched.since v4.0

TypeScript define()

If you prefer defining tables in code instead of YAML, use define(). This gives you full control over column types and render logic. Call it before init().

db.define('agents', {
  columns: {
    id: 'TEXT PRIMARY KEY',
    name: 'TEXT NOT NULL',
    persona: 'TEXT',
    active: 'INTEGER DEFAULT 1',
  },
  render(rows) {
    return rows
      .filter((r) => r.active)
      .map((r) => `## ${r.name}\n\n${r.persona ?? ''}`)
      .join('\n\n---\n\n');
  },
  outputFile: 'AGENTS.md',
});

The render field accepts a function (like above), a built-in template name ('default-list', 'default-table', 'default-detail', 'default-json'), or a template spec with hooks.

Schema-only tables

If you only need a table for data storage (no rendered context file), omit both render and outputFile. Lattice creates the table and gives you the full CRUD API without generating any files during render.since v0.17

// Schema-only — no rendered output
db.define('session', {
  columns: {
    id: 'TEXT PRIMARY KEY',
    agent_id: 'TEXT NOT NULL',
    started_at: 'TEXT',
    token_count: 'INTEGER DEFAULT 0',
  },
});

// Full CRUD works as normal
const id = await db.insert('session', { agent_id: 'a1', started_at: new Date().toISOString() });
const row = await db.get('session', id);

Constructor

Three ways to create a Lattice instance:

// From YAML config (recommended)
const db = new Lattice({ config: './lattice.config.yml' });

// From a database path (define tables in code)
const db = new Lattice('./app.db');

// In-memory for tests
const db = new Lattice(':memory:');

// With options
const db = new Lattice('./app.db', {
  wal: true,              // WAL journal mode (default: true)
  busyTimeout: 10_000,    // ms to wait on locked DB (default: 5000)
  renderSkipsEmpty: true, // skip read+write for spec-less tables on render() (default: false)
  security: {
    sanitize: true,
    auditTables: ['users', 'credentials'],
    fieldLimits: { notes: 50_000, bio: 2_000 },
  },
});

init() / close()

init() opens the database, creates any tables you defined, and runs migrations. Call it once when your process starts. Call close() when you're done.

await db.init({
  migrations: [
    { version: 1, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
    { version: 2, sql: 'ALTER TABLE tasks ADD COLUMN priority INTEGER DEFAULT 0' },
  ],
});

// Migrations run once each — safe to call init() on every process start.

db.close(); // call on shutdown

You can also apply migrations after init using migrate(). This is useful when migrations are defined separately from your init call, or when plugins add their own tables.Migration.version accepts either a number or a string (e.g. a semver tag or date-based identifier).since v0.17

await db.init();

// Apply migrations after init
await db.migrate([
  { version: 3, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
  { version: '2026-04-01', sql: 'ALTER TABLE tasks ADD COLUMN priority INTEGER DEFAULT 0' },
]);

CLI commands

The CLI is bundled with the package. Run commands with npx lattice.

since v5.7 The commands below are the schema and rendering group. npx lattice --help is the full list and the authority on flags: it also covers init, workspace, database, schema, questions, ask, model, account, cloud, connector, ingest, import, search, and update — walked through with worked examples in Lattice 5.7 — without the browser.

npx lattice generate --config ./lattice.config.yml --out ./generated --scaffold

npx lattice render --config ./lattice.config.yml --output ./context

npx lattice reconcile --output ./context --protected SESSION.md

npx lattice status --output ./context

npx lattice watch --output ./context --interval 3000 --cleanup --protected SESSION.md

npx lattice gui --config ./lattice.config.yml

npx lattice reindex docs

npx lattice index status

npx lattice doctor --fix

Reading & writing records

Standard operations for creating, reading, updating, and deleting records. All methods are async and return Promises.

// Create a record — returns the generated id
const id = await db.insert('task', { title: 'Write docs', status: 'open' });

// Update or create by primary key
await db.upsert('task', { id: 'task-001', title: 'Updated title', status: 'done' });

// Update or create by any column
await db.upsertBy('user', 'email', 'alice@example.com', { name: 'Alice' });

// Update specific fields on an existing record
await db.update('task', 'task-001', { status: 'done' });

// Fetch a single record by primary key
const task = await db.get('task', 'task-001');

// Delete a record
await db.delete('task', 'task-001');

// Insert and return the full row
const newTask = await db.insertReturning('task', { title: 'Write docs', status: 'open' });
// newTask = { id: 'generated-uuid', title: 'Write docs', status: 'open', ... }

// Update and return the updated row
const updated = await db.updateReturning('task', 'task-001', { status: 'done' });
// updated = { id: 'task-001', title: 'Write docs', status: 'done', ... }

insertReturning() and updateReturning() combine a write with a read in a single call, returning the full row including any auto-generated fields (UUIDs, defaults, timestamps).since v0.17

Queries & filters

Query records with filters, sorting, and pagination. Filters support these operators: eq, ne, gt, gte, lt, lte, like, in, isNull, isNotNull.

// Find open tasks with priority >= 3, newest first
const urgent = await db.query('task', {
  where: { status: 'open' },
  filters: [{ col: 'priority', op: 'gte', val: 3 }],
  orderBy: 'created_at',
  orderDir: 'desc',
  limit: 20,
});

// Count matching records
const openCount = await db.count('task', { where: { status: 'open' } });

Natural-key operations

When you identify records by a name or slug instead of a UUID, these methods handle the lookup automatically.since v0.11 They work on any table, including tables not registered with define().

// Create or update a record by its name
await db.upsertByNaturalKey('agent', 'name', 'Alice', {
  role: 'engineer', status: 'active',
});

// Update only the fields you pass (leaves others untouched)
await db.enrichByNaturalKey('agent', 'name', 'Alice', { title: 'Senior Engineer' });

// Look up a record by name
const alice = await db.getByNaturalKey('agent', 'name', 'Alice');

// Get all non-deleted records
const agents = await db.getActive('agent', 'name');
const count = await db.countActive('agent');

// Soft-delete records that are no longer in a source file
await db.softDeleteMissing('agent', 'name', 'agents.yaml', ['Alice', 'Bob']);

// Link / unlink records in a junction table
await db.link('agent_skill', { agent_id: 'a1', skill_id: 's1' });
await db.unlink('agent_skill', { agent_id: 'a1', skill_id: 's1' });

Seeding from files

Load records in bulk from YAML or JSON files. Lattice upserts by natural key, links junction table entries, and soft-deletes anything that's no longer in the source data.since v0.13

import { parse } from 'yaml';
import { readFileSync } from 'fs';

const rules = parse(readFileSync('rules.yaml', 'utf8'));

await db.seed({
  data: rules,
  table: 'rule',
  naturalKey: 'title',
  sourceFile: 'rules.yaml',
  linkTo: {
    targetAgents: {
      junction: 'rule_agent',
      foreignKey: 'agent_id',
      resolveBy: 'name',
      resolveTable: 'agent',
    },
  },
  softDeleteMissing: true,
});

Rendering basics

Rendering is how Lattice turns database rows into text files. There are four methods depending on what you need:

// One-shot render
const result = await db.render('./context');
// { filesWritten: ['context/AGENTS.md'], filesSkipped: 2, durationMs: 12 }

// Render + process agent output
await db.sync('./context');

// Watch with auto-cleanup
const stop = await db.watch('./context', {
  interval: 5_000,
  cleanup: { removeOrphanedDirectories: true, protectedFiles: ['SESSION.md'] },
});

// Render + clean up orphaned directories
await db.reconcile('./context', {
  removeOrphanedDirectories: true,
  protectedFiles: ['SESSION.md'],
  dryRun: false,   // set to true to preview without deleting
});

Files are written atomically (write to temp, rename). If a file's content hasn't changed, Lattice skips it — so re-rendering is cheap.

Entity directories

defineEntityContext() creates a directory for each row in a table.since v0.5 You declare which files each directory should contain and where the data comes from. Lattice handles querying, directory creation, and cleanup.

db.defineEntityContext('agent', {
  // How to name each directory (e.g. agents/alice/)
  slug: (row) => row.slug as string,

  // Apply to all relationship sources
  sourceDefaults: { softDelete: true },

  // Global index file
  index: {
    outputFile: 'agents/AGENTS.md',
    render: (rows) => rows.map((r) => `- ${r.name}`).join('\n'),
  },

  // Files inside each agent's directory
  files: {
    'AGENT.md': {
      source: { type: 'self' },
      render: ([r]) => `# ${r.name}\n\n${r.persona ?? ''}`,
    },
    'TASKS.md': {
      source: { type: 'hasMany', table: 'task', foreignKey: 'agent_id',
                orderBy: 'created_at', orderDir: 'desc', limit: 20 },
      render: (rows) => rows.map((r) => `- ${r.title}`).join('\n'),
      omitIfEmpty: true,
      budget: 4000,
    },
    'SKILLS.md': {
      source: {
        type: 'manyToMany',
        junctionTable: 'agent_skill',
        localKey: 'agent_id',
        remoteKey: 'skill_id',
        remoteTable: 'skill',
      },
      render: (rows) => rows.map((r) => `- ${r.name}`).join('\n'),
      omitIfEmpty: true,
    },
  },

  // Merge all files into one combined file
  combined: { outputFile: 'CONTEXT.md', exclude: [] },
  // These files (written by agents) are never deleted during cleanup
  protectedFiles: ['SESSION.md'],
});

Source types

Each file in an entity directory gets its data from a "source". There are six types:

The hasMany, manyToMany, and belongsTo sources accept optional query options: softDelete, filters, orderBy, orderDir, and limit.since v0.6 Set sourceDefaults on the entity context to apply the same options to every source.

Render templates

Instead of writing a custom render function for every file, you can use a built-in template.since v0.9

render: {
  template: 'entity-table',
  heading: 'Skills',
  columns: [
    { key: 'name', header: 'Name' },
    { key: 'level', header: 'Level', format: (v) => String(v || '—') },
  ],
}

render: {
  template: 'entity-profile',
  heading: (r) => r.name as string,
  fields: [
    { key: 'status', label: 'Status' },
    { key: 'role', label: 'Role' },
  ],
  sections: [
    { key: 'skills', heading: 'Skills', render: 'list',
      formatItem: (s) => s.name },
  ],
}

render: {
  template: 'entity-sections',
  heading: 'Rules',
  perRow: {
    heading: (r) => r.title as string,
    metadata: [{ key: 'scope', label: 'Scope' }],
    body: (r) => r.rule_text as string,
  },
}

Cleanup & reconciliation

When you delete an entity from the database, its directory becomes an orphan. Use reconcile() to remove it. Lattice tracks which directories it created using a .lattice/manifest.json file, so it only touches directories it owns.

Files listed in protectedFiles are never deleted. If an entity is removed but its directory still has protected files, Lattice removes only its own files and leaves the directory intact with a warning.

Reverse-sync (v0.16+)

AI agents frequently edit rendered context files directly. Without reverse-sync, those edits are destroyed on the next render. Add a reverseSync function to any EntityFileSpec to parse external edits back into the database before re-rendering.

// In your entity context definition:
'AGENT.md': {
  source: { type: 'self' },
  render: ([r]) => `# ${r.name}\n**Role:** ${r.role}\n`,
  reverseSync: (content, entityRow) => {
    const match = content.match(/^# (.+)$/m);
    if (match && match[1] !== entityRow.name) {
      return [{ table: 'agent', pk: { id: entityRow.id }, set: { name: match[1] } }];
    }
    return [];
  },
}

// Control via reconcile options:
await db.reconcile(dir);                             // reverse-sync enabled (default)
await db.reconcile(dir, { reverseSync: 'dry-run' }); // detect but don't modify DB
await db.reconcile(dir, { reverseSync: false });      // skip entirely

Lattice stores SHA-256 hashes of rendered content in the manifest. On the next reconcile, it compares hashes to detect modifications. Only files with a reverseSync function are checked — files without it are overwritten as before.

File loopback (v3.4+)

When the GUI is serving a workspace, editing a rendered .md context file on disk is captured back into the database through the normal write path — so the change lands in the changelog (versioned and undoable) and shows up live, exactly as if it had been made in the GUI. Frontmatter and body key: value fields round-trip automatically; an edit that can't be safely parsed (free-form or custom render) is surfaced as a notice rather than guessed at, so a lossy render can't corrupt a row. Render echoes are suppressed via the manifest, so there is no write loop.since v3.4

// Round-trip frontmatter + body `key: value` edits from the rendered
// tree back into the DB. Changelog-aware: pass `apply` to route each
// update through a versioned write, `useDefault` to round-trip without a
// hand-written reverseSync. Render-written files are recognized as echoes.
const result = await db.reverseSyncFromFiles('./context', { useDefault: true });
console.log(result.filesChanged, result.updatesApplied);

Seamless GUI auto-update (v3.4+)

Launched from an npm install, lattice gui runs as a small supervisor that silently installs the latest published version before opening, and keeps checking in the background. When a new version lands it relaunches the server on the same port; the open tab reconnects, notices the version changed, and reloads onto the new build — no manual refresh or reinstall. A git checkout or npx copy is left untouched.since v3.4

New GET /api/version and GET /api/update/status report the running version and the update state.

Writeback pipeline

The writeback pipeline watches files that agents write to and parses their output back into the database. You define which files to watch, how to parse them, and where to store the results.

db.defineWriteback({
  // Watch all SESSION.md files across agent directories
  file: './context/agents/*/SESSION.md',

  // Parse new content since last offset
  parse(content, fromOffset) {
    const newContent = content.slice(fromOffset);
    const entries = parseMarkdownItems(newContent);
    return { entries, nextOffset: content.length };
  },

  // Save each entry to the database
  async persist(entry, filePath) {
    await db.insert('event', {
      source_file: filePath,
      ...(entry as Record<string, unknown>),
    });
  },

  // Prevent duplicate processing
  dedupeKey: (entry) => (entry as { id: string }).id,
});

// db.sync() renders context AND processes writeback files
await db.sync('./context');

By default, writeback offsets are held in memory and lost on restart. For persistence across restarts, plug in a SQLite state store:since v0.12

import { createSQLiteStateStore } from 'latticesql';

// Offsets and dedup keys survive process restarts
const stateStore = createSQLiteStateStore('./state.db');

db.defineWriteback({
  file: './context/agents/*/SESSION.md',
  stateStore,
  parse(content, fromOffset) { /* ... */ },
  async persist(entry) { /* ... */ },
});

SESSION.md pattern

SESSION.md is a convention for agent-written output. Context files generated by Lattice are read-only. SESSION.md is the one file per entity directory where agents can write structured entries that get ingested back into the database.

Agents write structured entries in this format:

---
type: write
timestamp: 2026-03-25T10:30:00Z
op: update
table: agent
target: agent-id-here
reason: Completed deployment task.
---
status: idle
last_task: deploy-api
===

Lattice ships parsers for this format:

import { parseSessionWrites, parseSessionMD, applyWriteEntry } from 'latticesql';

// Parse write entries only
const result = parseSessionWrites(sessionContent);
for (const entry of result.entries) {
  // entry.op, entry.table, entry.target, entry.fields, entry.reason
  const applied = applyWriteEntry(db.db, entry);  // pass raw SQLite connection
}

// Parse all entry types (events, learnings, writes, etc.)
const all = parseSessionMD(content, startOffset);
// all.entries: SessionEntry[], all.lastOffset: number

Write hooks

Run code after a record is inserted, updated, or deleted. Useful for keeping denormalized fields in sync, fan-out updates, or computed columns.since v0.10

db.defineWriteHook({
  table: 'agent',
  on: ['insert', 'update'],
  watchColumns: ['team_id', 'division'],  // only fire when these change
  handler: (ctx) => {
    // ctx.table, ctx.op, ctx.row, ctx.pk, ctx.changedColumns
    updateTeamCounts(ctx.row.team_id);
  },
});

Hook errors are caught and routed to error handlers — they never crash the caller. Multiple hooks per table are supported.

Token budgets

Limit the rendered output of a table to a token budget. When content exceeds the budget, rows are pruned by priority and a truncation footer is appended.since v1.3

db.define('tickets', {
  columns: { id: 'TEXT PRIMARY KEY', title: 'TEXT', updated_at: 'TEXT' },
  render: (rows) => rows.map((r) => `- ${r.title}`).join('\n'),
  outputFile: 'TICKETS.md',
  tokenBudget: 4000,            // max estimated tokens (~4 chars/token)
  prioritizeBy: 'updated_at',   // keep most recent rows when pruning
});
// Output: "- Fix auth bug\n- Deploy v2\n\n[truncated: 47 of 123 rows rendered, ~3800 tokens]"

prioritizeBy accepts a column name (sorted descending) or a (a, b) => number comparator. When omitted, rows at the end of the query result are dropped first.

Relevance filtering

Dynamically filter rows based on the current task context. Only relevant rows are rendered.since v1.3

db.define('knowledge', {
  columns: { id: 'TEXT PRIMARY KEY', topic: 'TEXT', body: 'TEXT' },
  render: (rows) => rows.map((r) => `## ${r.topic}\n${r.body}`).join('\n\n'),
  outputFile: 'KNOWLEDGE.md',
  relevanceFilter: (row, ctx) =>
    ctx ? String(row.body).toLowerCase().includes(ctx.toLowerCase()) : true,
});

// Set the current task — only matching rows are rendered
db.setTaskContext('deployment');
await db.render('./context');

// Clear context — all rows rendered again
db.setTaskContext('');

Enrichment pipeline

Transform rows between filtering and rendering. Add computed fields, cluster by category, or summarize large datasets — without modifying the underlying data.since v1.3

db.define('incidents', {
  columns: { id: 'TEXT PRIMARY KEY', severity: 'TEXT', title: 'TEXT', created_at: 'TEXT' },
  render: (rows) => JSON.stringify(rows, null, 2),
  outputFile: 'incidents.json',
  enrich: [
    // Add computed age field
    (rows) => rows.map((r) => ({
      ...r,
      _age_hours: Math.round((Date.now() - new Date(r.created_at).getTime()) / 3600000),
    })),
    // Summarize if too many rows
    (rows) => rows.length > 100
      ? [{ _summary: `${rows.length} incidents, ${rows.filter(r => r.severity === 'P0').length} critical` }]
      : rows,
  ],
});

Reward memory

Track which data is useful to your agents. High-reward rows are prioritized in rendering; low-scoring rows can be auto-pruned via soft-delete.since v1.3

db.define('tips', {
  columns: { id: 'TEXT PRIMARY KEY', tip: 'TEXT', deleted_at: 'TEXT' },
  render: (rows) => rows.map((r) => `- ${r.tip}`).join('\n'),
  outputFile: 'TIPS.md',
  rewardTracking: true,   // auto-adds _reward_total, _reward_count columns
  pruneBelow: 0.3,        // soft-delete rows with reward < 0.3
});

await db.init();
const id = await db.insert('tips', { tip: 'Use batch inserts for bulk data' });

// After the agent confirms this tip was useful
await db.reward('tips', id, { relevance: 0.9, accuracy: 1.0 });
// _reward_total = 0.95, _reward_count = 1

// Second signal — running average
await db.reward('tips', id, { relevance: 0.5 });
// _reward_total = 0.7, _reward_count = 2

Semantic search

Enable embedding-based semantic search on any table. Bring your own embedding function — Lattice stores vectors in a companion SQLite table and computes cosine similarity in JS. No external vector database required.since v1.3

import { Lattice } from 'latticesql';

db.define('docs', {
  columns: { id: 'TEXT PRIMARY KEY', title: 'TEXT', body: 'TEXT' },
  render: (rows) => rows.map((r) => `## ${r.title}\n${r.body}`).join('\n\n---\n\n'),
  outputFile: 'DOCS.md',
  embeddings: {
    fields: ['title', 'body'],
    embed: async (text) => {
      const res = await openai.embeddings.create({
        input: text, model: 'text-embedding-3-small',
      });
      return res.data[0].embedding;
    },
  },
});

await db.init();
await db.insert('docs', { title: 'Deploy guide', body: 'How to deploy to production...' });

// Search by meaning, not keywords
const results = await db.search('docs', 'ship to prod', { topK: 5, minScore: 0.7 });
for (const { row, score } of results) {
  console.log(`${score.toFixed(2)} — ${row.title}`);
}

As of 5.0, a native vector index accelerates search() and hybridSearch() and keeps itself in sync with writes — see Native vector search for tuning, ops commands, and cloud-member behavior.

since v5.7 This no longer has to be done in code. fts: and embeddings: are config keys, so a workspace driven from YAML can turn search on — see Retrieval from the workspace file.

Writeback validation

Validate agent-written data before persisting. Reject low-quality or inconsistent entries with scoring and threshold-based gating.since v1.3

db.defineWriteback({
  file: './agent-output/*.md',
  parse: (content, offset) => ({
    entries: [content.slice(offset)],
    nextOffset: content.length,
  }),
  persist: async (entry) => { /* save to DB */ },
  validate: async (entry) => {
    const text = entry as string;
    const hasFields = text.includes('## Title') && text.includes('## Body');
    return {
      pass: hasFields,
      score: hasFields ? 0.9 : 0.1,
      reason: hasFields ? undefined : 'Missing required sections',
    };
  },
  rejectBelow: 0.5,
  onReject: (entry, result) => {
    console.warn(`Rejected: ${result.reason} (score: ${result.score})`);
  },
});

Connect Claude

Open Settings → User → Assistant. The primary action (v3.3+) is Connect with Claude — an Authorization-Code + PKCE flow that links your Claude Pro / Max / Enterprise subscription, so the assistant runs on your own account with no API key to paste. It works out of the box (the public OAuth client is built in) and uses a loopback callback derived from the GUI's own origin. Prefer a raw key? Expand Advanced — use an API key instead and paste an Anthropic API key (or set ANTHROPIC_API_KEY); keys are stored encrypted in the native secrets entity. Every endpoint/client value is overridable via ANTHROPIC_OAUTH_*.

since v5.7 None of this requires a browser on the machine you are configuring. lattice model subscription prints the URL to approve — open it in a browser anywhere — and lattice model code <code> finishes with the code that page showed. An OpenAI-compatible endpoint connects with lattice model connect and the key piped in on --key-stdin. See A machine with no display.

Chat + cross-turn memory

The rail runs a Claude tool-calling loop streamed over SSE. The model can list, read, create, update, link, delete, and revert rows in the active database — deletes are guarded and reversible, restored from version history just like a manual delete. Every edit goes through the same audited, undoable mutation path as a manual edit — it lands in the activity feed and the version history and can be reverted. Conversations persist in the native chat_threads / chat_messages entities.

Replies render inline object-link pills: when the assistant references a record it retrieved, the [label](lattice://<table>/<id>) link becomes a clickable pill that opens that row via the mode-aware navigator (it only links ids it actually saw, preferring the user-facing record). Each data change the assistant makes surfaces as an activity card in the live feed — an operation icon, a human summary, and a duration — collapsed by type across objects (“Deleted 19 tables”, “Removed 49 rows across 9 tables”). Read-only calls produce no card, and the feed is per-conversation: each turn persists its data-change events and replays them when you reopen the thread.

The assistant remembers what it read across turns: earlier tool calls and their results (row ids included) are replayed into the model context, so a follow-up such as “now update that row” reuses the id it just listed instead of guessing. Replay is bounded to recent turns within a size budget and is secret-redacted; set LATTICE_CHAT_REHYDRATE=false to disable it. Reads are deterministically ordered, so listing a table twice returns the same rows.

The assistant knows the record you're viewing: when a file or row detail is open, the chat passes it as context, so “delete this file” or “summarize this” act on it directly. It is a hint only — every action still flows through the same permission-gated tools. It can also answer questions about Lattice itself (e.g. “what is private mode?”) via a lattice_help tool that searches Lattice's own documentation rather than guessing or querying your data.

The Context Constructor (file & text ingest)

Drag files onto the rail, click the upload button, or paste text or a URL. Each source is referenced, not copied (a native files row), extracted (text directly; PDF, Word, PowerPoint, Excel, OpenDocument, EPUB, and RTF parsed natively in-process — no external CLI; images by Claude vision; a pasted URL is crawled for readable text), summarized with Claude Haiku, and organized — classified against your existing records and linked, auto-creating the files_<entity> junction when none exists. A source that fits nothing (at higher aggressiveness) becomes a new notes object. New objects, enrichment, links, and junctions are all reversible via the version history.

Library API

The same intelligence is a first-class, GUI-independent API (inert without an LLM client), importable from latticesql:

import { organizeSource, describeImage, crawlUrl, enrichKnowledge } from 'latticesql';

// Sort a source into your own schema: summarize + classify + link,
// creating a new object only when nothing fits. Inert without an LLM client.
const result = await organizeSource(db, {
  text: 'Acme Consulting — signed MSA, net-30, effective 2026-01-01',
  client, // your LlmClient
});

Plus the summarizeText / classifyLinks primitives. sharp + file-type (images) and jsdom + @mozilla/readability (crawler) are optional, lazily-loaded deps.

Inference Aggressiveness

A single Conservative ↔ Aggressive slider (Settings → Assistant) tunes how much the assistant extrapolates: the model sampling temperature, how liberally the ingest classifier proposes links, and whether ingest auto-creates a missing junction (gated at ≥ 0.25) versus only suggesting it. Default 0.5; settable via PUT /api/assistant/aggressiveness.

Voice (optional)

The composer's mic dictates on-device — speech is transcribed in your browser by Whisper (WebAssembly), so it needs no API key or setup and audio never leaves your machine. There is no voice-provider choice in the UI; the mic is shown whenever a microphone is available (disabled with a tooltip when none is). Keyed cloud transcription (OpenAI Whisper / ElevenLabs) stays available to API callers via POST /api/assistant/transcribe for backward compatibility.

New tools (v3.4+) — get_row_context & add_column

Two new assistant tools reduce round-trips and allow schema edits from chat.since v3.4

Prepared statement cache

since v1.4

Lattice automatically caches compiled SQLite prepared statements. Repeated calls with the same SQL reuse the compiled statement instead of recompiling on every invocation. DDL statements (CREATE, ALTER, DROP, PRAGMA) bypass the cache. The cache clears automatically on close() and after schema or migration changes. No API changes required.

Batch entity query resolution

since v1.4

Entity context rendering pre-fetches related rows for all entities in a single WHERE IN (...) query per source, replacing the previous per-entity query pattern. hasMany, manyToMany, and belongsTo sources are batched automatically. custom and enriched sources fall back to per-entity resolution. IN clauses are chunked at 500 parameters to stay under SQLite's limit. No API changes required.

Render change detection

since v1.4

Lattice tracks per-table write version counters. Use isDirty() to check if any table has been written to since the last render, and markDirty(table?) after escape-hatch writes.

// Custom polling loop that skips redundant renders
setInterval(async () => {
  if (db.isDirty()) {
    await db.render(outputDir);
  }
}, 5000);

// After direct DB writes, mark dirty
db.db.prepare('UPDATE tasks SET status = ?').run('done');
db.markDirty('tasks');

Migration validation

since v1.4

Pass a validateMigrationSQL function in InitOptions to validate all pending migration SQL before any migrations execute. If validation fails, no migrations run and an error is thrown. Multi-statement migration SQL is fully supported.

await db.init({
  migrations: [
    { version: 1, sql: 'ALTER TABLE tasks ADD COLUMN due_date TEXT' },
  ],
  validateMigrationSQL: (sql) => {
    if (sql.trim().length === 0) return { valid: false, errors: ['Empty SQL'] };
    return { valid: true };
  },
});

Reports

Build time-windowed reports from your data. Useful for daily summaries, activity digests, or status dashboards.since v0.14

const report = await db.buildReport({
  since: '24h',   // or '8h', '7d', or an ISO timestamp
  sections: [
    { name: 'tasks', query: { table: 'task' }, format: 'count_and_list' },
    { name: 'events', query: { table: 'event', groupBy: 'type' }, format: 'counts' },
    { name: 'alerts', query: { table: 'event',
      filters: [{ col: 'severity', op: 'lte', val: 2 }] }, format: 'list' },
  ],
});

report.sections.forEach(s => console.log(`${s.name}: ${s.count}`));

Markdown helpers

Helper functions you can use inside render functions to reduce boilerplate.since v0.6

import { frontmatter, markdownTable, slugify, truncate } from 'latticesql';

// YAML frontmatter with auto timestamp
frontmatter({ agent: 'Alice', skill_count: 5 });

// Markdown table from rows
markdownTable(rows, [
  { key: 'name', header: 'Name' },
  { key: 'status', header: 'Status', format: (v) => String(v || '—') },
]);

// URL-safe slug
slugify('Alice Smith');   // 'alice-smith'

// Truncate to character budget
truncate(longContent, 4000);

Auto-update

For applications that manage their own updates at runtime, autoUpdate() checks npm for a newer version and installs it automatically. Call it once at startup, before initializing Lattice.since v1.1

import { autoUpdate } from 'latticesql';

// Call at app startup — checks npm, installs if outdated
const result = await autoUpdate();
if (result.restartRequired) {
  process.exit(0); // Let process manager restart
}

Safe to call on every startup — skips if already on the latest version. Pass { quiet: true } to suppress console output.

interface AutoUpdateResult {
  updated: boolean;
  packages: Array<{ name: string; from: string; to: string }>;
  restartRequired: boolean;
}

Events

Subscribe to lifecycle events for monitoring and audit logging.

db.on('audit',     ({ table, operation, id, timestamp }) => { /* ... */ });
db.on('render',    ({ filesWritten, filesSkipped, durationMs }) => { /* ... */ });
db.on('writeback', ({ filePath, entriesProcessed }) => { /* ... */ });
db.on('error',     (err: Error) => { /* ... */ });

audit events fire on every insert/update/delete for tables listed in security.auditTables.

The three flows: migrate, join, invite

There are exactly three things you do with a cloud: migrate into one, join an existing one, or invite someone to yours.

• Migrate — point a local Lattice at a fresh, empty Postgres. Lattice copies your data in, installs RLS, and stamps you as the owner of every migrated row.
• Join — redeem the email-bound invite token the owner sent you (your email + the token). The token decrypts locally to your scoped credentials and connects directly; the member UI never handles a connection string, and there is no server to sign into.
• Invite — an owner (whose role holds CREATEROLE) provisions a scoped, NOSUPERUSER member role and mints a single, email-bound token carrying its credential to hand the new member.

All the cloud helpers import from latticesql. Migrate a local Lattice into a fresh cloud, then force RLS on each table:

import {
  Lattice,
  openTargetLatticeForMigration,
  migrateLatticeData,
  installCloudRls,
  enableRlsForTable,
  archiveLocalSqlite,
} from 'latticesql';

const encryptionKey = process.env.LATTICE_ENCRYPTION_KEY;
const cloudUrl = 'postgres://alice:secret@cloud.example.com:5432/app';

const source = new Lattice({ config: './lattice.config.yml' }, { encryptionKey });
await source.init();

const target = await openTargetLatticeForMigration('./lattice.config.yml', cloudUrl, encryptionKey);
await migrateLatticeData(source, target); // → { tablesCopied, rowsCopied }

// Owner-side RLS install. Protecting a table also records ownership of the
// rows it already holds, so there is no separate step to run first.
await installCloudRls(target);
for (const table of target.getRegisteredTableNames()) {
  if (table.startsWith('__lattice_')) continue;
  const pk = target.getPrimaryKey(table);
  if (pk.length === 0) continue; // unkeyable table — no per-row RLS
  await enableRlsForTable(target, table, pk);
}
target.close();

archiveLocalSqlite('./data/app.db'); // renames to .db.local-bak

since v5.7 backfillOwnership is no longer exported, and this loop used to call it before enableRlsForTable. Delete that call — recording ownership is now part of protecting the table, so there is no step left to run first. The remaining call keeps the arguments it always took, and a table protected by the old two-step sequence needs no repair. This matters more than it looks: an ESM import { backfillOwnership } from 'latticesql' now fails to load the module before any of your own code runs.

since v5.7 All three flows are also commands — lattice cloud migrate, lattice cloud join, lattice cloud invite — with the connection string and the invite token read from standard input rather than an argument. See Administering a shared workspace.

A member joins by connecting directly as their scoped role — that is the whole credential. probeCloud confirms the target is reachable and is already a Lattice cloud (RLS installed):

import { Lattice, probeCloud } from 'latticesql';

const url = 'postgres://lm_bob_a91c:the-password@cloud.example.com:5432/app';

const probe = await probeCloud(url);
// → { reachable: true, dialect: 'postgres', isCloud: true }
if (!probe.reachable) throw new Error(probe.error);
if (!probe.isCloud) throw new Error('Not a Lattice cloud yet — ask the owner to migrate into it.');

const db = new Lattice(url);
await db.init();
const visibleItems = await db.query('items'); // RLS-filtered to what this role may see

An owner invites a member by provisioning a scoped role and handing over its credentials. Member roles are created NOSUPERUSER NOCREATEDB NOCREATEROLE and added to the lattice_members group; the generated password is shown once:

import { Lattice, memberRoleName, generateMemberPassword, provisionMemberRole, revokeMemberRole } from 'latticesql';

// owner connection — must hold CREATEROLE
const db = new Lattice('postgres://alice:secret@cloud.example.com:5432/app');
await db.init();

const role = memberRoleName('bob');        // e.g. 'lm_bob_a91c' — collision-safe, ≤63 bytes
const password = generateMemberPassword(); // 48 hex chars
await provisionMemberRole(db, role, password);
// Hand off:  host / port / dbname  +  user=role  +  password — that blob IS the invite.

// Removing a member drops the role (their rows persist but become unreachable):
await revokeMemberRole(db, 'lm_bob_a91c');

since v4.0 The members group is now per-cloud, derived from the database and schema, so unrelated clouds that share one Postgres cluster no longer share a members group. Library consumers resolve it with memberGroupFor(db); this replaces the previous exported MEMBER_GROUP constant.

Sharing: private by default

Every row is private to its owner the moment it's written — the per-table trigger stamps visibility = 'private'. The owner opts a row into wider visibility: everyone (every member sees it) or custom (the owner plus an explicit grant list). Sharing runs through the owner-only SECURITY DEFINER function — Postgres raises for anyone who isn't the row's owner:

-- Make one row visible to every member of the cloud:
SELECT lattice_set_row_visibility('items', 'item-42', 'everyone');

-- Take it private again:
SELECT lattice_set_row_visibility('items', 'item-42', 'private');

-- Or grant just one member (sets visibility = 'custom'):
SELECT lattice_grant_row('items', 'item-42', 'lm_bob_a91c');
SELECT lattice_revoke_row('items', 'item-42', 'lm_bob_a91c');

From the library, setRowVisibility validates private | everyone before calling the function. The primary key is the row's canonical key string (composite keys are TAB-joined). Because sharing lives in out-of-band bookkeeping, opting a row in or out never touches your table's columns:

import { Lattice, setRowVisibility } from 'latticesql';

const db = new Lattice('postgres://alice:secret@cloud.example.com:5432/app');
await db.init();

// Only the row's owner may call this; Postgres raises otherwise.
await setRowVisibility(db, 'items', 'item-42', 'everyone');

Per-column audiences (experimental)

RLS is whole-row. A layered primitive takes it to the cell: declare an audience on a column and Lattice generates a cell-masking view <table>_v beside the base table. The audience is a +-joined (OR) set of clauses — role:<name>, subject:<col>, source:<col>, or everyone — each compiling to a session_user-keyed SECURITY DEFINER predicate.

person:
  fields:
    id: { type: uuid, primaryKey: true }
    name: { type: text }
    comp: { type: text, audience: 'subject:subject_role+role:hr' }
    subject_role: { type: text }

Members SELECT the view (base SELECT is revoked); a masked cell reads as NULL. App roles are owner-managed via lattice_assign_role(member, role) — members can't self-promote. A column with no audience behaves exactly as before. Off by default.

S3-backed file bytes (opt-in)

A files row is shared by RLS like any other row, but by default a file's bytes live only on the uploader's local disk — so another member can SELECT the row yet can't fetch the content. Enabling S3 for the cloud closes that gap: uploaded bytes also go to an S3 bucket under a content-addressed (<prefix>/<sha256>) key, and any member who can see the row pulls them down in the viewer.

Access rides entirely on the files-row RLS — the serve route does db.get('files', id) as the member's own scoped role, so a row RLS won't let them read returns 404 before S3 is ever touched. The bucket credential is least-privilege — GetObject + PutObject, no ListBucket, no Delete — and is per-member, machine-local, and encrypted (it never lives in the shared database). This is app-mediated access control; see the full caveats in docs/cloud.md.

Chat system prompt (owner-set)

A cloud owner can set a chat system prompt that's bundled into every member's assistant chat for that workspace — house style, domain facts, a fiscal calendar, whatever the team should always have in context. It's stored in the shared DB (__lattice_cloud_settings) and reached through owner-gated SECURITY DEFINER helpers; members never see the control or the prompt value through the UI or API. On a local SQLite workspace there are no members and nothing to keep secret, so the editor is hidden.

Realtime + offline editing

A per-row AFTER INSERT trigger fires pg_notify('lattice_changes', …) carrying only metadata (table, pk, op) — never row content — so a connected GUI can refetch the affected row through RLS. SQLite databases are unchanged; LISTEN/NOTIFY is Postgres-only.

Offline editing is preserved as a client-side local edit queue: edits made while disconnected are held locally and replayed when you reconnect. It is a client behavior only — not tied to any replica or sync server (there is no server). On reconnect the queued writes go to the cloud as your role and land under the same RLS rules as any other write.

Per-viewer rendered context (v3.4+)

On a cloud, the background render reads every table through the member's row-level-security connection and per-column masking view. The rendered markdown a member's assistant reads off disk contains only the rows they may see, with owner-only columns blanked and any per-viewer enrichment folded in; it re-renders promptly when sharing changes. Owners and local single-user workspaces render the full tree unchanged.since v3.4

Cloud resilience & search (v3.4+)

since v3.4

Full-text search on migrated clouds: when you migrate a local Lattice into a cloud, the full-text index is now built automatically, so search works immediately. Call db.rebuildFtsIndexes() on any cloud to rebuild from scratch.

Open-time convergence is per-table fault-isolated: when opening a cloud, if a table is unmanageable (owned by a different Postgres role), it is skipped with an actionable reason instead of failing the entire workspace. Warnings are surfaced via GET /api/dbconfig as convergeWarnings. Use POST /api/workspaces/reload to re-register entities without restarting.

Raw Postgres URL healing: a raw postgres://user:password@host/db string in a workspace config is healed on open into an encrypted ${LATTICE_DB:label} credential reference, so the secret no longer lingers in cleartext on disk.

GUI cloud endpoints

lattice gui drives all three flows from the browser. The relevant endpoints are localhost-only, same model as the rest of the GUI:

v3.4 additions:

The full architecture — the RLS / role model, the three flows in detail, the S3 and system-prompt designs, per-column audiences, and the sharing API — lives in docs/cloud.md.

Machine-local config at ~/.lattice/

Files outside any Lattice DB, so switching projects doesn't cost your identity or your scoped credentials:

Native secrets + files entities

Every Lattice opened by lattice gui automatically registers two framework-shipped tables. Available to any Lattice via the public helper:

import { Lattice } from 'latticesql';
import { registerNativeEntities } from 'latticesql/framework/native-entities';

const db = new Lattice(
  { config: './lattice.config.yml' },
  { encryptionKey: process.env.LATTICE_ENCRYPTION_KEY },
);
registerNativeEntities(db);
await db.init();

// 'secrets.value' is encrypted at rest:
const id = await db.insert('secrets', {
  name: 'OPENAI_API_KEY',
  kind: 'api-key',
  value: 'sk-...',           // stored as 'enc:<base64>' in SQLite
});
const row = await db.get('secrets', id);
console.log(row.value);      // → 'sk-...' (decrypted)

Encryption uses a new TableDefinition.encrypted field — the same shape as the existing entity-context option, now extended to plain define() tables. attachBlob(srcPath, latticeRoot) writes any file into a content-addressed store at <root>/data/blobs/<sha256> and returns metadata suitable for a files row. On a cloud with S3 enabled, those bytes also go to the bucket.

Computed tables

A computed table is a live, read-only view built from the tables you already have. You describe what each field is — a copied column, a calculation, an AI-derived value, or a total across linked rows — and Lattice compiles that description into a SQL VIEW and registers it as a queryable table. The values are never copied: every read reflects the current state of the records the view is built from. Everything is additive and opt-in — a workspace with no computed: section behaves exactly as before.

Definitions live in a top-level computed: section of lattice.config.yml, next to entities:. Each computed table has one base table (a declared entity or another computed table) and a set of named fields; the view always projects the base's primary key as id first:

computed:
  ticket_summary:
    base: tickets
    fields:
      title: { kind: alias, source: title }
      team: { kind: alias, source: assignee.team.name } # tickets → people → teams
      is_urgent: { kind: calc, expr: 'priority >= 3', type: boolean }
      sentiment:
        kind: ai_classify
        input: body
        prompt: How does the customer feel in this message?
        labels: [happy, neutral, frustrated]
      tag_count: { kind: aggregate, via: ticket_tags.tag, fn: count }

The five field kinds

AI fields are materialized once — and never stale

AI fields never re-run a model at read time: outputs are materialized once into bookkeeping tables the view LEFT JOINs, so reads are always deterministic SQL. A fill pass runs when a definition with AI fields is created or changed, and on demand (Refresh values in the builder). An AI value is cached against the exact input values it was derived from — when a source row changes, the join misses and the field reads blank for that row until the next refresh: you may see a gap, but never a stale value. Unchanged rows keep their cached values (no re-billing for what's already known), and a changed prompt, label set, input list, or model invalidates exactly that field's cache — through any edit path, including a hand-edited config.

Building one in the GUI

The Tables explorer's Computed Tables column carries a + New button that opens the full-page builder (#/computed/new): name the view, pick the base table, and add fields — each row is a name plus a kind (Copy a field, Calculation, AI category, AI text, Total across links) with pickers listing every reachable column, including dotted paths through linked tables. Preview dry-runs the definition against your live schema and shows up to 20 sample rows — each field stamped ✓ or ✕, with the compiled SELECT under a collapsed Definition (SQL) block — and nothing is created or saved by a preview. Create registers the view and persists the definition to the workspace config; editing an existing view adds Refresh values (streaming per-field progress) and Remove. A hand-edited config and a GUI-built view are the same thing. The explorer draws a dashed connector from the base table to the computed view and lists the base as upstream lineage.

Read-only, audited, revertible

A computed table's rows are derived, not authored: the GUI marks the view with a Computed badge, shows where the values come from, and offers no editing affordances — and the server refuses any direct write with a message pointing at the source tables or the definition. Deleting or renaming a source table is refused while a computed table reads from it. Create, update, and delete are audited and land in version history — undo re-creates a removed view from its captured definition. A definition that fails to compile is fault-isolated: recorded and reported, never blocking the workspace from opening. On a team cloud, the view compiles with per-viewer row-visibility predicates, members are granted SELECT, and definitions publish through the shared schema — mutating verbs are owner-only.

The assistant drives the same loop through four chat tools — preview_computed_table, create_computed_table, update_computed_table, refresh_computed_table — preview-first, with every mutation audited and undoable exactly like a builder action. Over HTTP, the GUI server exposes GET/POST/PUT/DELETE /api/computed-tables[...] plus /preview, /fields?base=, and /:name/refresh (an NDJSON progress stream). For library consumers, the compiler and fill engine are public API: ComputedTableDef, compileComputedTable, registerComputedTables, runComputedFill, parseCalcExpr / emitCalcExpr, and Lattice.isComputedTable / getComputedTableNames.

Analytics view & dashboards

The app splits into two views. Analytics is the new landing surface: a Dashboards sidebar, a tab strip of open dashboards, and the assistant docked on the right. Configure is the Inputs · Model · Outputs workspace. The top-right header button toggles between them, and each side remembers its last location; boot and workspace switches land on Analytics.

Dashboards are a first-class object. A dashboard is a live visual page — charts, tables, key numbers — authored from a plain-language spec via the assistant's create_dashboard / edit_dashboard tools, stored in a native dashboards table, rendered in a sandboxed, no-network frame, and shareable per-row exactly like any record. The page body is writable only by the authoring tools — no other write path can plant executable content — and is redacted from assistant reads. Each open dashboard is a closable, deduped tab; when the strip can't fit, trailing tabs collapse into an overflow menu that always keeps the active tab visible.

Dashboards are live and self-healing. A page reads its data at load time through the sandboxed bridge — including lattice.sql(...), a read-only, capped, single-SELECT surface for aggregations — so a dashboard always shows current data, never a snapshot. When the data model changes underneath one (a rename, a delete, a merge — from the assistant or the schema tools), every consuming dashboard is re-authored against the new schema automatically in the background: each repair lands as an ordinary activity-feed update and the open page live-reloads; a repair that cannot run keeps the previous page and says so.

The docked assistant is quieter for non-technical users: it discusses only what goes into a dashboard and what it shows, structural work happens silently behind a single transient status line, and a plain-text answer with no dashboard is a first-class outcome. Existing assistant-authored HTML pages migrate from files into dashboards on the next open (same id — sharing grants and ownership preserved); the old floating assistant panel is retired in favor of the Analytics dock.

Native vector search substrate

Self-maintaining. Once a native vector index exists, inserts, updates, and deletes mirror the affected row into it incrementally (on the same background path as the embedding write); refreshEmbeddings reconciles the index after a bulk backfill; and search() verifies the index is in sync with the stored vectors before using it, otherwise falling back to the exact in-process scan — so a drifted index is never silently served: at worst a slower query, never a wrong result. The SQLite-side sqlite-vec index build is atomic, so an interrupted build can't leave a half-filled index that looks complete.

Tunable. Optional knobs, all defaulting to prior behavior: embeddings.index = { m, efConstruction } sets the pgvector HNSW build parameters, and search() / hybridSearch() accept efSearch for query-time search breadth. Opt-in half-precision index storage via embeddings.index.quantization = 'halfvec' (pgvector ≥ 0.7) roughly halves the index's memory while the embeddings store stays full precision, so the scan fallback remains exact.

Observable. An internal registry records each built index's dimension, parameters, source count, and build time; an auto-rebuild after a bulk refresh reuses the recorded parameters. Ops CLI: lattice reindex <table> rebuilds, lattice index status reports per-table health, and lattice doctor --fix rebuilds any index it reports as stale.

Cloud-safe member search. Semantic and hybrid search now work for scoped cloud members, confined to the rows they may see: the vector arm reaches the embeddings store only through a SECURITY DEFINER function that returns just the chunk vectors for visible rows, scored in-process. The member scan is exact (no recall loss) and has no over-fetch channel by which a member could infer the existence of hidden rows. Routing is automatic — owners and local callers are unchanged.

Big changes, undone in one step

The assistant makes changes of any size, as long as they can be undone. Clearing a field across thousands of records, unlinking, merging, cleaning up duplicates, deleting records to the trash — all of it goes ahead, and all of it comes back with a single Undo. A change made across many records is one entry in your history, so reversing it is one action rather than a repair job.

What matters is whether a change can be taken back, not how big it is. A recoverable change to ten thousand records is safer than an unrecoverable change to ten, so that — not a record count — is where the line sits.

The one thing it still declines is a permanent removal: a delete that does not go to the trash, spanning several objects or a large number of records. That is refused outright, before anything happens, and it tells you what it was about to touch so you can do it deliberately yourself. There is no “are you sure?” to talk it around — approval is never inferred from a conversation.

Clarification questions

Ask when marginal, act when confident. One threshold — the machine-local clarify threshold, default 0.6, settable via PUT /api/assistant/clarify-threshold — governs when an automated inference asks instead of guessing: at or above the threshold it acts silently; between the floor (threshold ÷ 2) and the threshold it asks a short, information-seeking multiple-choice question (always with a free-form “Other”); below the floor it drops the inference as noise.

Questions are always about what the data means or is for, never about storage mechanics — and answers are enrichment, not just disambiguation: an informative answer is persisted onto the object it describes (as a table or column definition, a row value, or lineage detail), so the knowledge outlives the conversation. Pending questions surface in one place — interactive cards above the composer in the Analytics assistant dock, with a notification dot on the header trigger while the view is hidden. Answering executes the question's deferred action and enrichment writes through the audited mutation paths; a failed execution leaves the question pending with the error shown on the card.

since v5.7 The dock is no longer the only place they surface. lattice questions list / answer / dismiss — and listPendingQuestions / answerQuestion / dismissQuestion from the package — drain the same queue, so an automated pipeline is not blocked indefinitely on a question only a browser could reach.

The assistant also gains an in-turn ask_user tool for genuine uncertainty about intent, and background producers use the same channel: file-ingest object extraction asks (at most 2 questions per ingested file) instead of creating anything when its target-entity decision is marginal, and the structured importer's link inference queues questions for marginal links — see Import intelligence.

Import intelligence

Excel formula capture. Reading a workbook now also summarizes each column's formulas (per-sheet, per-column normalized patterns plus an example). Cell values still import from the cached formula results exactly as before — the formula text feeds only the computed-field proposals below.

Computed fields are detected and built automatically. An import creates them for you — nothing to tick: a calc field when a spreadsheet column is computed by one dominant row-local formula (≥ 90% of the column's rows) that translates into the sandboxed calc grammar, and — sparingly — a classifier field for a category-named text column, seeded with a starter label set from the most frequent values (at most 1 per table and 3 per import, with no model calls at proposal time). They are created after import as live computed tables named <entity>_computed through the same audited path as the builder; the raw source columns import as plain values regardless, and a computed-create failure warns without failing the import.

Link inference: act, ask, or drop. A reference column whose values resolve to another table at or above the clarify threshold is linked exactly as before. Candidates between the floor and the threshold — which previously auto-created a junction — are no longer created: the column imports as a plain scalar and a short clarification question is queued instead (at most 5 per import, highest confidence first). Answering “Yes, connect them” creates and fills the junction from the already-imported rows (idempotent); “No” or dismissing does nothing; a free-form answer is saved as the column's definition. Below the floor, candidates are dropped as noise.

Databases & MCP servers as inputs

Connect an external database. A credential connector imports an external Postgres-family database (AWS RDS Postgres, Supabase, or generic Postgres): it introspects the schema and imports the tables as connected data types under the Source tier, with single-column foreign keys materialized as graph edges so rows arrive already linked. Imports are bounded (keyset/offset paged with a hard page cap), imported table names are namespaced per connection, and credentials plus the introspected schema live only in the machine-local encrypted store.

Read-only, enforced in depth. A connected database is a data source — Lattice must never be able to write to it. Every pooled connection starts with default_transaction_read_only = on (the server itself refuses writes), and the connection wrapper additionally refuses any non-read statement (SELECT/WITH/SHOW/EXPLAIN only) before it touches the network. The connect dialog takes host/port/user/password/database fields only — raw connection strings are not accepted — and recommends a read-only database user.

since v5.7 Attaching one is scriptable. lattice connector connect-database takes the same fields as separate flags — never a connection string — with the password piped in on --password-stdin; connector list, sync, reconnect-database, and disconnect tend it afterwards, and a sync in which every source failed names each one and exits non-zero. Authorizing an MCP server that uses OAuth is deliberately not a command — see What still needs a person.

MCP-backed connectors. Connectors are powered by the Model Context Protocol: Lattice runs as a local MCP client and pulls a server's read tools in as connected data types. Everything runs on your machine — a remote server is reached over Streamable HTTP or SSE with that server's own OAuth (tokens stored in the machine-local encrypted store), and a local server runs as a stdio child process; nothing is routed through any cloud middleman. Ships with Gmail, Google Calendar, and Google Drive (typed schemas), Jira and monday.com (pre-pointed at their hosted endpoints), Trello, and a generic “custom MCP server” connector you point at any URL. Connector data keeps the same conventions as before — typed connected tables, per-member private visibility, full-text search, graph edges, and rendered context.

GUI: Inputs · Model · Outputs, graph & auto-update

The Configure view is three columns. Inputs (files, connectors, databases), Model (a Graph tab and a Tables tab — a tiered schema explorer organized as Inputs / Derived Tables / Computed Tables, with field-level detail, lineage, and relationship edges), and a Markdown view of the rendered context tree that mirrors the Tables list exactly. A Folders view shows objects as folders whose rows appear as files; Wire links two tables many-to-many and Merge moves one object's rows into another — by click-to-pick or drag, audited and restorable from history, with inbound links carried across inside a single transaction.

Live force-directed graph. A dependency-free force-layout engine (many-body repulsion, degree-biased link springs, collision resolution, alpha-cooled integration — DOM-free and unit-testable) drives a live SVG renderer with continuous animation, drag-to-pin, pan, pinch/wheel zoom, neighbor highlight, zoom-to-fit, and incremental fly-in growth as objects are created during ingestion. It powers every GUI graph surface, and the graph is clamped to the viewport so panning or dragging can never push it off-screen.

Data provenance for every object. GET /api/provenance?table=<t> (and a per-row variant) traces where an object's data came from across three tiers — raw (uploaded files, connectors), computed (Lattice-created artifacts, imports), and observation (AI / learning-loop edits). An object's page defaults to this provenance view, and a row's detail view gains a collapsed, lazy-loaded “Data provenance” panel. Provenance is backed by a dedicated __lattice_lineage table plus a source column on the audit log, so lineage survives re-renders. Reads are bounded.

Auto-update, visible on every surface. GET /api/update/status reports a surface-aware action — upgrade-in-place for an npm install, restart-to-update for the desktop app — so the version chip shows the right affordance everywhere, and the desktop app surfaces an “Update available — Restart to update” hint (a read-only probe of the release manifest; nothing downloads or relaunches until you act). To pin the running version — for testing, air-gapped, or reproducible-demo runs — pass --no-auto-update (or set LATTICE_NO_AUTO_UPDATE=1, or autoUpdate: false on startGuiServer): the update poll never runs. A development / linked checkout is badged vX.Y.Z (dev) so a stale dev build can't be mistaken for an auto-updating install.

Also in 5.0: all data — the files table, connector-synced tables, and imported database tables — renders real per-record markdown contexts (with secrets and conversation tables hard-excluded at the derivation); render progress is shown per file in the Markdown tree; records, files, and artifacts share one page with a Formatted | Markdown toggle whose Markdown side is an editable, round-trippable textarea; undo/redo state is computed with bounded queries; the GUI shell is served compressed; and the local server gains CSRF / DNS-rebinding hardening (state-changing requests require a same-origin request and a loopback Host; binding to a non-loopback address requires an explicit --allow-remote opt-in).

Lattice Cloud (hosted)

Prefer not to self-host? Lattice Cloud runs it for you. Sign in at latticedesktop.com with Google or email — every workspace is a cloud workspace (there are no local accounts in the hosted product), backed by a managed Postgres with the same Row-Level Security and scoped roles described above. The GUI, workspaces, and external-database connections are exactly the open-source experience, hosted.

Sharing is account-mediated. Invite a collaborator by email; they accept by signing in with their own Lattice Cloud account, and Row-Level Security confines each member to the rows they own or are granted — no connection strings are ever shared.

Lattice tokens instead of an API key. The hosted assistant runs on prepaid Lattice tokens — no Claude API key to bring or manage. Add credit with a card; usage is metered per turn at cost plus a small margin, and your balance is shown in Billing.

Getting started

1. Sign in at latticedesktop.com with Google or an email address — a workspace is created for you automatically on first sign-in.
2. Open it to launch your hosted GUI — the same interface as the desktop app, already connected to your managed Postgres. External databases connect the same way they do locally.
3. Invite collaborators by email from your account; each accepts by signing in with their own account and is Row-Level-Security-scoped to the rows you grant.
4. Add credit in Billing to enable the assistant. Turns are metered against your Lattice-token balance; at a zero balance the assistant pauses until you top up, while the rest of the workspace keeps working.

A machine with no display

Start with a root, not a directory scan. A root is never picked up by searching upward from the current directory — --root is the whole answer, and every command that operates on a workspace accepts it. That is deliberate rather than strict: the master key and the encrypted credential store are the most sensitive things on the machine, and inferring them from whatever root happened to sit above the working directory would let a leftover checkout hand a process working credentials for a database it was never meant to reach. The root anchors that store too, so it does not come from the surrounding environment either.

npx lattice init --root /srv/lattice --name Ops

Which model answers is a property of the machine. It is stored encrypted, every surface reads the same one, and these verbs work before any workspace exists — which is when you need them. An endpoint is asked to answer before it is kept, so a wrong URL or a dead key changes nothing.

# An OpenAI-compatible endpoint. The key goes in on standard input:
# passed as an argument it would be readable from the process list and
# kept in your shell history.
printf '%s' "$OPENAI_API_KEY" | npx lattice model connect \
  --base-url https://api.openai.com/v1 \
  --model gpt-4o-mini \
  --key-stdin

npx lattice model status
npx lattice model status --json

A Claude subscription or a Lattice account needs a person to approve a consent screen — but not a browser on this machine. Start it here, approve the printed link in a browser anywhere, paste the code back:

npx lattice model subscription        # prints a URL to approve
npx lattice model code <code>         # finish it with the code that page showed

npx lattice account signin            # same shape for signing the machine in
npx lattice account code --code-stdin < code.txt

model use picks which configured backend is active (anthropic or openai_compat), model test asks it to answer once, and model disconnect drops an endpoint, account, or subscription. account code is the one place a one-time code has a --code-stdin form; model code takes it as an argument.

Getting data in

# A spreadsheet, CSV, or JSON export becomes real tables and rows.
npx lattice import /srv/exports/accounts.csv --dry-run   # writes nothing
npx lattice import /srv/exports/accounts.csv

# Documents. A folder can be registered as a source, or walked once.
npx lattice ingest /srv/handbook              # register it, then walk it
npx lattice ingest /srv/handbook --once       # walk it without registering
npx lattice ingest sources                    # what is registered here
printf 'Renewal terms are net-30.\n' | npx lattice ingest --stdin --title "Renewal terms"

An import never overwrites the last one. A file with no date of its own is filed under today's, so re-running next month appends a snapshot rather than clobbering what is there. --as-of sets the file-level date explicitly and --as-of-column names the column the source dates each row by. --mode schema creates the tables without the rows, --sheet picks one sheet of a workbook, and --dry-run reports what it would create and writes nothing.

Workspaces, databases, and shape

npx lattice workspace list
npx lattice workspace create Staging
npx lattice workspace use Staging
npx lattice workspace rename Staging --name "Staging EU"
npx lattice workspace delete "Staging EU" --yes

npx lattice database list
npx lattice database create archive
npx lattice database delete archive --yes

npx lattice schema link deals --to accounts     # → deals.accounts_id
npx lattice schema links                        # prints the reference unlink takes
npx lattice schema unlink deals.accounts_id
npx lattice schema describe deals.amount --text "Contract value in USD."

--yes is a flag, not a prompt, on purpose. These run unattended, and a prompt in that setting is a hang rather than a safeguard — so the confirmation is something a script states up front. Both delete verbs resolve the named thing before they check the flag, and deleting the last database in a workspace is refused outright. Unlinking is soft: the column and its values stay, so the change reverts from version history and the command returns an undo id. An empty --text "" clears a definition; omitting --text is a usage error rather than a silent no-op.

Gating a deploy on health

doctor exits non-zero when the report contains an error-severity issue, so it works as a CI or deploy gate. It also fails honestly now: a database nothing was assessed on reports nothing_to_diagnose at error severity, instead of reading an empty expectation list as a clean bill of health. A schema that was fully read and simply configures no search reports no_retrieval_configured at info and stays healthy — those are different facts and they now look different.

npx lattice doctor --config /srv/lattice/Workspaces/Ops/workspace.yml
npx lattice doctor --json
npx lattice update --check    # reports the published version; never installs

Administering a shared workspace

A cloud is a shared Postgres database secured by row-level security — see Cloud for the model. There is no server to run: permission is the Postgres role you connect as, so every verb below is refused by the database itself when you are not allowed it. The commands bind no port and open no browser.

Secrets go in on standard input, or in the environment. A connection string carries the owner password and an invite token decrypts to a database login — both are readable from the process list by anyone on the machine, and both are kept in shell history, if you pass them as arguments. Use --url-stdin / --token-stdin, or set LATTICE_CLOUD_URL / LATTICE_INVITE_TOKEN.

The lifecycle, in the order you actually perform it. On the owner's machine:

# 1. Check the database before pointing anything at it.
npx lattice cloud probe --url-stdin < db-url.txt
# Reachable:  yes
# Database:   postgres
# Cloud:      no — nobody has secured this database as a cloud yet

# 2. Move this workspace onto it. Data is copied in, row security is
#    installed, and the local database is kept alongside as a backup.
npx lattice cloud migrate --url-stdin < db-url.txt

# 3. Confirm where you stand.
npx lattice cloud status
# Connected:  <your role>
# Security:   installed
# You are:    the owner — you can invite and remove

# 4. Invite somebody. The token is printed once and is bound to that email.
npx lattice cloud invite --email teammate@example.com

On the member's machine. Redeeming an invite creates a new workspace pointed at the shared database. The token decrypts locally to that member's own scoped database login, so a connection string is never handed over and never has to be:

npx lattice cloud join --token-stdin --email teammate@example.com < token.txt

npx lattice cloud status
# You are:    a member — you see the rows shared with you

Back on the owner's machine — deciding what each member can see, and removing them when they leave:

# Grant one row to one member, or set its audience outright.
npx lattice cloud share --table accounts --pk <row-id> --to teammate@example.com
npx lattice cloud share --table accounts --pk <row-id> --visibility everyone
npx lattice cloud share --table accounts --pk <row-id> --visibility private

# Take a grant back.
npx lattice cloud share --table accounts --pk <row-id> --to teammate@example.com --revoke

npx lattice cloud members
npx lattice cloud revoke teammate@example.com   # by role, email, or display name

cloud secure is the entry point for a Postgres you already have data in — it turns that database into a cloud rather than migrating a local workspace onto it. It is owner-only and idempotent: run twice and the second run re-converges row security and member access rather than failing. cloud revoke drops the member's role, so their credential stops working; their rows persist and become unreachable.

status, members, and probe take --json, which is the form to branch on in a script:

npx lattice cloud status --json
# {
#   "dialect": "postgres",
#   "secured": true,
#   "role": "…",
#   "standing": "owner",
#   "warnings": []
# }

Every verb also takes --config <path> (defaulting to the active workspace) and --root <dir>, so one host can administer several workspaces without switching the active one.

Asking the assistant from a script

lattice ask runs one full assistant turn — the same assistant the app runs, over the same tools, against the same workspace, with the same permissions and the same refusals. Nothing about it is a reduced version. It answers and it acts: a change it makes is audited and reverts from version history exactly like one made by hand.

npx lattice ask "Which accounts renewed this quarter?"

The answer goes to stdout and nothing else does. Which tools ran goes to stderr, so watching a turn work costs nothing on the pipeline and the answer alone survives a redirect:

#!/usr/bin/env bash
set -euo pipefail

# Progress on stderr stays visible; only the answer lands in the file.
npx lattice ask "Summarize this week's new accounts." \
  --root /srv/lattice > /srv/reports/weekly.txt

# Or take the whole turn as data — the answer plus every tool it ran.
npx lattice ask "How many accounts renewed?" --json > /srv/reports/turn.json

The exit code is the whole signal, so it is strict. Zero means the turn did the job it was given. It is non-zero when the turn failed, when no model is connected, when the assistant asked a clarifying question instead of acting (nobody is there to answer it, so the job will not become done), when it ran out of tool rounds with work still outstanding, when the outcome ledger has something to report about what did not happen, and when there is no answer at all. The reason is always written to stderr as well, so a person watching sees which of those it was.

A turn that asked a question leaves it queued. That queue used to be reachable only in the browser, which meant an automated pipeline could be blocked on it indefinitely:

npx lattice questions list
npx lattice questions list --json
npx lattice questions answer <id> --text "Renewals are counted on the signature date."
npx lattice questions dismiss <id>

The same capabilities as functions

Every command above is a thin wrapper over an exported function. An embedder calls the function directly — no HTTP, no server process, no subprocess. Open a workspace with openConfig, hand its primitives to runAssistantTurn, and dispose of it when you are done:

import { openConfig, disposeActive, runAssistantTurn } from 'latticesql';

const active = await openConfig(
  '/srv/lattice/Workspaces/Ops/workspace.yml',
  '/srv/lattice/Workspaces/Ops/Context',
);

try {
  const result = await runAssistantTurn(
    {
      db: active.db,
      feed: active.feed,
      validTables: active.validTables,
      junctionTables: active.junctionTables,
      softDeletable: active.softDeletable,
      configPath: active.configPath,
      outputDir: active.outputDir,
      // Undo is scoped to the session that authored the write, so keep this
      // stable if a later process should be able to reverse what this one did.
      sessionId: 'nightly-report',
    },
    { message: 'How many accounts renewed this quarter?' },
  );

  if (!result.ok) throw new Error(result.error);
  // A turn cut off at the step cap still produces prose. Do not report success on it.
  for (const warning of result.warnings) console.warn(warning);
  console.log(result.text);
  console.log(result.tools.map((t) => t.name)); // what it actually ran
} finally {
  await disposeActive(active);
}

streamAssistantTurn is the same turn as an async generator, yielding the event sequence a browser receives — for a caller that wants to show progress. runAssistantTurn is that generator drained.

The cloud verbs are functions too. These take a Lattice connected as the role whose permission applies, which is what makes them safe to expose:

import { Lattice, probeCloud, cloudStatus, listCloudMembers, shareRow } from 'latticesql';

const url = process.env.LATTICE_CLOUD_URL!;

const probe = await probeCloud(url);
// → { reachable: true, dialect: 'postgres', isCloud: true }
if (!probe.reachable) throw new Error(probe.error);
if (!probe.isCloud) throw new Error('Not a cloud yet — the owner has to secure it.');

const db = new Lattice(url);
await db.init();

await cloudStatus(db);
// → { dialect: 'postgres', secured: true, role: '…', standing: 'owner', warnings: [] }

await listCloudMembers(db);   // the roles on this cloud, with their standing

const [row] = await db.query('accounts', { limit: 1 });
await shareRow(db, { table: 'accounts', pk: row.id, visibility: 'everyone' });

await db.close();

The rest of the surface follows the same shape: secureCloud, inviteMember, redeemCloudInvite, removeMember, grantRowAccess, migrateWorkspaceToCloud for the cloud; ingestPath / ingestText / readImportSource / applyImport for data; connectDatabaseSource / refreshStaleSources for connectors; createDatabase / deleteDatabase / renameWorkspace / deleteWorkspace for the workspace registry; addUserLink / removeUserLink / setColumnMeta for shape; listPendingQuestions / answerQuestion / dismissQuestion for the clarification queue; and checkForNewerVersion for the update check.

A refusal arrives as an Error carrying a code, not an HTTP status. That is what lets one function serve a request, a command, and a library call without any of them re-deriving the rule. Read it with workspaceErrorCode, ingestErrorCode, connectorErrorCode, or assistantTurnErrorCode — each returns the code, or undefined when the error came from somewhere else, so an unrelated failure is never swallowed:

import { deleteDatabase, workspaceErrorCode } from 'latticesql';

try {
  // configPath: any config in the workspace. target: the one to remove.
  await deleteDatabase({ configPath, target });
} catch (e) {
  if (workspaceErrorCode(e) === 'last_database') {
    // "Cannot delete the only database." — remove the workspace instead.
    return;
  }
  throw e; // not ours; do not absorb it
}

The codes are small, closed sets: invalid_request, not_found, last_database for workspaces; too_large, outside_roots, source_unreachable among those for ingest; setup_failed, import_failed, source_rejected among those for connectors; and no_model_connected / empty_request for an assistant turn — the two situations a caller must distinguish before a turn starts, which is why they are thrown rather than returned as an outcome.

Retrieval from the workspace file

since v5.7 Search settings can be written in the config. Before this they were reachable only from define(), so anyone running Lattice from a YAML config had search permanently off with no way to turn it on — see Semantic search for the code form.

entities:
  article:
    fields:
      id: { type: uuid, primaryKey: true }
      title: { type: text, required: true }
      body: { type: text }

    # Opt into the full-text index. `true` indexes the entity's text columns;
    # name them yourself when the default is not what you want.
    fts: true
    # fts:
    #   fields: [title, body]

    embeddings:
      fields: [title, body]
      url: https://api.openai.com/v1/embeddings
      model: text-embedding-3-small
      apiKeyEnv: EMBEDDINGS_API_KEY
      # Optional: timeoutMs, chunk: { maxChars, overlap, minChars },
      # maxScanChunks, index: { m, efConstruction, quantization }

There is no default endpoint. Embedding a field means sending its contents somewhere, and that is never implied — url is required. The key is named by environment variable and never written in the file; a config that puts one in embeddings.apiKey is rejected with exactly that reason, because config files are shared and versioned.

npx lattice doctor --config ./workspace.yml
#   article: rows=0 fts=100% emb=100%

npx lattice search "renewal terms" --table article --topk 5 --explain

What still needs a person

Three things genuinely cannot be done by a command, and it is worth being exact about which:

• Applying a desktop update — it needs the desktop shell that restarts itself. lattice update --check reports what is published and what this copy could do about it, and never installs.
• An operating-system file picker — it needs a person at a screen. That is precisely why every other door takes a path instead.
• A provider's consent screen — a subscription, an account sign-in, an OAuth MCP server. But this is not a requirement on the machine being configured: start it there, approve in a browser anywhere, paste the code back. Authorizing an OAuth MCP server is deliberately not a command, because it ends at a consent page and there would be nothing for the command to finish; the library is explicit about it, with connectSource handing back the URL to approve and stopping, and completeMcpConnection finishing once the code comes back.

What that breaks, concretely. An upsert of a row that is not already there — and most visibly, a first sync of a connected external source into a shared workspace, which lands zero rows, because a sync upserts every incoming row. It reaches every scoped member, and any owner whose Postgres role does not carry the privilege that bypasses row security. The owner case has a workaround — ALTER ROLE <role> BYPASSRLS, which itself requires a superuser. There is no workaround for a member, and that privilege must never be given to one: bypassing row security is exactly what keeps one member's rows out of another member's reach.

This behavior is inherited, not new — 5.7 is where it was written down. One thing did change, and it cuts both ways: a table created outside the browser, by a script or a command or an embedder, is now secured the moment it is registered. Previously it stayed unsecured until an owner next opened the browser app, so a table made any other way was born readable by everyone. That exposure is closed — and it also means an upsert into such a table, which used to succeed precisely because the table had no row security on it, now meets the refusal above.