GrayMatter

Retrieval coverage and contribution evidence explains complete-list discovery, explicit reuse versus write verification, and receipt-to-decision-to-artifact-to-test reporting without invented savings.

GrayMatter Lite is a real open-source memory product for one person or one workspace. It runs locally or on infrastructure you control and includes the same useful product loop from the first launch: sign in, create durable memory, retrieve it through MCP, import or export portable KnowledgePacks, and connect local or hosted agent profiles.

It is not a time-limited trial and it is not a hollow demo. The Lite boundary is the canonical ThorAPI YAML under openapi/bundles, ./vaix builder, Spring/H2 backend, embedded dashboard, MCP server, starter KnowledgePack, Docker definition, tests, and public documentation in this repository.

Choose hosted or local memory

Start with the setup page. GrayMatter Cloud is the recommended default: use your valkyrlabs.com account and the native plugin sign-in to connect hosted memory. Tenant and account permissions are enforced by the hosted service.

Local GrayMatter Lite needs no valkyrlabs.com signup. In ValorIDE, click Connect Local GrayMatter Lite on the welcome screen (or use that command from the command palette), then select this source folder if asked. ValorIDE starts Lite and verifies the local account and MCP memory tools. Choose Ollama, LM Studio or your own model provider separately. The first source build may need toolchain and dependency downloads; installed local memory works offline.

For other clients, use the source commands below. Hosted workflows, ecommerce and application hosting use their authenticated ValkyrAI services.

Install local Lite in one command

On macOS or Linux:

git clone https://github.com/ValkyrLabs/GrayMatter.git
cd GrayMatter
./vaix setup

./vaix setup uses Java 17+, Maven, and Node 20+ already on the machine when possible. Missing toolchains are downloaded privately under .vaix/runtime; nothing is installed system-wide. The command builds standalone Lite using public Maven dependencies, creates one local profile, starts the services, and verifies the connection. A hosted account or private ThorAPI generator is not required. Explicit schema generation remains available through ./vaix generate. Setup starts:

• dashboard and sign-in: http://localhost:8787
• HTTP MCP: http://localhost:3333/mcp
• durable H2 data: .graymatter-lite/data

Retrieve the generated local credentials only when you need them:

./vaix credentials
./vaix doctor

Other source commands:

./vaix generate     # compose canonical YAML and generate Spring + TypeScript
./vaix generate --extension ./application-domain.yaml
./vaix regenerate   # generate, clean-build, and run the acceptance suite
./vaix build        # build the Spring/H2 backend
./vaix test         # backend, bootstrap, docs, release parity, and MCP contracts
./vaix run          # foreground backend
./vaix up           # background backend + HTTP MCP
./vaix stop

See Schema regeneration for the authoritative files, generated/handwritten boundary, extension contract, and failure rules.

Docker

The Docker path builds from the committed source; it does not depend on a closed prebuilt GrayMatter backend image:

export GRAYMATTER_ADMIN_PASSWORD='choose-a-strong-local-password'
docker compose -f deploy/docker-compose.lite.yml up --build

The same dashboard is available on port 8787 and MCP on 3333. H2 data is kept in the graymatter-lite-data volume.

What is included

• one local user/workspace with Basic-auth sign-in and a switchable local profile;
• durable MemoryEntry creation, read, hybrid search, and H2 persistence;
• local vector indexing with optional loopback Ollama embeddings, plus bounded Bifrost context, citation pointers, source rechecks, and retrieval receipts;
• the existing Valkyr dashboard, memory workbench, telemetry, and SWARM status;
• the bundled stdio/HTTP MCP server for Codex, OpenClaw, Claude, local-model hosts, and other MCP-compatible clients;
• signed .gmkp KnowledgePack import plus whole-memory KnowledgePack export;
• a vetted starter KnowledgePack with GrayMatter, Valkyr SWARM, ValkyrAI, and ThorAPI setup/product/support knowledge, inserted idempotently into local H2;
• local and hosted named profiles plus explicit read-only blended retrieval;
• source builds through ./vaix and container builds through Docker Compose;
• AGPL-3.0 source, public docs, Issues/Discussions support, and a private security-reporting path.

The legacy environment and filesystem identifiers retain LIGHT for backward compatibility. The public product name is GrayMatter Lite.

Profiles and blended memory

./vaix setup registers graymatter-lite-local without changing an already active hosted identity. Profiles store routing metadata in profiles.json; hosted sessions remain in the platform credential vault and local Basic-auth passwords live in a mode-0600 secret file outside the repository.

scripts/gm-profile add-local graymatter-lite-local \
  --api-base http://localhost:8787/v1 --password-stdin
scripts/gm-profile add cloud --from-current
scripts/gm-profile use graymatter-lite-local
scripts/gm-profile blend graymatter-lite-local cloud
scripts/gm-query "release rules"

Blended reads execute independently under each profile and preserve profile and account-fingerprint provenance. Every write fails closed until a single profile is selected. MCP exposes the same read-only federation for memory query/read/health tools; mutating or unsupported tools return a read-only recovery result until one profile is selected. Restart the MCP process after changing the persistent profile selection.

Local models

GrayMatter is model-neutral. Ollama, LM Studio, llama.cpp, and other local-model hosts connect through MCP while the model process remains separate from the H2 memory and authorization boundary. See Local models for stdio and HTTP examples. Local Workflow execution uses two MCP boundaries: Valkyr SWARM owns registration and signed Workflow coordination; GrayMatter owns memory, context, and durable evidence. Model-only LM Studio/Ollama nodes are signed-workflow-only inference workers, never general command runners.

Imports, exports, and starter knowledge

Use the dashboard or API:

curl -u "admin:$GRAYMATTER_ADMIN_PASSWORD" \
  -o graymatter-lite-memory.gmkp \
  http://localhost:8787/v1/knowledge-packs/export

scripts/gm-knowledge-pack-import ./graymatter-lite-memory.gmkp

The starter pack source is committed at templates/graymatter-light-bootstrap/local-server/src/main/resources/knowledgepacks/graymatter-lite-starter.json. Startup signs it with the same self-contained integrity contract used by normal KnowledgePacks, imports it through the production importer, and stores its records in the local H2 database. Repeated starts are idempotent.

Valkyr SWARM

The starter KnowledgePack explains the supported SWARM path. Install the peer open-source product when this machine should register, heartbeat, receive exact-target control commands from an authenticated ValkyrAI mothership, and return durable receipts:

git clone https://github.com/ValkyrLabs/ValkyrSWARM.git
codex plugin marketplace add "$PWD/ValkyrSWARM"
codex plugin add valkyr-swarm@valkyr-swarm

SWARM preserves the canonical human approval gates for outbound sends, production deploys, merges, and supervised service restarts. GrayMatter Lite does not invent a second command bus or allow an agent to approve itself.

Documentation and community

• GrayMatter Lite guide
• Local model compatibility
• KnowledgePacks
• MCP server
• Architecture
• Contributing
• Community support
• Security policy

Use GitHub Issues for reproducible defects and GitHub Discussions for support, ideas, KnowledgePacks, and integrations. Report vulnerabilities privately as described in SECURITY.md.

---

GrayMatter Platform: the AI brain for your entire business

Valkyr GrayMatter™ turns your applications, documents, workflows, conversations, and institutional knowledge into a living, searchable intelligence layer.

It is more than a vector database and more than chat memory. GrayMatter combines durable memory, real-time search, structured business data, and relationship-aware reasoning into one secure system that agents, people, apps, and APIs can use together.

A memory system that actually remembers context

GrayMatter stores durable decisions, preferences, tasks, research, procedures, conversations, and operational knowledge as structured, attributable records—not disposable chat history. Every memory can carry source, scope, tags, relationships, timestamps, ownership, and provenance, so the system can distinguish a personal preference from a company policy, a current task from historical context, or a source document from an AI-generated summary.

It supports agentic memory across Codex, OpenClaw, SageChat, workflows, and connected tools, giving every authorized agent a shared understanding of the business without flattening everything into an untraceable prompt.

Hybrid search: keyword, vector, graph, and structured data

GrayMatter search is designed to answer both simple and difficult questions:

• Exact search for names, IDs, tags, commands, products, and records
• Full-text search across documents, titles, fields, and parsed uploads
• Semantic/vector search for concepts, intent, and meaning
• Relationship-aware search across ThorAPI entities and knowledge-graph links
• Structured filtering by type, date, category, tags, tenant, workflow, project, and more
• Retrieval receipts with citations, evidence, quality signals, and answer-policy guidance

That means a user can search the public website, an authenticated workspace, or SageChat through the same intelligence foundation—while each surface receives only the information it is permitted to see.

PostgreSQL-native vector intelligence

GrayMatter uses PostgreSQL as the durable operational foundation and pgvector as a high-performance semantic-search accelerator.

SemanticIndexEntry remains the portable canonical record, while tenant-scoped pgvector projections provide fast nearest-neighbor retrieval. The platform supports configurable embedding providers, including production OpenAI embeddings and deterministic local fallback embeddings for offline/development environments. It also tracks model, provider, dimensions, index health, stale rows, reindex guidance, and degradation state.

This gives teams a practical path from ordinary relational data to production-grade semantic recall without adopting a separate proprietary graph or vector silo.

A real knowledge graph, built from ThorAPI

GrayMatter understands that knowledge is not a pile of documents—it is a network.

ThorAPI entities and their OpenAPI-defined relationships can be indexed after successful writes, producing safe semantic evidence across workflows, applications, customers, tasks, files, procedures, agents, goals, notes, and more. GraphLink records make important relationships explicit: supports, blocks, references, derives from, relates to, and beyond.

This enables relationship-aware retrieval such as:

• “What is related to this customer issue?”
• “Which workflow depends on this integration?”
• “What evidence supports this decision?”
• “Which documents, memories, and tasks explain this result?”

The system is designed for bounded graph traversal, citations, and future graph analytics without allowing the index or graph to bypass ThorAPI authorization.

SageChat and document intelligence

SageChat can turn uploaded files and parsed documents into searchable evidence rather than leaving them stranded in storage.

Files are processed into bounded, provenance-rich semantic material: source file, parser output, chunks, pages or offsets, source hashes, extraction quality, and relationships to the business objects they support. GrayMatter can then surface that evidence in SageChat with citations and explainability—helping users understand not only an answer, but where it came from.

Plugins, skills, and MCP: intelligence everywhere

GrayMatter is packaged as an installable plugin and skill system for Codex/OpenClaw-style agents. It supports secure, platform-vault-backed authentication, agent registration, schema awareness, durable memory reads and writes, graph access, retrieval receipts, and operational health checks.

Its MCP capabilities make GrayMatter available to compatible AI clients through typed tools such as memory write, memory query, memory read, retrieval with receipt, graph inspection, schema discovery, and authorized entity access.

Plugins and skills give each agent the same durable organizational context; MCP makes that context usable from external AI environments and workflows.

Credits designed for real AI operations

GrayMatter includes a credit and entitlement model for higher-order memory and AI operations. This supports sustainable usage of semantic retrieval, embeddings, reindexing, and advanced intelligence features while retaining clear operational controls.

The platform distinguishes public-safe experiences from private operational tooling, so commercial or account-management detail does not leak into public AI surfaces.

Secure by design

GrayMatter does not treat search as a shortcut around security.

Generated ThorAPI RBAC and ACL remain the source of truth for every result, snippet, count, facet, graph path, file citation, and recommendation. Tenant context is resolved server-side. Sensitive fields, credentials, binary data, and protected audit information are excluded from indexing. Search, vector retrieval, graph navigation, and agent tools are all designed to fail closed when authorization is uncertain.

The outcome

GrayMatter is becoming the intelligence layer that lets every Valkyr product, workflow, app, agent, and document participate in a shared, secure, searchable brain:

• A memory system for agents and teams
• A knowledge graph for the business
• A hybrid search engine for websites and workspaces
• A vector intelligence layer on PostgreSQL
• A citation-backed research and reasoning engine for SageChat
• A portable, ThorAPI-native platform for plugins, skills, MCP tools, and AI automation

GrayMatter is an installable OpenClaw skill and MCP service for:

• primary durable memory
• shared object-graph state
• live organizational schema awareness through the ValkyrAI api-0 OpenAPI

It lets an agent move beyond local files and isolated chat context. Once authenticated, GrayMatter is the agent's exclusive primary durable memory: it persists durable memory, inspects the live business schema, and operates inside the organization's RBAC-scoped data environment.

ChatGPT developer-mode app

The submission surface is the existing Node MCP adapter in mcp-server/, running in hardened public-app mode. It keeps api-0 as the memory, ContextPage, procedure, receipt, RBAC, ACL, and tenant source of truth.

Public deployment contract:

• Production MCP URL: https://api-0.valkyrlabs.com/graymatter/mcp
• Compatibility URL: https://api-0.valkyrlabs.com/mcp
• Protected-resource metadata: https://api-0.valkyrlabs.com/.well-known/oauth-protected-resource
• Authorization-server metadata target: https://api-0.valkyrlabs.com/.well-known/oauth-authorization-server
• Development MCP URL: http://localhost:3333/graymatter/mcp

The production MCP and OAuth URLs above are deployment targets, not a claim that the reverse proxy and OAuth authorization server are already live. Verify them before creating or submitting the ChatGPT app.

Required public-app environment:

export PORT=3333
export GRAYMATTER_MCP_MODE=hosted-multi-tenant
export GRAYMATTER_PUBLIC_APP=true
export GRAYMATTER_PUBLIC_RESOURCE=https://api-0.valkyrlabs.com
export GRAYMATTER_PUBLIC_MCP_PATH=/graymatter/mcp
export GRAYMATTER_OAUTH_ISSUER=https://api-0.valkyrlabs.com
export GRAYMATTER_OAUTH_JWKS_URI=https://api-0.valkyrlabs.com/oauth2/jwks
export GRAYMATTER_ALLOWED_ORIGINS=https://chatgpt.com,https://platform.openai.com
export VALKYR_API_BASE=https://api-0.valkyrlabs.com/v1
node mcp-server/index.js

Do not configure VALKYR_AUTH_TOKEN, VALKYR_JWT_SESSION, GRAYMATTER_TENANT_ID, or X-Valkyr-Token on the public multi-tenant service. Each request must carry the current user's OAuth bearer token. Public mode validates issuer, audience, lifetime, RS256 signature, required identity claims, and tool scopes; it then forwards only the bearer token to api-0 and never forwards caller tenant or owner identifiers. OAuth metadata, JWKS fetches, and request-body reads inherit the request's shared execution deadline and stop when it expires. HTTP request bodies and local stdio messages fail before JSON parsing above GRAYMATTER_MCP_MAX_REQUEST_BYTES (default 1 MiB, capped at 16 MiB); health and bounded errors return content-free executionLimits.

To connect a developer version in ChatGPT:

1. Deploy the endpoint over HTTPS, or expose the local server with Secure MCP Tunnel or another HTTPS tunnel.
2. In ChatGPT, open Settings → Security and login and enable Developer mode.
3. Open Settings → Plugins and select the plus button.
4. Enter GrayMatter, the description Persistent, secure memory and shared context for AI agents., and the HTTPS /graymatter/mcp URL.
5. Complete OAuth linking and verify that exactly eight tools are discovered.
6. Start a new chat, add GrayMatter from the composer, and run the representative prompts in SUBMISSION_CHECKLIST.md.
7. After tool metadata changes, redeploy and use Refresh on the developer-mode app.

Run the automated public-app contract with two isolated reviewer accounts:

GRAYMATTER_MCP_URL=https://api-0.valkyrlabs.com/graymatter/mcp \
GRAYMATTER_TENANT_A_TOKEN='redacted' \
GRAYMATTER_TENANT_B_TOKEN='redacted' \
GRAYMATTER_TEST_RECEIPT_ID='authorized-receipt-id' \
scripts/smoke-test-public-mcp.sh

Release surfaces

GrayMatter ships as three related but independently usable surfaces:

• MCP service: mcp-server/ runs as an HTTP/SSE service for Claude.ai, Claude Code, Cursor, ChatGPT Apps SDK, and any MCP-compatible host, and also supports node mcp-server/index.js --stdio for plugin-managed MCP launch.
• Codex plugin: .codex-plugin/plugin.json exposes this repo as the graymatter plugin with the standalone skill plus .mcp.json, so Codex can discover both the instructions and the MCP server.
• Standalone OpenClaw skill: graymatter.skill packages SKILL.md and the required scripts for OpenClaw install, activation, hosted api-0 use, and GrayMatter Light local mode.

If a GitHub sparse/root install only brings down root files, run ./graymatter-bootstrap on macOS/Linux or .\\graymatter-bootstrap.ps1 on Windows. Both restore scripts/ and mcp-server/ from the bundled graymatter.skill archive.

Quick start

macOS/Linux:

git clone https://github.com/ValkyrLabs/GrayMatter.git
cd GrayMatter
./install.sh

Windows PowerShell:

git clone https://github.com/ValkyrLabs/GrayMatter.git
Set-Location GrayMatter
.\\install.ps1

The installer automatically connects this checkout to Codex when the Codex CLI is available, installs the plugin, and opens one native GrayMatter sign-in window. macOS uses a single AppKit dialog; Windows uses one WinForms dialog backed by Windows Credential Manager. The first screen offers GrayMatter Cloud (api-0), Local GrayMatter Lite (localhost:8787), Local ValkyrAI (localhost:8080), and Other self-hosted server, with an editable server URL and masked password field. A rejected login returns to the same flow with the username preserved and a clear correction message. ValkyrAI/Cloud passwords are sent only to the selected login endpoint and are never printed or saved; the returned session is stored in the platform credential vault. Local HTTP is allowed only on loopback; remote instances require HTTPS. GrayMatter Lite uses its existing local account authentication and keeps local credentials in a private mode-0600 profile file.

Returning users sign in immediately. New users choose Create Free Account, finish the dedicated GrayMatter Cloud signup page in their browser, then return to the still-open connection window and sign in with the username they created. Recover Account opens the dedicated username/password recovery page. For a local or self-hosted instance, choose its connection before entering credentials. GrayMatter Lite accounts come from ./vaix setup; ValkyrAI accounts come from that instance's signup or administrator. Open Instance opens the selected server instead of Cloud signup. A Cloud account is optional. The selected server is saved as an account profile for the next plugin launch, and its credentials stay separate from hosted accounts. Browser redirects, clipboard tokens, and manual JWT handling are never part of normal sign-in: the native window exchanges credentials with the selected ValkyrAI server and captures its session from the response body, headers, or secure cookies.

Sign-in identity example:

Username: your-username

The happy path deliberately has only four visible stages: downloading plugin, performing signup/login, authenticating, and GrayMatter plugin ready. It does not require jq or manual JWT handling. Advanced OpenClaw operators can run scripts/gm-activate afterward for the complete smoke-test, agent-registration, and schema-sync bootstrap.

scripts/gm-activate is the preferred first-run path. It checks for updates, signs in, stores the session in Keychain when available, validates the install, registers the agent, and writes a bounded startup-preflight artifact after invariant retrieval, authenticated capability discovery, and live OpenAPI freshness checks.

Before task planning, code edits, production-affecting actions, or answers based on project history, agents must immediately run the invariant preflight for the current workspace/product:

scripts/gm-invariant-preflight ValkyrAI signup acl thorapi aspectj

MCP hosts that cannot shell out should call graymatter_invariant_preflight. Returned decision records tagged as invariants, security, RBAC/ACL, generated-code, AspectJ, vaix/vai, testing, or product names are binding operational rules. Missing or degraded retrieval is never permission to ignore known durable rules.

The required preflight is broader than a keyword search. It must look up invariants, rules, instructions, prior session context, personalization, business truth, personal truth, and organizational truth before the agent begins work. New user-provided corrections, procedures, preferences, and invariants must be written to GrayMatter during the session and read back by ID to confirm persistence.

For ValkyrAI, ValorIDE, GrayMatter Light, and ThorAPI-generated application work, agents should prefer repo launchers over direct build shortcuts: ./vaix build, ./vaix test, ./vaix run, and repo-documented ./vai flows preserve ThorAPI generation, AspectJ weaving, heap defaults, local H2/runtime flags, and end-user operational behavior. Signup, ACL/RBAC, and generated API fixes should normally be proven with ./vaix run on localhost:8080 plus the frontend on localhost:5174 before using production only as a comparison point.