Mempersist

Mempersist is a clean-room, Cloudflare-native long-term memory service for AI conversations. It preserves original ChatGPT exports and normalized conversation graphs in R2, catalogs them in D1, builds disposable lexical and semantic indexes, and exposes compact retrieval and intentional writes through MCP.

It does not silently capture ChatGPT traffic, extract replacement “facts,” provide a SaaS billing layer, or make search indexes canonical. “Unlimited” means no application message quota; Cloudflare limits and billing still apply.

Architecture

ChatGPT conversations.json / MCP writes
                  |
          validation + IDs
                  |
          R2 canonical archive  <------ export / recovery
                  |
             D1 catalog
                  |
        Cloudflare Queues
          /             \
   D1 FTS5          Workers AI BGE-M3 -> Vectorize
          \             /
       normalized hybrid search
                  |
       HTTP + OAuth-protected MCP

R2 is the source of truth. D1 holds operational metadata and the derived FTS representation. Vectorize is disposable. A canonical write succeeds before indexing is queued, and an indexing failure never reports that durable memory was lost.

See ARCHITECTURE.md, SECURITY.md, and docs/operations-and-recovery.md.

Prerequisites

• WSL2/Linux, Node.js 22+, Yarn 1.22, and Wrangler 4.x
• A Cloudflare account with Workers, D1, R2, Vectorize, Workers AI, and Queues available
• Wrangler OAuth authentication: yarn wrangler whoami

Use Yarn only.

Setup

yarn install
cp .dev.vars.example .dev.vars
yarn types:bindings
yarn db:migrate:local
yarn dev

Set a long random MEMORY_API_TOKEN in .dev.vars. Local D1 and R2 are simulated; Workers AI and Vectorize bindings are remote in the main configuration. Unit and integration tests do not call remote AI.

Cloudflare provisioning

Provisioning is intentionally manual and must be explicitly authorized. Follow docs/cloudflare-resources.md, then add the real D1 database_id returned by Wrangler to wrangler.jsonc. Never invent IDs or reuse unrelated account resources.

Set the production secret without putting it in source:

yarn wrangler secret put MEMORY_API_TOKEN

Apply migrations and deploy only after review:

yarn db:migrate:remote
yarn deploy:dry-run
yarn deploy

ChatGPT import

Export data from ChatGPT, extract conversations.json, then:

MEMPERSIST_URL=http://localhost:8787 \
MEMPERSIST_TOKEN='your-token' \
yarn import:chatgpt /path/to/conversations.json

Files up to 16 MiB use direct streaming upload. Larger files use 16 MiB R2 multipart parts. The Worker hashes the completed object, preserves it unchanged, detects exact duplicate exports, and processes at most 25 conversations per queue turn. Check progress with:

MEMPERSIST_TOKEN='your-token' yarn import:status <import-id>

See docs/chatgpt-import.md.

MCP

The Streamable HTTP endpoint is https://<worker>/mcp. Interactive clients such as ChatGPT use OAuth 2.1 authorization-code flow with PKCE: the consent page takes an email, sends a single-use magic link, and completes the connection only after the link is opened. Existing emails reconnect to their archive; a new archive is created after the first link. The owner archive is bound to vhie1046@gmail.com across all of its namespaces (personal, astara_alt_v2, coding/mempersist, test/mempersist-blackbox); entering that email reconnects to the same data. Each account can own multiple namespaces, and the same namespace name may exist in different accounts with fully separated data. Developer scripts and the CLI may keep sending MEMORY_API_TOKEN as a bearer token for the owner archive.

Email authentication uses the EMAIL send binding. The primary endpoint sends from AUTH_EMAIL_FROM (noreply@mempersist.codifiedtech.id); the legacy endpoint keeps using LEGACY_AUTH_EMAIL_FROM (noreply@mempersist.nextostaging.net).

The public site, OAuth pages, and magic-link email support English and Bahasa Indonesia. Use the language switcher to persist a browser preference; otherwise MemPersist uses Accept-Language and falls back to English. API, MCP, and CLI contracts remain English.

Dashboard

Open /login for passwordless access to the server-rendered dashboard. It includes archive totals, a deterministic memory map, revision-pinned canonical conversation reading, display-name editing, and a streamed lossless export of current memory. Namespace emptying runs asynchronously. Account deletion has a seven-day cancelable grace period and makes writes read-only while pending.

See docs/dashboard.md and ADR 0028. No frontend framework, extra dependency, or additional Cloudflare resource is required.

For the deployed Worker, add https://mempersist.codifiedtech.id/mcp as a custom MCP app in ChatGPT Developer mode. ChatGPT discovers OAuth automatically, opens the consent page, and stores the issued access/refresh tokens. Existing connections keep working after upgrades without re-authorization. Clients already configured with https://mempersist.nextostaging.net/mcp remain supported; changing one to the primary endpoint requires one new authorization. Do not paste MEMORY_API_TOKEN into ChatGPT's connector settings.

Available tools:

• memory_search
• memory_get_context
• memory_get_conversation
• memory_get_conversations
• memory_list_conversations
• memory_list_revisions
• memory_resolve_conversations
• memory_build_context
• memory_list_namespaces
• memory_stats
• memory_store
• memory_append
• memory_replace
• memory_restore_revision
• memory_copy_conversations

memory_store and memory_append accept optional tags (lowercased, deduplicated, up to 20); memory_search filters by tags with AND semantics and returns each conversation's tags. See docs/mcp.md and ADR 0013.

• memory_delete_conversations
• memory_empty_namespace
• memory_import_status

Search returns compact references; call memory_get_context only for selected results. See docs/mcp.md.

For known memories, memory_get_conversations returns up to 20 ordered compact pages within 48 KiB, including explicit errors and continuations. Single reads accept format: "compact"; canonical output remains the default. memory_resolve_conversations resolves up to 20 known conversation owners by exact title without semantic search, returning conversation IDs, current revision IDs, and live tags. memory_list_revisions returns the immutable revision history of one owned conversation (metadata only, newest first, cursor-paged), so a client can pin and read any earlier revision with memory_get_conversation instead of relying on a retained write receipt. memory_restore_revision restores an owned conversation to any historical revision with base_revision_id optimistic concurrency, recording an immutable head transition without creating redundant canonical revisions. memory_copy_conversations performs a lossless canonical R2 copy of 1–20 owned conversations into another owned namespace using a required idempotency_key and attaching first-class derivedFrom provenance, rather than a compact-message restorable via memory_store. Store/append/replace/restore/copy accept verify: true to reload the committed R2 revision and return compact readback with separate indexing/verification status. See the RP workflow and reviewable runtime-rule amendment.

For prompt and task execution, memory_build_context compiles a deterministic, revision-pinned context pack from 1–20 required canonical conversations (exact title or conversation ID, full or tail mode, active or all branch, and optional follow pointer-expansion fields) and up to 8 optional hybrid-search retrieval requests within caller-specified token and serialized-byte limits (at most 49,152 bytes / 48 KiB). It pins required current revisions before loading canonical bodies, follows exact structured pointers (such as current_scene and active_arc.owner) deterministically without semantic fallback, deduplicates source messages structurally on (conversation_id, revision_id, source_node_id), orders content deterministically by authority (explicit required, then required expanded, then optional expanded, then retrieved), priority, score, and stable IDs, and greedily fits whole messages. If required content alone exceeds either budget, it returns a bounded diagnostic with suggested minimums without leaking canonical text. Context packs are extractive-only, include full message provenance and an optional deterministic compiled text projection, have a deterministic pack_id, and perform no writes or generative summarization.

Quality gate

yarn verify

This runs formatting, lint, strict TypeScript (Worker and the web/mindmap/ browser project), the memory-map bundle freshness check, unit/MCP/retrieval tests, Workers-runtime D1/R2 integration tests, and a Wrangler deploy dry run. No command deploys unless yarn deploy is invoked explicitly.

After changing anything under web/mindmap/, run yarn build:mindmap so the generated src/mindmap-bundle.ts matches its sources.

Operations

yarn admin search 'api.internal.example'
yarn retry <job-id>
yarn reindex
yarn verify:integrity

Reindexing reads canonical R2 data; the ChatGPT export does not need to be uploaded again. D1 migrations are numbered SQL files and must be applied through Wrangler—never by dashboard drift.