Automation vs. Augmentation
Tools for thought are, as you would expect, tools that assist in human thinking. Much of the AI space has been about automating things we already do. But attempting to automate human decision making and our communication seems more deleterious than assistive. Instead of automating our lives, replacing what’s so closely human with computers, what if we used them to augment ourselves, our lives, enrich us, and help us learn and think more clearly? To know ourselves, our patterns, to learn more about our interests. To increase our understanding, in a time when information has never been so much a torrent.
We’re stuck in this ‘perpetual now’, unable to make connections and maintain deep threads. Self-awareness and understanding is being eroded by this relentless flow of information and the automation of deeply human things like decision, thought, and communication.
To be seen, to be recognized, to be understood. It’s not something that most automation software can do for you. Of course they can hit their scratchpad for a profile, or read a directory loaded with facts. But how does it handle evolution, change, connection? How does it understand priorities? How does structure inform its understanding? Does awareness accumulate? Or does it actually emerge from ongoing structure building, from cultivation and curation and maintenance?
What is the point of memory?
A lot of apps offer ‘memory’, and most of them do recall facts pretty well. But the nuance is in how they evolve, and how insight emerges. A lot goes into building a knowledge management system. How it handles change, and how it integrates new memories with existing mental models. Structure brings order to information. It allows things to emerge. Centrality, bridges, paths, and threads emerge over time, not being explicitly coded or defined at any singular point. Can it traverse connections and information, navigate structures, analyze networks, to bring additional dimensions of understanding that go deeper than fact retrieval, but onto being capable of emergence?
The reality is that intelligence is not a monolithic phenomenon. It’s an emergent one. It emerges over time, with experience, and never in isolation. It’s a phenomenon in the mind that emerges from the communication of specialized subsystems, a hybrid of process applied over a substrate. Thinking patterns applied to networks. The mind is also not entirely structural, it’s also associative. It finds patterns with explicit links and relationships, but also through similarity of experiences or memories. Something might remind you of another thing, an experience might feel similar to something you’ve experienced before. You may recall by traversing relationships, causal, or discrete, but you may also feel for similar situations, or find patterns that emerge by similarity. On sight of a leaf’s venation your mind may surface similar expressions by its similarity, rather than its relation to it, like trees and lightning.
User Ownership
There are memory solutions like Supermemory which are still very cool, but being cloud-hosted services you have to actually pay monthly fees to maintain access to your memory bases due to cost structure of hosting.
Your agent memories are yours to accumulate, grow, and tend to. The app’s data is stored locally on your device. Meaning if your subscription stops, you still have what you built. And because it’s not hosted by a central party, you won’t find your memories in a data breach/leak. If you want, you can run scripts to re-structure the data to ‘upload’ into other apps if you so wish. and we don’t offer or require metered inference; if you choose to run inference locally in the app, the data never leaves your device. User ownership is key to trusting that your memories won’t be seen in logs, unsafely stored by third-parties, so you can actually share more and get more out of your memorybases/knowledgebases.
Scaffold is a cognitive infrastructure for you to build continuity on your terms.
Approaches to Memory & Persistence
Common patterns we see are scratchpads and filesystems. e.g. Claude Code’s memory.md, Hermes agent’s memory.md/user.md, Obsidian vaults + Claude Code, etc. But these are primitives developed for programming, for storage, well suited to coding agents. You want keyword search in a codebase because you’re looking for every instance of a function or variable with the same name across a codebase for example. It’s precise, It’s fast, and It’s simple. And as a result, it’s a relatively elegant solution for its purpose.
Keyword search and grep works well for programming usecases because of the clean deterministic nature of code; if you know what you’re looking for you’ll find it. Keywords are brittle in the real world when you need it for a diversity of situations. It’s not possible to know what exact keywords were used to compose a memory or experience.
But Memory is relational, and it’s associative. We use similarity, and relationships, to recall past perceptions and memories in order to navigate the present moment in relation to past experiences. We jump from one concept to the next, linking them together as we find and assemble meaning. These both rely on a navigable space with dimension.
Similarity allows experiences to be found that are similar but not exact in wording like keyword search. That’s well established, but in a memory system it helps in finding similar memories. It lets you find memories by meaning, instead of language used. You can take a retrieved memory and find its neighbours, or use cluster analysis to feel out regions.
In a knowledge graph, its actual structure contains meaning. Relationships let you navigate, understand greater context and relationships, find paths, bridges and connectors, etc. It lets agents build a mental model of sorts. A network of structured information.
Navigability is not at play in filesystems; the structure doesn’t carry much meaning itself. Scratchpads and memory files need size limits; some have maximum amount of content to avoid breaking under scale. Contradictions can be created and kept without reconciliation processes. If you don’t know exactly what you’re looking for, and a memory wasn’t stored with those exact keywords, it fails to find nearest, next best result, and provides no path to getting there other than trying more keywords.
Cognitive Scaffolding
Memory should be a first class object. Its patterns should be fit for purpose. Memory is often associated with retrieval, search and recall, but memory is not a search problem, it’s a coherence problem. Vectors and graphs are small elements of a larger solution.
Failure modes of memory systems; bringing in irrelevant information, having stale information, storing incorrect information and continuing to cite it. These are painful enough or disruptive enough for a lot of people to choose to just disable memory features. But the implementation of it, the actual nuances that go into building these tools or scaffolds for ai, are at fault. Agent continuity is still an unsolved problem.
Intelligence involves knowing what is actually salient enough and worth connecting/bringing in to context. Consistency, congruity, coherence, these are difficult problems in building a long-term memory solution. Memory is often thought of as storage, as accumulation, but it’s way more about integration and reconciliation. Edit/update, marking corrections, finding similar (including opposite takes), and resolving them at write-time preserves reasoning at a later date for the actual task at hand. Contradictions shouldn’t be able to exist in the first place.
Scaffold is built on fundamental insight on how memory works, research dating back to the 70s, Endel Tulving, articulated different memory systems, two of which are episodic and semantic memory. Dated experiences, and structural factual knowledge. These are well represented by vectors for episodic, and graphs for semantic.
Trees and containers don’t organize that, they lack structure they don’t have much dimension compared to the mind that has spaces that can be navigated where structure and dimension carries meaning
Memory accumulates, but it also resolves, reconciles, and integrates with existing understandings. Memory and knowledge and experience evolves over time. And new information is processed in relation to what’s already understood.
Integration & Reconciliation
At write-time, new information being stored should be cognizant of what exists at that location. Memory storage then shouldn’t be blind, it should sample the region it’s being saved to, and reconsidered. If nothing’s there, store it. If something is, consider whether new information is redundant, genuinely new, or whether it can be merged into what’s known already. It should, at write-time, be able to update other similar memories as necessary. Integration and reconciliation means applying multiple levels of inference at the time of writing.
Scaffold takes the proposed episodic memory and vectorizes it, but before saving to the space, it checks first for highly similar pre-existing memories/neighbours to the proposed memory. If there’s nothing in that region, it saves new. But if similar memories are close in proximity or similarity to the new proposed one, then an analyzer agent is presented those highly similar memories alongside the new one, and given the opportunity to skip if redundant, merge if additive, store new if genuinely unique, as well as update fields on neighbors to reflect the new information and refine the boundaries of each respective memory. This means when you save a new coding preference, the agent’s aware of existing preferences when it forms that new memory.
Same for knowledge graph, if entities exist that contain similar keywords, it presents it to a graph analyzer who performs similar integrations, giving it the opportunity to add the new entity/relationship, skip redundant, merge into existing, remove other relationships that may no longer be true, convert existing relationship types based on the new information, or update or append fields of existing. Corrections are appended with dates, enabling the agent to know when it changed its mind or understanding, instead of forgetting what it once thought.
Memory Quality
Write time integration keeps the memorybase/knowledgebase healthy in-use, but memory health management is still beneficial. In the graph for example, supernodes can form as one entity develops a great number of relationships, overwhelming the LLM and reducing structural integrity and purpose of a graph. Commands like /maintenance, /tidy, /connect fire off tasks to either break a super node into major facet nodes and reparent relationships to the facets, preventing overload upon lookup of a heavy node (such as the user-node), meaning the agent can get user’s relationships, find the facet node of let’s say Career, before finding the detailed relationship/node that contains the answer, instead of opening the usernode and getting overloaded with every relationship they have. /Connect finds disconnected or low-connection nodes and creates a task for the agent to integrate them into the graph by creating new nodes and relationships, keeping them connected to the main network so they can be found during traversal. These tasks can run at night, but also can be kicked off by the user.
Scaffold
Scaffold is a specialized system of agents, tools, tasks, and processes to enable agent learning, knowledge and experience. It not only learns from conversation and interaction, but it also intakes multimodal inputs like YouTube videos, blogs, websites, pdfs, markdown files, audio notes, and studies them progressively, saving information, structure, insights, and observations through the process, building a mental model of the world across two parallel subsystems and databases; a vector store and a knowledge graph. It works with local inference, with a few suggested models and allows any huggingface GGUF model to be input to use depending on your hardware, and it supports providers OpenAI, Anthropic, and Openrouter via API keys. Data is stored on your machine, so you don’t have to pay monthly to retain your agent’s knowledge and experience. It exposes the main search, store, and study tools via MCP locally so that programs on your machine like Cursor and Claude Code can delegate and query and access the memories and knowledge base, bringing in cross-application continuity.
The result is an agent with continuity, knowledge, memory, and experiences you can access in app, or across different clients. It can know you personally, it can read books and papers, maintain temporal awareness, track your evolution or a project’s evolution, help you ideate and draw connections between different subjects or sources. Imagine letting it read Clean Code by Robert C. Martin, and having your Cursor or Claude Code agents reference, retrieve, and reason over its main points, alternating between inspecting your code and identifying smells and bringing in lessons from the book to improve or clean your codebase. Imagine feeding it Steve Jobs interviews and talks on YouTube to ask it for Job’s feedback on your product. Imagine being able to ask it how your week went, how you’ve evolved, tracing back through time to pivots, rationales, updates. Imagine being able to ask it what insights it gathered from reading a book, what was surprising at first read, vs what they think of it now.
Continuity
The result ultimately is an LLM with continuity. No matter what model, or what app you’re using, you can tap into this shared experience, this shared awareness. Conversations and work sessions become part of a greater whole. Your agent develops a sense of ‘self’, an understanding of the user, its past actions, and experiences. It develops deep, long-term memory and knowledge, it’s able to build mental models. It’s able to refer to them when it makes sense. It can connect the dots between concepts, find paths between different ideas, and reference similar situations it’s seen in the past.
It can recall your evolution, entering its context on what’s been, whether you’re using Scaffold, or integrating it in Claude Code, Cursor, Hermes Agent, or any other client. It has temporal awareness, and can genuinely learn.
You can feed it PDFs, studies, articles, videos, audio notes, photos, screenshots, and it can progressively consume the content piece by piece, storing insights and learnings for later citation and reference.
You can tell Cursor to save what was accomplished at the end of a chat. And it can reference this at a later date. Your agent then becomes something capable of experience, of memory, of collection and recollection. It becomes an entity itself, ‘one’ that can exist not just in different conversation threads, but different apps. It becomes a tool for better understanding, for self-awareness. Something that serves a purpose higher than automation and replacement, but augmentation and elevation.