For the past few years I have been working a lot with graphs. Having some skin in the game around graphs helps you understand the bigger picture when particular trends appear. Right now, the latest trend is what gets called “graph engineering“, a term popularized by Peter Steinberger, the author of OpenClaw.

The latest premise is that we may be transitioning from loops to graphs.

The idea is simple if we look at it from one perspective: through the lens of time, and through how we have become used to working with LLMs.

Moving in a line

In the early stages of LLM development, problem solving was mostly linear.

You moved through time toward a solution like this:

---> LLM ---> Prompt ---> LLM ---> Prompt---> .... And to eternity, or until the problem is solved

This is how the initial versions of what we now call a harness looked.

Initially, it was just a single prompt: you put some text into the LLM, and the LLM returned the most probable continuation, which often led to a correct solution. Early versions of harnesses had sliding windows, so only a few previous messages were kept in context.

After a few interactions, the beginning of the conversation had completely moved out of the context window.

You were the person pushing the LLM toward a solution. The two of you followed a single thread of work together. It was the bare bones era of 2023. The autonomy of work was still mostly in your hands.

The work was not glamorous and automatic. It meant you were making the decisions and doing the quality control yourself. The model was useful, but it was like a very confused intern with great self-esteem.

Moving in a loop

The limitation of the line is that you have to make the line work and push the line further. You need to guide the LLM toward the solution.

In the meantime, LLMs gained access to tools and MCP servers, which gave them hands to operate more autonomously. This meant the LLM was now mostly capable of solving smaller tasks from start to finish.

That also meant you were the bottleneck in accessing their potential output. This is where loops come in.

Obviously, if you are moving in a loop, there is some circular condition that keeps repeating.

There is a start state and an end state, so it is not a perfect circle. It is closer to a state machine that your LLM moves through until it decides it has completed the task and breaks the loop.

So it is a state machine with a fairly rigid structure that predefines the states, something like this:

This picture is from the Anthropic Claude agentic loop docs

At this stage, an interesting dynamic appears. The two of you are no longer really collaborating in the same way. The model is trying to find the solution to your problem, and you step in only when needed while it mostly runs on its own inside the loop. You are no longer the bottleneck for every step of the process.

The LLM, or agent if you prefer that term, is actively trying to solve the problem posed by your question. This means the agent has taken some autonomy from you, or more precisely, you have delegated it.

It is autonomous in the way any semi-automatic machine is autonomous: once started, it can run through a sequence without further supervision, but it still depends on someone else to guide it correctly, define success, and clean up when it does something stupid. Think of an automatic lawn mower, except it is not really autonomous, only semi-automatic.

So the nature of work changes. You stop manually advancing every step of the line and start managing boundaries around what it can do. This already shows how powerful loops are.

The typical harness today has a loop. You can also configure different loops and tasks to run on a schedule. For example, every hour it could check which PRs were assigned to you and review them in a semi-supervised way with the tools you gave it.

Moving in a graph

One of the biggest changes happens when you drop fixed structures such as a state machine and move into the domain of graphs. Graphs are strong causality engines because they can represent states that do not yet exist and remain unknown until the future unfolds. Thinking about the lawn mower for a moment, imagine it wandering into your neighbor’s yard and either getting you paid or getting smashed to pieces. You do not know the outcome in advance.

I have written before about why I think the graph model is the most natural fit for these kinds of AI workloads in What is the best data model for AI?.

This matters for the nature of the agentic AI world. If you think about a current LLM as operating token by token, it can take wildly different directions if you change just a few tokens. Each of those directions can be intentional, shaped by you as the composer and coordinator trying to steer the model elsewhere.

Each of those directions is a particular version of the path. At any point, you could have chosen a different route by changing only a few tokens. This yields a graph of traces, changes, and internal thoughts that can be biased toward a particular better solution.

Now imagine an army of LLMs running in loops, solving different problems while cooperating with one another in graph space.

That means you have a subgraph of loops running to solve a particular task, and then those loops create new subgraphs, potentially without end.

Something small like this:

           [Goal]
          /  |   \
         /   |    \
    [Loop A][Loop B][Loop C]
      /   \      \      \
     /     \      \      \
 [Loop A1][Loop A2][Loop B1][Loop C1]
     /   \     ...    ...      ...
    /     \
...      ...

This is where autonomy of work stops being partial to your hands and starts becoming an organizational problem. Once many agents can branch, delegate, and recombine, the human is no longer supervising individual steps. The human is supervising a system of evolving components and failures that happily produce more of themselves. You can model that system only with a graph.

If the graph is specifically organizing context for agents to consume, update, and act on, then it also overlaps with what I called a context graph.

In that world, your job is less about doing the work directly and more about deciding which work deserves autonomy at all. Some tasks benefit from branching, while some of them will fail flat on their face.

All of this happens while you are out drinking a macchiato and wondering whether your future of work is in jeopardy because at that point you have a self-problem solving army of agents working on a graph.

The language debate was one of the most interesting debates engineers have had. It is fascinating how much energy has gone into debating languages, and how often that turns into an egoistic debate.

People devote their entire identities to the programming language they know and are used to, and that is a very natural thing to do. The programming language and ecosystem in which you work make you feel like a worthy member of a particular community. Humans love to feel useful.

On the other side, there is an enemy growing inside: the ego. The human ego can be a very fragile thing, and people are mostly unaware of it and the limits of their thinking. Once someone criticizes a programming language, they feel personally attacked. That is somewhat understandable, but it can turn a technical discussion into a complete mess.

As soon as we get emotional about deterministic and machine things, we are probably in an egoistic debate, not a language debate.

I obviously do not want to debate language, but rather the features it brings to the table. Before that, the growing question today is this:

What is the programming language today?

English, by Karpathy

Obviously, Andrej Karpathy’s tweet, “The hottest new programming language is English”, feels a bit like an epiphany because it captures the direction.

As more and more code is written by LLMs and less and less by humans, a few interesting things start to happen.

We are probably getting less attached to the programming language we are spending our time in, and questioning our purpose and existence a bit more. That means we will switch more easily and reconsider what is actually optimal.

On a more technical note, we get massive gains in prototyping and refactoring in any language. Humans play a smaller role.

There are some technical differences in token spend, review time, correctness, and so on, but I am thinking more about the bigger picture.

You can generate a million lines of code in a couple of hours and completely refactor a whole codebase.

If you have a good test setup, solid coverage, and a benchmark harness, you will immediately see the drawbacks or gains from that refactor.

I had been planning to write this blog and already had an initial draft ready, but what happened with Bun, a million changed lines in six days, a port from Zig to Rust, pushed me over the edge to actually finish it.

The pros and cons of programming languages need to be completely reevaluated.

Back to the language debate

If we start thinking about language features and comparisons, most of them are probably obsolete because they were built around the human bottleneck.

One of the big distinctions is whether languages are compiled or interpreted. Interpreted languages are usually easier to learn, handle more things for you, and require less thinking. This was ideal when humans were writing the code and needed to prototype fast.

If you ask an LLM for a comparison of statically typed and compiled languages versus dynamic and interpreted languages, you would get something like this:

Static: C, C++, Rust, Go, Zig · Dynamic: Python, JS, Ruby, PHP

Fast prototyping is critical. You usually sacrifice performance and security for a faster pace and lower development time, and this leads to a more vibrant ecosystem of tools built on top of a poor foundation. This was the default because of project budgets, time, and engineers.

Keep in mind that I do not have anything against dynamically interpreted languages. I write Python all the time, but it is a liability.

Predicting the unpredictable

As our world is quite unpredictable (LLMs were not a thing a few years ago), I am going to make a prediction for the fun of it.

I think the future for any software is tied to statically typed and compiled languages for specific platforms, since there is no longer a measurable benefit in prototyping speed. The only argument I can think of against this is the existing ecosystem and libraries. But think about it: refactoring something into another language has basically become cheap.

I do not think dynamic languages will disappear. People and agents will probably use them in some capacity.

Turns out that a lot of manual work has moved toward using LLMs and agents to drive some parts of our daily workflows. At some point in your work you are probably using them, because they are highly effective and, frankly, addictive. If you are not, you are probably in some state of Nirvana.

Your work starts by opening your terminal, favorite agentic IDE, or a GUI like CoWork. I am assuming you are using some SoTA LLM model and wrapped in an agentic harness. If you are doing this in the web interface, you are probably about a year behind what is currently possible. I know that sounds a bit like AI-bro advice, but I mean it.

The key is that the agent running your prompt has access to tools, skills, and MCP servers it can operate on. That means it has hands to perform tasks for you. That combination of model, harness, and tools is what makes it an agent.

Good, now we get to the fun part: the agent doing some work for you. Jake has sent you a message, you have seen the start of it and a link inside, and you decide that the agent should deal with it:

Take a look at the message Jake sent to the infra channel. There is a link in the message. Review it and do as much work as you can autonomously.

The big question is: how should the agent think about this task?

Context engineering

The agent operates purely on context. It is like a working memory it can process, and that memory is highly limited, so it needs to stay lean and optimized. You have given the agent a task through a prompt it needs to interpret, but there is not much context in it.

So the question is: what is the context in this case?

It is actually:

• Infra channel
• Message and message content
• Link and link context
• Review task
• Permission to operate autonomously

That is the context you see from the prompt you have written. There is also a system prompt and more, which goes under the topic of prompt engineering, and I will not go into that rabbit hole here, but it is important that different levels of prompts exist.

Now the question becomes: what context is not visible from your prompt? As an employee in a company, you probably carry a lot of context based on the current state of the company, so the task sounds easy to you.

Here are some examples of what is not visible:

• Who are you inside the organization, what are your role, responsibilities, actions you can take?
• What is Jake’s role, responsibilities, and actions he can take?
• What decision is driving Jake’s message?
• How does the task contribute to company goals, metrics, or the unit as a whole?

That is the kind of context that can be captured, and it goes into the topic of context engineering because it is exactly the context that is currently missing from your prompt.

The question is: where are you getting that context from, and how are you retrieving it?

Context graphs

At this point in time there are several approaches you can take to solve this problem of retrieval, but if we are talking about enterprise knowledge bases, that knowledge is stored in multiple systems and formats, and the question is how all of it connects under a single umbrella. Just think about how you would connect documentation to a Git repository and an internal chat system, and there is already sufficient chaos to handle.

As I have discussed before, it turns out that the graph model is great for modeling in the AI space due to its flexibility and its ability to represent structure. You can pretty much express any problem as a graph problem. These days, graph databases support vectors, text, edge, and property indexes, so it is easier to connect different sources while optimizing storage for actual search.

This leads me to a naming point: if we are solving a context-engineering problem by using a graph as the model, we are talking about a context graph. My definition is simple: a context graph is a knowledge graph organized specifically for agents to consume and update. It is an AI-first graph solution to a given problem and a use case. I use “knowledge graph” here to clarify that it is modeled with intent, not just complete chaos.

In the end, humans can remain operators of intent while agents handle execution, search, and storage with context graphs.

Context graph use cases

Use cases are already appearing everywhere, but in my current state of mind, here are some examples with short descriptions and questions:

• Agent Memory graphs -> What is the work the agent has done prior to this? What do agents already know about this problem?
• Agent Decision trees/traces -> Storing, retrieving, and evolving decision traces. What were the decisions and why?
• Agent Actions graphs -> What actions can the agent take, what is the right action to take, and what responsibility or ability is attached to that action?
• Agent Code graphs -> How are source files and components composed together, and what are their dependencies?
• Skills graphs -> What skills need to evolve or be used to solve a particular problem?
• Agent entity/ontology/data graphs -> What is the actual data or problem the agent is working on, and what problem is it trying to solve relative to organizational data?
• …

Obviously this list can keep going, so in my mind there is no single context graph use case. It is really about the application of graph models and graph algorithms to solve context-engineering problems for agents.

Technical challenges and ambiguity

We are still mostly operating in single-player mode, where you run agents toward a goal using graph data, memory, tools, and components that are mostly local to you. The harder problem is making this work in an environment where multiple agents, teams, and systems are operating at the organizational level (multiplayer mode). That is where context graphs become much more valuable, and much harder to build at the same time.

Turns out from time to time we really like to explain some theoretical concept with different names, which introduces ambiguity. The reason usually comes from the background of the author, or from the fact that the theory itself stands on shaky legs, so people reuse the same theory in conjecture while trying to fit it to the actual narrative they are working through.

Thinking now about GraphRAG, I see context graphs as the broader framing. GraphRAG usually describes a retrieval pattern: use graph structure to improve retrieval and generation. A context graph is a wider operational concept that covers things from agents first perspecitve.

That is why I think context graph is the more useful term for the workloads we are starting to face. The point is not just to retrieve better context. The point is to organize the environment around the agent in a way that makes reasoning and action more reliable and efficient.

Some of the sources I came across while thinking about context graphs before writing this blog, but I find them missing the bigger picture:

• Context Graph
• Context Graphs: AI’s Trillion-Dollar Opportunity

I quite recently discovered the Bitter Lesson by Rich Sutton. It opened my eyes in so many different ways, I would recommend reading it. There are different points there, you will probably see something else, but the bigger point for me is that any type of human sophistication (in technical systems) usually loses to the general search/compute methods that scale to infinity and are convex in nature, compared to the concave approach of human sophistication.

The most visual example is in the computer vision space where you had a bunch of sophisticated math models like SIFT that were made to detect robust features to do something bigger, but were completely replaced by convolutions in neural nets.

This brings me to something I have been thinking about for quite some time now. As you are probably aware, there are a bunch of different data models implemented in different database systems. The question is: what data model is the most suited for AI (LLM driven agents)? Thinking from the perspective that the actual data model will introduce the least human sophistication and enable search and compute to scale.

We are probably a few years away from AGI or the LLM bubble bursting (it will be interesting to read this in a few years). That means we still need to somehow manage the data models that are most common to humans and LLM driven agents.

Everything I am tinkering with at the moment is around the actual data model on the logical and representation level (software), not the actual storage model (hardware). At the point when AGI hits, who knows how data would be represented then, probably in some synapse-style baked into the hardware. Who knows what the memory of that system will look like, but I guess we won’t abandon the current deterministic computations in software on the data from databases since it is much superior to the human brain in deterministic computation, and we still need a human-readable format we use today.

Shortest history of data models

An obvious start could be: what do we have currently in the store?

If you prefer the short history of how things evolved:

• Flat files (1950s and 1960s)
• Hierarchical Model and Network model (1960s and 1970s)
• Relational model (1970s)
• NoSQL key-value, document, object, graph (2000s)

Each of these periods was an era of discovery and adoption. In each of the periods, some of the models showed their strengths and weaknesses, but it was super early days for each of the data models. Graph was initially started in the 1960s and later revisited in the NoSQL era. Flat files seem to be like where skills are today. But there was one that scaled: the relational model was the prime winner so far.

The main reason is that it was the most suited for what was needed at the time: a decent level of structure and rigidity to power the simpler businesses that were starting to pop up. Pair that with the time we spent working, improving, and breaking the relational model in the last 50 years, and it grew to be the most reliable and widely adopted data model out there that most companies and engineers use. The relational model scaled and it works today; it is not outdated by any means. The most recent example is OpenAI scaling ChatGPT to 800 million users via Postgres.

It is important to know that network effects, serendipity, and human bias play an important role in the development of technology. You won’t pick a key-value or a graph database if you haven’t used it before or didn’t have exposure to it. Usually the choice is something that happens by accident, you find you need some feature or the current system becomes too rigid, and then you go and pick something from your mind.

Think about it this way: if you just need to smack a nail into the wood, you may pick the first rock you find on the ground, not the hammer 30 meters away. Don’t fool yourself that you are always picking the optimal tool.

Picking the optimal data model

Well, the obvious way we could start applying the bitter lesson is the level of structure we have in our database systems. The more structure we need to provide, the more the system is locked into our way of thinking about it. Think about it this way: if you have a box you can write into, you have already created the box and limited it to a box. Now ask yourself, how are you envisioning AI?

Well, I am envisioning that it will be able to go over some of my data that is completely messy, and without me doing anything it will be able to reason and figure out how we got to that point (scaling search and compute). Now this is what you would ideally want in your daily life and in business, the data model that learns (compute) as you go and can enable scalable search on the fly. The search is important since you don’t want to spend any resources you don’t have.

What I described above is flexibility of the system to represent any data point, relationship, or structure. So the data model needs to be flexible and represent some structure at the same time.

If we take a look at the data systems, we would have the following structure and flexibility:

Relational models can also be quite flexible, but the schema needs to be created upfront. This means before putting data in you need to create the box, which defeats the purpose of freely moving AI. In the example of OpenAI and scaling Postgres, they are not allowed to change the schema. They can’t even change the box they have created to write into. I am not mocking here, I am impressed by what they did, but it strikes the direct point I am making: the relational model is not so flexible.

Object, document, and key-value stores are in the same bucket since their capability to represent structure is limited.

Vectors in themselves are not in the table since I don’t consider them a data model. Rather, they are a compression or search index for some data. More on that in the next blog.

You will probably notice that I am infinitely biased towards Graph data structures because I work at Memgraph, but that is not the point here. The point is trying to find the best model to represent flexibility of schema and structure between different things that happen in life. Graph model seems to fit perfectly into this based on the metric I am thinking about.

Metric of data models

Tinkering with the data model is all about the metric that data model needs to serve. My personal metrics are being able to represent structure and flexibility. These two metrics in my mind are the most important ones.

Of course, implementing this into the transactional, scalable and robust systems is a different challenge, but it should move us to the ideal system that holds the data model designed natively for AI.

The opposite question can also be valid: what is the metric to evaluate the capability of a data model?

Are there other metrics you would take into consideration?

We live in fascinating times of mighty LLMs that can help boost productivity and simplify day-to-day tasks. The speed of adoption of LLMs is a story that will constantly be compared to everything that came before and after them.

It is essential to highlight and understand the fundamentals and flaws of this technology. Let’s jump to the big question around LLMs that became quite controversial: Can they reason? My short answer is no. Let’s dive into why.

Reasoning

To even discuss reasoning, we would first have to define it and agree on what reasoning is. Let’s distill this into what we refer to as a typical thinking process: you are presented with some information, can think about it, and conclude something.

The practical example would be something like this: The S&P 500 sank 15% percent in the three months.

Now, in your brain, you start to ponder that information and connect it to the events that happened in that period. Some notable things that could pop into your mind are the Trump tariffs, lower-than-expected BDP in the US economy, global conflicts, etc.

In a pure form, this is reasoning about specific information in your brain. If I follow up on the question and ask: “What will the stock market do tomorrow?” you will probably start thinking hard about this.

In other words, you would read projections, news, global tax, and tariff policies. You would invest time and effort into constructing a thesis under different assumptions.

Usually, the more effort you invest, the higher the probability you will be correct.

In a pure form, it is reasoning based on the information you were given, have in your memory, or acquire over the web.

The nice thing about reasoning is that our brain is the ultimate reasoning powerhouse (most of the time :D), and it runs on beer and bananas, it does not require 1 Megawatt of electricity to be trained

Reasoning and LLMs

LLMs cannot reason, they can mimic reasoning that sometimes works and sometimes does not. You can call it primitive, low reasoning, or use any other way to package it, but it is far from your reasoning capabilities.

This starts with the basis of what LLMs actually are: They are super capable text predictors with a lot of memory. Memory is constructed based on what they have been trained on, and small “thinking memory” is the context you are passing with your question.

Let’s put this into practical example if your LLM was trained on this data:

We have different colors for objects: blue, red, green, purple, yellow, etc.
The ball can be red.
The ball can be blue.
The car can be purple.
The flag can be purple.

Now, if I ask: “What color can a ball be? Give me three answers.”

The LLM computation will give a higher probability to the obvious answers:

1. The ball can be red.
2. The ball can be blue.
3. The Ball can be ...

The third answer would be some of the possible colors, because more objects are purple and purple has a higher frequency, it would probably give the third answer as purple. But we all know that it can be any color since it is not specified.

If you want more details about this, one of the best explainers about why LLMs is a text predictor, I would recommend jumping on the 3Blue1Brown channel and watching a series about neural networks, specifically about the LLMs and transformers.

By design, they are not made to think but rather to predict the next word (token, to be exact).

Well OpenAI docs seem to suggest otherwise.

This is a snippet from their docs page:

Reasoning models, like OpenAI o1 and o3-mini, are new large language models trained with reinforcement learning to perform complex reasoning. Reasoning models think before they answer, producing a long internal chain of thought before responding to the user. Reasoning models excel in complex problem solving, coding, scientific reasoning, and multi-step planning for agentic workflows.

The image from the same docs page illustrates this process:

They are saying their models can perform complex reasoning.

Well, they do not reason. They generate a thesis on how to answer questions, summarize that generation into reasoning and output, and chain this multiple times, which improves the text generation and makes it more successful. Something you will hear as Chain of Thoughts.

So, they are mimicking reasoning where you generate text related to the text you are asking for to try to come up with a solution. It is like a person saying the facts out loud when someone asks you a question or creates a theory about something and uses theory construction to present the facts. At any step of the process, you are hoping the theory won’t go sideways.

If we go back to the LLM that was trained on ball colors:

Question asked: What color can a ball be?

Chain of thought:
The ball can be blue
The ball can be red

Output:

Therefore, the ball can be red or blue.

This would be a good Chain of Thoughts:

Here is how it could fail:

Question asked: What color can a ball be? Give me three answers
Chain of thought:

The ball can be blue
The ball can be red
The car can be purple.


Therefore, the ball can be blue, red, and, I guess, purple.

If you do not trust my examples, give a shot to your reasoning LLM, I got this on ChatGPT with reasoning:

If you do not trust me, one of the best men to describe this is Yann LeCunn, Chief AI Scientist at Meta.

I admire the engineering behind modern LLMs and find them helpful. If you provide the proper context, they won’t hallucinate. That is why we, in Memgraph are focused on building cool Graph and LLM-powered AI apps.

As they say, the harder it is to explain and argue how bad or incapable technology is, the better it is. Arthur C. Clarke’s Third Law:

Any sufficiently advanced technology is indistinguishable from magic.