New AIs – Path to Godlike Intelligence?

Energy-Based Models: Can We Judge the Whole Story Instead of One Word at a Time?

Autoregressive language models are very good at finishing the next word. That is their gift and their trap.

Next-token prediction is like finishing someone’s sentence at a dinner party. Sometimes charming. Sometimes useful. Sometimes wildly presumptuous. The model asks, “What word should come next?” over and over until a paragraph appears. But coherence is not only a local event. A sentence can look good word by word and still collapse as a whole.

Energy-based models ask a different question: does this whole configuration make sense?

Instead of treating generation as a left-to-right typing exercise, energy-based approaches assign a score to a possible state. Lower energy means “more plausible.” Higher energy means “something is off.” It is not just grammar. It is compatibility. Does the claim fit the premise? Does the explanation fit the evidence? Does the ending belong to the beginning?

That shift matters.

NVIDIA’s work on Energy-Based Diffusion Language Models is a good example. The idea is not simply to make text faster or stranger. It is to combine diffusion-style generation with an energy function that evaluates sequences more holistically. Text becomes less like a single-file line of tokens and more like a noisy draft being repeatedly cleaned up.

That sounds technical. It is. But the psychological metaphor is obvious.

We do not heal a conversation by fixing one word. We look at the pattern. Tone, memory, timing, fear, expectation. A person can say every “right” word and still be avoiding the truth. Likewise, a model can produce every locally probable token and still generate nonsense with excellent posture.

Energy-based thinking tries to expose that gap.

One newer direction, Boltzmann-GPT, pushes the separation even further. Its premise is basically: the mouth is not the brain. The system separates an energy-based world model from the language generator. One part models what is plausible. Another part says it fluently.

I like that distinction.

Humans often suffer because our world model and our mouth are fused. We feel threatened, so we speak threat. We feel shame, so we produce certainty. We confuse the reaction with the truth.

A system that separates “what do I believe is structurally plausible?” from “how do I express it?” has more room for correction. If the inner model is wrong, you can intervene there. If the language is clumsy, you can intervene there. When everything is one giant blob, you get confident nonsense and no obvious place to repair it.

That is not just a machine learning problem. That is Tuesday.

Diffusion Language Models: Parallel Refinement Instead of One-Track Thinking

Diffusion models changed image generation by starting with noise and refining toward structure. Text is harder. Language is discrete. You cannot blur a word the same way you blur a pixel. Still, the basic idea is powerful: generate by refinement, not by one-way commitment.

Masked diffusion language models do this by filling in, revising, and denoising multiple tokens at once. The draft is not sacred. The model can reconsider.

That is the interesting part.

InclusionAI’s LLaDA 2.1 continues this direction with diffusion language models that support more flexible decoding and token-level editing. Instead of only turning masks into tokens, LLaDA 2.1 also uses token-to-token refinement. In plain English: the model can change what it has already written while generating.

This is closer to how actual thinking works.

We rarely build a belief one word at a time. We update clusters. A memory shifts, then the meaning of an argument shifts, then the identity story attached to it shifts. If you have ever had a real apology land in your body, you know this. It does not merely change one sentence. It reorganizes the room.

Inception Labs’ Mercury 2 pushes the practical side of diffusion language models. The pitch is speed: parallel refinement, lower latency, and output that feels closer to instant. Their claim is that diffusion-style generation can be much faster than traditional autoregressive decoding while staying competitive in quality.

That matters because latency is not a cosmetic issue. Speed changes how we use intelligence. A slow model feels like a consultant. A fast model feels like an extension of thought.

But speed is not wisdom.

Fast confusion is still confusion. A model that can revise in parallel may become more useful, but it does not automatically become more grounded. Revision is only valuable if the system has a good signal for what “better” means.

That is why the energy-based and diffusion threads belong together. Diffusion gives you a process for revising. Energy gives you a way to judge the shape of the revision.

One asks, “Can we change many things at once?”

The other asks, “Did the whole thing become more coherent?”

Good therapy needs both.

State-Space Models: Long Memory Without Melting the Machine

Transformers are powerful, but attention is expensive. At long context lengths, the classic attention mechanism starts to look like trying to maintain eye contact with everyone you have ever met.

State-space models offer another path. Architectures like Mamba-2 aim for efficient sequence modeling with linear-time behavior. They are not just trying to remember more. They are trying to make remembering computationally survivable.

NVIDIA has explored Mamba-based and hybrid architectures, including support in NeMo. The appeal is simple: keep useful long-range information without paying the full quadratic attention bill every time.

In human terms: can I remember what happened forty minutes ago without frying my nervous system?

Long context is seductive. It feels like wisdom. It is not.

Capacity is not discernment. You can remember every insult from 2009 and still learn nothing. You can store a million tokens and still miss the point. More memory gives a system more material to work with. It does not guarantee that the system knows what matters.

That distinction will become more important as models get longer memories. The next frontier is not merely “can the model remember?” It is “can the model prioritize, compress, reinterpret, and let go?”

Humans struggle with the same thing.

Some memories are context. Some are clutter. Some are open loops pretending to be identity.

HOPE and Self-Modifying Systems: Adapt While Running

Google’s work on Nested Learning and HOPE points toward another frontier: systems that adapt while operating.

HOPE is described as part of a nested-learning paradigm, where models maintain hierarchical memory and update internal learning processes across different timescales. The broad idea is important: intelligence is not only what happens during training. Some learning has to happen during use.

That sounds obvious because humans do it constantly.

You walk into a conversation with one model of the other person. Ten minutes later, if you are paying attention, that model changes. “Oh, they are not trying to control me. They are trying to feel safe.” Rule shift. Behavior shift.

Or the opposite happens. You misread a facial expression, update around fear, and spiral into a completely false story. Also a rule shift. Also a behavior shift.

Self-modification is not automatically noble. It is just powerful.

A system that can update itself during inference may become more adaptive, more personal, and more capable across long tasks. But it also raises the old question in a new form: what governs the update?

If the inner algorithm rewrites itself around noise, it drifts. If it rewrites itself around fear, it defends. If it rewrites itself around curiosity, evidence, and constraint, it grows.

The mechanism is neutral.

The direction is everything.

The Hybrid Future: A Patchwork Organism

The future probably does not belong to one clean architecture.

It will not be “just transformers” or “just diffusion” or “just state-space models” or “just energy-based systems.” The frontier is already becoming hybrid. Transformers for flexible representation. State-space layers for efficient long-context processing. Diffusion-style refinement for parallel generation and editing. Energy functions for global coherence. Adaptive memory for continuity across time.

It will be messy.

Of course it will. Evolution is messy. Minds are messy. Relationships are messy. Anything that survives contact with reality becomes a patchwork.

Single-paradigm purity feels clean, but it fails under pressure. Pure logic fails. Pure emotion fails. Pure memory fails. Pure speed fails. Integration works better.

The same is true in machine intelligence.

Autoregressive models gave us fluency. Diffusion models are giving us revision. Energy-based models may give us better global judgment. State-space models may give us longer usable memory. Self-modifying systems may give us adaptation during the act of thinking.

None of this ends suffering.

That is not cynicism. It is hygiene.

Technology reduces certain constraints and exposes others. Faster models expose the poverty of our prompts. Longer-context models expose our inability to decide what matters. More coherent models expose how often we prefer pleasing answers to true ones.

AI is a mirror, but not a passive one. It accelerates whatever is already in the room.

If you are hopeful, notice what you hope for. More speed? More coherence? Fewer hallucinations? Less cognitive load? A machine that can hold the thread when you cannot?

Underneath the technical language is a human request: please make reality easier to work with.

And if you feel cynical, notice that too. Cynicism is often disappointed hope wearing armor. It says, “I knew this would fail,” because wanting it to work felt too vulnerable.

The architectures are changing because the old bargain is not enough. One-word-at-a-time prediction gave us astonishing fluency, but fluency is not the same as understanding. The next wave is trying to judge larger shapes: whole drafts, whole contexts, whole belief states, whole trajectories.

That is the right question.

Not “what word comes next?”

But:

Does this hold together?

Does this adapt without drifting?

Does this remember without drowning?

Does this speak from a model of the world, or only from momentum?

We keep iterating machines because we keep iterating ourselves. The architectures change. The old questions remain.

Watch the animated musical story / prediction of where things could be going and how we get there. Trigger warning: It starts out dark.