🌀 I
Tried to Explain a Picture That Contains Its Own World

I thought
describing an image would be easy. I had described photographs before.
There is a person. There is a building. Something is in the foreground.
Something else is in the background. If necessary, I can become extremely
adventurous and mention that an object is on the left side of the
image.

Then a colleague showed me an image that seemed to contain
another version of its own scene.

That immediately destroyed my
comfortable little description plan.

The picture was not simply
showing an object. Its geometry appeared to bend into itself. A region of
the scene seemed connected to another representation of the same visual
world, creating the impression that the image was somehow referring back to
itself.

I stared at it for a while and produced my first highly
technical analysis: “The image is very strange.”

It
was not wrong.

It was also not going to carry me very far through a
conversation about recursive geometry and generative models.

I
started with a more useful description: “The image refers to
itself.”

That gave me a simple way to describe
self-reference without immediately trying to explain the mathematics behind
it. I could say that the image looked unusual, that one part appeared
inside another part, and that the scene seemed to repeat itself.

But
the more closely I looked, the less accurate the word “repeat” became.

This was not ordinary repetition like copying the same photograph three
times. The structure had been transformed. Positions changed. Scale
changed. Straight relationships could become curved ones. The scene was
recognizable, but its geometry followed a different arrangement.

A
more precise description was: “The geometry of the image has been
transformed.”

That sentence was already much better than
“very strange.” It focused on the spatial structure of the image rather
than merely its appearance.

Then the mathematics arrived.

The
discussion involved a mapping between an ordinary source representation and
a distorted representation. In a simpler image-editing workflow, I might
create an image first and transform it afterward. But that can produce a
problem: structures that should meet naturally in the transformed geometry
may no longer connect convincingly.

In other words, “A
post-generation transformation can damage connections in the
image.”

That idea became surprisingly important.

If
I distort a finished scene only after generation, I am asking a geometric
operation to reorganize content that was never created with that final
geometry in mind. Lines can stop meeting properly. Objects can stretch in
awkward ways. Boundaries can become visually inconsistent.

My first
instinct was obvious: why not apply the transformation while the image is
being generated?

Unfortunately, generative models have their own
opinions.

A denoising model is trained to turn noisy intermediate
representations into increasingly plausible images. If I impose an unusual
geometric distortion during that process, the denoiser may interpret the
distortion as something that needs to be corrected.

“The
model tries to correct the distortion.”

The funny part was
that the model could be doing exactly what it had learned to do well while
simultaneously destroying the structure I wanted.

I was not asking
it to repair the geometry.

I was asking it to respect the
geometry.

That distinction moved the discussion into much more
precise territory. Instead of talking only about a transformation, I needed
to think about a geometric constraint.

The
generated image should not merely look attractive. It should remain
compatible with a prescribed spatial structure.

The problem could be
summarized like this: “The model should generate a plausible image
without leaving the prescribed geometry.”

That sentence
captured the tension surprisingly well. There were really two objectives.
The image needed to look visually coherent, but it also needed to remain
geometrically admissible.

Then my colleague introduced another
complication: the transformation was not necessarily invertible.

If
a transformation is invertible, I can conceptually move from one
representation to another and then recover the original. A non-invertible
transformation is more difficult because some information can be merged,
constrained, or lost.

So simply saying “apply the inverse” was no
longer enough.

We needed something more general: a
generalized inverse.

For a non-invertible
transformation, a generalized inverse can provide a way to move information
back toward the source representation even when an ordinary inverse is
unavailable. More importantly, it can be designed around the recursive
constraint of the transformation rather than treated as a generic
image-editing trick.

One useful way to describe its role is:
“The generalized inverse reconstructs a suitable representation in
the source space.”

The idea of a source
space became central to the entire discussion.

I began
thinking of the process as movement between two related spaces. In the
source space, the scene could develop in a comparatively ordinary,
untwisted representation. In the transformed space, the image could be
evaluated and refined according to the unusual final geometry.

This
was much easier to understand than imagining one model desperately trying
to generate a perfect recursive image in a single representation.

The process could alternate.

A few steps could improve the source
representation. Then the transformation could carry the current result into
the target geometry. Additional denoising could improve appearance and
local connections there. A generalized inverse could then return useful
information toward the source representation.

“The process
alternates between source space and transformed space.”

That one sentence contained the central intuition I had been
missing.

The source and transformed representations were not
competing versions of the image. They had different responsibilities.

The source space made it easier to develop recognizable content. The
transformed space made it possible to refine the image where the recursive
geometry actually mattered.

Eventually, I could describe this as an
interleaved process rather than a simple sequence: “Denoising steps
are interleaved with the transformation and its generalized
inverse.”

The operations are deliberately woven together.
We are not finishing one entire process and then beginning another. Scene
formation and geometric enforcement develop together.

That
distinction also changed how I thought about image generation.

A
conventional post-processing workflow says: first create the picture, then
distort it.

The more interesting approach says: allow the picture
and the distortion to influence one another while the picture is still
forming.

That sounds like a small procedural difference until you
consider what happens to structures crossing transformed regions. If two
parts of the image are supposed to meet under a recursive mapping, waiting
until the end may be too late. Their visual relationship needs to develop
under the constraint.

“The scene and its geometric
distortion develop together.”

Then came the idea of
projection.

Here, projection means a mathematical
operation that maps a candidate representation back toward the set of
images satisfying the geometric constraint.

An especially useful
property is idempotence.

An idempotent projection has the useful
property that applying the projection again does not continue changing an
image that is already in the projected set. Informally, once the
representation satisfies the relevant constraint, projecting it again
should leave it there.

“The projection is
idempotent.”

Short sentence. Considerably less short
explanation.

But even projection does not solve everything.

A
geometrically admissible image is not automatically a natural image from
the perspective of a diffusion model. If I force the denoiser to operate
only on heavily transformed representations, I may be asking it to process
inputs that differ substantially from the distribution it encountered
during training.

The transformed representation may be
outside the model’s training distribution.

This
explains why enforcing the geometry and obtaining good denoising are not
the same problem.

A projection could enforce a mathematical
constraint while the denoiser still struggled with an unfamiliar
representation.

That is why alternating between spaces is so
appealing. The model can spend part of the process working in a
representation closer to ordinary images, while other steps maintain and
refine the unusual target geometry.

As the discussion became more
technical, the important task was distinguishing the relationships between
the concepts precisely: inverse versus generalized inverse, plausibility
versus admissibility, transformation versus projection, and ordinary
denoising versus denoising under a recursive constraint.

Geometric
admissibility alone does not guarantee an input that is statistically
familiar to the denoiser.

Something can satisfy the geometry and
still look statistically unfamiliar to the model.

Source-space steps
can stabilize the untwisted scene structure, while target-space steps
refine coherence within the prescribed geometry.

At that point, I
was no longer merely describing a strange picture. I was describing a
computational strategy.

That was the most interesting part of the
entire exercise.

I had started with simple ideas like picture,
shape, strange, inside, and repeat. From there, the discussion moved
through transformations, model behavior, non-invertibility, generalized
inverses, projection, denoising, constraints, and interleaved
processing.

Eventually, the central question became how to optimize
visual plausibility and geometric admissibility together.

The image
had not become less peculiar.

I had simply acquired more precise
ways to explain why it was peculiar.

And somewhere between “the
image is strange” and “geometrically admissible projection,” a simple
visual description had wandered into a machine-learning laboratory and
decided to stay there.