Self-Supervised Learning of Universal Latent Spaces for Operator Learning

Sepehr Mousavi1,3,  Levi Lingsch2,3,  Siddhartha Mishra2,3,  Laura De Lorenzis1,3
1Department of Mechanical and Process Engineering, ETH Zurich   2Seminar for Applied Mathematics, ETH Zurich   3ETH AI Center
ETH Zurich Computational Mechanics Group CAMLab ETH AI Center

TL;DR  We pre-train one functional autoencoder (FAE) on synthetic functions only, freeze it, and reuse its latent space for every operator. It reconstructs PDE solutions and photographs on geometries it never saw, its latent space is interpretable, and operators trained on cached embeddings are orders of magnitude more efficient to train and to infer than state-of-the-art end-to-end models, without losing accuracy.

χ arXiv </> Code ◉ Models ❝ BibTeX
A painting of Daedalus, its contours and partitions.
0 PDE solutions
seen in pre-training: the backbone learns from analytic functions synthesized on the fly.
23/28 operator tasks
where Daedalus beats the strongest end-to-end baseline.
Details on every problem →
Performance

On one RTX 4090 (24 GB), with variable geometry and the partitioning included.

64×
Larger problems
Daedalus trains on problems of up to 4.2 M points (16K partitions) at a batch of 32. This is 64 times the largest that GAOT trains on with variable geometry and 16 times with fixed geometry.
13.6×
Faster on large problems
On the largest problem GAOT still trains on, Daedalus trains 13.6 times faster, and 3 times faster than GAOT with fixed geometry.
92×
Larger models
On dense problems (262K points, 4K partitions) at a batch of 32, Daedalus trains a 101M-parameter processor, 92 times the largest GAOT that fits.
4.9×
Faster on large models
At the size of the largest GAOT that fits, Daedalus trains 4.9 times faster.
1.6B
Model parameters
Daedalus infers with a 1.6B-parameter processor on 32 dense problems at once; the largest GAOT that fits has 285M.
537M
Points at once
Chunked encoding and decoding map functions of 537M points in 33.6M partitions. The largest problem that fits in one piece is 16 times smaller.
10×
Faster rollouts
A latent rollout advances the state in the latent space and decodes once, up to 10 times faster than decoding and re-encoding at every step.
See moreEvery performance benchmark: speed and memory against baselines over problem size, model size and space size, chunked inference and rollouts.
Projected training on eight RTX Pro 6000 (768 GB in total)

Projections: the memory measured on one RTX 4090 grows affinely with the partitions and the processor depth; extrapolated to 768 GB, 32 times one RTX 4090, assuming the memory is pooled by sharding the model and the batch across the eight cards.

projected
~160M
Largest problem in training
in more than half a million partitions per function, at a batch of 32
projected
~4B
Largest model in training
on problems of 262K points and 4K partitions, at a batch of 32
The method in a nutshell

From one function to a latent operator

One FAE, trained once and without labels, turns any function on any point cloud into points on a hypersphere and back. A small region of a field becomes one embedding; a whole field becomes one embedding per region, each seen in a common local frame; and an operator learns to move these embeddings across the hypersphere instead of the field itself.

Read the method in detail →

Interpretability

Directions in the latent space mean something

The chord between the embeddings of two anchor fields that differ by one operation is a direction. Added to the embedding of a different field, it applies the same operation. Scroll over the field or drag the slider.

Direction
anchor A
anchor A ·
→
anchor B
anchor B ·

Field it is applied to
c = 0
Advantages

What a frozen, universal latent space buys

∞

One for all

Partitions and their normalization carry a field into the familiar band of the training functions, provided it is sampled finely enough to be partitioned, so one backbone serves fluids, solids, waves and images on unseen geometries.

⚡

Scalable operator learning

A dataset is encoded once and an operator trains on its cached embeddings, so a training step no longer passes through the points, and far larger models train and infer on far larger problems. One backbone, trained once, serves every task at no extra cost.

✦

Interpretability

The whole latent space is used and decodes to valid fields, latent distances preserve the order of distances between the decoded fields, geodesics interpolate evolutions, and edit directions transfer.

↻

Equivariance by construction

The representation is exactly equivariant to translation, dilation and amplitude: the static frame stores them, the embedding carries only the shape.

Training throughput samples/s per GPU, log scale
Training throughput on one 24 GB RTX 4090 as the problem grows, in points per function (bottom) and partitions (top), at 256 points per partition and a batch of 32. Daedalus trains on the latent loss with input and output caches (stars), on the physical loss with an input cache (triangles) or end to end (circles). GAOT trains end to end with a point cloud per sample; with Fixed geometry on, it shares one cloud across the batch, and RIGNO joins it. A red cross under a line's last point marks where it runs out of memory; Memory shows the peak memory of the same runs. Hover for the values.
Mean relative L1 test error %
Relative L1 test error (the median over the test samples of a problem) and total training time of the operator on the frozen backbone and of two end-to-end baselines, averaged per family of the 28-problem benchmark (problems per family in parentheses) and over all problems. A bar that reaches the end of an axis is cut there. RIGNO-12 is not trained on the steady compressible flows, so its overall mean covers 24 problems.
Physics in the latent space

Do latent geodesics follow the physics?

An incompressible flow evolves over 20 steps. The anchors are Δt snapshots apart; every state in between is decoded from the great circle (SLERP) between the two nearest anchor embeddings, without any operator. On the globe, the physical evolution traces a curve, and the geodesics cut across it.

Latent hypersphere

physical evolution geodesic between anchors snapshot active anchors drag to turn

Physical evolution t = 0

ground truth

Geodesic estimate

anchor

Error of the geodesic estimate relative L2

geodesic vs. truthreconstruction flooranchors
Right now
0%
geodesic error
0%
reconstruction floor

Between anchors close in time, the geodesic stays within a few percent of the true flow: the embedding preserves the geometry of the evolution. Spread the anchors (Δt = 10) and the shortcut drifts off the physical path: over longer intervals, following the physical path is the job of a learned operator.

Live demos

Run the backbone on your own input

Nine demos run the frozen 128-dimensional analysis backbone on a small CPU server. Every demo comes with pre-computed samples that show at once; your own inputs are queued and run one at a time.

Transition

One field turns into another on the sphere.

Open →

Latent directions

Move an edit from two anchors to a new field.

Open →

Latent search

The nearest of 2,300 functions in the latent space.

Open →

Derivatives

Exact gradients of the decoded field.

Open →

Reconstruction

Type or paint a function; see it encoded and decoded.

Open →

Upsampling

From a coarse grid or a few pixels to a fine grid.

Open →

Inpainting

Paint holes; decode every pixel from the rest.

Open →

Outpainting

See a field through a window; decode it beyond.

Open →

Perturbations

Turn one embedding at random; decode the variants.

Open →
Arcade

Train your eye on the latent space

Three mini-games built on the ideas of the paper test your understanding of its concepts, each with a leaderboard of the scores of other players.

Freaks

Guess a field's effective frequency by eye.

Play →

Tiles

Put a blend of four embeddings back in order.

Play →

Divide and Conquer

How many partitions does a field need?

Play →
Articles

Learn more

Read our articles to learn more about Daedalus. Every article discusses a specific topic and includes interactive visualizations.

Method overview

A short tour for a first read: partitions and their frames, the functional autoencoder, and the operators learned on its latent space.

Read →

Operator learning

Operators learned on the frozen latent space: their accuracy across datasets, their speed and their memory.

Read →

Interpretability

Geodesics between embeddings, directions that carry an edit to other fields, and a slice of the latent sphere.

Read →

Inference perks

Rollouts that stay in the latent space, and chunked inference for problems larger than the device.

Read →

Effective spatial frequency

One number for how hard a field is, and the measurement showing that it, not the kind of function, decides the error.

Read →

Functional autoencoder

The encoder and the decoder, the update that keeps every embedding on a sphere, weight-shared depth and the two backbones.

Read →

Reconstruction

Physics fields, photographs and curves the backbone never saw, and how their error falls with the partition count.

Read →

Partitioning

How a point cloud is cut into partitions by k-means started from a space-filling curve, and how overlap widens them.

Read →

Partition normalization

The local frame of every partition, the static features it stores, and the invariances it gives the codes.

Read →

Derivatives

The exact gradient of any decoded field, with one forward-mode pass per coordinate axis.

Read →
Cite

BibTeX

@article{mousavi2026daedalus,
      title={Self-Supervised Learning of Universal Latent Spaces for Operator Learning},
      author={Sepehr Mousavi and Levi Lingsch and Siddhartha Mishra and Laura De Lorenzis},
      year={2026},
      eprint={2611.05735},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2611.05735},
}