Ente Photos now processes your photos up to ten times faster. For the first time, our ML runs on the GPU. And everything around the models is now lean Rust.
On a recent iPhone, running all models together takes only ~40ms per photo, 10x faster than before. Photos already on your device now index in a fraction of the time.
Why local speed matters so much to us
Everything in Ente is end-to-end encrypted. We can't see your photos, so we can't run ML on them for you. Your phone does the work other services do in a data center.
That makes local processing the whole game. Every model we ship has to run over your entire library, on your device, without draining your battery or getting in your way. The faster it runs, the more we can do with it, for you.
Not just a switch
Telling ONNX Runtime (ORT) to use the GPU is a one line change. Getting a fast model out of it is not.
CoreML and WebGPU (the GPU accelerators) only accelerate the model operators they know. The rest goes back to the CPU. A model that bounces between the two is often slower than one that stayed put. So we rewrote our models.
Our face embedding model has 33 PReLU activations. WebGPU has no kernel for them, so each is now two plain ReLUs: Relu(x) - alpha * Relu(-x). Same math, but now it doesn't bounce between the GPU and CPU.
Our CLIP model had 54 expanded GELU expressions consisting of multiple operators. Each is now one fused Gelu operator that CoreML and WebGPU both understand.
Our face detector had a dynamic batch dimension. It's pinned to one now, so the shape arithmetic folds out of the graph.
Every rewrite is scripted and verified against the original model. Same numbers out, far less work to get them.
The work around the model
Model inference is only part of indexing a photo. Before the model, the photo has to be decoded, resized and turned into a tensor. After inference, raw outputs have to become faces, boxes and embeddings. All of that runs in Rust, and most of our optimizations went there.
Our HEIC decoder got a round of speed work. HEIC, the iPhone default, now decodes twice as fast.
As for preprocessing, we made sure a decoded photo lives in one buffer for the whole pipeline, across models. Every stage borrows it, nothing copies it. Faces are aligned straight out of the decoded pixels, and any resize only touched the pixels that actually need to go into a model. Resize buffers are allocated once and reused. It's these small choices made everywhere that add up.
Postprocessing follows the same principle. Model outputs are read straight from ONNX Runtime's memory, in whatever precision the GPU returned them. We allocate only for what we keep.
Pre- and post-processing now take a few milliseconds per photo on iOS. Negligible even compared to the improved model inference.
One pipeline, every platform
This is the payoff of our Rust rewrite, which started with a vector database and now covers our cryptography too. It's far from done, but it's already paying off.
Rust compiles natively to every platform we ship on. So mobile and desktop now run the exact same ML code. Desktop moved over a week after mobile, with the GPU enabled on macOS, Windows and Linux.
As the rewrite continues, more of Ente will work this way. Feature parity by construction, and faster product development everywhere.
Tests, tests, tests
The other enabler is less glamorous, but crucial: tests. Lots of them.
ML indexing used to be one of the scariest parts of our codebase, right after upload and encryption. Not because it was slow, but because bugs here are silent. A subtle preprocessing mistake produces embeddings that look fine, even when they're not. They get stored, synced, and quietly make search worse. So every change used to come with extensive manual testing.
That's fixed. Every pull request now checks our Rust ML code against ground truth from the original Python models. Shipping something that breaks indexing is now close to impossible.
The safety net extends to your phone too. GPU drivers in rare occasions can return plausible-looking garbage while reporting success. So on every model load, the app runs a verification inference and compares it with the expected output. If the GPU is off, we fall back to the CPU.
With these tests in place, we could optimize as aggressively as we wanted. Every optimization was automatically verified before it shipped.
Full steam ahead
Local processing has gone from something to be careful around to something we can build on. Analyzing what's on your device is fast, verified and identical across platforms.
The Rust rewrite continues, and there's more on the way. The code is on GitHub. What we build on top of this is partly up to you. Come tell us on Discord what you'd want first.