A few months ago, Zac and I won the "Future of Health" track at UK Hack — a16z's London hackathon with Mistral AI.
The idea was dumb in the best way: point your phone camera at your face, and an LLM tells you your biological age, heart rate, sleep quality, and whether you look like you've been making good decisions lately.
No hardware, just your face
Most health tracking requires hardware. Wearables, blood tests, clinic visits. We wanted to see how far we could get with just a phone camera and a good model.
Turns out, pretty far.
Mistral's Pixtral-8b can look at a 10-second video of your face and extract a surprising amount of signal. Skin texture, micro-expressions, eye movement patterns. Feed it the right prompts and it starts outputting things that actually correlate with clinical biomarkers.
We built a Next.js app that captures video, ships it to a Python backend, runs it through Pixtral, and returns a full health dashboard. Functional age, cardiovascular indicators, cognitive engagement score.
LLM meets old-school CV
Mixture of models. Pixtral handles the high-level stuff — skin texture, facial structure, signs of aging. But for the real biomarkers, we layered in traditional computer vision techniques that don't need an LLM.
Pupillary movements give you cognitive load and neurological signals. Blood flow detected through subtle skin color changes (remote photoplethysmography) gets you heart rate and variability. Eye movement patterns reveal sleep quality and fatigue.
The LLM ties it all together — interpreting the signals, generating the narrative, making sense of the numbers. But the actual extraction is a mix of old and new.
20 hours later
- Frontend: Next.js on Vercel
- Backend: Python/Flask on Heroku
- Database: Supabase
- Model: Mistral Pixtral-8b
- Build time: ~20 hours
Brilliant or insane?
LLMs are weirdly good at reading faces. Not in a creepy way — in a "this is probably useful for healthcare" way. The outputs were consistent enough that we could validate them against traditional approaches for functional age estimation.
The judges seemed to agree. We took home the health track, which felt good given we'd spent most of the night arguing about whether this was brilliant or insane.
Probably both.
Bonus: Parliament
A few weeks later, we got invited to drinks at the Houses of Parliament with MPs and winners from the other tracks. Not bad for a weekend hack.
