VisualSounds: Eye-Gaze Detection

VisualSounds is the first building stone of a larger idea I’m assembling quietly.

What I’m sharing now is the foundation: an iOS system that detects where your eyes are looking on the screen, built with the iPhone TrueDepth / Face ID camera stack, ARKit face tracking, personalized on-device gaze calibration, and live inference. Think of it as learning to read attention itself — fixations, saccades, looking away — from the device you already hold.

What it unlocks is still under wraps. I’m only giving glimpses: a new kind of media consumption experience, one that listens to where attention lands and answers back through text-to-audio transformers. More soon.

For now, the stone on the table is gaze — precise, personal, on-device.

Repository (private).

#iOS #ARKit #TrueDepth #EyeTracking #OnDeviceML #Swift #TextToAudio #Media