Gyroscope-steered shaders: making a WebGPU lab fun on phones

I built a WebGPU shader lab around a desktop mental model: move the mouse to nudge the metaballs, drag to spin the Mandelbulb, type a number to switch scenes. It was great on a laptop. On a phone it was dead. There was no pointer to track, no keyboard to type with, and the only "interaction" was an accidental scroll that did nothing because the canvas captured the gesture. A shader lab that's only fun with a mouse is a shader lab that 60% of my visitors never actually play with.

The fix wasn't a responsive layout. It was treating mobile interactivity as its own design problem.

Touch falls out of Pointer Events for free

The first win cost almost nothing. The render loop reads a mouse uniform — a normalized 0..1 position the shaders use to steer the scene. I was already listening on pointermove rather than mousemove, and Pointer Events unify mouse, touch, and pen into one stream. A finger drag fires pointermove with the same clientX/clientY shape a mouse does.

The one thing that broke it was the browser's default touch handling — a drag on the canvas scrolled the page instead of reaching my listener. Setting touch-action: none on the canvas style fixes that: it tells the browser to hand all touch gestures to my code instead of interpreting them as scroll or zoom. After that, a finger-drag nudges the scene exactly like a mouse, no extra code path.

Tilt the phone to steer

Dragging works, but the thing that actually feels like magic on a phone is steering with the device itself. Phones have a gyroscope, and the deviceorientation event surfaces it as three angles. I only use two: gamma (left-right tilt) and beta (front-back tilt).

The mapping takes a comfortable hand-held range and squashes it to 0..1:

function handleOrientation(e: DeviceOrientationEvent) {
  const gamma = e.gamma ?? 0; // left-right, -90..90
  const beta = e.beta ?? 45;  // front-back, -180..180
  const nx = (gamma + 35) / 70; // -35..35 -> 0..1
  const ny = (beta - 20) / 50;  // 20..70 -> 0..1
  tiltTarget = {
    x: Math.max(0, Math.min(1, nx)),
    y: Math.max(0, Math.min(1, ny)),
  };
}

Note that the handler writes to a tiltTarget, not directly to the mouse uniform. Raw gyro data is jittery — feed it straight in and the scene vibrates. So the render loop lerps the live mouse value toward the target every frame, with a frame-rate-aware smoothing factor:

const a = 1 - Math.exp(-dt * 6.0);
mouse.x += (tgt.x - mouse.x) * a;
mouse.y += (tgt.y - mouse.y) * a;

The 1 - exp(-dt * k) form is the trick worth keeping: it gives the same perceived smoothing whether the device runs at 60fps or 30fps, because it's integrated against real elapsed time rather than a fixed per-frame constant.

The iOS permission gesture

On iOS 13 and later, gyro access is gated. You can't just add a deviceorientation listener and start reading angles — Safari ignores it until the user explicitly grants permission, and that grant has to come from inside a user gesture (a tap), not on page load. The API for this is a static method on the event constructor that returns a promise resolving to granted or denied.

The toggle handles all three worlds — iOS that needs the prompt, Android that just works, and devices with no gyro at all:

const DOE = window.DeviceOrientationEvent as typeof DeviceOrientationEvent & {
  requestPermission?: () => Promise<"granted" | "denied">;
};
if (typeof DOE.requestPermission === "function") {
  const res = await DOE.requestPermission();
  if (res !== "granted") {
    showToast("Tilt permission denied", 2200);
    return;
  }
}
window.addEventListener("deviceorientation", handleOrientation);

The feature-detect on requestPermission is doing real work: it only exists on iOS Safari. On Android the method is absent, so we skip straight to adding the listener. On a denied grant we surface a toast and bail cleanly rather than silently doing nothing.

Only show the control where it works

A tilt button on a desktop is noise — there's no gyro to steer with. So the button doesn't render at all unless the device is both touch-capable and exposes the orientation API:

const hasOrientation = "DeviceOrientationEvent" in window;
const isTouch = navigator.maxTouchPoints > 0;
setTiltSupported(hasOrientation && isTouch);

The maxTouchPoints > 0 check matters because plenty of desktops technically have DeviceOrientationEvent defined but no actual sensor. Requiring a real touch device filters those out. This detection runs in an effect, client-side only, because there's no window or navigator during server rendering.

Haptics, guarded

Every meaningful action — switching scenes, toggling a mode, unlocking something — fires a short vibration via the Vibration API. A scene switch is a single tick; a bigger event is a pattern like [10, 30, 10]. The catch is that iOS Safari doesn't implement navigator.vibrate at all, so the helper has to be a guarded no-op:

function haptic(pattern: number | number[]) {
  if (typeof navigator === "undefined") return;
  const nav = navigator as Navigator & {
    vibrate?: (p: number | number[]) => boolean;
  };
  if (typeof nav.vibrate === "function") {
    try { nav.vibrate(pattern); } catch {}
  }
}

On Android you feel the feedback; on iOS the calls evaporate harmlessly. Either way the calling code never has to think about it.

A demo mode for the curious-but-passive

Some visitors won't tilt or drag at all. For them there's an auto-cycle demo mode: a setInterval that advances to the next scene every eight seconds, with a tiny progress bar under the active tab so it doesn't feel like a glitch. Any manual interaction — a tab tap, a number key — cancels it. It turns the lab from "thing you have to operate" into "thing that performs for you," which is the right default when someone is just watching.

The lesson I keep relearning: mobile interactivity is not a media query. The layout adapts with breakpoints, but the interaction model has to be rebuilt around the inputs the device actually has — a finger, a tilt, a buzz — not a shrunk-down version of the one it doesn't.