Hand Tracking

The concept

Hand tracking detects 21 key landmarks per hand — wrist, joints, fingertips. WebXR Hand Input API exposes this data. The distance between thumb and index (pinch) is the standard selection gesture. In p5.js, we simulate these landmarks to visualize the algorithmic logic.

The code

// Hand tracking simulation: 21 landmarks
// Real: WebXR Hand Input API → frame.getJointPose()
// Simulation: animated hand skeleton
let joints = [];
function setup() {
  createCanvas(400, 280);
  // Simulate 21 hand landmarks (simplified)
  for (let i = 0; i < 21; i++) joints.push({x:0,y:0});
}
function draw() {
  background(11, 15, 20);
  let cx = mouseX, cy = mouseY;
  // Thumb tip (landmark 4) and index tip (8)
  let pinchDist = 30 + sin(frameCount*0.05)*20;
  let thumb = {x:cx-pinchDist/2, y:cy-20};
  let index = {x:cx+pinchDist/2, y:cy-20};
  // Draw simplified hand
  stroke(55,227,195,180); strokeWeight(2); noFill();
  line(cx,cy,thumb.x,thumb.y);
  line(cx,cy,index.x,index.y);
  fill(55,227,195); noStroke();
  ellipse(thumb.x,thumb.y,10,10);
  ellipse(index.x,index.y,10,10);
  // Pinch indicator
  let pinch = pinchDist < 35;
  fill(pinch ? [255,200,0] : [100,100,120]);
  text(pinch ? 'PINCH' : '---', cx-20, cy+30);
}

Going further

Explore other free courses on the Workshops page.