AR Interaction Design — Level 2: Intermediate

Abstract: This intermediate course deepens AR interaction design skills with a focus on spatial interaction patterns, ergonomics, structured user testing, and multi-anchor scenes. Students work with ARKit/ARCore (via Unity AR Foundation or 8thWall) and tackle a cultural mediation or educational AR project. Outline: Spatial UX patterns → Ergonomics & comfort → Multi-anchor scenes → Spatial audio → Structured user testing → Iteration based on data → Visual design in AR → Portfolio project (cultural mediation).

Learning Objectives

By the end of this course, students will be able to:

1. Analyze (analyzing) an existing AR experience against criteria of affordance, comfort, and spatial coherence.
2. Design (creating) advanced interaction patterns: near/far interaction, dwell, voice, body-locked vs world-locked UI.
3. Implement (applying) a multi-anchor AR scene with spatial audio and state transitions.
4. Conduct (applying) structured user tests and extract actionable insights.
5. Iterate (evaluating + creating) a prototype based on real test data.
6. Produce (creating) a documented, presentable, and testable AR cultural mediation project.

Module 1 — Spatial Interaction Patterns (2h)

Concept

AR interaction patterns differ from 2D interface patterns through their spatial dimension and physical constraints.

Fundamental patterns:

Reference: Apple Human Interface Guidelines — Spatial UI: https://developer.apple.com/design/human-interface-guidelines/spatial-ui

Lab 1.1 — Pattern Inventory

Steps:

1. Choose 2 known AR applications (Pokémon GO, IKEA Place, Google Lens, or other)
2. For each, list all interaction patterns present (5 min of active testing)
3. For each pattern: name it, describe the trigger, describe the feedback
4. Compare the two apps: which patterns does one have that the other lacks?

Mini-exercise: Design on paper the interaction flow (user flow) for an AR experience of your choice — 5 steps max, each step with its main pattern.

Module 2 — AR Ergonomics and Comfort (2h)

Concept

AR imposes specific physical constraints. Ignoring them leads to tiring or painful experiences.

Main problems and solutions:

Visual comfort zone: between 0.5 m and 10 m from the user, at eye level (±30°).

Source: Caudell, T., & Mizell, D. (1992). Augmented Reality: an application of heads-up display technology to manual manufacturing processes. HICSS, pp. 659–669.

Lab 2.1 — Ergonomic Audit

Steps:

1. Take the Level 1 prototype (or an existing prototype)
2. Have 2 people test it for 5 minutes each
3. Measure: at what height does the user hold their phone? Do they raise their arms? Do they move their feet?
4. Identify 1 ergonomic problem and propose a design correction

Mini-exercise: Draw the "sweet spot" of your AR experience: where in physical space is the interaction optimal? At what distance, at what height?

Module 3 — Advanced Spatial UI (2h)

Concept

In AR, the graphical interface must coexist with the real world. The four types of spatial UI (covered in the Bible) have distinct use cases.

Design rules:

• World-locked (default): best spatial coherence, avoids fatigue
• Body-locked (e.g. palette on wrist): reserved for frequently used tools
• Head-locked: avoid except for critical, transient alerts (< 3 seconds)
• Diegetic: prefer when the interface can be part of the narrative

Typography in AR:

• Minimum size: 80 pt at 1 m distance (experimentation needed per platform)
• Semi-transparent background behind text if the environment may be busy
• Contrast: white on dark background OR black on light background — avoid bright colors on variable backgrounds

Reference: Microsoft MRTK UX Guidelines: https://docs.microsoft.com/windows/mixed-reality/design/typography

Lab 3.1 — Prototyping a Spatial UI

Steps:

1. Choose a scenario: AR museum guide, assembly assistance, or plant exploration
2. Design 3 UI variants (world-locked / body-locked / diegetic)
3. Paper prototype each in 5 minutes
4. Have 2 people choose the most natural variant — note their reasons

Mini-exercise: For your capstone project, define: which UI type is most appropriate and why in 3 sentences.

Module 4 — Multi-Anchor Scenes (2h)

Concept

An AR experience can contain multiple objects anchored to different surfaces, image anchors, and objects persistent across sessions.

Multi-anchor scene architecture:

• Each anchor has a unique identifier
• Objects reference their parent anchor
• Transitions between anchors are animated (fade, movement)
• Persistence (cloud anchors) requires a backend service

With A-Frame + AR.js: html <!-- Two different markers trigger two different objects --> <a-marker type="pattern" url="pattern-a.patt"> <a-sphere position="0 0.5 0" color="blue"></a-sphere> </a-marker> <a-marker type="pattern" url="pattern-b.patt"> <a-cone position="0 0.5 0" color="red"></a-cone> </a-marker>

A-Frame marker generator: https://jeromeetienne.github.io/AR.js/three.js/examples/marker-training/examples/generator.html

Lab 4.1 — Scene with 3 Markers

Steps:

1. Generate 3 custom markers (simple images)
2. Create an A-Frame scene that displays a different object on each marker
3. The 3 objects form a coherent narrative ensemble (e.g. 3 steps of a process, 3 states of the same object)
4. Test with all 3 markers printed on the same A4 sheet

Mini-exercise: How do you indicate to the user that there are other markers to find? Design the affordance (without explicit text).

Module 5 — Spatial Audio in AR (2h)

Concept

Spatial audio (3D audio) anchors sounds in physical space. An AR object on the left emits from the left; as the user approaches, the sound grows louder.

Why spatial audio is critical:

• Compensates for the absence of haptics
• Directs attention toward off-screen objects
• Reinforces the AR object's presence

In A-Frame: html <a-box position="-2 1 -3"> <a-sound src="#mysound" autoplay="true" positional="true" rolloff-factor="1"></a-sound> </a-box>

Free audio sources (CC0): https://freesound.org · https://pixabay.com/music/

Lab 5.1 — Spatial Sound Feedback

Steps:

1. Take the scene from Lab 4.1 (3 markers)
2. Add a different sound to each object (3 sounds < 2 sec, CC0)
3. Sounds trigger on hover of each object
4. Test: does the sound come from the right place in space?

Mini-exercise: Identify 3 situations in your capstone project where sound can replace or complement visual feedback.

Module 6 — Structured User Testing (2h)

Concept

A structured user test produces comparable data across participants. Methods suited to AR:

Think-aloud protocol: the user verbalizes what they are doing and feeling in real time.

Eriksson, M., & Wiberg, M. (2003). Towards a theory of mobile work.

Structured observation grid:

• Task completed: yes/no
• Completion time
• Number of errors
• Hesitation moments (> 3 sec)
• Spontaneous comments

SUS (System Usability Scale): standardized 10-question questionnaire (Brooke, 1996) that calculates a score from 0 to 100.

Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In Jordan et al. (Eds.), Usability Evaluation in Industry. Taylor & Francis. https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

Lab 6.1 — Structured Test with SUS

Steps:

1. Define 2 concrete tasks for your prototype (e.g. "Place the blue object on the table" + "Activate the red object")
2. Prepare the observation grid (columns: participant / task 1 completed / time / errors / hesitations)
3. Test with 3 people
4. Have each tester fill in the SUS questionnaire after the session
5. Calculate the average SUS score

Mini-exercise: Compare your SUS score with benchmarks (< 50 = poor, 50–70 = marginal, 70–85 = good, > 85 = excellent). Identify the most frequent problem in your data.

Module 7 — Data-Driven Iteration (2h)

Concept

Iteration in AR design follows the classic cycle: prototype → test → analyze → correct → re-test. The AR-specific feature is that corrections may touch visual design, interaction logic, AND the target hardware simultaneously.

Prioritizing corrections:

• Blockers (error preventing task completion): fix immediately
• Major friction (confusion, delay): fix before the next test
• Cosmetic (preferences, aesthetics): defer

AR design sprint: 1 week to test an interaction hypothesis — inspired by Jake Knapp, Sprint (2016).

Lab 7.1 — One Documented Iteration

Steps:

1. Take the results from Lab 6.1
2. Identify problem #1 (most frequent/blocking)
3. Propose 2 different solutions on paper (2 variants)
4. Implement the chosen solution in the prototype
5. Re-test with 1 person: is the problem resolved?

Mini-exercise: Document the correction cycle: observed problem → solution hypothesis → change made → re-test result. 1 paragraph, factual.

Module 8 — Visual Design in AR (2h)

Concept

AR objects coexist with real, variable, and unpredictable photographic environments. AR visual design must be robust to this variability.

Principles:

• Adaptive contrast: AR objects must remain legible on both light and dark backgrounds
• Drop shadow: simulating a shadow beneath the AR object significantly reinforces depth perception
• PBR materials (Physically Based Rendering): respond to estimated ambient lighting → visual coherence
• Avoid pure white and pure black: too much contrast with natural environments

In A-Frame, reflective material: html <a-sphere material="color: #888888; metalness: 0.5; roughness: 0.2; envMap: #envmap"></a-sphere>

Accessibility check: WCAG 2.1 AA-compliant colors (contrast ratio ≥ 4.5:1 for text).

Lab 8.1 — Visual Prototype Review

Steps:

1. Test your prototype in 3 different environments (indoor artificial light, indoor by window, outdoor in shade)
2. Photograph each situation: are the AR objects legible?
3. Identify visual failures (crushing brightness, loss of contrast)
4. Apply 2 visual corrections: simulated drop shadow + less saturated material

Capstone Project — "AR Cultural Mediation"

Brief

Design and prototype an AR cultural mediation experience for an existing space (museum, library, public space, classroom). The experience must enrich understanding of an object, a place, or a concept.

Constraints

• Technology: A-Frame + AR.js or 8thWall (WebAR)
• Target duration: 3–5 minutes for a user
• At least 2 different interaction types (tap-to-place + spatial audio, or marker + dwell)
• Tested with a minimum of 3 users (SUS + observation grid)
• Iterated at least once based on test results

Deliverables

1. Functional prototype (link or files)
2. Design report (3–5 pages): concept, user journey, 5 design decisions with justification, test results (raw data + analysis)
3. Presentation (10 min): demo + test results + reflection on what worked / didn't work

Evaluation Rubric

Readings & Resources

• Apple HIG — Spatial UI: https://developer.apple.com/design/human-interface-guidelines/spatial-ui
• Microsoft MRTK Design: https://docs.microsoft.com/windows/mixed-reality/design/
• Brooke, J. (1996). SUS: A "quick and dirty" usability scale. https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
• Freesound.org (CC0 sounds): https://freesound.org
• AR.js docs: https://ar-js-org.github.io/AR.js-Docs/
• 8thWall: https://www.8thwall.com
• Knapp, J. (2016). Sprint. Simon & Schuster.

→ Previous level: AR Interaction Design — Level 1 → Next level: AR Interaction Design — Level 3 → Custom workshop: Book a 30-min call — free