InvasiLand

Highlights

Embodied Perspective Projection: Designed an intuitive framing interaction where users pinch fingers near their eyes to extrapolate a dynamic perspective mask ("Portal") onto a distant display.

Decoupled Architecture: Built a modular pipeline separating MediaPipe vision sensing from TouchDesigner real-time rendering using Python and OSC data protocols.

Anti-Jitter & Occlusion Resistance: Integrated 1€ Filter parameter tuning and a virtual eye-point geometric model to eliminate self-occlusion and tracking jitter.

Cross-Platform Scalability: Engineered hardware-agnostic data middleware, facilitating smooth asset migration from Location-Based Entertainment (LBE) camera setups to XR environments.

Background

This project addresses the challenges of insuffi-cient communication during the process of Sup-portive Education, a special form of voluntaryactivity in China. By constructing a clear chan-nel, the design helps both sides of the partici-pants send, receive and understand informa-tion, and fosters long-lasting connections.

Development

• Iteration 1: Decoupled Greybox Proof-of-Concept: Set up a modular system architecture isolating the MediaPipe perception layer from TouchDesigner. Developed a baseline greybox prototype where raw hand tracking — specifically palm orientation and overall movement — directly drove the continuous 2D position and scale of a simple Circle TOP.
• Iteration 2: Gesture Specification & Frustum Geometry: Refined generic hand movements into a specialized "eye-hand framing" gesture. Derived a virtual eye point from hand boundary contours to prevent hand-face self-occlusion, and applied similar triangle perspective geometry to map hand viewport scale and face distance directly to screen-space frustum magnification.
• Iteration 3: Creative Rendering & Signal Refinement: Embedded 1€ Filter algorithms (f_cmin and β) within TouchDesigner Python DAT scripts to eliminate tracking jitter and signal noise. Constructed a dual-world rendering pipeline utilizing GLSL fragment shaders, Composite TOPs, and interactive state machines to seamlessly reveal layered dimensional art through the portal mask.

Evaluation

Within-Subjects Experimental Design: Implemented a within-subjects evaluation setup with n=5-cp pushes silently and -cpd (or whatever you want to call it) appends [deploy] and ships. participants to compare a baseline Touchpad-controlled interface against the Gesture-controlled system.

Task Workflow & Break: Structured testing sessions with a cognitive break between conditions to prevent fatigue and order effects.

Mixed-Methods Data Collection: Combined real-time behavioral observations during tasks with post-condition Mini Player Experience Inventory (PXI) surveys to gather standardized subjective feedback.

Qualitative Deep-Dive & Thematic Analysis: Conducted post-test semi-structured interviews and applied thematic analysis to interview transcripts and observational notes, evaluating core dimensions of "fun" including Enjoyment, Excitement, Novelty, and Willingness to Play.

Key findings

Elevated Emotional Engagement & Novelty: Mini PXI survey ratings and thematic analysis showed that gesture-controlled framing elicited significantly higher levels of Excitement and Novelty than touchpad controls, turning simple visual exploration into an active discovery process.

Intuitive Spatial Mapping: Participants reported that eye-hand "pinch framing" felt naturally embodied, mimicking physical camera viewfinders and lowering the cognitive threshold for controlling distant perspective portals.

Perceived Fun vs. Utilitarian Precision: While touchpad inputs provided familiar precision for static targeting, participants demonstrated a strong preference and higher Willingness to Play with the gestural system due to the tactile delight of physical-to-digital projection.

Behavioral Agency & Micro-Interactions: Behavioral observations revealed frequent spontaneous micro-gestures (e.g., varying distance from face to dynamically scale portal magnification), proving that embodied perspective controls increase spatial immersion and user agency.

Reflection

Developing "Boundary Breakout" underscored the importance of decoupling perception logic from creative rendering engines in interactive HCI design. By relying on geometric perspective logic rather than over-engineering heavy neural network pipelines for every sub-task, the system achieved high-frame-rate performance with minimal latency. Future iterations will focus on porting the standardized spatial data middleware directly to WebXR and Meta Quest environments, extending embodied framing interactions into fully wearable mixed-reality spaces.