Siming Wang

3D previs has validated the need for directorial control, but non-3D users still have to build a scene first. Starting from one reference image, I use segmentation and depth estimation with iPhone AR to move objects, plan camera moves, and generate prompts for AI video.

Starting from the feedback gap in ASL learning - learners can understand a demonstration but struggle to judge whether they performed it correctly - I built an XR learning and real-time translation prototype from zero to one. It combines gesture recognition, micro-gestures, speech recognition, and an animation state machine in Unity and Meta Quest, and was iterated into the version presented at AWE USA 2025.

Built a multi-arm Unity simulation with SO-101 digital twins, YOLO vision, real-time telemetry, and a robotics data-collection pipeline during OpenAI Build Week. The prototype supports parallel testing and structured pick-data capture for robotics training.

To solve multi-system state synchronization and reliability problems in live AI interaction, I built a voice-triggered OSC state machine with Unreal Engine MetaHuman and TouchDesigner. During a three-day public exhibition at Inspace Edinburgh, the installation recorded 422 interactions and 287 completed experiences.