Jack Yang

Jack Yang

Building real-time AI agents and multimodal systems

I’m a builder interested in how people and computers talk to each other. Currently at Microsoft Teams AI, I build Facilitator, an agent that joins live, multi-person meetings.

In undergrad, I double-majored in Computer Science and Communication at UW–Madison, then built XR products at Holos and worked on cross-platform UI, avatar customization, rendering, and performance at Microsoft Mesh. I am also pursuing an M.S. in Computer Science at Georgia Tech with a specialization in human-computer interaction.

Email Me Resume Schedule a Call
Location
Seattle, United States
Email
GitHub
jackyangzzh
Medium
jackyangzzh
LinkedIn
jackyangzzh

Experience

– present

Software Engineer II, Teams AI at Microsoft

Highlights

  • Build Facilitator, Microsoft's conversational AI agent for live, multi-party Teams meetings. It produces real-time notes, tracks agendas and decisions, and answers questions during the conversation for Microsoft 365 Copilot customers.
  • Contribute across Facilitator's core agent systems, including conversation orchestration, observability, tool calling, context management, and multimodal LLM work.

–

Software Engineer II, Mixed Reality (Mesh) at Microsoft

Highlights

  • Engineered the shared UI framework used by Microsoft Mesh and Avatars in Teams for millions of users. Built reusable interaction patterns, input abstractions, accessibility support, and spatial UI components used across Mesh product teams.
  • Owned the avatar rendering and performance pipeline across HoloLens, Quest, web, and desktop. Profiled CPU and GPU frame budgets, then used LOD switching and GPU instancing to maintain target framerates on constrained hardware.
  • Shipped avatar customization and character-building systems backed by data-driven content pipelines. Supported weekly releases and localization in more than 20 languages.

–

AR/VR Interaction Engineer at Holos

Highlights

  • Built AR and VR training features for gesture recognition, hand tracking, and in-context guidance.
  • Implemented multiplayer networking, 3D model ingestion, and physics-based interactions.
  • Worked with leadership on the product and technical work that helped secure a $750K U.S. Air Force contract and a place in the Techstars accelerator.

Research

–

XR Researcher (part-time) at Kats Research Group

Highlights

  • Developed a computer vision model that simulates color vision deficiency in hyperspectral images. It reached about 90% agreement in pilot color tests and ran in a color-calibrated Oculus Rift demo.

–

Graphics Researcher (part-time) at UW Graphics Group

Highlights

  • Built a real-time VR control loop mapping hand gestures to remote robotic-arm manipulation, plus a Unity UI for replaying arm motion from 15+ lab-experiment datasets.

–

HCI Researcher (part-time) at UW HCI Group, People and Robots Lab

Highlights

  • Added speech recognition and NLP-driven interaction to children's reading-companion robots; validated in an eight-family trial.

Certifications

Machine Learning Specialization from Stanford University

Human-Computer Interaction Professional Certificate from Georgia Tech

Computer Vision Nanodegree from Udacity

VR Development Professional Certificate from UC San Diego

Virtual Reality Specialisation from University of London

Interaction Design Specialization Certificate from UC San Diego

Skills

AI & Agents
Keywords:
  • LLMs / Agents
  • RAG / MCP / A2A
  • Python / PyTorch
  • OpenCV
Real-Time & Spatial Systems
Keywords:
  • Unity / C#
  • Unreal / C++
  • visionOS / Swift
  • Babylon.js / WebXR
  • Vulkan / OpenGL
Product & UI Engineering
Keywords:
  • Interaction Design
  • React / TypeScript
  • Three.js / JavaScript
  • Azure / Cloud Services

Education

–

M.S. in Computer Science (HCI) from Georgia Institute of Technology

Courses

  • Human-Computer Interaction specialization; in progress, expected May 2027.

–

Bachelor of Science from University of Wisconsin–Madison

Courses

  • Double-majored in Computer Science and Communication.
  • TA for CS 559 (Computer Graphics) for 2 semesters.

Publications

Passive frequency conversion of ultraviolet images into the visible using perovskite nanocrystals

Co-authored a peer-reviewed paper on a passive system that converts ultraviolet images into visible light using perovskite nanocrystals.

Breaking Binocular Redundancy Through Virtual Reality

Wrote a technical report and released an open-source VR demo that simulates static and dynamic binocular rivalry.

Effective Leaders: Four Attributes That Underpin The Core Characteristics of Effective Leadership

Contributed the article “Critical Conversation” to Mike Cameron’s collected volume.