Software engineerSeattle

Jack Yang

AI agents for live conversations

Jack Yang
View projects Email Me Resume PDF (opens in a new tab) Schedule a Call (opens in a new tab)

I build systems that have to hold up inside a live conversation. At Microsoft Teams AI, I work on Facilitator: a meeting agent that needs to know when to contribute and when to stay out of the way.

Before that, I built training tools at Holos and worked on avatars, UI, and rendering for Microsoft Mesh. The thread goes back to a VR interview coach I built in 2020. I’m still interested in the same question: how can a computer join a conversation without taking it over?

Location
Seattle, United States
Email
GitHub
jackyangzzh
Medium
jackyangzzh
LinkedIn
jackyangzzh

Experience

– present

Software Engineer II, Teams AI at Microsoft

Highlights

  • Build Facilitator, a meeting agent for Microsoft 365 Copilot that takes live notes, tracks agendas and decisions, and answers questions in Teams.
  • Contribute to conversation orchestration, tool calling, context management, observability, and multimodal LLM features.

–

Software Engineer II, Mixed Reality (Mesh) at Microsoft

Highlights

  • Built reusable UI components, input handling, and accessibility support for the shared framework behind Microsoft Mesh and Avatars in Teams, products used by millions of people.
  • Owned the avatar rendering and performance pipeline across HoloLens, Quest, web, and desktop. Profiled CPU and GPU frame budgets, then used LOD switching and GPU instancing to maintain target framerates on constrained hardware.
  • Shipped avatar customization and character-building systems backed by data-driven content pipelines. Supported weekly releases and localization in more than 20 languages.

–

AR/VR Interaction Engineer at Holos

Highlights

  • Built AR and VR training features for gesture recognition, hand tracking, and in-context guidance.
  • Implemented multiplayer networking, 3D model ingestion, and physics-based interactions.
  • Worked with leadership on the product and engineering work that helped Holos secure a $750K U.S. Air Force contract and a place in Techstars.

Research

–

XR Researcher (part-time) at Kats Research Group

Highlights

  • Developed a computer vision model that simulates color vision deficiency in hyperspectral images. It reached about 90% agreement in pilot color tests and ran in a color-calibrated Oculus Rift demo.

–

Graphics Researcher (part-time) at UW Graphics Group

Highlights

  • Built a real-time VR control loop mapping hand gestures to remote robotic-arm manipulation, plus a Unity UI for replaying arm motion from 15+ lab-experiment datasets.

–

HCI Researcher (part-time) at UW HCI Group, People and Robots Lab

Highlights

  • Added speech recognition and NLP-driven interaction to children's reading-companion robots; validated in an eight-family trial.

Certifications

Machine Learning Specialization from Stanford University

Human-Computer Interaction Professional Certificate from Georgia Tech

Computer Vision Nanodegree from Udacity

VR Development Professional Certificate from UC San Diego

Virtual Reality Specialization from University of London

Interaction Design Specialization Certificate from UC San Diego

Skills

AI & Agents
Keywords:
  • LLMs / Agents
  • Retrieval-Augmented Generation (RAG)
  • Model Context Protocol (MCP)
  • Agent-to-Agent (A2A)
  • Python / PyTorch
  • OpenCV
Real-Time & Spatial Systems
Keywords:
  • Unity / C#
  • Unreal / C++
  • visionOS / Swift
  • Babylon.js / WebXR
  • Vulkan / OpenGL
Product & UI Engineering
Keywords:
  • Interaction Design
  • React / TypeScript
  • Three.js / JavaScript
  • Azure / Cloud Services

Education

–

M.S. in Computer Science (HCI) from Georgia Institute of Technology

Courses

  • Human-Computer Interaction specialization; in progress, expected May 2027.

–

Bachelor of Science from University of Wisconsin–Madison

Courses

  • Double-majored in Computer Science and Communication.
  • TA for CS 559 (Computer Graphics) for 2 semesters.

Publications

Passive frequency conversion of ultraviolet images into the visible using perovskite nanocrystals

Co-authored a peer-reviewed paper on a passive system that converts ultraviolet images into visible light using perovskite nanocrystals.

Breaking Binocular Redundancy Through Virtual Reality

Wrote a technical report and released an open-source VR demo that simulates static and dynamic binocular rivalry.

Effective Leaders: Four Attributes That Underpin The Core Characteristics of Effective Leadership

Contributed the article “Critical Conversation” to Mike Cameron’s collected volume.

© Jack Yang