Jack Yang
Software Engineer II · Microsoft Teams AI Based in Seattle

Jack Yang: Real-time AI for real conversations.

I’m Jack Yang, a software engineer at Microsoft Teams AI building real-time multimodal agents for Microsoft 365 Copilot. Before that, I built avatar and XR systems for Microsoft Mesh.

Book time with me Résumé LinkedIn
or email jackyangzzh@gmail.com

About

Jack Yang, Software Engineer at Microsoft Teams AI
Now
Software Engineer II, Microsoft Teams AI
Before
Microsoft Mesh, Holos
Studying
M.S. in Computer Science (HCI), Georgia Tech
Based in
Seattle

I build real-time AI agents at Microsoft Teams AI, where I work on Facilitator, the Microsoft 365 Copilot agent that runs inside live Teams meetings. My focus is the orchestration, tool execution, context management, and multimodal features that let an agent keep up with a conversation as it happens.

Before Teams AI, I engineered the shared UI framework adopted by every product team at Microsoft Mesh and was a key contributor to its avatar rendering pipeline, shipping weekly in 20+ languages to millions of users. Earlier, I built AR/VR interaction and multiplayer systems at Holos, a Techstars startup, and helped it secure a $750K U.S. Air Force contract.

Outside work, I created FakeHaul, an AI-generated marketplace where every checkout is $0.00, covered by The New York Times and Fast Company. I’m also completing an M.S. in Computer Science (HCI) at Georgia Tech, where I research how AI systems interpret evidence in conversations. My research project, Ground Tracker, examines how later confirmation can hide a lack of support for an earlier action; the manuscript is under consideration at Dialogue & Discourse. I enjoy connecting low-latency systems with generative models, and I care as much about how the interaction feels as how the system is built.

Selected work

All 7 projects
Showreel My work in 20 seconds.
  1. FakeHaul checkout graphic showing a $1,248.50 cart reduced to $0.00
    01Personal2026

    FakeHaul

    A fictional marketplace I built with coding agents and offline AI pipelines, where every checkout ends at $0.00.
  2. Microsoft Mesh immersive event with avatars gathered around a shared presentation
    02Professional2022–2024

    Microsoft Mesh & Avatars for Teams

    The shared UI framework every Mesh product team adopted, plus avatar editing and rendering across mobile, desktop, and AR/VR.
  3. Holos immersive training scene with an aircraft cockpit model and interactive instruction panels
    03Professional2019–2022

    Holos

    Hand interaction, multiplayer networking, and 3D content workflows I built for Holos’s AR and VR training platform.

Experience

Résumé (PDF)
Work
Research
Study
Current role Expected Select a bar to jump to the entry

Industry

  1. to PresentNow

    Microsoft Teams AI

    Software Engineer II
    • Build Facilitator, a meeting agent for Microsoft 365 Copilot that takes live notes, tracks agendas and decisions, and answers questions in Teams.
    • Design the turn-taking layer of a proactive agent, deciding when and how the AI speaks into a live group conversation without interrupting.
    • Contribute to conversation orchestration, tool calling, context management, observability, and multimodal LLM features.
  2. to
    Case study

    Microsoft Mesh

    Software Engineer II, Mixed Reality
    • Engineered the shared UI framework behind Microsoft Mesh and Avatars in Teams for millions of users, defining reusable interaction patterns, input abstractions, accessibility integration, and spatial UI components adopted by every Mesh product team.
    • Built avatar customization systems with weekly releases in 20+ languages. Satya Nadella showed the outfit-selection flow I built at Microsoft Ignite.
    • Key contributor to the avatar rendering and performance pipeline across HoloLens, Quest, web, and desktop. Profiled CPU and GPU frame budgets, using LOD switching and GPU instancing to maintain target framerates on constrained hardware.
  3. to
    Case study

    Holos

    AR/VR Interaction Engineer
    • Built core AR and VR training platform capabilities, including hand tracking, gesture recognition, multiplayer networking, and 3D model ingestion workflows.
    • Worked with leadership on product and engineering execution, helping Holos secure a $750K U.S. Air Force contract and a place in Techstars.

Research

  1. to PresentNow

    Georgia Institute of Technology

    HCI Researcher
    • Built Ground Tracker, a transcript-analysis system that separates language-model perception from deterministic grounding rules.
    • Designed a 264-dialogue controlled benchmark to study how later confirmation can obscure missing evidence at the time of an earlier action.
    • Authored Preserving Action-Time Grounding Judgments in Transcript Trackers; manuscript under consideration at Dialogue & Discourse.
  2. to
    Case study

    Kats Research Group

    XR Researcher
    • Developed a computer vision model that simulates color vision deficiency in hyperspectral images with 90% accuracy, and ran it in a color-calibrated Oculus Rift demo. Co-authored an arXiv publication.
  3. to
    Case study

    UW Graphics Group

    Graphics Researcher
    • Built a real-time VR control loop mapping hand gestures to remote robotic-arm manipulation, plus a Unity UI for replaying arm motion from 15+ lab-experiment datasets.
  4. to

    UW Human-Computer Interaction Group

    HCI Researcher
    • Added speech recognition and NLP-driven interaction to children’s reading-companion robots.
    • Designed and built a collaborative-filtering recommender system, successfully trialed with eight families.

Education

Degrees

  1. to Expected

    Georgia Institute of Technology

    M.S. in Computer Science (HCI)
    • Human-Computer Interaction specialization.
  2. to

    University of Wisconsin–Madison

    Bachelor of Science
    • Double-majored in Computer Science and Communication.
    • TA for CS 559 (Computer Graphics) for 2 semesters.

Certifications

Publications

  1. Journal of Optics (IOP Publishing)

    Passive frequency conversion of ultraviolet images into the visible using perovskite nanocrystals

    Co-authored a peer-reviewed paper on a passive system that converts ultraviolet images into visible light using perovskite nanocrystals.

  2. UW–Madison technical report

    Breaking Binocular Redundancy Through Virtual Reality

    Wrote a technical report and released an open-source VR demo that simulates static and dynamic binocular rivalry.

  3. Strategically Yours

    Effective Leaders: Four Attributes That Underpin The Core Characteristics of Effective Leadership

    Contributed the article “Critical Conversation” to Mike Cameron’s collected volume.

Writing

All writing
  1. I leave AI models constantly. I have never once left the platform. Medium (opens on Medium)
  2. I built a fake shopping app. Nothing ships. It might be the most useful thing I’ve made. Medium (opens on Medium)
  3. My Top 5 Books of 2025 Medium (opens on Medium)
  4. Making a Comeback Antaeus AR (opens on Medium)