Ishani Chakraborty

About

I am a Computer Vision Researcher in Microsoft Mixed Reality, pursuing problems in Semantic Understanding, Embodied AI and AI-assisted Collaboration. My interests lies in applying geometry to constrain visual perception through physical, geometric and multi-view consistency in 3D environments. Some of my projects include developing 3D object detection algorithms for Virtual Object Overlay, NeuralScan with the Synthetics team for generating high-fidelity object scans, and more recently, the 3D Spatial Instructions with the D365 Guides team for Augmented Reality Step-by-Step Guides. Extending these learnings to the HoloAssist research initiative alongside Microsoft Research has been an especially fulfilling endeavor. Prior to my role in the Mixed Reality team, I was at Microsoft AI & R where I developed the Multi-Object Tracking system for Azure’s Spatial Analysis API.

My previous stint was as a researcher at SRI International in Princeton, NJ, where I participated in a range of DARPA and IARPA funded projects, including the Aladdin project for Large-scale Video Understanding. I received my Ph.D. in Computer Science from Rutgers University, specializing in Computer Vision and Probabilistic Graphical Models. Throughout my career, I have participated in delivering successful prototypes, won research grants and primary author of numerous publications and patents.

Links to some of my recent projects:

• HoloAssist (opens in new tab): A large-scale egocentric human interaction dataset benchmark.
• Virtual Object Overlay (opens in new tab), Precise 3D Pose Estimation of small and medium sized objects for interacting in Augmented Reality.
• Semantic Geometric Label Fusion (opens in new tab) developed on top of Hololens2forCV codebase.
• Re-animator (opens in new tab): Converting everyday objects into digital assets using 3D Neural Rendering techniques.