Microsoft Research India: Celebrating 20 years of innovation

Microsoft Research India launched in Bangalore

Microsoft Research announced the launch of its operations in India. The Bangalore lab, the third Microsoft Research facility outside the United States, will conduct long-term basic and applied research. Microsoft Research India (MSRI) will initially start investigating the areas of geographic information systems (GIS), technologies for emerging markets, multilingual systems and sensor networks.

UIs for Low-Literate Users

One of the greatest challenges in providing information and communication technology access is that about 775 million people in the world are completely non-literate and many are able to read only with great difficulty and effort. Even though mobile phone penetration is growing very fast, people with low literacy have been found to avoid complex functions and primarily use mobile phones for voice communication only. Using “Text-Free UIs” and design principles we developed through research and rigorous user evaluations, we designed three PC and mobile phone-based applications for 1) job-search for the informal labor market; 2) health-information dissemination; and 3) mobile-phone-enabled banking and payments.

Digital Green: Participatory Video for Agricultural Extension

Digital Green (opens in new tab) is a research project that seeks to disseminate targeted agricultural information to small and marginal farmers in India by using digital video. The unique components of Digital Green are: (1) a participatory process for content production; (2) a locally generated digital video database; (3) a human-mediated instruction model for dissemination and training; and (4) regimented sequencing to initiate new communities. Unlike some systems that expect information or communication technology alone to deliver useful knowledge to marginal farmers, Digital Green works with existing, people-based extension systems and aims to amplify their effectiveness. While video provides a point of focus, it is people and social dynamics that ultimately make Digital Green work. (This technology was transferred to digitalgreen.org (opens in new tab).)

99Dots

99DOTS (opens in new tab) is a technology-enabled project focusing on medication adherence for anti-tuberculosis (TB) drugs. Treatment programs wrap each anti-TB blister pack in a custom envelope, which hides phone numbers behind the medication. Patients can only see these hidden numbers after dispensing their pills. After taking daily medication, patients make a free call to the hidden phone number. The combination of the call and the patient’s caller ID yields high confidence that the dose was “in-hand” and they took the dose. Patients receive a series of daily reminders (via SMS and automated calls). Missed doses trigger SMS notifications to care providers, who follow up with personal, phone-based counseling. Real-time adherence reports are also available on the web. (This technology was transferred to 99DOTS.org (opens in new tab).)

Mélange

The goal of Project Mélange is to understand the uses of and build tools around code-mixing. Multilingual communities exhibit code-mixing, that is, mixing of two or more socially stable languages in a single conversation, sometimes even in a single utterance. This phenomenon has been widely studied by linguists and interaction scientists in the spoken language of such communities. However, with the prevalence of social media and other informal interactive platforms, code-switching is now also ubiquitously observed in user-generated text. As multilingual communities are more the norm from a global perspective, it becomes essential that code-switched text and speech are adequately handled by language technologies and intelligent agents. In 2020, we proposed the first benchmark for code-switching, GLUECoS, which spans 7 NLP tasks in English-Hindi and English-Spanish. We also released the first code-switched NLI dataset in Hindi-English based on Bollywood movie dialogues. Although we continue working on various aspects of multilingual systems and code-switching, the primary focus of our group is on Project ELLORA.

HAMS: Harnessing AutoMobiles for Safety

In the Harnessing AutoMobiles for Safety, or HAMS, project, we use low-cost sensing devices to construct a virtual harness for vehicles. The goal is to monitor the state of the driver and how the vehicle is being driven in the context of the road environment. We believe that effective monitoring leading to actionable feedback is key to promoting road safety. As part of the project, we have also explored several use cases for HAMS. One of the earliest we prototyped was a fleet management dashboard, which allowed a supervisor to view safety-related incidents offline. We have also piloted HAMS in the context of driver training, in collaboration with the Institute of Driving and Traffic Research (IDTR), run by Maruti-Suzuki, the largest passenger car manufacturer in India. More recently, we have been working with several state transport departments on using HAMS for automated license testing.

Accessibility and assistive technology

The work on accessibility at MSR India has spanned the range from spatial audio with HoloLens to the use of feature phones to reach children with vision impairments and a spectrum of tangible toys to enhance numeracy for them, to a quiz platform for the Deaf or Hard of Hearing community, with an overarching new methodology called Ludic Design for Accessibility. Most of this work has been focused on the people with disabilities (PwDs) in the Global South and firmly rooted in the lived experience of PwDs by strong partnerships with disabled peoples’ organizations. The interdisciplinary nature of the work needed to address the complex challenges has naturally attracted a diverse set of people to work on these projects.

Digital Labor: Project Karya

Karya (opens in new tab) aims to enable supplemental income opportunities for people in low-income and marginalized communities by connecting them to AI-enabled digital work. We have built a smartphone-based digital work platform that makes a wide variety of language-based digital tasks accessible to people in low-income communities, particularly in rural India. The open-sourced platform is being used for several major data collection efforts in India. In addition to being a source of income, participating in such a platform can boost the digital skills of the workers. Our recent research also shows that digital work can be a great mechanism to deliver knowledge and skills to users. This technology was transferred to Karya.in (opens in new tab).

Always Encrypted

Always Encrypted is a feature designed to protect sensitive data, such as credit card numbers or national identification numbers (e.g. U.S. social security numbers), stored in Azure SQL Database or SQL Server databases. Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the database engine ( SQL Database or SQL Server). As a result, Always Encrypted provides a separation between those who own the data (and can view it) and those who manage the data (but should have no access). By ensuring on-premises database administrators, cloud database operators, or other high-privileged, but unauthorized users, cannot access the encrypted data, Always Encrypted enables customers to confidently store sensitive data outside of their direct control. This allows organizations to encrypt data at rest and in use for storage in Azure, to enable delegation of on-premises database administration to third parties, or to reduce security clearance requirements for their own DBA staff.

Gandiva: Scheduler for DNNs

Gandiva is a cluster scheduling framework that uses domain-specific knowledge of deep learning to improve the efficiency of training deep-learning models in a GPU cluster. By co-design of the cluster scheduler and the deep-learning framework (e.g. pyTorch), Gandiva is able to communicate richer information and exercise richer control between the two layers, enabling better scheduling.

Blockene

Blockene is a new blockchain architecture that reduces resource usage at member nodes by orders of magnitude, requiring only a smartphone to participate in block validation and consensus. Despite being lightweight, Blockene provides high throughput and scales to millions of participants. Blockene consumes negligible battery power and data in smartphones, enabling millions of users to participate in the blockchain without incentives, to secure transactions with their collective honesty. Blockene achieves these properties with a novel split-trust design, based on delegating storage and gossip to untrusted nodes. Blockene provides a throughput of more than 1,000 transactions per second. It can run with very low resource usage on smartphones, pointing to a new paradigm for building trustworthy, decentralized applications.

DoWhy: Causal Reasoning for Designing and Evaluating Interventions

Today’s computing systems can be thought of as interventions in people’s work and daily lives. But what are the outcomes of these interventions, and how can we tune these systems for desired outcomes? In this project we are building methods to estimate the impact of changes to a product feature or a business decision before actually committing to it. These questions require causal inference methods; without an A/B test, patterns and correlations can lead us astray. We have used some of our latest research to build a software library, DoWhy (opens in new tab), that provides a unified interface for causal inference methods and automatically tests their robustness to assumptions. Refer to the paper and the software library on GitHub (opens in new tab) for more details.

Sankie

Project Sankie infuses data-driven techniques into engineering processes, development environments, and software lifecycles of large services. Sankie’s goal is to consume data from static and dynamic features of a system, learn from them, and provide meaningful insights that can be used to make decisions for developing/reviewing, testing, deploying, monitoring, and root-causing.

Easy Secure Multi-party Computation

Easy Secure Multi-party Computation (EzPC) is an open-source framework aimed at accelerating the AI model validation process while also ensuring dataset and model privacy. It’s the result of a collaboration among researchers with backgrounds in cryptography, programming languages, machine learning, and security. EzPC is based on secure multiparty computation (MPC)—a suite of cryptographic protocols that enable multiple parties to collaboratively compute a function on their private data without revealing that data to one other or any other party. EzPC makes it easy for all developers, not just cryptography experts, to use MPC as a building block in their applications while providing high computational performance.

ELLORA: Enabling Low Resource Language

The main goal of ELLORA is to positively impact underserved communities by enabling language technology to create economic opportunities, build technological skills, enhance education, and preserve local language and cultures for future generations. ELLORA aims to do this in three ways: (1) Create new/Innovative methodologies for data design and collection, e.g., gamification of data collection and crowdsourcing; (2) Design new techniques and technology framework/architecture for low-resource languages, and build speech and NLP systems for low-resource languages; (3) Carry out at-scale deployments of language technology applications that impact the community.

Technology and Empowerment | India

Technology and Empowerment (TEM) is a research umbrella at Microsoft Research India. We build and study socio-technical systems that seek to have a meaningful impact on the daily lives of underserved people and communities. Our interdisciplinary team includes technologists, social scientists, linguists, and designers, united in a passion for how technology can genuinely empower otherwise marginalized populations. Over the years, we have worked across a number of domains, including health, education, language technologies, accessibility, social media and society, future of work, road safety, sustainability, financial inclusion, agriculture, and more. Our work often includes partnerships with NGOs, startups, and government organizations to deploy and evaluate our solutions in the real world. Though our presence in India offers a powerful context for our research, much of our work has also been applied in other parts of the world. The TEM research area is closely related to the Center for Societal Impact through Cloud and Artificial Intelligence (SCAI). Research in TEM often leads to prototypes, pilots, and exploratory studies. As projects become more mature, they often draw upon SCAI to synergize scale-up activities.

EdgeML

EdgeML is an open-source machine learning library for enabling privacy-preserving, energy-efficient, off-the-grid intelligence in low-resource computing devices. EdgeML enables even tiny, resource-constrained IoT devices to run machine learning algorithms locally—without connecting to the cloud—while eliminating concerns about latency or energy and ensuring privacy and security. With EdgeML, classical machine learning tasks such as activity recognition, gesture recognition, and regression can be efficiently performed on tiny devices like the Arduino Uno, with as little as 2 kilobytes of RAM.

Dependable IoT

Billions of “internet of things” (IoT) devices are currently deployed in farms, factories, and smart buildings and cities to monitor various kinds of data. Dependable IoT aims to provide a simple and easy way to remotely measure and observe the health of a sensor, and to empower users to specify their acceptable data quality threshold driven by the application requirements. This project offers the ability to automatically generate a sensor fingerprint, which captures the unique electrical properties of the sensor, such as voltage and current. This can be measured alongside the data being captured on the IoT device in real-time.

LITMUS: Linguistically Inspired Training and testing of MUltilingual Systems

Transformer-based language models have revolutionized the field of natural-language processing (NLP), have shown great improvements in various benchmarks, and are being used to power many NLP applications today. Multilingual versions of these models could serve many low-resource languages, for which labeled data is not available, by using the zero-shot paradigm. However, deployment challenges remain, including the question of evaluating the models across a wide variety of languages that may not be present in standard evaluation benchmarks. The goal of Project LITMUS is to discover strategies to evaluate massive multilingual models and to suggest data collection and training strategies to improve their performance.

Reliable Machine Learning

The Reliable Machine Learning project addresses unified questions of model stability, fairness, and explanation. We believe that fundamental connections exist between stability (generalization), fairness, and explainability of an ML model. Having one without the other two is not useful: all three should be met for an ML model to deliver its stated objective in a high-stakes application. If a fair and explainable model is not stable across data distributions, its stated properties can vary over time and across domains. Similarly, stable and fair models that cannot be explained are difficult to debug or improve. And a stable and explainable model without fairness guarantees may be unacceptable for many applications.

Extreme classification

Extreme classification is a growing research area in computer vision that focuses on multi-class and multi-label problems involving large numbers of labels (ranging from thousands to billions). Applications of extreme classification have been found in diverse areas, including recognizing faces, retail products and landmarks as well as image and video tagging. Extreme classification reformulations have led to significant gains over traditional ranking and recommendation techniques for both machine learning and computer vision applications, leading to their deployment in products used by millions of people worldwide. To foster research in extreme classification, we have released datasets, codebases, benchmarks, and other useful resources at The Extreme Classification Repository (manikvarma.org).

VeLLM

While LLMs help people address real-world challenges and applications in various domains, a digital divide exists that may exclude large populations from contributing to and benefiting from this technological revolution due to factors such as language, income, digital awareness, and access to information. To address this issue, Project VeLLM (UniVersal Empowerment with Large Language Models) is focused on developing a principled approach to enable inclusive applications of LLMs for all languages and cultures worldwide. This interdisciplinary project, conducted in collaboration with partners across Microsoft, is addressing fundamental research problems that present barriers for making LLMs inclusive for everyone. Shiksha copilot, an AI-powered digital assistant that frees teachers in India for more student mentoring and professional development by helping them quickly create comprehensive, age-appropriate lesson plans, is a good example of VeLLM in action.

CodePlan

Tools like GitHub Copilot, which are powered by LLMs, Tools like GitHub Copilot, which are powered by LLMs, offer high-quality solutions to localized coding problems. Repository-level coding tasks are more complex and cannot be solved directly using LLMs, because code within a repository is interdependent and the entire repository may be too large to fit into the prompt. We frame repository-level coding as a planning problem and present a task-agnostic framework, called CodePlan, to solve it. CodePlan synthesizes a multi-step chain of edits (plan), where each step results in a call to an LLM on a code location with context derived from the entire repository, previous code changes, and task-specific instructions. CodePlan is based on a novel combination of an incremental dependency analysis, a change may-impact analysis, and an adaptive planning algorithm.

HealthBots

Studies show that more than 70% of patients and their caregivers experience anxiety prior to undergoing an invasive treatment. Additionally, over 80% require timely, trustworthy, detailed, and accurate information about their treatment. Providing such information could alleviate pre- and post-operative anxiety. To address this issue, we designed and developed chatbots, powered by state-of-the-art generative AI models fine-tuned on the doctor’s provided knowledge base. These HealthBots aim to help patients and their caregivers get answers to their questions regarding pre- and post-treatment. The bots are designed to be multimodal, supporting both voice and text interactions, and also multilingual.

PromptWizard

The large language models (LLMs) that are transforming AI rely on prompts to produce relevant and meaningful outputs. But creating prompts that can help with complex tasks is a time-intensive and expertise-heavy process, often involving months of trial and error. PromptWizard (PW) is a research framework that automates, streamlines, and simplifies prompt optimization, combining iterative feedback from LLMs with efficient exploration and refinement techniques to create highly effective prompts within minutes. Central to PW is its self-evolving and self-adaptive mechanism, where the LLM iteratively generates, critiques, and refines prompt instructions and in-context learning examples in tandem. This process ensures continuous improvement through feedback and synthesis, significant gains in task performance, and a substantial reduction in API calls, token usage, and overall cost.

Microsoft Research India 20th Anniversary