{"id":657531,"date":"2020-05-12T09:41:54","date_gmt":"2020-05-12T16:41:54","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=657531"},"modified":"2021-06-23T08:40:17","modified_gmt":"2021-06-23T15:40:17","slug":"wheres-my-stuff-developing-ai-with-help-from-people-who-are-blind-or-low-vision-to-meet-their-needs","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/wheres-my-stuff-developing-ai-with-help-from-people-who-are-blind-or-low-vision-to-meet-their-needs\/","title":{"rendered":"Where\u2019s my stuff? Developing AI with help from people who are blind or low vision to meet their needs"},"content":{"rendered":"
Microsoft AI for Accessibility is funding the ORBIT research project, which is enlisting the help of people who are blind or low vision to build a new dataset. People who are blind or low vision can contribute to the project by providing videos of things found in their daily lives. The goal is to improve automatic object recognition to better identify specific personal items. The data will be used for training and testing Artificial Intelligence (AI) models that personalize object recognition. In contrast to previous research efforts, we will request videos rather than images from users who are blind or low vision, as they provide a richer set of information. <\/strong><\/p>\n To help us with our research, we have conducted a pilot study to further investigate how to collect these videos, and we are currently recruiting users who are blind or low vision in the UK to record videos of things that are important to them. Visit the ORBIT dataset homepage (opens in new tab)<\/span><\/a> for more information on the study and how to sign up.<\/strong><\/p>\n To maintain privacy and confidentiality, users\u2019 contributions to the pilot study, and to all future phases of this research, are anonymized and checked before being included in the dataset. Any videos containing information that could lead back to the identity of the users are removed.<\/em><\/p>\n Smartphones are really useful in making visual information accessible to people who are blind or low vision. For instance, the Seeing AI (opens in new tab)<\/span><\/a> app allows you to take a picture of your surroundings in scene mode, and then it reads aloud what things are recognized in the picture (for example, \u201ca person sitting on a sofa\u201d). AI recognizes objects in a scene easily enough if they are commonly found. For now, however, these apps can\u2019t tell you which of the things it recognizes is yours, and they don\u2019t know about things that are particularly important to users who are blind or low vision. For example, has someone moved your keys again? Did your white cane get mixed up with someone else\u2019s? Imagine being able to easily identify things that are important to you or being able to easily locate your personal stuff.<\/p>\n Apps like Seeing AI use artificial intelligence techniques in computer vision to recognize items. While AI is making great strides toward improving computer vision solutions for many applications, such as automated driving, there are still areas where it does not work so well\u2014personalized object recognition is one such area. Previous research has started to make some advances to solving the problem by looking at how people who are blind or low vision take pictures, what algorithms could be used to personalize object recognition, and which kinds of data are best suited for enabling personalized object recognition.<\/p>\n However, research is currently held back by the lack of available data to use for training and then evaluating AI algorithms for personalized object recognition. Most datasets in computer vision comprise hundreds of thousands or millions of images. But at the moment, the datasets available for personal object recognition are from tens of users and contain maybe hundreds of images. In addition, there has been no effort to collect images of objects that may be particularly important to users who are blind or low vision. Providing a larger dataset for researchers and developers to use to build better AI systems could be a game changer in this area, for people who are blind or low vision in particular but also for everyone.<\/p>\n By funding the ORBIT project, Microsoft AI for Accessibility hopes to help researchers construct a large dataset from users who are blind or low vision, which will help further advance AI as it relates to personalizing object recognition. Researchers from City, University of London (opens in new tab)<\/span><\/a>, Microsoft Research, and University of Oxford are collaborating in this effort. Collaborators include the authors of this blog post along with Toby Harris, Mobile App Developer at City, University of London, Katja Hofmann (opens in new tab)<\/span><\/a>, Principal Researcher at Microsoft Research Cambridge, Luisa Zintgraf (opens in new tab)<\/span><\/a>, PhD student at the University of Oxford and Research Intern at Microsoft Research Cambridge.<\/p>\n Unlike previous research efforts, we will collect videos since they provide a richer set of information than images. Our research is also focused on providing realistic testing data so that any new algorithms can be rigorously evaluated. We anticipate that our dataset might be useful for implementations in existing apps, like Seeing AI, and also in novel wearable systems like Project Tokyo (opens in new tab)<\/span><\/a>, but our team is keen to future-proof the dataset for new applications that are yet to be imagined. The dataset will be made publicly available for download in two phases: Phase 1 will include about 100 users and thousands of videos, while Phase 2 will gather data from about 1,000 users and contain more than 10,000 videos.<\/p>\n