Shiri Azenkot on Accessible Virtual Realities
What I'd like to see more of is looking at how we can make the mainstream technologies and any application accessible to everyone Welcome to get killed. Our weekly podcast on ethics in a digital world may name. Is Jim Lob. I'm visiting journalist at Cornell Tax Digital Life Initiative and I'm your host. Our guest today is serious and Gut. She's an assistant professor of information science at the Jakobsen Technion Cornell Institute at Cornell Tech her research focuses on accessible technology and recently she. We started looking more closely at accessibility. In virtual augmented and mixed realities I sat down with her in November at Cornell attack and I began by asking her what brought her to this area research and why accessibility was important to her. I have I have always had a connection with accessibility. I have a disability myself. have low vision. And that's a visual impairment. That can't be fully corrected with eyeglasses glasses or contact lenses so it means that I have difficulty seeing but I still have a lot of vision. I'm not blind and growing up. I was always given all of these. Various assistive technologies and I always hated them I advocates. They were always designed in a way that was incredibly unattractive active to look like medical devices. They were different than what my friends were using. They were really expensive. You know there was no choice involved. They couldn't pick out what was desirable to me and I was stigmatizing to be perfectly honest so I didn't want anything to do with assistive technology. I didn't want anything to do with low vision with disability with blindness I was I was totally anti this whole thing altogether and I went to college legit. I studied computer science and then I became a software developer and at some point I started to wonder what was next for me. I felt like what I was doing. Really didn't have the kind of impact on the world that made me happy that made me feel fulfilled so I started getting interested. Did in going back to Grad School in computer science and doing more research and I came across accessibility as a research area and the work that I found it really really resonated with me because these researchers were saying that they were describing the same things that I had experienced. They were saying that current assistive technologies were not designed in way that was usable that provided a good user experience. They stigmatized the user. Hit me suddenly that this this is not necessarily okay. The way things had to be it clicked all of a sudden you now look more specifically at extended reality technologies so also known as X.. Art Technologies the incumbents things. Virtual augmented mixed realities. Can you just for the people who would not know exactly what these mean. Can you just give us us a sort of an overview of what that encompasses and what that means. Sure so ex are is one of many terms for this new New Paradigm of computing. So it's basically the idea that we have virtual elements that are embedded in the real world so a lot of people have probably seen pokemon go though. This is An augmented reality applications. So you have these pokemon which are virtual right and you you find them embedded or are located in the real world around you so this is one form of czar and then if you take this to the extreme you can yourself can be completely really immersed in a virtual world so you can put on an Oculus or some other kind of Vr headset and feel like you are more or less uh in another world. So what's the difference. Exactly between augmented a virtual and mixed reality so there are different definitions. One that I like to use. It says that makes reality. Is this entire continuum and on one end we have reality. And all the way on the other end we have Virtual tool reality with virtual reality like I was just describing VR. You're wearing a headset for example. And you're immersed in this virtual world different world and anywhere in between only part of your environment has virtual elements. It could be something small like Pokemon that appears or or we you and I could be sitting in a virtual room but I could still see your face so your faces real but then everything else you have. An avatar body could be virtual rule so that would be much closer on that continuum to the virtual reality end of it. And and so you. You believe that. These technology has a great potential for accessibility enhancing accessibility to people with different types of disabilities. Why is that? Why does that? Specific technology hold so much promise in that field. Well it's two things really so i. I believe that this technology is becoming mainstream. So what I mean by that. Is that right now. You know we typically use our phones throughout the day we typically sit in front of our computers. You know a lot of us might have a fitbit or some other device that's honest a wearable device and what what we see now. There's there's been a ton of research on virtual and augmented reality throughout the last few decades. But now it's starting to gain some traction action and it's becoming mainstream. This is coming to everyone to consumers. And we're GONNA see a huge shift. In general of how people are using technology how people are interacting with computing devices. What motivates me in in relation to accessibility is really two things? The first one is the need to make this mainstream technology fully the accessible. If we don't make sure that a blind person can use virtual reality application or a deaf person can experience virtual tool experience then these populations these people will just be marginalized from all kinds of different Experiences and applications of this new technology. So that's that's on the on the one hand and on the other hand I also think that this new type of computing technology can solve a lot of accessibility needs so so for example. Let's say a person with low vision might have difficulty Way Finding you might have difficulty getting from campus purpose to a nearby store for sighted person. You can okay you can easily pull out your phone pullout Google maps and you can use it to navigate their but for a person with low vision. It might be very difficult difficult to see the map on their phone and keep track of their environment and walk all at the same time. So it's a great opportunity to be able to use augmented reality so we can augment the users path with this virtual indicator of where they can go so that's just one example of a need of a certain population of people with disabilities that I think can be addressed with this new type of technology cularly. Well yeah more specifically used to As you were just starting to say you study ways to design for people with with low vision. You've done a lot of work on that. Now I understand from your own history but why these group of people also why not look look the blind or the blind and visually impaired in general. That's a good question So yes I do myself have low vision. But that's not the only the reason that I study low vision so I did in fact start designing technology for blind people people who have no vision when I was in graduate school but what I was finding was that there's actually very little research on technology for people with low vision and unload vision in general at the same time. There's a ton of research on technology technology for blind people and I thought this was really interesting and I think that one of the reasons for this is that it's much easier to understand blindness you you can close your eyes and more or less experience. What a blind person is experiencing at a certain point in time but then when it comes to low vision? It's incredibly difficult to to describe what someone sees and I know this personally my entire life people have been asking me. What can you see and I really don't know how to answer that question because first of all I don't have a baseline I don't know what you can see and second of all it changes with time and space and context and there's so much more to seeing than you know whether I can recognize a letter on a chart or whether I can recognize you from I'm across the hall? So Vision is incredibly complicated. So that's a major challenge but it's also it's incredibly interesting and it made you the perfect person to understand. Try to grapple. Yes so I do think I have unique insight here and I think that's incredibly important when doing research I think it's important important to leverage. What whatever you personal experiences in insights? We have to our research into what you explain that I find interesting. Is that for these people bowl. Of course there's the possibility of using the technology that would be developed for the blind but they would prefer solutions that enable them to use their vision right. What we've been seeing over and over again in our research is that people with low vision typically used their vision and they rely on their vision and we can leverage their vision so vision is a powerful sense one of the main challenges and this work that we do in general? Is this question of how can we. I understand people's People's visual perception in experiences in different contexts and then how can we leverage it. How can we design in a way? That really uses the vision that you have have and also suits your preferences. Let's dive into some example of the innovations that you've built in the backfield you've worked and I just picked up on two things that I thought were I believe interesting from your research You've worked on augmented reality to facilitate stare navigation. I I thought that was really interesting. And you build a tool cold. Hugh seem Designed to help people with low vision on conducting daily activities like shopping stores. So could you just elaborate on these to tell us what you were trying to solve in. How what that? Technology does in those two cases. Sure so I'll start from QC So one of the first things that we started to do is to you just understand what challenges and what tools people with low vision are experiencing in using in their daily life and daily tasks. So we did a study where we observe people as they walked to a nearby store and bought a specific product. We told them find a nearby pharmacy. Chrissy and purchase this very specific tylenol products in this task had many sub tasks in it. Interestingly in this study we found that that people experienced a lot of different challenges so that was the not so surprising part and possibly the most severe challenge did they experienced in. This task was finding the actual product on the shelf so once they had found the pharmacy nearby and they had found the correct correct. I'll like the aisle that had all these medications. We realized that there were no tools that they use. That could help them with this task. This is what is called a visual search task. Ask So it's finding unknown item among set of destructors so there are lots of items on the shelf spill a kind of similar by design. And if you're cited did what you're looking for will pop out at you so for example when you're walking you're constantly doing a visual search for obstacles if you're looking around for a bathroom you know you're going to glance lance around and the bathroom sign of one it'll pop out right but then if you have low vision this becomes very very difficult almost prohibitively so slow so so people were taking one product at a time. They were examining it possibly using a magnifying glass to read what it was and then putting it back and then taking taking the next one and doing the same thing in this was extremely slow and also strenuous and we saw that the tools that exist magnify of various forms phone applications etc.. They all help you see details and that's a different visual tasks that's not visual search. So there was this gap so oh. Qc was our first attempt to address this gap so we were looking at this. Particular application of product search. So here are idea was well if the person with low vision can't do the visual search and locate the item then let's use computer vision but now the question is. How do we communicate kate that to the user in a way that's convenient and doesn't feel as you were saying earlier doesn't feel clumsy or that you don't like exactly so we AH decided to explore this idea? This communication between the computer and the user through augmented reality cues there have actually been different applications in the past. Ask that solve this type of problem and again they focus on blind people. So they'll give the user feedback with sounds so like they'll give you beeps or something saying that tells you where your hand should go but for people with low vision we can do this much more efficiently at least this hypothesis. So we designed visual cues to leverage the vision that they have and we drew on some cognitive psychology literature and also some of our knowledge on different types of vision conditions and then we designed augmentation reality system that can locate the a specified product. And then it it overlays these cues over what you're seeing so that you your attention. Chen is drawn to the target suggest that our listeners. Visualize it you would have. There were several Q.. Options so you would have the product appear with. What kind of cues? Yeah so. We designed several Q.. So for example one of them was a set of lines. Eight lines they each line starts from from appoint. Around the periphery of the display and then they converge at the target product. So that if you're wearing let's say google glasses or innocuous aura known on what you were doing that you would see the product with the lines around it so you would be drawn to this immediately exactly. Was the idea that these would be applicable. Oh for people in the future on say Google glasses or other augmented glasses that they could wear all the time today with augmented reality with Google Glass Sir with a bulky headset. That's this is not a feasible solution. But what we're really trying to do is look into the future what we will see in what we're seeing starting to happen already is that there will be smart glasses. I'm wearing greg standard glasses right now. In some number of years these will be instrumented with a digital display. A microphone a speaker. You know various other sensors a camera at this time. We want these devices to be effective and to be ready to be inclusive as well exactly exactly and so what. What about the other example that I was just bringing up this example of stairs? Can you just quickly tell us what you try to do there and how it worked. Yeah so oh. This is one of the first ideas that I had like going all the way back to when it was a Grad student so You probably have noticed that a lot of stairs have this contrast stripe towards the edge and sometimes they don't and sometimes they have a contrast right but it's not very contrasting so I was thinking can't use augmented reality to to put virtual actual contrast stripes on the stairs before actually implementing these contrasting virtual contrasting stripes and augmented reality. So we I did a user study where we actually wanted to observe people with low vision as they walked up and down different staircases here on campus and what we found was that it was really most important to communicate the location one of the first and the last stare. So you did add this contrast on the first and last Yeah so then what we did was we designed a system That enhanced the first the last stare a lot more than the middle stairs and we also experimented with different ways of adding animations like flashes of light for example to the first and last there so that we made sure they attracted people's attention to avoid tripping. What about other types of disabilities? He's I know you mostly look at visual impairments but are you or maybe your students. Trying to address other types of disabilities or starting to do some work right now for students with learning learning disabilities specifically math courses. So that's been very interesting so we're trying to understand what challenges students with learning disabilities his face in math. And we're looking at the technologies that are being used in the classrooms now and whether and how they support students with a range of of abilities extended reality is definitely a hot topic in tech right now. A lot of people say twenty twenty will be the big year for extended reality. There is a lot of innovation happening especially in the the entertainment side. Let's very cool video games and stuff like that but what about accessibility in the field do you feel like this is also a hot the topic in the field or not yet So I think that there are many researchers right now and also many companies That are interested in seeing how we can design these technologies to address specific accessibility use case. Oh like QC and like the stair project so looking at how we can design and application specifically for population of people with disabilities. But what I'd like to see more of is looking at how we can make the mainstream technologies and and any application accessible to everyone. A lot of your research is is looking at a enabling systems. Are you looking yourself or is that maybe the next except for you looking into how to make the mainstream applications of V of x are available and accessible inclusive to these people so like those school video games I just talked about all those things that if technologies inclusive we would want everyone to be able to use them. Yes definitely so this is going to be a major focus of my research moving forward and this is also why we started the Xsara access initiative. What is this? It was a a seminar or conference this summer. Can you tell us a little bit. More about voted and why it came to be. Yeah so we started out by organising the X. axis symposium and. This is a conference that I put together with Larry Goldberg and a few other colleagues from Horizon Verizon Media and the goal of the extracts the symposium was to bring together people from different sectors so we brought academics people from industry in a variety of companies represented People from government and also from nonprofit to bring together people who were interested in accessibility and also in x are and to set an agenda for how to make extra accessible to all we all know. Oh that this is a technology that's coming just like you said and we all know at the same time that it has a major accessibility challenges. So we wanted to catalyze more more work in this area so it was this summer in New York. What have you seen? Do you think that's creating. Momentum is a movement of excess ability emerging so this was he was a great success and we had about one hundred and twenty people which was at capacity so we had to wait list. There was a lot of interest. People are very excited. Everyone realizes this is an extremely ainley important problem. So after the symposium we decided to establish the X. Axis Initiative and during the symposium the output of the symposium was a list of goals we put together an agenda. Basically just like I was saying for what needs to be done. And so the the the goal of the initiative is to work on this agenda to make it happen The initiative has several components to it. So there's going to be a research component where we're going to be doing research on making extra accessible and we also have a working groups so these are groups of people from The different sectors people who attended the symposium who are personally interested and passionate about this area and bring them together in different focus areas. It's like looking at how to make the hardware accessible looking at how to make The content accessible specifically looking at how to establish standards and guidelines around accessibility and several other groups were organizing facilitating these working groups. And we really really want Xsara Access Dot Org they initiative in general to be a hub for everything to do with Xsara and ability and this is just the start this assist in research and are indeed exactly. Yeah and so would you say there's an ex our access movement emerging or is that a too early to tell or is that your hope that is certainly the hope I would definitely say there's a movement emerging. There's a lot of excitement that we've seen and people are definitely silently energized and we've had so much interest and follow through with the working groups. I always end up my interviews by asking guests. What is the one thing they would change if if they could if I gave them a magic wand I would like to see more people with disabilities in the technology industry Right now they're extremely underrepresented and I think that fundamentally until we have diversity of the people creating the technology of creators of leaders. Then we're not gonNA have have full and complete accessibility of our products. What can listeners do to help technology be more inclusive in the future? I think that one key aspect aspect of this and this came through loud and clear in our symposium also is awareness being aware of the fact that accessibility is important that there is a diversity of user. Zor needs out there Diversity of people and abilities and just like anything else. Technology should be designed for people with all kinds of different abilities. One more thing I want to mention a lot of times disabilities left out of the diversity conversation and I think it's very important to make sure that it's included looted. It's a very good point. Well she isn't God. Thank you so much for the time. That was a really great conversation. Yeah thank you very much. That was good code collaboration with Cornell Text Digital Life Initiative this podcast is produced hosted and edited by yours truly career. Shrapnel is our mix engineer so he saw an is our music composer. Thanks for listening and if you liked it tell your friends. Share this episode and more more social media leaks five stars on itunes. To help other lease knows find us but most importantly come back next week. I'll be here and.