Facial Recognition Technology and the Challenges it Poses for Privacy

Cho: First, please introduce yourself. Tell me your name. Who are you? Where are you from? What do you do?

Katina Michael: Okay. Katina Michael. I am a full professor at Arizona State University. I work in the School for the Future of Innovation in Society and the School of Computing Informatics and Decision Systems Engineering. Currently, I'm speaking to you from Wollongong Australia because of COVID. But my main job is at Arizona State University. I'm also the director of the Society Policy Engineering Collective at ASU. I also do voluntary work for the Australian privacy foundation and I'm funded by the national science foundation in America and previously funded by the Australian Research Council in Australia.

Cho: So tell me about, more about your work at the Arizona State University.

Katina Michael: So I look at the way that technology impacts society and rather than. Allowing engineering and technology to be rolled out and then implications to happen. We are trying to change this way of the cycle. We are trying to introduce a methodology that says society comes first. Society should determine collectively which technologies will be rolled out. Why they will be rolled out how they will be designed, what questions they will answer. So rather than engineering driving society, we're looking at how society can shape technology and deployments. And so if we're looking at, for example, an emerging technology like facial recognition, which is getting a greater and greater footprint in the world. We are asking how have these technologies being deployed by whose authority by whose request and in which values. So we are trying to encourage values by design or ethical alignment and cultural awareness in the co-design, rather than the imposition of a technology on society. We are trying to say, well, look. Let's ask the people, what do they want? Let's ask the people to help us design better systems, whether it's for criminality, whether it's for dialogue, where, whether it's for, national security, whatever it's for. How did the people want to approach this? And what are the main values is that are associated with a culture and a market and even minority groups. How do we give voice to those who are underrepresented in a society? Whether it's people with disability, people with non-English-speaking  backgrounds or people who are migrants, people who are refugees, people who are belonging to a subgroup, how do we give people a voice at the table to determine the technologies that will ultimately affect them every single day of their life, rather than allowing technology to drive the change?

Cho: So why do you get the idea of the like focusing on people? Because like most of the engineering expert maybe they only focused on the technical parts of the technology. Why do you have this idea?

Katina Michael: Long ago I had come from an industry background. We deployed mass scale broadband networks and narrow band networks. I worked for a telecommunications vendor in the mid to late nineties. And we really did not pay much attention to societal factors or what I call human factors. And when I became an academic, yeah. I realize that 70% of large scale IT implementations fail. I asked the question to my students. Why do large scale implementations fail of big tech or they fail because people don't want to use them and adopt them. We used to have them mentality when we rolled out. For example, uh, high speed networks. We rolled out mobile. If any cloud computing building that will come. This was the mentality of AT & T and large scale of US providers just roll it out and we'll get customers and we'll get money and we'll become rich. Now the mentality is changing because we can see that a lack of consultation with citizenry or with the actual users of a system will inadvertently mean that it will fail unless it is nation state imposed. So far country says citizens, you live in my jurisdiction, you live in my legislative boundary. You will uptake this because I'm telling you, you will. Then citizenry have little choice when we are talking about more democratic and less autocratic nations. More democratic nations tend to say, not always, but tend to say, look, we have all this technology available. We can, are consulting you in what is the best way to go forward and you can adopt, but it's not mandatory. It's not obligated at the same time. It does not mean that in democratic States, all technologies that are introduced are successful. Absolutely not. I think there are equal failures, the independent of the economic system, they were, there can be failures in any deployment of emerging technologies. What has happened is there is a mismatch of the value system of the citizenry or the organization or the government agency with the actual users of a system. And so you're mismatching, the tech one size does not fit all.

Cho: When and how did you get interested in facial recognition technology?

Katina Michael: So, in my PhD, I studied automatic identification technologies and location based services. And, my husband and fellow collaborator, MG Michael conceived of the term, uberveillance in 2006, which was about identity, location, and condition of a human. And what we are saying at the moment is almost the enactment of uberveillance, uber, meaning above and beyond, and the veillance meaning to watch. So this extended, this hyper vigilant, watching this, this beyond just normal surveillance. And so in my thesis, I looked at different identification technologies. I looked at barcodes magnetic, stripe cards, smart cards, biometrics, and RFID technologies. And rather than saying, one of these technologies would be the winner of all identification. I realized there was a convergence and there was also a proliferation of all these technologies in society. And so today we may have our fingerprint tech and our facial image taken. We may have a SIM card, which has a smart cut embedded in our mobile phone. We may take a selfie to unlock our telephone. We may go to make a transaction at a mobile teller machine, an automatic teller machine at a point of sale. And we use a magnetic Stripe card. But what we are seeing is the convergence of these phones. Now becoming the way we transact becoming the way we prove our identity, becoming our location, tracking entities, becoming our condition monitoring devices. These phones have 14 different chipsets in them. They do different things. They can locate you. They have an image sensor. They can look at you, they have a condition sensor like accelerometers and altimeters and gyroscopes and magnetometers. They know how fast you're walking, they know how stressed you are. They know your temperature levels and everything. And so now what we have is this goldmine of information and perhaps the private sector saying, look, we have everything here. We know what you're doing. We know everything. We're using this for better service provisioning. But in actual fact, government agencies want this data. They want to know, where are the citizens? What are they doing? Are they in criminal activities? Are they safe and secure? Are they away from harm's way? Are they healthy? And so what we're seeing now are these mass large scale public interest technologies for the common good. But in natural fact, what we are trying to use to empower people is possibly working paradoxically to disempower people, say here, I grant you an identity. I grant you the ability to like, take this off and navigate from point A to B. But at the same time, I maintain the right to use that data on search and seizure or by a warrant or by checking, just in case you need some extra help, but in so doing, I remove your right to privacy. I remove your right to security. I remove, I disempower, although I'm trying to empower you, right? So I changed the laws. I changed the regulation. I changed the legislation. I changed the way processes occur.  I make things contactless because it's healthier like that. I allow you to use WeChat or Alipay. You don't need to carry an additional card. You don't even have to carry a wallet, just carry your mobile phone. And so I empower you because I'm giving you convenience. I'm giving you care, but underlying those two dimensions is controlled. And so by granting you the power to have convenience and care. I'm also disempowering you because I can control or at least I can monitor, I can track and monitor and trace your activities. And so that's becomes the paradox.

Cho: This is a really good question. So what are your concerns about facial recognition technology specifically?

Katina Michael: There are many concerns.  The first thing is that individuals can't change the way they look. You know, there are three main ways that you can change. You can put on weight or take off weights. I can have plastic surgery, for example. Or I can create coverings in my face, short term and longer term, but really I can't change my fingerprints unless I scrubbed them off. And I have no fingerprints. I can't change my face. You know, you can't change the color of my eyes, independent of what I try to do, and if I want to, it has to be extreme. I can put a mask to cover my face. I can wear glasses to cover. I can carry an umbrella to protect myself from being covertly monitored or unobtrusively monitored via various means. And what I guess I'm really worried about is racial profiling, discrimination, privacy invasion, human rights abuses, and all of these things that technology unfortunately can be used for bad. And so I then withdraw my right to be able to protest, my right to be able to march, my right to be able to voice my concerns. I lose my liberties. So the individual is faced with a loss of political freedom, a loss of a voice, a loss of being able to say," I don't want to go this way. Why don't you listen to me?" And this is perhaps the complexity of different types of economic systems by which I allow myself to enter myself into a hypervigilant or uberveillance society, where I'm constantly being watched. I'm not being left alone. I'm playing to a theater. Because I know the minute I leave my front door, I have to act a certain way. Otherwise the cameras may will say, "You know, you're looking a little bit odd today. You fit my profile and here you come over my way and I'll have a talk to you." And so I think facial technologies and automated recognition technologies, we don't understand the boundaries of surveillance. We don't understand what the limits of this are. We don't understand how they could be used or misused. We don't understand their own technical limitations, like misidentification of individuals. So I exist. I have enrolled in the system. I'm registered. I have not done anything wrong, but you. I think I saw you today, even though I saw another 14 people that looks like you. I think we all should go together and have a bit of a chat. And you told me whether I've made a mistake in your identification as a suspect of an antidemocratic rally or an autocratic rally or some kind of, of any smart city rally, whatever it might be. And this is where it becomes a little bit complex, because then we start to buckets and funnel people into minority groups.

Cho: You mentioned privacy is one of the concerns about the technologies. So can you tell me more regarding that concern?

Katina Michael: So when we have an privacy invaded, it could be informational privacy. It could be locational privacy. It could be bodily privacy. There are many different types of privacy. It's not just the right to be let alone. So when we look at different things, like what happens when my biometric is stored on a database, is that database secure? Who has control of that database? Where does it reside? Which companies are working in concert with government agencies to actually look at the database and to analyze it. How are we rolling out these databases? Do foreign nation States have access to those databases depending on the service. For example, if I'm in Australia and a piece of content is held on an Amazon server, am I subject to the legislation in America? Like the cloud act, which says, well, okay, you're an Australian citizen. Your biometric is stored on an American server. I'm sorry. It's not just the Australian government will have access. The American government will also have access because Australia is a Five Eyes nation subject to the laws that govern, the US if it is a US product. The other thing is when we're looking at privacy, looking at how much of our privacy is being invaded in a locational sense. Do I have the right to leave my house and not be tracked as I move around? Do I have freedom to visit geographically where I want to visit and to interact socially with people that I want to interact with or is my social physical network and also my locational, breadcrumb and location history and Chronicle. Being looked at for repeated patterns of movement. And this is a way that possibly I might be accused of something because I'm in the wrong place at the wrong time. Also assumptions may be built in. So when we talk about the pitfalls of uberveillance, we talk about misinformation, information manipulation and misrepresentation. It all comes back to. Here is this beautiful data set, allegedly it's perfect. It's not corrupted. And we know everything about you, every 30 seconds, where you went with your phone. And we know this because we are running a smart city trial and a pilot, and we have smart city lampposts. And those lampposts not only look at your location, but they actually also were able to denote your voice patterns because we have these beautiful technologies  now that can be used to access voice. And we know that you were a bit stressed out because you started to talk really quickly and you started to shout and you started to give all these commands. And this is the words that you used. There were antigovernment words. They were rallying people to do things that are like a dissident behavior.

Cho: Are there any solutions to protect people's privacy with facial recognition technology? Like maybe you mentioned your data may be stored in the United States server. So if the government, the Australian government, they set their servers in Australia only. So is that a solution for that problem to deal with the privacy? Do you think are there any solutions?

Katina Michael: So when we look at sociotechnical systems, because there's no such thing as a purely technical system, and that's what we study in the group that I run, there's a social component, there's a technical component and there's a regulatory legal component. So let's unpack these three things. The first thing is Institute systems that align with the social values of society, that's the first thing, rather than deploying a technology, let's ask before we deploy, let's gather the values and let's replicate these values in the design of the system. So it's inbuilt in the system itself. So if people want anonymity, yes, you can store the biometric. But so long as this is the limits of the use. If people want the identified data, okay. We have the identification if people want consent every time a camera takes a  snapshot of  them from CCTV. Then we build this in it's privacy and security by design, like the form of privacy commissioner  of Ontario, Ann Cavoukian has noted. Privacy and security by design. Others have called this engineering by design. But if we look at these things and don't bolt them on at the end, we implicitly put them in the design process. Then we can have a good product that is socially aware and matches the social value system. The next thing is the technical. Can we introduce privacy enhancing technology? Where is the data stored? If the data is just stored on my Apple iPhone. That might be okay. You know, they have an enclave, a secure enclave built into the operating system. Maybe that is enough. So we can look at privacy enhancing technologies. The third thing is we can look at tight access controls. We can look at enforceable regulation. We can look at the limits by using legislation to say, well, we can collect this data. With a warrant with a proportional reason to actually investigate somebody or some group or some event. But beyond that we have limits that's it, finished. We can't step on our own regulations. And what we're seeing at the moment, either countries have deployed mass scale technologies. Like the Aadhaar system in India, which requires biometrics and have not got preexisting legislative frameworks to be ready for the new technology, or we are seeing countries deploy technologies. And then think as an afterthought, maybe we'll change our existing laws to fit the new technology so we can abuse people's rights. So, okay. We have a new technology. Yes, we've got these laws, like the computer crime act or the cyber crime security act. What if we changed them a bit? So then we can do other funny things that we want to do, because now we have this new technology it's got new capabilities, let's water down or change in our favor the laws so that we can now infiltrate. And use these technologies to the maximum. We didn't know we could do A, we didn't know he could do B, this is a wonderful opportunity. Let's just change the existing laws. So a lot of people, when they don't want to go down the route of asking people, what do you think about the new technologies? Because there's no time to do that. They then look at the tech itself and they say, well, the technical limitations are obvious. We're not going to get privacy enhancing technologies. They cost too much. So they'd go to regulation. And usually it's this environmental view of a system, a technology system, which is most abused either. There are no laws and the laws are far lagging behind the introduction of the new technology. Or I have this existing laws, a patchwork of laws, and I tweak them and change them so that it suits me as the government. And so that is the interesting thing that is occurring today.

Cho: You also mentioned about smart city. So I'm quite interested in the term because more and more countries now they want to develop into a smart country, smart city. So what is smart city? Can you give me a definition about that?

Katina Michael: A smart city, in design can have different emphasis. I can have a knowledge exchange smart city. I can have a smart city that's based on carbon zero. I can have a smart city that provides law enforcement with total visibility. So there's less crime. I can provide a smart city infrastructure that encourages innovation. There are different smart city models. When we are talking about control in a smart city. Usually there has to do with law enforcement having real time or near real time data. And so the adoption of that data can give us snapshots throughout that day to follow on suspects, to look at what we call hits.  Trying to find an individual who has committed a crime, trying to lower and lesser the amount of crime in a particular area. Increasingly what we're seeing is the bringing together of all the different senses. We want AI almost to be like a human in its faculty. We want AI to see, we want AI to hear, we want AI to feel. We want AI to have human capabilities almost. And so we are seeing the erection of many smart city lampposts, and really as much tickets you can fill in the smart city lampposts that shovel and increasingly cameras don't look like cameras, microphones don't look like microphones. A condition monitoring devices don't look like that. Like what did they look like? They probably look like a power point or a charging device or some kind of nebulous thing. I often play tricks with my kids and I go spot the camera and often increasingly. These cameras are covert, they're unobtrusive. They're embedded in a frame of a building or in a lamppost or in a pole, you can't tell there are cameras, there are emotion detection now. And so if we are carrying smart devices, we carrying the token. Then there is this iteration, there's bunch of transactions occurring between the device I'm lugging around like the smartphone. Or a wristband or Apple wristwatch, or you name it, like we were proliferating in devices, a Fitbit, you know, a necklace, a smart necklace, whatever. But increasingly we're going to see interactions with these poles that may aid us in some ways. But in other ways, not be beneficial. What I heard is that increasingly we do use this smart lamppost for directions, for missing persons, for children with autism, for better access. For those who are disabled. I don't know how, but maybe we become more innovative in the right way that we are using the technology to help humans actually navigate the space around them, to open up areas, to have bud directional feedback flows with citizens. But unfortunately for the time being, we can't help ourselves. We are attempting to use these devices, not so much for knowing congestion and traffic and parking. But more so to know who is where and what they are doing. Again, I go back to the concept of uberveillance, this context, this assumption, because I know where you are. I know who you are. I know what condition you are in, I know what you're doing. And that is the biggest fallacy, we do not know because someone is somewhere. Because they are the person that allegedly that is the token carrier and we know what condition they're in. We can't, it's like trying to get into predict the profile of the person's mind by these external factors, which are really just technology sensors driving predictions. But we can't do that. We fall short because still the context is missing. We are trying to be omniscient. But at best we can have omnipresence. We fall short and that's when we have these three things occurring, misrepresentation, information manipulation. And this problem of misinformation, I accuse you of doing something because purely of where you are and who you're with.

Cho: What do you say to people who believe facial recognition technology is a good thing.

Katina Michael: I ask them to talk to people, everyday people and in plain language to describe why it is good. I think we could have some wonderful applications of facial recognition. Again, I go back to those people who are in the early stages of suffering or living with dementia. We could have wonder alerts. We could have systems to support people, perhaps. At the same time, we can't be ignoring that the same systems that we use for good can also be used for terrible reasons by those in power, to disempower. And that's the paradox here. When we have an absence of trust, we have a lack of transparency in what's going on. In this surveillance technology, we have a lack of understanding what legislation supporting the deployment of this technologies. Where are the boundaries? Where are the limits? So I ask people to think twice, because until we define how we want to live as a people, as a global community, as a local community, then we are allowing tech to drive our destiny, our trajectory. And it's the opposite back to my group, society. policy, engineering, not engineering, policy, society. We are thinking backwards. So we need the intersectional political dimensions where we talk to civilians and citizenry and people might say, but do we ask people where we should put a sewage pipeline? Do we ask people where we should put water pipelines? Do we ask them where they should be electricity conduits? Maybe not, but that actually forms as part of a residential community or a business community. You have to have access to basic amenities. And when we're talking about safety and security, we don't want to bend things that help us with national security, help us with our surveillance, help us with safety, but emergency management safety. And surveillance are all very different things. You know, I want systems that will encourage when someone is sick to get them help, to get the ambulance directly where they're located. I want this, I want to, if I had a child with autism who got lost in society, that if my neighbors could not find him because they went out looking that somebody else could maybe have identified him maybe through a camera. Yes, this is good. But when we are using technology for nebulous means, we are trying to, for example, in some places ban protest, ban the freedom of speech, ban religion or religious beliefs, ban political ideologies and freedoms ban ban ban. When we are trying to do this through the use of technology that cannot be healthy.

Citation: Katina Michael with CK Yeung, 21 July 2020, “Contribution to a practical assessment for a Masters in International Journalism” [radio documentary], Cardiff University, https://www.katinamichael.com/interviews/2020/7/20/facial-recognition-technology-and-the-challenges-it-poses-for-privacy