More Intelligent Tomorrow: a DataRobot Podcast

Bringing History and Foresight to Ethical AI - Meg Mitchell

March 02, 2022 DataRobot Season 2 Episode 6
More Intelligent Tomorrow: a DataRobot Podcast
Bringing History and Foresight to Ethical AI - Meg Mitchell
Show Notes Transcript

In this episode of More Intelligent Tomorrow, Meg talks to Michael Gilday about the opportunities in machine learning to create a more diverse, equitable, and inclusive future.

Dr. Margaret Mitchell (Meg) is a researcher in ethics-informed AI with a focus on natural language generation, computer vision, and other augmentative and assistive technologies. 

 Foresight is an indispensable tool for shaping and evaluating AI project outcomes. Instead of focusing on creating technology to improve something that already exists, a longer-term focus –one that is two, five, or ten years into the future–can help us understand what we should be working on today. It’s a fairly straightforward way of thinking, yet foresight is often brushed aside as incalculable. 

Foresight also can present a liability issue. If you’re working on a technology that will be less discriminatory, for example, that means your technology right now is discriminatory. Fear of impending regulation and of  misinterpretations that hamper development can cause a troubling lack of imagination within development teams.

Bringing in people who have a creative mindset or a different perspective can help technical teams see things in a more imaginative way. Science fiction writers, for example, are adept at bringing foresight into a project and help teams see how things might evolve over time. That, in turn, could help us be smarter about the kinds of development we do. But bringing people who are adept at foresight into a project, such as science fiction writers, creates an opportunity to think through how things might evolve over time. That, in turn, could help us be smarter about the kinds of development we do. 

Similarly, historians can shed light on patterns of development over time. Instead of focusing on  how rapidly technology is changing, they can offer a reflection on corresponding power dynamics and sociological changes that can also inform how we develop a technology.

A collaboration of humanities-oriented thinkers and science-oriented thinkers can help us think through the storyline of what a technology should be. There’s a need to focus not only on how well the model or system works in isolation but also how well it works in context. 

“Understanding how people use a technology–and therefore understanding people– is not something computer scientists are always good at. It requires different skill sets, which makes collaboration with subject matter experts critical.”

To really understand what it means to have AI in our social contexts, we need social scientists, anthropologists, and historians. So, how do we bring a diversity of voices and experiences into these technological challenges and conversations? 

“Now is a great time to focus our attention on the science of diversity and inclusion. We’re on a global scale we haven’t been able to see before. We have infinitely better access to different cultures and perspectives on differences and similarities like we’ve never had before.”

Listen to this episode of More Intelligent Tomorrow to learn about: 

  • The culture of ethical behavior and the bottom-up-top-down approach with regulators and corporations
  • How no-code solutions are removing barriers in AI and machine learning work
  • Malicious actors vs. irresponsible ones and why ignorance is the biggest problem we face
  • Gender bias and progress bringing more women into in tech and STEM
  • How transparency can be prioritized over the obfuscation that is prevalent right now



Michael Gilday:
Dr. Mitchell, thank you so much, and welcome to More Intelligent Tomorrow.

Meg Mitchell:
Thank you for having me.

Michael Gilday:
I'm so excited to have you here, and I'm just going to jump right into it. Is AI a fad?

Meg Mitchell:
There might be a question in whether certain types of AI are a fad, and we can make a distinction between AGI, which is artificial general intelligence, and then task-based AI. AGI might be more of a fad now than it has been in other times. So we had an AI winter, where people really didn't work on it because it was sort of seen as something outside of the realm of possibility, and so AGI sort of comes and goes. So maybe fad, and then not fad, and then fad again. I think task-based AI and the desire for task-based AI has been around a long time, and that's pretty steady. We're always sort of working as humans on ways to come up with automation, and task-based AI is a way of automating. So it's a natural step following the industrial revolution like that.

Michael Gilday:
So I want to jump into AGI later on, because that's always an interesting topic. But if it's not a fad, is it going to increase in relevance and importance in our lives?

Meg Mitchell:
Yeah. I don't see why it wouldn't. I feel like that's a boring answer. We've already seen over the past five years how fundamental changes in the technology with the technology getting better has led to the proliferation of AI technology in our lives, to the extent that we're not even evaluating the technology well, or really understanding what it's doing. It's already being and out there. People are so hungry for it. So I think we might hopefully see a little bit of a slow down as regulation comes in, as norms around development are further developed. I would say it will steadily in increase in our lives, and I don't see why it wouldn't.

Michael Gilday:
Are you excited by that prospect, or scared of it?

Meg Mitchell:
So I started working on ethics in part because I was terrified by the technology I was working on. I'm a computer scientist by training, and I was working a lot on creating human-like language, multimodal stuff, so taking a computer vision stuff and producing descriptions. And it wasn't until I started seeing the descriptions working, started seeing a glimmer of what AI would look like in the near future that instead of having a eureka moment, I had an "oh crap" moment, because I saw all the mine fields that were there, all the errors it was making, all of the ways that it was just picking up directly issues in the dataset.

Meg Mitchell:
And so I see that there are a lot of paths we can go down. There's not just one inevitable end of AI. There's a whole bunch of different ways it can go and it can be developed. I think maybe two years ago, we were going down a path that was terrifying to me. I think now, over the past couple years, there's been a lot more understanding about and demand for ethics and fairness and transparency, and some of these things that help AI to be going down a slightly better path. So at this point, I could kind of see it going either way. It really depends on how the push and pull works out between a push for profit and a push for human wellbeing.

Michael Gilday:
What are some examples of the things that give you hope in that?

Meg Mitchell:
There's a lot that can be done with using AI for assistive technology and augmentative technology. I'm a little hesitant to say that, because assistive and augmentative technology is often used by AI researchers as a kind of ethics washing or goodness washing to justify all kinds of different research. They can point to that one thing, like "Oh, well this could help people who are blind." And there's often an issue with that, where the people who would actually be using the assistive technology aren't actually part of the development process, so it's not even the right technology.

Michael Gilday:
But you were working in that space-

Meg Mitchell:
Exactly. Yeah, yeah. So I want to first say I think that anyone who responds with, "AI, it can be very beneficial in assistive and augmentative technology," should be approached with a little bit of skepticism, to dig into what do you mean when you say that. Personally, I've been working in health domain, assistive technology domains since 2005, so I have hopefully a bit of credentials in pushing AI in that way. So there are use cases where you understand language, where you use technology to understand people's language in order to, for example, help create different diagnoses for autism or Alzheimer's or things like that, doing sort of signal processing. And there's also uses like what I worked on later in my life, such as creating technology for people who are blind. So doing things like object detection, sometimes image description, although the task of image description or image captioning is largely not aligned with what blind people are asking for.

Meg Mitchell:
But I think that AI can definitely go in helpful directions in the assistive and augmentative domains, and ideally work in an augmentative way for everyone regardless of ability status to be empowered and enabled to be more on an equal playing field with everyone else, or have other sorts of things like helping us remove our biases when we're trying to write a letter of recommendation, things like that. So these sort of assistive approaches or augmentative approaches, both in terms of different ability statuses and its more typical ability statuses, I think that there could be a lot more done that will definitely be helpful.

Michael Gilday:
So I'm curious. You work for a while in the space. You get very excited, inspired. You start to then get concerned. What do you do at that point?

Meg Mitchell:
Yeah. So at the time I had this change of heart was when I was working at Microsoft Research. And so I started really pitching a new research direction, where instead of focusing in the here and now out on creating technology that improves over something that already exists, getting out a paper where you can show you have a table that your thing is the best, instead of looking at it in that very local way, looking at it in a more long-term way where you're sort of saying, "Where do I want this technology to be in 10 years?" If my goal is goodness or something positive for humans, where should it be in 10 years based on what I know now? And then move backwards from there in order to understand what you should be working on.

Meg Mitchell:
So I pitched this around to various colleagues. I was able to get connected to people working in the disability space at Microsoft, and from there, we were able to start collaborating on what it would look like to create technology for people who are blind. And it was at that point where I started seeing more of the connections between the work I was doing and how to start evolving it in a more beneficial way.

Michael Gilday:
So looking forward 10 years, the pace of innovation has increased so much that so many people are hesitant to speculate even a year or two from now. Do you still feel that confidence in looking down the road? Because I feel like it's becoming a more rarefied group that is willing to speculate.

Meg Mitchell:
Really? Are you finding that people are less likely? I see tons of speculation all the time, but I guess maybe I'm in different circles than you. But I feel like there's a lot of that, in part because that helps drive research. You have to be speculating all the time about where things could be going in order to make that happen, in order to do your research now.

Michael Gilday:
It might be in the corporate context, and what people are willing to go on record about, which is-

Meg Mitchell:
Exactly, yeah. So there's a liability issue, definitely, in what you say. And representing something that might then come back to bite you because, for example, you talk about how technology might be less discriminatory, that means that right now the technology is discriminatory. And so if you're at a company using this technology, you kind of don't want to get that out there as an idea. I do think that part of why there might be some people who are more hesitant is around the concern of upcoming regulation, how things could be misinterpreted to hamper different kinds of developments. I also think there is a troubling lack of imagination in a lot of people who are developing this technology. I like to talk a lot about the utility of foresight when working on AI, and how that can inform what you work on and how you evaluate.

Meg Mitchell:
And I think that's a fairly straightforward way of thinking, and yet I found with a lot of people who work on AI all the time, that idea is kind of brushed aside as unrealistic because we're bad at foresight, and rapid hindsight is much sort of... To which I respond, "Well, who is the 'we' in 'We are bad at foresight'?" That might be some of us. It's not all of us. And so the goal, ideally, is to really bring forward people who are quite good at foresight, who are quite good at thinking through how things might evolve over time, and use that to help inform the kind of development we do.

Michael Gilday:
So what would be examples of those kinds of people?

Meg Mitchell:
So science fiction writers are pretty amazing, and there's actually the show Black Mirror that has done sometimes an uncanny job at predict how things will go. They had one show about having a social scoring system, and having that possibly ruin your life. And since we've seen social scoring systems come out, there was another episode about natural language generation, and using the language of a dead loved one in order to sort of make them come alive again and talk to you. Since then, we've seen that technology be developed. So I think that once you start thinking through what the storyline could be as someone who could be a good sort of science fiction writer, then there's a lot to be learned there.

Meg Mitchell:
And that's also sort of a mixing of more creative ways of thinking, or maybe more humanities-oriented thinking versus maybe more science-oriented thinking. And to that point, historians are also quite helpful, because we've had very similar things happen again and again. So having a good grasp of patterns of how we've developed things over time and how people have responded is also really useful for informing what will happen or could happen next.

Michael Gilday:
That's a good point. People focus on the technology changing and how fundamentally different things are, but a lot of the sociology doesn't always change, or the power dynamics, or the things that feed into it.

Meg Mitchell:
Exactly. And we saw, for example, in work on fairness... So fairness and bias is something that has recently received a lot more attention in AI, and it's often sort of pitched as something new, but we actually see the same thinking down to exactly the same math coming around in the '60s and early '70s. So there's a period from about '67 to 1973 where a ton of the fairness definitions we use now for machine learning systems is identical. A ton of the math that we are using now is identical to the math that was being used at that time. And you can learn from that, because eventually it sort of hit a point in the '70s where there was a realization that there was so much of a disconnect between what the mathematicians and statisticians were doing and what systems of justice needed, that it sort of fell out of disfavor to approach the sort of measuring of fairness in such a mathematical way.

Meg Mitchell:
And we're maybe starting to have that conversation now in AI. People are understanding that fairness is different than justice, but that is a novel thing happening just within the past year or so in AI. And if we had paid more attention to the statisticians of the past, then this would be very old news, and we might even be further in our thinking because of it.

Michael Gilday:
It sounds like you're saying AI right now has an overreliance on math and algorithms, and maybe not enough viewpoints being brought in from the humanities or the creative side. What's your sort of position on-

Meg Mitchell:
Yeah, I would completely agree with that. I think AI, or... I do want to say machine learning. I have a little bit of difficulty saying AI [inaudible 00:13:22] machine learning. I know that if I say AI, it will be more understood as the concept. But just in my head, in order to really articulate what I'm saying, I have to sort of frame it more in terms of machine learning. So I would say that in machine learning, there's generally been a sense of developing in a vacuum. So you have your algorithm, you have your evaluation, and that's sort of all there is. Now that machine learning systems have been increasingly deployed, it's not just how well the model or the system works in isolation, it's how well it works in context.

Meg Mitchell:
And now you need to know how people use the technology, and that means you have to understand people. And that's not something that computer scientists are necessarily that good at, so it really requires a different kind of skillset, bringing in people like social scientists, anthropologists in order to understand what it means to have AI in context, in our social contexts, because that's where we are now.

Michael Gilday:
Well, I think also, talking about the Seeing AI project that you worked on, you indicated that maybe the needs of the visually impaired community weren't being met by the application of the project, which is... That's a typical problem we see nowadays, is solutions in search of problems, instead of the other way around, not bringing in the subject matter experts.

Meg Mitchell:
Exactly. I have this hammer. What can I hammer? And yeah, in the case of image descriptions for people who are blind, that was often used as a motivation in work on image captioning. The problem was that if you take a look at the kinds of things image captioning was doing, it would give you things like a man holding a soda, or a man holding a soda can. And then talking to people who are blind, they'd be like, "Okay, if I'm standing somewhere holding a soda can, I know that I'm holding a soda can. I need to know how many steps there are going down a staircase so I don't fall. Can you please count steps instead of describing to me a world that I already know?" And so that's a very...

Meg Mitchell:
Someone who has vision sort of bias, where you think, "Oh, of course you want to describe the world in this way," but then the image captioning task turns out to be only tangentially related to what's useful for blind people. So yeah, this has happened again and again. And I would say now one of the concerns would be around climate change, and whether machine learning developers are so excited about climate change that they're going to try to use their hammers to do whatever, when in fact the EPA or some other sort of source of experts actually on climate might be looking for something very different. And hopefully, there's more communication between the two who fields to figure out what would be the most helpful.

Michael Gilday:
How do we surmount that gap? Or how do we create incentives that bring the right folks, a variety of diversity of voices and experience into these challenges and conversations?

Meg Mitchell:
I think a lot of it has to do with what's incentivized in terms of the papers that get accepted, the reviewing rubric for those papers, the kinds of awards and grants given, the kinds of things you're promoted for, the kinds of things that you're liable for. People will generally do work that's aligned to the incentives. And so if you start to have, for example, reviewing rubric that says this author has demonstrated an understanding of the relevant field for this technology, then you start to have researchers actually talking to the relevant people. If it'll get your paper rejected not to have it, then absolutely you're going to start paying attention. If you want to incentivize something like inclusion, then you can imagine performance reviews and promotions saying that, "Has this person created an inclusive environment? Have they brought in all the relevant people or lots of different perspectives?"

Meg Mitchell:
And then once that becomes part of performance reviews and promotions, people are going to be awesome at it. So it's really... What are the prizes? What are the good things? And if you have these kinds of goals as what people should be working towards, then people do work towards them generally.

Michael Gilday:
Technology companies are really good at solving some of the hardest problems in the history of humanity. But for some reason, even though many of the large ones have had a commitment for over two decades, they can't seem to figure out how to become more diverse or inclusive.

Meg Mitchell:
Yeah, yeah. I can speak to maybe more recent time more than I can speak to this historically, but I would say that society has more or less tended towards greater inclusiveness. And so if you have, over time, people who are younger growing up in an environment where they come to understand inclusiveness more than people who are older than them, you have a corresponding situation in tech, where the people who are older and have sort of been around longer are the ones in power, and the people who are sort of younger and have had more experience with understanding what inclusion is and diversity is have a lot less power. They're just starting to come in to the tech industry. And so there's a really unfortunate power dynamic there, where the people at the top are not incentivizing the right things for what we know now in the current time.

Meg Mitchell:
And ideally, there could be some sort of, I don't know, labor law or some sort of rights afforded to workers who are not in the top positions of power to be able to say something about what should be incentivized and what shouldn't. But I think it's going to have to come to a evolution of different technologies over time, evolution of different companies over time, with each step bringing in sort of more people who have become more familiar with inclusion. And then hopefully, then the next tech company is people who are even greater at inclusion, and things like that. I think it does help that we're all talking on a global scale in a way that we haven't been able to do that before. So we do ideally have better access to different cultures and understanding similarities and differences and things like that, so that does give me some hope about working towards greater inclusion. But I think as long as we have these power dynamics where the higher you go, the more exclusion there is, the current state of the art in inclusiveness will always lag what the general social understanding is.

Michael Gilday:
You sound optimistic to me, and I think that is great, despite the current challenges.

Meg Mitchell:
Right. Well, you have to be optimistic if you're working on it, or you just won't do anything, just be depressed and stare at a wall instead.

Michael Gilday:
Just described my life.

Meg Mitchell:
Yeah.

Michael Gilday:
So you brought up something interesting about potential regulation, and it made me think about your job as an ethicist. So you are chief ethics scientist, is that correct?

Meg Mitchell:
So I'm not an ethicist. I'm a computer scientist, and I try to work on ethics. I try to apply ethics as best I can, but I definitely don't deserve the credit of being an ethicist.

Michael Gilday:
In your lead role as an ethics scientist, chief ethics scientist at Hugging Face, I saw something where you were talking about top down versus bottom up, and regulation, which is not here but is coming, but it is a critical element, but also people's sort of values and the bottom-up approach, and how those collectively can inform sort of successful ethical approaches. Do you want to dig into that philosophy?

Meg Mitchell:
So there's a lot to unpack there. So the bottom-up top-down idea as it applies to regulation hinges on this idea that people who are outside of tech, often regulators... I'm thinking of people like regulators. They are not so aware of the nitty-gritty details of how the technology is developed and how different kinds of developments can affect the output. They are familiar with output, because that's what people see day to day. And so by trying to regulate the behaviors of development, you run into a problem where you might be shooting yourself in the foot. You're trying to achieve something like non-discrimination, but the constraints you put on the development of the technology actually sort of ensures discrimination. So for example, you can't pay attention to gender. Well, we can't say we don't discriminate based on gender if we can't pay attention to gender.

Meg Mitchell:
So there's something to be said for the fact that I think regulators understand a lot about the output of the systems, and can have some sort of regulatory insights into how the output should be. So it should be fair, or it should be robustly tested, or all these sort of different things. While internally, within tech companies and for developers, they understand how the development works, and so could inform in a sort of self-regulatory way what are the right behaviors, or what are the right development practices in order to meet these higher-level goals? So that's a place where sort of top down from within a company does a bit of self-regulation towards... Oh, I should say bottom up within a tech company towards top-down goals set by regulators can hopefully sort of work together.

Michael Gilday:
Talk to me about the difference between legal compliance and a culture of sort of ethical behavior, especially since legal compliance is probably not where it needs to be, and definitely not uniform.

Meg Mitchell:
Well, one of the issues with legal compliance is that you can create horrible harms that are technically legal. And so as long as you are staying within something that an argument can be made is legal, then you can just do horrible things, and then you can do a horrible thing to a worker, and then that worker has no rights because it was legal. For example, your company might decide to put out horribly disparaging comments about you that imply that you're leaking confidential information, and as long as they don't say explicitly leaking confidential information, they use some other word like exfiltrate... As long as they kind of mess with the wording, you can make an argument that maybe they're actually describing something, and they're not trying to be disparaging. They're just trying to be clear or something. And so legal constraints can actually work the workers, I think. Or legal compliance can work against the workers, in that it takes away their rights because something could be argued to be legally compliant, and that's-

Michael Gilday:
Just purely theoretical, though.

Meg Mitchell:
Purely theoretical, yeah. I'm like a science fiction writer. I'm just trying my best to be creative. So legal compliance is sort of a very different beast than thinking about values and doing the right thing in terms of human morality and stuff like that. It's definitely the case that law is somewhat informed by human values, but law is also very informed by lobbyists and business interests, and so things that are legal aren't necessarily ethical, and vice versa. So there's definitely a tension there, especially if you're trying to move a company to be more values-informed, but there's no incentive in the law. Or the law would even potentially come to harm the company based on more alignment to the values. There could be some intersection of utility. There definitely is some intersection between law and values, but they're not the same thing at all, and I'd say the intersection is maybe a bit smaller than we would like.

Michael Gilday:
Are you a sci-fi fan?

Meg Mitchell:
Yes. But if you're going to quiz me on something or ask about something, I'm probably not going to know the answer.

Michael Gilday:
I stopped giving quizzes before I started. I'm a sci-fi fan, going back since I was very young, and part of why I've had such an attraction to technology. But I think what's interesting, especially what you were talking about, is it life imitating art or art imitating life... A lot of dystopian science fiction, sometimes we ignore the dystopia, like a Ready Player One, and just look at the fun parts, and ignore the fact that the whole world has fallen apart. But it has served as cautionary tales that have helped guide the development of technology as well.

Meg Mitchell:
That's definitely true.

Michael Gilday:
A lot of times for the good, maybe. You don't get credit for the ditches you don't drive into. How important do you think that role is to sort of imagine out the future?

Meg Mitchell:
Right, yeah. So I think people who are good at foresight and good at creating the story or the path of how to get there, that can be for better or for worse. Or rather, I should say the foresight could be towards positive outcomes or towards negative outcomes. The ability to have this kind of storytelling or forecasting ability is critical across all the sort of ways that AI can turn out, and in fact provides us with examples of the kinds of paths that AI might follow. There's oftentimes these discussions about whether AI will turn out to be good or bad, but if we're paying attention to what a lot of more creative people are saying about all these different forms of AI, all these different things in the future, then we actually see that there are a lot of different paths we could be going down that some are maybe more dystopian than others.

Michael Gilday:
I mean, all technology is a double-edged sword, and one of the things that we try to do here is try to look at... We can't mitigate at all the negative things that'll happen, but how can we bend the arc towards a more intelligent tomorrow? How can we either inspire or incentivize collectively? What are those solutions? Because AI has the potential to do incredible things, along with the potential to do some pretty horrible things if left unchecked. When you think of a more intelligent tomorrow, whether it's 5, 10, 20 years, what are those areas that you think could be substantively better?

Meg Mitchell:
Yeah. I don't know that I agree that the best goal or a very good goal is a more intelligent tomorrow. I might say something more generic, like a more beneficial tomorrow, or something like that.

Michael Gilday:
But beneficial to who?

Meg Mitchell:
Yeah. Well, intelligent in what ways? Is this a "We're going to overthrow the humans" kind of intelligence? Because maybe that's not what we want.

Michael Gilday:
I don't think that's more intelligent. If you can find people who will agree that's intelligent-

Meg Mitchell:
Oh. Rather, I should say the details of how to do that would require a much greater intelligence than AI currently has. Just sort of being autonomous, completely autonomous, requires much greater intelligence than what AI currently has. So that can go for better or worse. It can go in positive directions and negative directions. But I will say that one of the ways that I think AI will go, that I'd like to see AI go in that's more positive, is around lowering the barrier to entry of working on AI. So currently, there's a bit of a bottleneck where people who really care about the data, for example, so people who care about the rights of crowd workers who are labeling the data, people who care about data curation and data documentation, those sorts of people are not necessarily the people who have the engineering skills to run a query in a database of different kinds of data sets in order to extract some specific bit of information.

Meg Mitchell:
So there's a little bit of a bottleneck there between the skills needed to do something very fundamental and the skills needed to understand all the different aspects at play. And so one of the things I've been working on, and one of the reasons why I joined Hugging Face, is to try and come up with tools that allow people from totally different backgrounds to interact with data and interact with... I think there's a general tendency, or I should say there is more and more interest recently, in no-code solutions, so being able to build a model without code, being able to work with a data set without code. And I see that as something that can go quite positively if the no-code solutions are available for people who don't have coding skills, but do have social skills or the ability to understand people. Then I think that's a way that AI could evolve to become better, in some way is starting to evolve right now, and I would argue should evolve in order to help bring about more diverse perspectives and a deeper understanding of what the technology is.

Michael Gilday:
I totally agree. Another interesting thing is AI exists and is around us, but the people who are successful at it have sort of brute forced it with a lot of engineering power, when it's not sort of the magic thing people think it is. And we're just starting to maybe get the level down in terms of democratization that more people have access, but we're still staying within market forces, corporate enterprise space. How long or how far do we have to go before we can really start to bring that value to regular folks?

Meg Mitchell:
Yeah, the brute force issue has a lot to do with what tools are at your disposal. And so one of the areas where that has very recently been a problem is around the building of large language models, where training large language models has generally required the developers to be at a very large corporation in order to have enough resources to do the training. And we've seen very recently countries creating their own sort of super computers to democratize in order to allow people at universities or at schools to have some of the same access that people at larger companies have. So that's a little bit of a public approach that contrasts with a private approach. And so I think that direction of creating sort of more publicly-funded resources for AI is a good one to be going in, in terms of handling the sort of monopoly on brute-forced development that corporations have.

Meg Mitchell:
That said, I don't want to imply that governments are good and private corporations are bad. I think we've seen a lot of governments that might tend towards totalitarianism, and perhaps we don't want to let them have at their disposal lots of abilities of tech development. I don't know. But at least it's somewhat of a counterbalance, and opening it up to let people have more access, which could potentially be quite good.

Michael Gilday:
Yeah. A variety of funding sources and structures of incentive, I think, has... I'm assuming you are a fan of open source.

Meg Mitchell:
Yeah, yeah. There's a distinction to be made between democratization and responsible democratization. So some things that you might make widely available foreseeably can create a lot of harm, especially if they're not really well-understood in context or documented how they can be used in context. So facial recognition is an example of that, one where even if it's open source, I would like to have there be some sort of agreement about how it can be used. But in general, I would say that open sourcing and open sourcing paired with no-code solutions so you can play with the sort of general interface and you can go directly to the code that's underlying it, I think that's a really cool way to go, and opens the discussion up to tons of people.

Michael Gilday:
Do you think the philosophy of open source has had a major impact on the acceleration of some technologies?

Meg Mitchell:
Oh yeah, absolutely. That's true for operating systems as well. Red Hat has been open source, and it quickly became one of the dominant Linux operating systems. And it's the same with computer vision. It's the same with natural language processing. One of the sources of computer vision development has been OpenCV, which was a resource, or is a resource of code that people can use as building blocks for computer vision. Yeah, absolutely it sped up development.

Michael Gilday:
But then you've got things like, like you said, sort of open source technology and low code, and then you've got deepfakes of inappropriate images of folks.

Meg Mitchell:
There are always trade-offs. This is one of the things about working on ethics, informed AI as well, is that you find you can't really develop things that are 100% ethical in some way, 100% values that are good and no values that are bad. There's always trade-offs where if you want to do one thing, so democratize for people who have useful expertise, then there's the trade-off of democratizing for malicious actors. So I think that's always there. That's always going to be there. Part of the trick is being able to foresee that, to think through all the different kinds of users and all the different kinds of ways something can be used, and trying to develop things like licenses that at least help somewhat mitigate the issue.

Meg Mitchell:
And then similarly, you might want to come up with aspects of the technology that makes it easier for malicious actors to be found. So for example, watermarking. We might want to develop technology for deepfakes, or might want to develop generative models, but in that they can be used for deepfakes, we might also want to think about generative models that have some sort of processing in them that plays the role of a watermark, maybe some second-order characteristics or something like that, so we can really start to understand who's using this and pretending these things are real and who isn't.

Michael Gilday:
What do you think is a bigger danger in the future, malicious or bad actors, or irresponsible actors?

Meg Mitchell:
Definitely irresponsible. There's arguably a subset of people are malicious. I would say all of us are irresponsible in our own ways. And not just irresponsible, but I would say also just sort of unknowledgeable about the issues, because the machine learning world has largely not shared the issues. Documentation isn't even really a thing yet. So intended use and directions for use don't really exist.

Michael Gilday:
People share the information. Sometimes they lose their job.

Meg Mitchell:
Exactly. Yeah, that's definitely an issue as well. Sometimes there might be an incentive not to share the relevant documentation, but then that creates situations where even people who would like to be responsible are using the technology in irresponsible ways or problematic ways without realizing it. It takes a lot out of understanding to use technology in a social context really well. And without that, I think generally we will see that people are using technology problematically or somewhat irresponsibly if for no other reason that they don't know what the responsible usage would be.

Michael Gilday:
So I'm curious. I'm making an assumption here, but where did your love of tech sort of start in your life?

Meg Mitchell:
It's hard o pinpoint an exact moment. I was a user of Prodigy, for anyone who remembers Prodigy. It was one of these first internet-based systems, and I was just fascinated by that. I was fascinated that you can input things, and then somehow it went somewhere, so that they would have competitions of writing captions on little pictures and stuff. And I was just amazed that I could write something in my computer, and then it was part of a larger competition. There were other people doing it too, and they were all sort of coming together in some way. So I was super fascinated by that, and I also got super fascinated moving from Prodigy to AOL and things like this. I started getting more interested in user interactions and how things are designed, and I started doing a couple things. One was I taught myself HTML and JavaScript, really easy starter languages.

Meg Mitchell:
So this is early middle school or something, like 11, just to... I just wanted to make fun things that people could play with. I made a website where you could ask Satan what the future was... Like a Magic 8 Ball, but it was all just terrible. So I would just have a lot of fun doing things like that. I would also waste a lot of time in DOS. And that's mostly wasting time at that point, but I liked making things go. I liked how there could be communication between people from all different places. And so I would say that late elementary school, early middle school was when I really started to understand what tech is and become very fascinated about it in a ton of different ways.

Meg Mitchell:
And then I never really thought of it as something I would follow as a job, because I think a lot of women have this experience where you're like, "Well, I'm not a tech person. I might code every day, but literature... I'm good at spelling. That's my thing." It wasn't until much later in my life that I realized, "Oh no, actually I can actually do this as a living, and I would make a lot more money than I would as someone who majors in literature, probably, so maybe not a bad path to follow." But it took a long time to realize that.

Michael Gilday:
Why do you think that sort of gender line is there on self-identifying with tech? It used to be... And I'll tell you, I grew up as a techie, but we were just all nerds. Then there became different strata of nerds, and some were cooler than others, but it has been traditionally kind of masculine.

Meg Mitchell:
Yeah, yeah. Well, part of it is your social groups and what's incentivized in your social groups. So I've often been in situations where there was... There's a separation at a party between the men, and the women are sitting around talking about stuff, and the men are over there doing something like making music, or showing each other tricks on their skateboard, or whatever it is. And that distinction always really fascinated me, that there seemed to be somewhat of an incentive when you're aligning with men that you're doing things and showing off things or creating things, whereas with women, it was more of everyone's on the same page communicating about things. And so that definitely has been a factor, at least in me feeling self-conscious about some of the stuff that I've found really interesting, because it has aligned more with men I know, at least in terms of the cultures that are put forward, than with women.

Meg Mitchell:
So there's a cultural aspect there, where I think at this point, or at least maybe in the '90s and such, the culture was really accepting of men coding things and working together to code things and showing it off to one another, whereas there was less so with women. And that also goes to general stereotypes about being ladylike, and it's your job not to make any waves and to be supportive of the men, rather than leading them in any sort of ideas.

Michael Gilday:
Do you think we're making progress in that space, especially women in tech and STEM?

Meg Mitchell:
So we've done better in the past. It's, I think, common knowledge now that the first computers were programmed by women. We saw in... I think it was the early '90s, there was much more representation of women in technical roles, and then the representation of women went down quite a lot. There was much less women through the '90s. And then now, I'd like to think there's a little bit of an uptick. It seems like there might be. The numbers are hard to understand, because when you're using self-reporting from a company on their gender breakdown numbers in different sort of roles, they'll group together technical and non-technical roles as long as it's in a technical part of the organization, and say that, "This is our percentage of technical women." It's like, "Well, technical in the sense that we're in the tech organization, but 100% of the people with assistant in their name is a woman. So that's a problem." I think it's a little bit harder to tell than I would like, but I don't know. I honestly don't know.

Michael Gilday:
But how telling is it that, like you mentioned, women's contributions was written out of the narrative for a very long time in the space? And how important is storytelling that's based on fact, but ensures that the stories get out there and inspire other people?

Meg Mitchell:
Yeah, yeah. Historical writing is super important, and I think one of the large issues is around our desire for alpha hero narratives. And I think we as humans, I suppose, or at least from the cultures I'm familiar with, have had a preference for hero narratives where there's one hero, and it's usually the good guy, and it's usually a guy, because we associate being heroic with having lots of muscles and fighting things or whatever, and that's traditionally a more masculine thing. So there's a strong desire for alpha hero narratives, and that affects everything that gets recorded in history and who sort of gets erased and marginalized, especially if we're only looking for a subset of things in who we consider to be heroes.

Michael Gilday:
Well, the other thing about the alpha hero narrative is if you remove those heroes and you've undervalued the whole entire movement, people lose hope.

Meg Mitchell:
I mean, there's an issue with having only one versus having multiple people be recognized. It's an issue that's really close to my heart recently, because journalists only have so many words they can put in an article. And so we've seen that there's a lot of women and people with marginalized characteristics involved in recently speaking up against tech. So Kate Rotondo is one, Ashley Gjøvik is one, Chelsey Glasson is one, Cher Scarlett is one, Aerica Shimizu Banks, Ifeoma Ozoma, but then you only know a subset of these names. Even if they're all working on the same things, some of them have more connections, for example, and so they become more of who the narrative is about, and less all the other are people at play.

Meg Mitchell:
And it is a constraint, but ideally we could say, "These three people are awesome. They're all women and they're awesome at the same time," as opposed to really going towards the "It's just the one." So I think that ideally, we could move more towards that kind of thing, but I think we're up against a lot of human culture that might make that difficult, and literal limitations on how many words can be written in an article. And this is where I hope historians are paying a lot of attention, or journalists who are hoping to write books are keeping notes on everything in order to paint a more holistic picture.

Michael Gilday:
How do you stay current?

Meg Mitchell:
How do I stay current in what sense?

Michael Gilday:
How do you discover who's doing things that are inspirational and moving the needle and expand your aperture beyond, I don't know, the largest of the mainstream communications?

Meg Mitchell:
Yeah. So honestly, Twitter I have found really useful. A lot of people have come to my attention via Twitter that absolutely would not have been on my radar. I also pay a lot of attention to who the authors are on papers I like. If you're someone who is very familiar with the conferences where people are publishing, then it's a lot easier for you to understand who a lot of the players are, versus being someone who reads articles on the papers that come from different conferences. Which is going to be most people, but that gives you a very limited sense of what the technology being worked on is and who the players are. And I'm someone who pays a lot of attention to who authors are, even when they're not at a top university, or what's considered a top university, as those tend to get more attention. And so that's been a way that I've found really cool people that have otherwise sort of flown under the radar, so both through social media and through paying attention to who's publishing.

Michael Gilday:
So you were involved in the development of the concept of model cards?

Meg Mitchell:
So model cards were inspired by Timnit Gebru's work on data sheets for data sets. One of the fundamental ideas there is that for hardware, you have documentation called data sheets. We should have something similar for data sets. And at the time she introduced that, I had been working on protocols for robust evaluation of machine learning models, the kinds of things to document or have as artifacts. And it wasn't really being accepted as an interesting direction to go in, but once Timnit sort of provided me directly with this idea of a data sheet, it became clear to me that, "Oh, if I just make everything I'm saying be an artifact, and an artifact that can be launched in the tech industry where launches is what everybody wants, then people will be into it."

Meg Mitchell:
We spent a while thinking of a name, and so data sheet, and then trying to make a parallel for models, we ended up with model card. And then her paper was called Datasheets for Datasets, so we called our paper Model Cards for Model Reporting, trying to sort of do a sister paper where we built off of the work and then really drove home what it is to document a machine learning model and why it's important. And I've worked on that internally at Google, I've consulted on that for other companies, and obviously published on it and talked about it a lot.

Michael Gilday:
What are some of those key important things that you need to convey about models?

Meg Mitchell:
Yeah, yeah. So one of the key things that's fairly common in work on ethics and fairness is disaggregated evaluation. We might say in sort of general terms that when we evaluate a model, we want to know how accurate it is. What is the accuracy? Accuracy isn't really the metric to be using, but we can use that term because it's sort of generally understood. So that's an aggregate number. When you have some evaluation data set and then you get the accuracy of the model based on it, that's one number. We can also say we're going to take this evaluation data set and we're going to split it up into different pieces, where this part of the data set is the part that has a lot of representation of women, this part of the data set is the part that has a lot of representation of men, and we're going to give the accuracy on those two parts.

Meg Mitchell:
And now this is a disaggregated evaluation. We're starting to show how this breaks down. So one of the key components of model cards is doing a disaggregated evaluation, and then the question is: What are the different sort of layers that should be disaggregated? So that has to be motivated in the model card. Another question is: What are the metrics to use? Why are those important? So that has to be documented in the model card. And then there's a lot of additional stuff that relates to usage in context and trying to minimize irresponsible use or problematic use, so including the intended use, including the limitations, including ways that might be misunderstood, or things that are out of scope, really thinking with foresight about all the ways this model could be used in context, and speaking to what makes sense and what doesn't make sense.

Meg Mitchell:
You might similarly have... If you have a bathroom cleaner, you might say, "Caution: Not to swallow." You can imagine something similar with models, "Caution: Does not actually work if people's eyes are closed," or whatever it is. And so those kinds of things can minimize problematic uses, and that's basically a summary of all the bits of the model card. The sort of foresight thinking about how it can be used, documentation relevant to that, and documentation relevant to the evaluation of the model.

Michael Gilday:
So these kinds of examples of transparency, why is transparency and avoiding black boxes important in AI and ML?

Meg Mitchell:
Right. So transparency is something that leads to accountability. So if you have a technology that's problematic in some way, and you release it with a documentation that it is problematic in some way, that you're transparent about how the model works, then if someone is affected negatively by that, you're now liable for that negative outcome. You're now accountable for that negative outcome. But it's also something that incentivizes good behaviors. So if you have to report a disaggregated evaluation, and your evaluation shows that this system only really works well for men, then unless it's targeted at men, you're probably not going to release it until you can show it works relatively well for sort of all genders.

Meg Mitchell:
So having to be transparent means that your development processes are going to be things that are okay to be transparent about, which means that you're going to do a lot better work. So both the effects of transparency and the incentives of transparency end up doing a lot for creating better development, or rather I should say end up doing a lot for creating good kind of models, and end up doing a lot for people in understanding how things should be working and what things are not okay, and not the way that the technology should be used.

Michael Gilday:
Do you think transparency is valued or prioritized enough yet?

Meg Mitchell:
Not at all. I would say obfuscation is prioritized right now.

Michael Gilday:
That's not a strategy long-term, right?

Meg Mitchell:
I don't know, maybe. You can obfuscate until you're told otherwise. I think part of the concern around obfuscation also comes to liability. So since laws have generally been lagging development, it becomes stressful to be transparent about things that might end up being illegal. A lot of this is new territory, so you don't know until it's argued in court. So you kind of don't even want to make it an option to do something where it turns out that it's illegal. So using an example of how well a system works for different genders, it might be the case that if there's a strong difference, you've now come up against discrimination law. And so there's an incentive not to be transparent, because there's a concern that the law there would actually very much harm you by being transparent. And this is part of why we need regulation and law to catch up a bit to where we are, because there does need a little bit of a safety net for people being transparent, organizations being transparent to know that they won't be punished for that in some way.

Michael Gilday:
Do you think there's a potential advantage to people who work in highly regulated industries like healthcare, financial services, where they are forced to be more rigorous, and in the long-term may succeed in other industries where the regulation isn't? Obviously the regulation for AI just isn't there, but there is laws for protected classes and that kind of stuff.

Meg Mitchell:
Yeah, yeah. I think it's an advantage. The main concern is around stifling innovation, and that's actually a problem for the health domain. Or that is an area where the health domain and the technology domain actually intersect in a difficult way, because in the US, a lot of medical data is protected by HIPAA or under HIPAA constraints, and so you can't share it. But in AI, you need the data in order to make development in order to develop further, and in order to share things, in order to be reproducible. So it can be a limit. Having lots of regulations and such can create a lot of constraints on what gets developed and how fast it gets developed. That said, by working in constraints, we come up with newer, greater ideas. So it's sort of a... Once you have the set of constraints you're working within, you're going to build technology or build solutions that meet those constraints that you might not otherwise think of if those constraints weren't there.

Michael Gilday:
Do you think there's an education problem or a data literacy problem about... There is a lot of fear with data, and rightfully. It's not one-size-fits-all, and there is a lot of great that could be done with anonymized data in healthcare, like people like the Broad Institute and what they're doing, trying to end single-cell mutation diseases, that kind of stuff. But they need a lot more voluntary contributions, but people are scared. And the irony is people don't want to give up any of their data unless someone lets them see cute pictures of cats with their friends, and then they want to give up all of-

Meg Mitchell:
Right. Take it all. Take it all! Yeah. Yeah, that's definitely true. I think there is good reason to be concerned, because we don't have technology right now that can guarantee privacy or anonymization, especially in larger big data where you don't have people going through it by hand. We also are just starting to discover that machine learning models that have been trained on data, personal data, copyrighted data can be queried in order to extract that data back out. So I think that's sort of been a concern that people have had, maybe not fully understanding the mechanics of it, but this concern that if I have data and it goes into a model, someone can get that data.

Meg Mitchell:
And we're starting to see that that's actually maybe a valid concern. It involves understanding what the risks are in sharing the data that you have, and this goes to foresight and thinking through all the problematic uses and beneficial uses of your data. I actually like giving information about my demographics, because I think it helps with things like disaggregated evaluation. It isn't personally identifying, it's not uniquely identifying, but it is enough personal information that I think the benefits outweigh the risks.

Michael Gilday:
But that's an important distinction that people don't necessarily understand.

Meg Mitchell:
Right, right. Yeah, and I think that if it were possible to be more clear about how the data is used, not just in terms of along terms of service sort of thing that nobody really understands, but in something like a model card with clearly visualized details, so the actual output of the data or the actual artifact that comes from the data as opposed to a description of that, I think then it becomes a little bit more easy to understand why the state is useful and how it will be... But without that, when everything is still sort of opaque and obfuscated, I'm just giving my data somewhere, and what's going to happen? I don't know. How could it be used problematically? I don't know. Better err on the side of keeping private and being careful. So with more transparency in what the actual outputs of that data is, how it actually comes to be used, I think there could be more interest in data sharing.

Michael Gilday:
Is there one sort of either lesson you've learned or one thing that you've done that you've been most proud of, whether it was a big leap or an earned journey, or you just feel good about the impact?

Meg Mitchell:
Yeah. I think maybe the thing I'm most proud about is the team I was able to build at Google with my co-lead Timnit Gebru. We really set out to create an inclusive environment. It can be really hard to do that in a sort of exclusive environment, exclusionary environment where there's a lot of hostility, where there's disagreement that you need interdisciplinary perspectives. But we really hoped we could create sort of a safe space so that people who, for example, were social scientists felt comfortable being at the table alongside computer scientists, as well as people with different demographics that are underrepresented in tech, so being a woman or LGBTQ, different kinds of things like that.

Meg Mitchell:
We really wanted to create a space where everyone felt included, and I think we did a really good job of that. It's not easy. There's a lot of trickiness in how you represent yourself, because that really informs... That goes below you. So learning how to be vulnerable at the right times and these kinds of things, at the same time balancing that with doing very good, rigorous work, especially in an environment that is not very supportive of what you're doing, that can be really tricky. And so given the obstacles we were up against and what we were able to create, I think probably the Ethical AI team is the thing I'm most proud of.

Michael Gilday:
Do you think that is a model for more effective sort of teamwork in the future?

Meg Mitchell:
Yeah, absolutely. Yeah. A lot of building up that group involved transparency. So if you're trying to be inclusive, there's some tension with what you're obfuscating, what you're not showing them. Especially as a manager, you might be involved in some discussions about whatever, and they're not, and so you kind of don't share that. And yet by sharing that, you are bringing people into the table and into the discussion in a way they wouldn't otherwise be, and showing them just how much you value their input. You don't even have to give me input on this, but I would love to know what you think so that I can bring this back up through the hierarchy or whatever. So I think that there are a lot of skills and techniques that both Timnit and I learned and developed in creating the team that would scale to other places, and ideally continue to create larger and larger inclusive organizations.

Michael Gilday:
How are you going to spread the word about that?

Meg Mitchell:
I'm very loud on social media. For better or worse, I say what I think. That's always been either a strength or a weakness. People say that I'm very candid. So people that like me say I'm very candid. People that don't like me say I'm brutally honest. I mean, being a bit of a public figure, embracing that to the extent that I can in order to talk about this stuff is definitely part of how I'm trying to create a path forward that's more inclusive. And then also, joining a company that's just starting up and is a company that other companies are looking to as a sort of model, that's a really great opportunity to build up an inclusive culture, be transparent about the processes they're using. And then hopefully, if it's like a hot and exciting company, people just sort of follow suit.

Michael Gilday:
But I got to ask you: How did it feel to get everything sort of lined up and feel really good about the progress, and then have at least your important part torn down?

Meg Mitchell:
It felt negative. I would say it's a negative experience. I've been in tech a long time. I've worked in computer science a long time, so I'm very used to barriers and obstacles. I'm used to the feeling of being punched in the gut. So unfortunately, I already had a bunch of strategies I use to deal with situations that can be traumatic, so those skills were in full swing. But I think that the team that's in there now is doing a great job, and I'm happy that they're all having successful careers. I think all of us are doing relatively well in terms of our careers. Yeah, it's okay. There's a lot more discussion about labor law now, and ethics, so there's always positives. There's not always positives. In this case, there are some positives too.

Michael Gilday:
Yeah. Do you think overall, the dialogue has gotten a jumpstart that it may not have gotten because of-

Meg Mitchell:
I like to tell myself that, yeah. There was already a trend towards the general public paying attention to fairness and ethics. I think having some faces put to it was definitely helpful. So I'd like to think that it further catalyzed the discussion and brought in some nuances that were otherwise sort of missed, like the interaction between ethical AI and diversity inclusion, and how tightly connected those are. So I would hope so, but I don't know the other paths that we could have followed, the alternate realities, so hopefully.

Michael Gilday:
Well, I for one applaud you and appreciate your optimism, your enduring optimism and fortitude, and we need more folks like that. So I want to thank you so much. Thank you for being on the podcast. This was awesome. I really enjoyed the conversation.

Meg Mitchell:
Yeah, I enjoyed it too. Appreciate it.

Michael Gilday:
Awesome, thanks.

Meg Mitchell:
Thanks. Bye.