In this episode, we have a conversation with Rana el Kaliouby, CEO and Co-Founder of Affectiva, about emotional AI, bias in AI and her new book, Girl Decoded. Rana is a leader in her views on ethical AI and how to design AI systems for humans—two topics that are near to our hearts! We hope you enjoy this conversation as much as we did.

Note we open with a short conversation about IBM’s announcement about no longer selling general purpose facial recognition technology—an unusual step for a big tech platform to cancel a product line for ethical reasons.

In this episode we have a conversation with Renée Cummings, Founder & CEO of Urban AI, about the issues and opportunities for AI in urban settings. While we recorded this before the current protests, our conversation with Renée couldn’t be more timely as we talk about the use of AI in law enforcement and recruiting. We think Renée’s specialized focus on AI in urban settings will be essential to understand as we, as a society, seek to bring people together and rise up out of the conflicts and economic strife that are hurting so many today.

This is a Sonder Scheme newsletter, written by me, Helen Edwards. Artificiality is about how AI is being used to make and break our world. I write this because, while I am optimistic about the technology, I am pessimistic about the power structures that command it. 

If you haven’t signed up yet, you can do that here.

Don’t forget….

This summer we’ve carved out time for a virtual summer camp for middle and high schoolers who want to learn to how to build an AI start-up. And because we know that everyone has had enough time on a computer, we will have kids out and about doing data collection, market research and other activities that aren’t all about Zoom. We’d love it if you can help spread the word and share.

Microsoft researchers recently published one of the most exciting advances in AI this year, IMO. I’m not referring to the company’s announcements regarding huge new NLP neural networks using semi-supervised learning, or the company’s new supercomputer, or the new tools for fairness in AI, although these are all impressive and useful. The new research is in human-machine collaboration.

Human-machine collaboration has been a hot area for years but, outside of social robots, it’s been more the domain of non-technical people than technical people. There simply hasn’t been much in the way of a scientific methodology for engineers to take a hybrid approach. While machine learning engineers focus on the frontier of mathematical and computational capabilities, in the absence of methods for human-machine team work, technology has generally marched on in isolation of the human factors. So while we can talk about “augmentation” of human skills, the reality is often different; shitty automation, sub-optimal task design, algorithmic aversion, hidden bias.

New research changes this. Now we have the math to train models in hybrid human-machine systems, taking into account what it means to consult a human.

Here it is: (or one sample of it)

Got that? Great.

In a new paper called “Learning to Compliment Humans,” Eric Horvitz, director of research at Microsoft, along with fellow researchers from Microsoft and Harvard, explain how they have formalized the math to design AI that has humans and machines work better together. It’s this mathematical formalization that makes this paper so important because now there’s no excuse for AI practitioners not to design AI systems that leverage the best of human and machine capabilities at the same time.

The methods presented are aimed at optimizing the expected value of human-machine teamwork by responding to the shortcomings of ML systems, as well as the capabilities and blind spots of humans. 

The standard way of training models which are used to compliment human decision making is to train in isolation—have the model produce the highest accuracy in isolation before putting it in front of a human. This new approach is different—the model is trained in such a way that it is forced to consider the distinct abilities of humans and machines. Training takes into account the “cost” of consulting a human and uses well-established AI techniques such as backpropagation to encode the unique skill of a human alongside the capability of the machine.

Think of it as a formal way to teach a machine how to make the trade-off between what it needs to learn to do to be accurate at a task, but isn’t inherently good at compared to a human.

This is a totally different mindset as well as technical approach.

• it optimizes the combined performance of the human-machine system, increasing the accuracy of task overall.

• joint training allows for smaller models and helps a model focus its limited predictive ability on the most important regions (of hyperspace), while matching humans’ top abilities with where the AI can afford to be less accurate.

• opens up a new design opportunity—tuning for asymmetric loss, where the impact of a false negative is much more than the impact of a false positive, for example.

The researchers tested their joint training approach on two problems: identifying galaxies using citizen science and diagnosing metastatic breast cancer in pathology slides.

For the star gazing application, joint models which optimize for complementarity uniformly outperformed fixed models; by anywhere between 10 and 73%. For the cancer task, improvements were up to 20%.

But the math allows the researchers to go further and to explore what factors are most influential. Because a smaller capacity model has more potential bias (because it represents less complex hypotheses and can’t fit the “truth” as well), there has to be a “tighter fit” between training and team performance. In theory, it’s possible to just throw more data at the problem and make more complex models but in practice this increases the risk of overfitting, so having simpler, better performing models is more useful. This approach helps with tuning for this and hints at new ways to value human expertise in training and developing AI.

The value of complimentary training is especially high when there is an asymmetry between error costs. In cancer diagnosis, for example, a false negative is much worse than a false positive—missing a cancer that’s present is way worse than subjecting someone to unnecessary intervention and anxiety. This technique allowed researchers to show that using a combined system is particularly valuable when these costs are asymmetric—the gap between the joint and fixed models grew as any asymmetry grew. This finding surely should have a big impact on algorithmic design in medical systems.

Humans and machines make different kinds of mistakes. This research was able to identify and quantify this affect as there was a very clear structure to the human error. In one experiment, a large portion of the human errors were concentrated in a small portion of the instances, identified with only two features. The joint model could then prioritize this region. While this came at the expense of lower accuracy in a different region, this was where the human had almost perfect accuracy so the overall effect was still an improvement.

The distribution of errors incurred by the joint model shifts to complement the strengths and weaknesses of humans.

There is now a mathematical way to train AI to take in a human’s knowledge and skill as the AI learns. This means that AI practitioners are now able to optimize the human-machine team performance when interactions between humans and machines extend beyond querying people for answers—say in settings where there are complex, interleaved interactions and that include different levels of human initiative and machine autonomy. 

We see opportunities for studying additional aspects of human-machine complementarity across different settings.

This is a huge step scientifically but it’s one that’s also important psychologically. Now human-centered AI can be expressed in a way that machines truly understand. That should have everyone excited.

A lot has happened this week so I’m skipping the extras.

In this episode, we open with a chat about Facebook’s acquisition of GIPHY and what the company may be trying to learn with its AI (hint: hidden meanings) and then we move on to a great conversation with Maria Axente of PWC about ethical AI at PWC and in her non-profit work with groups like UNICEF.

If you’re enjoying our podcast, please share with your friends, subscribe and give us a like—we’d appreciate your help spreading the word.

This is a Sonder Scheme newsletter, written by me, Helen Edwards. Artificiality is about how AI is being used to make and break our world. I write this because, while I am optimistic about the technology, I am pessimistic about the power structures that command it. 

If you haven’t signed up yet, you can do that here.

Special announcement…

This summer we’ve carved out time for something really important and close to our hearts: a virtual summer camp for middle and high schoolers who want to learn to how to build an AI start-up. We’re looking for enthusiastic, self-starters who are interested in what’s possible at the intersection of entrepreneurialism, AI and design. If you can, we’d love it if you could share in your network and help us bring human-centered AI to a Sonder Scheme Junior “Shark Tank.”

We’re also looking for someone to sponsor a couple of scholarships in each class for kids who would otherwise not be able to attend so if you have any ideas, get in touch.

If necessity is the mother of invention, then Facebook’s AI has been on extra duty since Covid as it tries to identify and understand hidden meaning.

AI now proactively detects 88.8 percent of the hate speech content we remove, up from 80.2 percent the previous quarter. - Facebook, May 2020

Moderators, as contractors, can’t work from home due to security concerns, so removing hate speech has relied more on AI while human moderators prioritize Covid misinformation. Covid is new, so there’s relatively less data available for AI to learn from which makes humans even more important. But humans are not a sustainable strategy for Facebook. Facebook is simply too big, too fast and too diverse.

Content moderation is a frontier for AI research. Developments in content moderation will drive a wave of new AI capability, which will be of huge value in Facebook’s core business of micro-targeting ads.

Hate speech is complex for AI; it’s a small proportion compared to the billions of posts that are not problematic; it’s often multimodal, it can be ironic or sarcastic which is tough for AI, and the people posting it are deliberately trying to avoid detection by doing things such as manipulating words or creating ambiguous text within a broader context. Haters use dog whistles to try to hide the meaning from anyone who doesn’t understand the codes—from those who doesn’t understand the hidden language.

Of course, if Facebook can use AI to solve hate speech, then it can use the same AI for a lot more. Which is a big incentive.

This week the company made a couple of interesting moves. It announced the acquisition of GIPHY:

GIPHY, a leader in visual expression and creation, is joining the Facebook company today as part of the Instagram team. GIPHY makes everyday conversations more entertaining, and so we plan to further integrate their GIF library into Instagram and our other apps so that people can find just the right way to express themselves. - Facebook

And it published a blog focusing on the AI research direction for hate speech. The details help us understand Facebook’s AI strategy in the context of moderation as a pain-point, but give insight into the potential reach of this AI in future.

To better protect people, we have AI tools to quickly—and often proactively—detect this content. - Facebook

I think there’s a way to put these two things together because GIPHY isn’t just about giving “people meaningful and creative ways to express themselves,” it’s also about giving direct access to data for AI to learn about the creative ways that humans express themselves in non-explicit ways—irony, humor, sarcasm, sleight-of-hand, juxtaposition.

And the work from Facebook’s AI team lays out how and where this data could be made even more valuable.

The unique challenge of hate speech

The real challenge for AI as a moderator is that humans are incredibly adept at manipulating language within the context of other media, especially when they don’t want to get caught. For humans, all media is mixed—vision, language, sound. Memes can use text and images or video together. The text alone can be ambiguous but when it’s combined with the image, the statement takes on another meaning, for example:

Credit: Facebook

But Facebook is showing how slick it is at hybridizing AI and developing ways to recognize usage in multimodal situations.

Facebook’s hybridization of leading-edge AI

In 2019, researchers took the Google BERT model for language and combined the technology with a vision system to create Vision-and-Language BERT (ViLBERT). Researchers used visual representations to pre-train ground truth then enhanced the language models, essentially building a joint visual-linguistic representation. The innovation links the models, having them reason jointly between vision and language with separate streams for vision and language processing that communicate with each other through transformer layers. 

The most important development from an AI perspective is the introduction of self-supervised learning models into production. The best way to think of self-supervised learning is a kind of “fill in the blanks” learning style. It’s fairly new and pioneered by Facebook’s chief AI scientist, Yann LeCun. What makes self-supervised learning different is that the goal is to have an AI to learn to reason. AI learns by gradient-based learning and self-supervised learning makes reasoning compatible with gradient-based learning. AI learns by filling in missing information so it develops a representation of the world and then it can be used to reason more generally about a task, like classifying language. Just as babies learn through observing the world before they undertake a task, self-supervised learning has AI do the same.

If AI can understand English, it can understand everything

Another theme that’s fascinating is language-universal structures, where AI can detect meanings that are similar in different languages. To be honest, this is kind of spooky—even with languages that have very little training data, the AI can build a model based on the structure of other, more prevalent languages. This graphic illustrates how hate speech in different languages is represented in a single, shared embedding space.

Credit: Facebook

Facebook now really does have the babel fish. It’s like having a translator—one who has their own agenda—involved in every conversation. It’s a strange feeling to see all of human language boiled down to common structures in high-dimensional space.

This allows models like XLM-R to learn in a language-agnostic fashion, taking advantage of transfer learning to learn from data in one language (e.g., Hindi) and use it in other languages (e.g., Spanish and Bulgarian). - Facebook

Fusion models: making a superintelligence and not calling it one

Self-supervised learning has enabled Facebook to build a vast array of fusion models that combine images, video, text, people, interactions and external content which then can all be mathematically represented across all languages, regions and countries.

Credit: Facebook

It really is the ultimate way to combine all of Facebook’s data. Actually all data, period. Which, of course, is a much bigger opportunity, by an order of billions.

This allows us to learn from many more examples, both of hate speech and benign content, unlocking the value of unlabeled data. - Facebook

If they called it the “brain of the world,” we’d all be terrified. But instead it’s AI for removing hate speech. And GIPHY, the data set that represents memes as a new form of human communication is a perfect resource to add to AI’s reasoning. Lucky it’s called meaningful and creative sharing.

Facebook’s AI frontier is formidable. Right now, it’s targeted at hate speech, which we all agree is a good thing. But AI that works on hate speech—which presents a relatively small but technically sophisticated portion of content on the platform—will be massively more powerful on the billions of posts across the world, especially when it can understand all the ways that humans try to stay opaque to the machine.

Where will they aim it next? And once Facebook’s AI understands how humans communicate with hidden meaning—through things like hate speech and sarcasm—how will the AI’s communication with humans change? Will we be unknowingly influenced by the AI because it’s expressing things to us in hidden ways?

Also this week:

• Since the pandemic and lockdown we have launched our Studio for human-centered AI design, re-crafted all our workshops for use by distributed teams, innovated on design thinking for couples with our coupled-centered design system for COVID confinement (free books), doubled down on ethical implementation of AI in general and in HR in particular. If you haven’t checked out our products and services recently, please do.

• Clearview AI in NZ, from RNZ. NZ’s experience mirrors the fundamental concern; individuals in law enforcement can trial this without “higher ups” knowing about it. This is the new “move fast and break things,” where privacy commissioners or other watchdogs for democratic public decision making aren’t part of any trials. “Official emails released to RNZ show how police first used the technology: by submitting images of wanted people who police say looked "to be of Māori or Polynesian ethnicity", as well as "Irish roof contractors".” Plus, an interview between Kim Hill and Kashmir Hill, a technology reporter for the New York Times who has been following the company.

• An essay in Nautilus from the inventor of the Roomba on how designers of robots need to be creative at understanding what a human is doing to complete a task and why they do it. It’s a delight to read. “Robots and people may accomplish the same task in completely different ways. This makes deciding which tasks are robot-appropriate both difficult and, from my perspective, great fun. Every potential task must be reimagined from the ground up.”

• ICYMI, an article in NYmag that everyone’s talking about: future of higher ed, by Prof Galloway.

• Great discussion on All Tech Is Human on emotional AI with Rana el Kaliouby and Pamela Pavliscak.

• Article in the MIT Tech Review about how humans need to step in for failing AI. “Machine-learning models that run behind the scenes in inventory management, fraud detection, and marketing rely on a cycle of normal human behavior. But what counts as normal has changed, and now some are no longer working.” Called it.