This is a Sonder Scheme newsletter, written by me, Helen Edwards. Artificiality is about how AI is being used to make and break our world. I write this because, while I am optimistic about the technology, I am pessimistic about the power structures that command it.
We are starting Artificiality Pro, a paid subscriber version of this newsletter, where our focus is on the frontier of AI and its interaction with human intelligence.
If you haven’t signed up yet, you can do that here, for both free and paid. If you like this post, share it with a friend :)
This week’s Artificiality is a bumper issue. There’s so much going on and so much changing. Here’s a quick take on what’s in this week’s newsletter:
Pandemic privacy.
How COVID-19 will reveal gaps in the EHR (electronic health record).
How AI isn’t up for this (yet) and what kind of technology solutions people need right now.
The opportunity to collect real-time data and why it’s so vital.
Why human experience and collaboration is more valuable than AI predictions right now.
The biggest gap in US health information system - death stats.
The US health system is about to be stress-tested like never before. Underneath the physical infrastructure and network of people lies an information infrastructure and a human-machine decision system. Is this system ready? How will it fail? What will it reveal about opportunities and barriers to effective AI in health?
Public good v privacy
A technology response to the current pandemic is important but it presents another challenge that we are unprepared for: redefining privacy. Just as we are unprepared for the outbreak itself, we are unprepared for the consequences of the technology response.
The most effective testing and tracking likely means that some privacy is lost. For example, when someone is tested positive, that person’s locational information is released to the public and so that people who may have been exposed can get themselves tested.
Just as medical practitioners are making up the rules as they go, privacy watch dogs and regulators should too. We’re all distinctly aware that times of crisis can usher in fundamental shifts in privacy. What were originally emergency measures become the new standard. In anticipation of this, Privacy International has set up an international tracker to monitor important measures, so that people can return to them in future.
Many measures are based on extraordinary powers, only to be used temporarily in emergencies. Others use exemptions in data protection laws to share data. Some may be effective and based on advice from epidemiologists, others will not be. But all of them must be temporary, necessary, and proportionate. - Privacy International
This is especially relevant if we look to big tech to provide part of the solution. Already, there are significant tensions between public health and privacy as governments turn to facial recognition and geolocation; Palantir is working with the CDC on data collection, as is Crimson Hexagon a company that scrapes Facebook, Instagram and Twitter. K Health is in talks with the CDC about aggregating data to map where people are showing signs of the virus.
We absolutely must do everything possible to control the spread of COVID-19 but we need to be vigilant on other things too. Soshana Zuboff writes in Surveillance Capitalism how the response to 9/11 allowed platforms to develop extraordinary surveillance capabilities. We can’t let today’s crisis be the platform for privacy erosion with vast, unaccountable AI and commercial entities securing a future moat for personal health data.
Trade-off: information exposure v movement restriction
Another way to look at the tension between personal privacy and public benefit is through the lens of individual rights. South Korea’s success in controlling the spread is just as much about its acceptance of surveillance as it is about its testing regime, according to Jung Won Sonn, Associate Professor in Urban Economic Development, UCL.
South Korea is the most surveilled country in the world. In 2010, everyone in South Korea was captured an average of 83.1 times per day and every nine seconds while traveling. This is likely to be far higher now. South Korea’s testing strategy is successful because sitting behind it is a huge surveillance network that combines CCTV and the tracking of bank card and mobile phone usage, to identify who to test in the first place.
Here’s how it works:
Banking: by tracking transactions, it’s possible to draw a card user’s movements on a map.
Mobile phones: Phone companies require all customers to provide their real names and national registry numbers. This means it’s possible to track nearly everyone by following the location of their phones.
Facial recognition (both AI and human): CCTV cameras also enable authorities to identify people who have been in contact with COVID-19 patients.
Combine the data: Nearly all potential patients can be found and tested by overlaying these three data sources. A new patient’s movement can be compared against those of earlier patients. This reveals exactly where, when and from whom the new patient was infected.
This information is made public through various smartphone apps, websites and text messages, which help people avoid hotspots.
In many ways, this is an overexposure of private information about people’s movements. But it is actually an effective way for the authorities to gain public trust, which in turn is important in preventing people from panicking. - Jon Won Sunn
South Korea’s public health information systems are an extension of the smart city infrastructure the country has built. The response to its repurposing is specific to the culture and demonstrates the interconnectedness of health, tech, politics and society.
Information is for billing not real-time care
In an article in Stat News, Eric Perakslis, Ph.D., Rubenstein Fellow at Duke University and Erich Huang, M.D., chief data officer for Duke Health raise the alarm on US hospital information systems’ readiness for the current crisis.
The EHR is not designed to give a clinician a cohesive picture of the patient. There is no fast way for clinicians to see an essential timeline of a patient. Tabs are split into sections - problems, medications, imaging. There’s friction that takes time to overcome - time that clinicians won’t necessarily have.
The undeniable fact that electronic health record systems are designed to track and bill procedures rather than provide optimal patient care is likely to be on full display as the health system becomes increasingly saturated with Covid-19 patients. - Perakslis and Huang
Prediction-action pairing is vital; keep it simple
AI systems for healthcare need a tight coupling between a prediction and the recommended treatment path. A National Academy of Medicine report on the opportunity for AI in health refers to this as “prediction-action pairing” and it’s critical to effectiveness of AI in health. It’s also a complex design challenge and one that’s hard to make up on the fly.
In the Stat News article, the authors advocate going back to simple ideas; say, an app that simplifies workflow and uses decision trees with a tight focus on COVID-19 diagnosis, that can be downloaded onto clinicians’ phones and updated continuously.
A simple app that guides frontline clinicians through a decision tree for evaluating and managing potential Covid-19 cases could reduce confusion and variation in care. - Perakslis and Huang
The information about this disease is changing rapidly - geographic information, comorbidities, risk factors and new symptoms. Current EHR systems update, at best, every quarter, which means it could be a while before there is a reliable predictive model embedded in current systems.
And while there are a number of organizations developing AI apps and AI-enabled symptom trackers for consumers to check their symptoms, it’s proving to be challenging to update models as new information emerges.
Data and real-time info is a huge opportunity
Research activity is going crazy and researchers need information as close to real-time as possible. A simple app could be very effective as a data gathering tool.
Because disease outbreaks are also times of intense research activity, well-designed apps may improve digital data collection and help research occur in a way that is less disruptive to clinical care. - Perakslis and Huang
We shouldn’t forget that humans are especially useful in the health information ecosystem. Scribes cut in half the time it takes to document a patient encounter. Scribes are a luxury in normal times but may prove to be essential in a crisis.
Causality + human connection > correlation + automation
AI relies on data. If there’s thin data then AI can’t do much. AI doesn’t understand causality which is of supreme importance in medicine. It’s critical to have a theoretical basis for why something might or might not work. There is no substitute for expertise and human experience.
Right now, the most valuable systems might enable clinicians to share information in an efficacious and privacy-secure way. This isn’t an AI opportunity per se, but it would bring in the data rapidly and lay the foundations for new models. Jennifer Ellice, an ER doctor in LA, said this via Twitter:
Death in real-time
And here’s something I had never considered - there is no robust information system for tracking deaths. The authors of the Stat News article claim this is one of the biggest gaps in the health information flow in the US.
A health infrastructure that cannot properly track death is unprepared to manage catastrophes. - Perakslis and Huang
It’s chilling to read that in Italy doctors are forced to ration access to critical care. The state of Washington is also preparing for this. As someone who lives in an immune-compromised household, this idea is personally terrifying. But “first come, first served” doesn’t work when hospitals are in crisis.
In a public health emergency, you shift from a focus on individual patients to how society as a whole benefits, and that’s a big change from usual care.” - George L. Anesi, critical care specialist at the University of Pennsylvania, per The Washington Post.
This idea is an important one for AI - under business-as-usual, AI can optimize for the individual. But with COVID-19, AI has to optimize for society, with all the contingencies and redundancies that it takes.
Dean Sittig, professor at the School of Biomedical Informatics in The University of Texas Health Science Center, told me that an important role of AI in health could be to optimize use of scarce shared resources.
Perhaps the most valuable AI to be developed in health is one that tells us who is going to live and who is going to die. - Dean Sittig.
This is all quite grim. There will be many lessons. The crisis will reveal vulnerabilities and gaps that experts have known about for a long time. The only silver lining is that perhaps now we will start listening to those who have always highlighted the interconnectedness of our health and social systems, the value of being prepared for the worst and leaving enough fat in the system to provide contingency. AI’s big role in this has been to optimize everything to the bone - supply chains, human attention and behavior, finance - and now there’s very little cushion.
Also this week:
An update on Porkbun from last week’s newsletter. Leanne Carroll, who alerted me to the company’s Name Spinner got in contact with me with this:
“I sent them some emails asking to chat with them about the product and shared examples of what I was seeing. Their CTO got back to me and let me know they've decided to disable the Name Spinner until they have the time to build something without the bias issues. A minor victory! Apparently they were using the Wordnet database which I'm sure many people are and clearly it has some awful association issues.”
A Sonder Scheme article on how developers and programmers who try to become AI developers suffer from imposter syndrome because of a lack of conceptual understanding of AI and a dearth of practical support in AI development tools.
Plus two members-only articles from Sonder Scheme - new research on how humans judge machines and how to map human and machine roles. New canvases are included and available for download in the membership which will help you work through the concepts and produce practical work product.
From MobiHealthNews, a comprehensive list of tech's role in tracking, testing, treating COVID-19. This will be worth tracking over the next few months to see what sticks ITRW.
This interactive model is worth spending some time on. You can experiment with variables such as infectiousness, social distancing measures etc. It’s not meant to be about COVID-19 per se, but more diseases processes in general but it’s handy to be able to play around with the sliders and develop a more intuitive sense of “exponential transmission.”
And surprise, surprise, Ring’s surveillance doesn’t actually reduce crime, despite the cheery anecdotes. Cnet takes a close look at the data.
Finally, I recommend this wonderful piece by my friend Ephrat Livni in Quartz; The lessons of cherry blossoms are most relevant during the coronavirus pandemic.
