Mumbai: In Code Dependent: Living in the Shadow of AI, award-winning Indian-British journalist Madhumita Murgia, artificial intelligence (AI) editor at the Financial Times in London, recounts meeting a woman who set up a company that watched the data-labelling industry grow from its early years.
Data-labelling is the process of identifying raw data including images, text files, videos, etc, categorising the data, and adding informative labels to each in order to provide context that enables a machine learning model to learn from it. The woman’s company operated in Kenya, Uganda and India, where thousands of workers were engaged in this laborious and repetitive labelling work, a kind of back-end spine for AI models.
Such low-paying employment has lifted 50,000 people in East Africa out of poverty by offering digital work, including for dependents of workers, the company said in 2022.
“The great false hope of Silicon Valley is automation,” the founder told Murgia. “But we’re only pretending—it’s actually humans behind it.”
It is one of scores of examples from across the world through which Murgia tells the story of how AI is impacting ordinary people, exploiting cheap labour markets, exacerbating structural social inequities and more.
The book describes the impact of technology that marks children as future criminals, an app that brings medical diagnoses to a remote tribal community, deepfake sexual imagery that devastates women, among other tales, many of them cautionary stories on what could go wrong.
India, a major market and a labour market for AI developers, has been bullish about the sector, with government data stating that there was a 109.6% rise in private and public expenditure on AI during 2018, to reach US$ 665 million. By 2035, AI has the potential to add $ 1 trillion to India’s economy, according to the NITI Aayog, the government think tank.
Asked by Microsoft founder and tech-billionaire Bill Gates what technological advancements excited him, Prime Minister Narendra Modi said on 29 March 2024 that he was interested in how AI may be used for good, among other things.
Murgia told Article 14 in an interview that countries in the global south must call for accountability from data-labelling companies and their customers, the big-tech billionaires. “Several economists feel this is the time to hold the big tech companies to account because they need these people,” she said.
Murgia also touched upon gendered applications of AI, the concerns about data privacy and surveillance, and the heartening impacts from the use of AI in healthcare. The book is published by Picador in India and is shortlisted for the inaugural 2024 Women’s Prize for Non-Fiction.
Excerpts from the interview.
One thread that runs right through your book is about social inequities and discriminations exacerbated by AI. What were the most worrying examples of this sort of inequality that you came across in your travels?
Almost every chapter is an example of this. I didn't start out with a plan to show this but it was just completely organically the people and the stories that I found.
Let’s start with the invisible workforce that helps build these systems. We expect that AI systems are autonomous and intelligent, but they need labelled examples of pictures, of words, of video, etc, in order to learn to find patterns. To do that, we need armies of people, humans, who set up banks of desks and computers and actually sit and label and kind of categorise objects for these systems.
In many cases, these people are automating their own jobs in the sense that they’re making themselves obsolete.
And because of how many people are required to do the job, it’s an expensive endeavour, involving thousands of people. Outsourcing is the obvious economic solution to this problem, and so, a lot of the data labelling is done in the global south.
In Argentina, Kenya and Bulgaria, all the companies that I visited were employing people from lower socioeconomic backgrounds.
In Kenya, it was mostly young people from Kibera, the largest slum in Africa. In Bulgaria, it was specifically refugees who had fled war, refugees from Iraq, Iran, Syria. The goal was to provide them economic stability. In many cases it had done so, but there was this huge inequality between their pay and that in the big tech companies, AI companies, consulting firms and banks that say this technology is going to inject billions, if not trillions, of dollars into our global economy.
On this end, you have people who are being paid, yes, a living wage; it’s not that they're being mistreated or that it's a toxic environment; but they are literally not even seeing what's coming out the other end of this pipe. One of the women I spoke to in Kenya compared this to luxury fashion and the people who make, say, a Gucci shoe in a factory in Southeast Asia, who have no idea that the little stitches they're making for $2 an hour end up becoming a $3,000 shoe. As someone said to me, in this case, this was a $3 trillion shoe.
This labour force is not benefiting from the upsides; they are not aware of or empowered by what they're building.
Another example is in Amsterdam, from where I write about a single mother who received a letter from the mayor's office saying her children were on a list of potential future criminals. An algorithm was picking out families, boys in particular, and overwhelmingly, these were boys from the North African immigrant communities in Amsterdam. Many happened to be single parent or single mother families, who were told they were on this list, that it was not supposed to be punitive, they were not supposed to be policed.
It was a very dystopian system, even if it had a good intention to begin with. The way it was enacted in practice excluded the mothers completely from the conversation, making the boys and their families feel scared and disempowered. That’s another way we see these inequalities play out.
Is racial profiling or ethnic profiling one of the big things that worries you about applications of AI in surveillance?
Particularly when it comes to criminal justice, policing and surveillance, the worry of profiling is real, for two reasons.
One, the technology is inherently biased from a technical perspective. We know now that facial recognition makes far more errors in identifying female faces or those with darker skin compared to Caucasian and male faces. Researchers haven't quite figured out why that is the case because they've tried to make the data equivalent with as many male and female faces, they've tried to use racial diversity, but they are finding that, for whatever reason, facial recognition systems all over struggle when it comes to identifying darker-skinned and female faces. You can imagine how this can be multiplied when it's being used at scale.
I give examples from the US, where African-American communities are over-policed. I also talk about an activist in Hyderabad who is Muslim and was profiled during the lockdown. The police took a picture of him, but for no reason. He was just doing errands with his father-in-law on his motorbike. He feels it's not so much that persons of a religious minority are profiled, but that it was often people on scooters, on the street who are being stopped and surveilled by these systems, compared to people in their cars.
The socioeconomic divides play out in such a visible way. With surveillance and policing, the profiling part of AI and how it scales it up so quickly is worrying.
Employment is one of the big discussions around AI in India, and everyone's worried about how the job market will be affected by AI. Is this something you came across elsewhere too, and how really is AI reshaping job markets?
This is a really live question. Generative AI or the GPT language models bring this problem into stark view because you can see how they're able to essentially produce creative outputs that humans do—writing, creating images, art, videos, coding. For decades, we’ve been training children to be engineers, learn how to be software programmers, and now AI is as good as the best programmers we have. It’s the first area where we're seeing this replacement of coders by AI.
This is a global issue, and people everywhere are worried.
I've spoken to everybody from voice actors to writers, journalists, graphic designers, video game artists in the US, etc. In Hollywood, the Writers Guild has been striking and part of the reason is that they are concerned that AI is coming for their jobs.
Ranging from creative spaces to white-collar jobs, desk jobs, where you can use AI tools to augment productivity on tasks, whether that's writing emails or summarising documents and interviews, transcribing, there is a real fear of where this leaves us as humans. In terms of where we're already seeing change, as I said, computer programming and coding is a place where there is genuinely high-quality AI. Many technologists I've spoken to recently say that they no longer code without AI; it does about 50% of the work.
We are yet to see replacement in some kinds of writing tasks, intellectual analysis tasks, whether that's lawyers, management consultants, strategists and so on.
The work for us over the next decade is to figure out what it is that humans do better.
In journalism, for example, the reality is that there are truths that companies or institutions may want to keep hidden. These are things that AI can’t surface, it’s only through human curiosity and empathy that we go out looking for stories—that is what is going to bring those stories to light.
I think we will evolve all jobs, whether mine or yours, it won't be the same just as you know, the jobs we do today are not the same as they were 20 years ago. I'm optimistic that new jobs will emerge.
Tell us a little more about the AI sweatshops in India or Kenya or Bulgaria. Do you see more of that happening? Could those be the only kind or large chunk of new jobs that will emerge?
I definitely think there will be more of this. Already, India is one of the centres for this type of work.
There’s a company that employs women near Kolkata who helped enable Alexa with voice snippets. There is going to be a huge appetite for more labelling as we continue to train AI models with a voracious appetite for data. It's the only way to increase technical sophistication, so there will have to be factories and thousands of people around the world doing this work, possibly in the developing world because of the costs.
The danger is that this stays hidden and they can continue to be paid the equivalent of $4 an hour, which may be the local market rate, which may allow them to put their children through school or pay for parents’ healthcare. But in no way does that pay bring them up to the level of fair payment that we should be looking at globally.
Why should someone in India, for the exact same job sitting at a computer and labelling images, be paid differently from somebody doing that in New Orleans or elsewhere?
Right now is the time for us to hold these companies to account particularly the customers of these labelling companies, which are some of the richest companies in the world, trillion-dollar companies such as Amazon, OpenAI, Microsoft, Google, and others, on a fair wage for this type of work in AI.
It is not equivalent to call centre work or factory work, and it shouldn't be the same wage as those. Several economists feel this is the time to hold the big tech companies to account because they need these people.
I don't believe that these are the only new jobs that will emerge from AI. I see really interesting, more efficient and productive ways of doing the jobs that we have today whether it is investigative journalism, or even artists using these tools. There is a way for us to use these tools as mentors of the things that we are good at. We will see a plethora of new ways in which we work alongside AI.
Has the discourse on ensuring that companies employing data workers for AI are held accountable, or is there any unionising of such workers?
A really good example of what's been happening on this, on which I spend an entire chapter of the book, is the people working in app-based companies such as Swiggy, Zomato, UberEATS, Uber, Deliveroo, Lyft, DoorDash and so on. You now have companies like these all over the world, in Indonesia, Kenya, Brazil, India, where we are now used to being able to just order things on an app and a person turns up. This is a millions-strong workforce globally, all very precarious and are essentially managed by algorithms, not by humans.
This can be very disempowering when you don't know why you have been given a job over somebody else, why are you being paid this wage, and there is no contract, no transparency in how they engage with their employer.
From around the world there are examples of human resilience and fighting back in gig work where unions are now playing a key role. These unions are local in different countries and different markets, but also they come together to meet and share data, knowledge and learnings, which really helps to empower them when they are faced with things like being kicked off an app for no reason, which may be done by an algorithm.
It is heartening to see that even in China where unions are illegal, there are informal workers’ committees that come together to exchange tips and tricks and advice, but to really fight back against what is a faceless system or a kind of corporate employer.
Companies are beginning to recognize this too. Here in the UK, for example, there was a Supreme Court ruling that said gig workers, Uber drivers and so on are workers, which means they're entitled to holiday, sick pay and other benefits, which makes them a lot less precarious. Once this happens in the UK, then spreads to Europe, Uber will have to start to behave more ethically, globally. I do see a movement towards that and not necessarily because these companies have suddenly realised their ethical responsibilities, but because of these activists, unions who are doing some very interesting work.
What about gender and AI? Do you see algorithms or apps reinforcing traditional gender roles and stereotypes, and how are women likely to be impacted?
There has been research on automation in the kinds of jobs that women tend to do more, such as caregiving roles, teaching, social work. These are job roles where AI is already being used to sort of replace workers, so I think that's an interesting body of work for me.
I discuss deep fakes at length in the book, something India is also very familiar with now. With generative AI being so much more sophisticated than the previous generation of tech, images and videos created through these models are really indistinguishable from reality.
People talk a lot about deepfakes in the context of politics, elections and misinformation, but really about 99% of deep fakes on the internet, according to research, is pornographic, and of that, 98% of the pornographic images being generated are of women. So it's clear here what the problem is.
Women already know what it's like to be female on the internet, to be harassed and bullied and targeted on social media through faceless trolls. We see this particularly when women are in the limelight, whether that's as activists, journalists, politicians; women with a voice seem to enrage the trolls—and it’s more of the same thing with deep fakes and generative AI.
What was shocking for me was the two stories that I spotlight, one from Sheffield in northern England, and one immigrant in Australia. They didn't have controversial views, one was a mother of a young child writing poetry about mountaineering; the other was an undergraduate lawyer at a university. But both had violent, graphic images of themselves on the Internet which were not real, which were deepfakes.
Deepfakes have a profound impact on the individual and on your identity, and though it abuses AI technology, there isn't any real regulation at a global level to protect women or people who are victims of this.
This is a place where yes, the technology can be weaponized by people. That’s always going to happen, but the regulatory failure is more shocking.
Do you connect the dots here with the fact that there just aren't enough women within the organisations that design these systems?
I do think the demographic make-up of the people who build the systems is particularly important in AI as compared to other industries. The reason I say that is because AI is a reflection of human society. It is trained using data from our lives, the things we write and generate, our behaviours, so it is a reflection of our society. That’s why all these biases crop up, because they kind of reflect the same historical issues in recruitment or criminal justice or healthcare.
It is not, however, just the data. It is also active decisions made by coders and engineers who decide on the trade-off—what is the output that we want from the system, and in conjunction with that, what are we minimising? It is the people who build these systems and the decisions they make and their ethics, morals and culture that actually becomes embedded in many ways and threaded into the systems and their output.
This is artificial intelligence, built by male coders. They see it as a sort of rational intelligence, or answers to questions. But there are so many types of intelligence, and emotional intelligence is another type which women can bring to these systems and which will make them so much richer. This is true of ethnic and geographic diversity too, because there is a big concern now that as AI systems output conversations, they have within them naturally embedded culture and ethics, and if it's going to be western systems that we're interacting with alone, the concern is we all start to go in towards one kind of belief system. We lose the nuances of all of the places that we come from.
What is the one thing that makes you feel most heartened by AI? What are the applications that you found to be the most promising? And will these impact the world's poorest and the most marginalised, and will they have equitable access to AI?
You can tell from my book that the most heartening example of application of AI is healthcare. I focused particularly on Dr Ashita, a doctor in a tiny village on the border of Maharashtra and Gujarat, mostly serving the tribal community of Bhils there. She’s helping to train and test an AI system that can diagnose tuberculosis.
That is just one example of how you can use an AI system to bridge access gaps in healthcare to reach people who you know have to travel hours to see a doctor. It is where we can genuinely plug some of the gaps that we have globally, such as in the NHS here in the UK where we have a huge shortage of radiologists.
You’re right that access will become the big question—who gets to avail of this? The hope is that it reaches people who need it the most, but the reality is often that this requires money to build and to deploy. If you end up having to pay for this then you are cutting out the same people who couldn't access human medical care in the first place. This is something that I try to bring up in the book.
I think that this is where the global south, countries who aren't necessarily building the technology from scratch, have a role to play where we find out what problems we have to solve in these countries. In India and the other countries that I visited in the Global South, people can have a voice and play a role in the embedding and implementation of AI.
(Kavitha Iyer is a senior editor with Article 14 and the author of ‘Landscapes of Loss’, a book on India’s farm crisis.)
Get exclusive access to new databases, expert analyses, weekly newsletters, book excerpts and new ideas on democracy, law and society in India. Subscribe to Article 14.