Delhi: In a piece published in the Times of India on 23 November 2022, authors Sanjeev Sanyal and Akanksha Arora pointed out that while India was ranked 93 on the liberal democracy index published by Varieties of Democracy (V Dem), a Swedish institute hosted by the University of Gothenburg, the Kingdom of Lesotho, which “suffered a military coup in 2014 and repeatedly has been under a state of emergency” was ranked 60. 

Echoing concerns about international rankings expressed by Prime Minister Narendra Modi’s government,  Sanyal, the principal economic adviser to the government of India, and Arora, deputy director of the economic advisory council to the prime minister, questioned the methodology of international surveys on democracy and press freedom where India has been falling since the Bharatiya Janata Party (BJP) came to power in 2014, and whether they were free from bias and susceptible to influence. 

Over the last few years, the Modi government has, either directly or through editorial interventions like those made by officials like Sanyal and Arora, similarly challenged a fall in other rankings related to Indian democracy, press freedom, and human rights. 

Challenging the downgrading by V Dem, the Economist Intelligence Unit (EIU) and the US-based Freedom House in 2022, the authors said international think tanks used anonymous experts with little transparency on how they were chosen, how they arrived at their conclusions and how cross-country comparisons are made. They said the rankings were based on “superficial and often skewed use of media reports”.

In an interview with Article 14, Staffan Lindberg, the director of Sweden-based V Dem Institute and the principal investigator of V Dem, said Sanyal and Arora's accusations were not true: V Dem did not use media reports for the rankings, used more than 30 experts and a scientific approach that was transparent and involved high math. Their objective was purely academic, not political. They were not biased against India or any other country, and it was impossible to influence them. 

Classifying India as an “electoral autocracy” from 2019, V Dem said last year “most of the decline occurred following BJP’s victory and their promotion of a Hindu-nationalist agenda”, and this year that “anti-pluralist parties are driving the autocratisation in at least six of the top autocratisers: Brazil, Hungary, India, Poland, Serbia, and Turkey”.

The 2021 report, Autocratisation Turns Viral, said that “India’s autocratisation process has largely followed the typical patterns for countries in the ‘Third Wave’ over the past ten years: a gradual deterioration where freedom of the media, academia and civil society was curtailed first and to the greatest extent.” 

V Dem attributed the downgrading to an “electoral autocracy” to a decline in the autonomy of the Election Commission of India, freedom of expression, media and civil society, noting that “censorship efforts are becoming routine”, and the use of “sedition, defamation and counterterrorism laws to silence critics” including academics and the students and activists who protested against the Citizenship Amendment Act, 2019. 

The Modi government reacted strongly to the demotion by V Dem and the Freedom House, which downgraded India from “free” to “partly free”, within days of each other last year. A month earlier, India slipped two positions on EIU’s democracy index to 53 out of 167 countries and was called a “flawed democracy”.

Chairman of the Rajya Sabha Venkaiah Naidu forbade a lawmaker from the Aam Aadmi Party to ask a question regarding the V Dem report, saying, “These are all political. They should look inward. All countries which are commenting on India should first look inward and then comment on India.” 

The ministry of external affairs said, “The political judgments of Freedom House are as inaccurate and distorted as their maps”. 

Calling it “hypocrisy”, the minister of external affairs S Jaishankar said, “...you have a set of self-appointed custodians in the world, who find it very difficult to stomach that somebody in India is not looking for their approval, is not willing to play the game they want to be played.”

“So they invent their rules, their parameters, they pass their judgments and then make out as though this is some kind of global exercise,” he said. 

After India slipped two positions on EIU’s democracy index to 53 out of 167 countries and was called a “flawed democracy” in 2021, the Modi government reached out to them to know about their methodology and sample size but received no response, the Hindustan Times reported. 

The government used the lack of response from EIU to ask the Rajya Sabha secretariat to disallow a question by a lawmaker of the Trinamool Congress on the democracy index, saying, “it involves information on trivial matters and raises matters not under the control of bodies or persons not primarily responsible to the government of India”.

While dismissing findings by international think tanks, the Modi government has consistently made efforts to improve its image and challenge the results. 

In 2020 after the Press Freedom Index, published by Reporters Without Borders (RSF), downgraded India two places to 142 among 180 countries, the Modi government set up a 13-member committee called the “Index Monitoring Committee” with two journalists, neither of whom stayed the course. Veteran journalist P Sainath’s dissent note said, “A fair and honest ranking would see India plumbing the depths below 142.”

Frontline reported that the ambassador of India to France met with officials from RSF in 2020 and presented India’s perspective. 

The ambassador reportedly said that “the real challenge in Jammu and Kashmir was not of press freedom, but the influx of messages made through WhatsApp, Facebook, Twitter, etc,” and “many incidents reported as attacks on journalists are often a consequence of the law and order situation in some area of India” and “this is often misrepresented as targeted attacks on journalists by the state in western media”.

The Print in 2020 reported that the government was planning a publicity campaign to “shape India’s perception for the domestic and global audience and publicise the problems, parameters and data sources of global indices”. 

In a letter to Christian Dore, the editor of The Australian newspaper in 2021, the Indian high commissioner of Canberra defended India’s response to Covid-19 in response to an article which said Modi had led India from a lockdown into a Covid apocalypse. They ordered Twitter to take down 50 posts critical of the government’s response to the pandemic. 

To the BBC, Paul Staniland, associate professor of political science at the University of Chicago, who has studied V Dem’s examination of India since 1947, said India’s ranking was lower during the Emergency in the mid-seventies, but there was no major decline between 1998 and 2004 when the BJP was in power, and “so there’s not an obvious anti-right wing bias” and “there is a lot of reason to think they capture important big-picture dynamics and trends”.

V Dem says it conceptualises and measures democracy with over 30 million data points from 202 countries and 3,500 country experts, laying out its expert coding process and recruitment of experts in their methodology document. 

"It is typical that a government like India would push back when they fall in the rankings," said Lindberg, referring to similar criticism of their findings. "Before that, it was Turkey. Now, I think Erdogan doesn't really care anymore. He has imprisoned opposition journalists anyway, so nobody is there to write about it."

"But you know it, I know it," said Lindberg. "It is not only us and our data. It's all over. Any international ranking will say the same about India that we do." 

The article questions how Lesotho, which suffered a military coup in 2014 and has repeatedly been under a state of emergency, can have a better score than India. Could you respond to this?

I think that is cherry-picking and based on some misunderstanding. It is true that Lesotho’s political history with multi-party elections since 1993 has been checkered. But in the last few years, the democratic development in Lesotho has been very good. And you can score high on a democracy index without necessarily having a long history. Scores don't say anything about the history. The other way around, if Germany had suffered a coup that was revealed the other day, then from one year to another, their score would have gone down to almost zero. 

Lesotho has had many years since 1992 when democracy has been good. They have tried out different electoral systems.  As for kingdoms, Sweden and Britain are also kingdoms. You can be a constitutional kingdom, but the king has no power. You can have a presidential constitutional system like Finland, where the president has no power. 

Other countries have scored higher than India, not only Lesotho. So do Nepal and Indonesia—to mention a few neighbouring countries. India is approximately at the same level as Hungary, also classified as an electoral autocracy. But India has scored higher than the Philippines, substantially higher than Pakistan and very much higher than Bangladesh. So, picking only Lesotho for comparison is misleading. 

I take your point that a country does not need a long history of democracy to rank high, but Lesotho had a military coup in 2014. That is not a long way back in history. We’ve not had a military coup.

That was very short-lived. They held elections again in 2015 that were free and fair, and they chose a prime minister who held another parliamentary election in 2017. You can recoup democracy in three years, depending on what happens after the coup. There are many instances of short-lived coups or attempted coups that do not have an impact on the longer trajectory. The 2014 coup was not good, but that doesn't say that you can't be a democracy in 2021. You can. 

But what about it being repeatedly under a state of emergency since the coup, as they write? Has it?

I haven't gone into those details with Lesotho, but laws of the state of emergency come in many forms and are practised by democracies as well. For example, France still has some after the terrorist Charlie Hebdo attack. Israel has had them for decades. Britain had it with regard to Northern Ireland for decades. State emergency laws are of a very wide variety. I'm not saying they are good, but just because they are in effect for one reason or another does not necessarily mean you are not a democracy or your democracy is in peril. Unless we talk about what type of state of emergency law it is, what it covers, how long it has been in place and what was the reason for putting it in place, you need to be very, very specific if you are going to say whether it is a problem for democracy or not. 

Why is India doing worse than Lesotho? 

I don't think the comparison with Lesotho is right. You can say why it is doing worse than Bhutan, Taiwan, Nepal, all countries or as democratic as Lesotho or better. That is a regional comparison that makes more sense for people in India. Picking on Lesotho has a little bit of a racist slur—one of those poor African countries. How can they be better than us? I don't like that. 

Let's take it back home. India is clearly better than Bangladesh and Pakistan but not as good as Bhutan and Nepal. The scores for India have been declining sharply over the past eight years. It's one of the countries in the top ten worldwide that have declined the most. It used to be classified as a liberal democracy ten years ago. It's declined because the executive power has become much more unconstrained by the judiciary, that is, ruling in favour of the government and the legislature, which allows the executive to do things. India is also doing much worse on media freedoms.

Media outlets in India are intimidated. The government is seeking to censor. There is a lot of self-censorship. Editors and journalists are afraid of sanctions from the BJP government. Academic freedom is also getting worse. Academics are afraid of sanctions from the BJP government. There are many reports of academics who have been pushed out. And civil society is also being more constrained because of government actions. You have discriminatory acts towards Muslims—the laws of acquiring citizenship. So, it is a broad swathe of things that have worsened.

With the 2019 election, the autonomy of the electoral management body worsened. The BJP government has come to exercise more control over the elections. That is a very worrying sign because that is at the heart of being able to throw out the government through elections. It is many areas of democratic rights and freedoms that have been negatively affected that we see in the data.

The article says the think tanks use four to six experts, and there is no transparency regarding how they are chosen and arrive at their conclusion. 

V-Dem Institute is not a think tank but an academic institution, and we do not use  four to six experts. In the case of India, it is between 30 and 40 that have coded for us in the past ten years. We don't ask them to code how democratic India is. We ask them to code a series of indicators for media, civil society, political parties, judiciary, and civil liberties—altogether, twelve areas. For each area, there should be a minimum of five experts. If someone is an expert on parties and legislatures in India, they are not necessarily an expert on civil society and media. 

We look for the academics who know the most and have published scientific articles on elections in India or the judiciary in India. They have verified scientific knowledge about this specific area for this specific country. Most experts are from the country they code, still active as academics, but there are also outside observers who come and do field work, etcetera. Or from a neighbouring country studying India. When you do expert surveys, you are not looking for an opinion. You are looking for knowledge. They have sound scientific knowledge about the area. 

What about transparency? 

We have a public protocol. It is a lie that our methods and processes are not transparent. It just means they have not read our methodology documentation that is available on the website. It specifies exactly how the process of recruiting experts is conducted and the criteria they should be matched against so that we can certify they are good enough, preferably the best-qualified people. 

We don't disclose the identity of the country expert. We have worked with a total of over 3,700 experts from 180 countries. We are bound by both European and Swedish laws and well-established research ethics to not disclose their identity because this could cause them and their families harm. An example of this is what is happening in India. In India today, you could be sanctioned if the government gets to know that you are coding for V-Dem. Another example is Turkey. When we started this in 2010, it wasn't dangerous to be a coder for V-Dem or speak up for democracy in Turkey. Today it is.

You could end up in prison. We can never predict which country will have a military coup tomorrow, and it becomes dangerous to be associated with us. Therefore, for ethical and legal reasons—in Sweden, where our office is—we are not allowed to disclose it by law because we have to protect them and their families and their relatives against potential sanctions. 

But the protocol for how we recruit them is completely transparent. And also how we aggregate them up. It is a complicated statistic because it is the gold standard latest, Bayesian Item Response Theory-modelling. We use supercomputers. It is almost like asking astronomers how you do your calculations. The code for this is also open for anyone to see, but it is complicated. If you have half a year, we can teach you about it. We are not astronomers, but we are the astronomers of social science. The whole code of how we do this in R and C++ is publicly available. Everything is open if somebody has access to supercomputers and wants to rerun these calculations to achieve cross-country comparability. But it is difficult to understand the math for non-experts, naturally.

The article says think tanks findings are based on a superficial and often skewed use of media reports. 

We use no media reports for our ranking. We have 450 indicators that are then aggregated into democracy indices. About half of those indicators are factual—how many political parties do you have, what does the electoral law say, what electoral system do you use, do you have a prime minister, what powers does the prime minister have by the law, what media freedoms are there by the law. The other half are these indicators of how much media freedom you have de facto, how much censorship there is, and how much corruption there is. Here the use of the scientific knowledge of specialised country experts is accepted worldwide as the gold-standard methodology. But we do not use media reports for the rankings. We refer to some media reports in our democracy report when we write a narrative about the rankings to provide some context around the data. 

Do you get pushback from other countries? 

Naturally. We typically get pushback from countries that are falling in the rankings. They don't like it. Our role as academics is to speak truth to power. We are a bunch of professors who have created V-Dem. We have tenure. We have no political interests when it comes to individual countries. We think democracy, democratic freedom and human rights are good. They are enshrined in the universal declaration of human rights, and we stand behind that. We think it is important to measure democracy properly and in a detailed, nuanced way so that when governments infringe on rights and freedoms, there is data to show they are doing it no matter what they say. We don't care what governments say because we are not in the business of politics, we are in the business of science.

It is typical that a government like India would push back when they fall in the rankings. Before that, it was Turkey. Now, I think Erdogan doesn't really care anymore. He has imprisoned opposition journalists anyway, so nobody is there to write about it. We have got pushback from Hungary. To some extent from Bolsonaro in Brazil. And some anti-democratic portions of Republicans don't like our ratings. But you know it, I know it. It is not only us and our data. It's all over. Any international ranking will say the same about India as we do. 

Although public, it is difficult to follow the methodology. Could you explain?

It is high math. It is the math that has been developed to get as true an estimate as possible. The IRT models, which are at the heart of this, were developed for educational testing and have been used over and over again. GRE scores, for example. You take 100 math questions and give them to 10,000 students, and you figure out which question, on average, is harder to answer than another question. Then they order the questions.  When doing a GRE you go in at mid-level. If you get the first two questions right, you get a harder question, but if you fail a question, you get an easier one, and so on. It can then figure out your mathematical ability compared to other students. It was developed to figure out how hard the questions are. We turned that around, and we tried to figure out how hard you are as a coder.

One question we have among our indicators is that if a journalist writes critically about the government, how likely is it that the journalist will be harassed? Zero means all the time; one means most of the time, two means about half of the time, and so on. That is the kind of question country experts get to answer.

But  also experts have different personalities that impact their assessments even if they have the same amount of information about a country. One is very strict and requires a lot of evidence to change from “zero” to “one”. Another expert is more easygoing, has a more optimistic view, and rates it as a “one”. They have the same information but different thresholds for which score they choose. If you go back to the methodology document, it is called differential item functioning. So if we didn’t adjust for this difference, countries would not be scored correctly. The IRT model calculates this mathematically, by comparing coders across time and between countries, figures out that the optimistic expert’s “one” is equal to the strict expert’s “zero” and can adjust that within countries and between countries. The “astronomer math” behind the Bayesian IRT models has been worked out over many years, so that country scores between countries become comparable. That is why we need supercomputers because this takes lots of calculations. Every variable is calculated 10-80,000 times. It is not taking simple means of experts’ scores. We are light years from that. 

Is there anything else you would like to say? 

In government responses, there have been allegations of connections to the US government. We have none. It is a purely academic project. Our core data collection funding comes from Sweden's national research council and the university here. Other professors only govern that research council. It is completely devoid of political and government ties. 

Get exclusive access to new databases, expert analyses, weekly newsletters, book excerpts and new ideas on democracy, law and society in India. Subscribe to Article 14.

(Betwa Sharma is managing editor of Article 14).