Much has been said about the terrifying models that in the spring projected such a staggering number of deaths from the novel coronavirus. 

In hindsight, as bad as the pandemic has been, it never even approached the dismal numbers suggested ‒ the very numbers that rationalized society-wide lockdowns in Italy, the U.K., New York City, and then in many other places as the pandemic spread.

What researchers have struggled with since then is how to measure the impact of various actions taken. Do we even know if what we’re doing is working? Where’s the evidence for that, and are there other things we ought to do instead?

Naturally, proponents of lockdowns have long said that strong government action prevented all kinds of horrors. If anything, the poor outcomes we had in the spring and the fall indicated that we didn’t do enough. Skeptics, on the other hand, said that lockdowns did nothing but harm our societies ‒ physically, economically, and mentally ‒ and that infection rate curves moved the way they did regardless of what strong-worded politicians implemented, and often before their strong policies took effect. The August NBER paper by Andrew Atkeson, Karen Kopecky and Tao Zha, ‘Four Stylized Facts about COVID-19’ spells out the uncomfortable position for most policy-makers: the virus seems to spread rapidly, kill selectively, and in no way responds to anything that well-meaning politicians have thrown at it. 

The general corona debate quickly became a battle of pointing to this or that country: Lockdowners picked Australia and New Zealand; skeptics picked Sweden and Taiwan. The angry feuds in political arenas and editorial pages were off to the races. Death rates in Sweden far outstripped those of its neighboring countries, a topic on which we already in August tried to bring some clarity. To an American and British audience who couldn’t tell Bergen from Ystad, or slurred Danish from Finnish diphthongs, higher death rates and weaker restrictions were conclusive evidence that Sweden’s slightly-more-open strategy had failed. Never mind that the Nordic countries may differ in other respects. One-variable statistical analysis at its worst while practically no one compared Sweden to the much worse-performing UK, Belgium or France.

Maybe countries greatly differed from one another in ways that would make such naive comparisons completely misleading: demographics, population densities, the size of the Covid-shock, the effect of government advice, the soft cultural values of how real people interact and how they responded to the pandemic. Besides, all these countries introduced so many new policies and behavioral changes that even those of us who tried to make sense of them quickly lost track. 

What we needed was an experiment, where all of those background differences were controlled for. Ideally, a jurisdiction with similar conditions operating on similar rules; where some of their areas locked down hard, while their neighboring counties, identical in every other way, did not. In a new article, one of us together with another co-author, did exactly that. The article, “Lockdown Effects on Sars-CoV-2 Transmission – The evidence from Northern Jutland,” by Kasper Planeta Kepp and Christian Bjørnskov is now available on MedRxiv. 

In late summer, a new mutation of the Sars-CoV-2 virus was discovered among mink farms in Denmark. That information suddenly became important in Danish debate in October, when researchers from the Danish Serum Institute warned against the mutation, and politicians demanded action. On November 4, the prime minister announced that in the Danish region of Northern Jutland, seven municipalities were to enter into extreme lockdown, enacting the usual battery of work-from-home, closing of commercial and leisure activities and closed public transport. Scattered among them, all in the same region of Northern Jutland, were four municipalities that didn’t; they remained under the then-fairly moderate rules in the rest of Denmark. In total 280,000 people and 126,000 jobs were affected by the extreme lockdown, as people were banned from crossing municipal borders to go to work.

Here was a golden opportunity to measure the infection impacts of very strict lockdowns. By comparing otherwise very similar municipalities ‒ language, culture, administrative region, geography ‒ the Danish professors could avoid the problems with identifying cause and effect that hampered cross-country observations. In addition, the lockdown of seven municipalities was not justified on different case numbers or spread of the virus, but only on a worry about a new mutation that subsequently proved to be unfounded.

Prior to the heavier lockdowns in our seven municipalities, there was no detectable difference between the two Northern Jutland groups. In the seven days before the lockdowns, the strict group had 0.15 positive tests per thousand inhabitants per day compared to 0.14 in the open group. In the spring, too, when far fewer people were tested, the former group experienced a total of 0.69 positive tests per thousand inhabitants while the open group saw 0.82 positive tests (all differences statistically insignificant). 

Treating the two groups as stand-alone units, Planeta Keep and Bjørnskov write that

“[W]e find no statistically significant differences between the two groups of municipalities prior to intervention. The strong similarity in infection rates at different timescales before the intervention strongly supports treating the lockdown as an actual quasi-natural experiment.”

In no statistical specification that the researchers run does the lockdown variable ‒ shifted by 4, 7, or 10 days to allow for an uncertain incubation period of the virus ‒ pass conventional significance tests for its impact on the number of infections. The only thing that seems to be driving positive tests in the North-Danish municipalities are the infections in previous days and weeks. 

As seen in Figure 1 of the paper, the number of Covid infections in the two groups was already falling before the onset of the heavy restrictions in the lockdown municipalities ‒ and it keeps falling just the same in both groups. In non-statistical terms: looking at identical counties, with as natural as natural experiments come, the researchers cannot detect any impact from lockdowns. Lockdowns don’t stop, slow down, or seem to affect the future spread of the disease in any way. 

Figure 1

What’s remarkable is that the study includes a big enough population to detect that change. It has similar test-and-control groups with hundreds of infections in each. There was a big push for mass testing in both groups, and so virtually no chance that testers did not detect a meaningful number of infections. The professors reflect on the study and describe it as

“[T]he most time- and space-focused empirical dataset available with sufficient statistical power, adequate and homogeneous control group, nearly complete testing, and with the smallest possible confounder pollution imaginable in a real setting.”

In great contrast to the terrifying projections from imagined models, this study showed real outcomes with real people going about their real pandemic lives. It could very well be that lockdowns work in some settings, in some jurisdictions, and under some conditions. But in a setting with voluntary compliance, high trust in government and lots of general information available to the citizen, such as across Denmark (and other Nordic and Northern European countries), lockdowns don’t seem to have added anything to prevent the spread. 

Whether this result is unique to a predominantly rural part of Denmark, or whether it translates to lockdown as a preventive policy more broadly remains to be seen. Because it has “lack[ed] actual empirical control cases for the same populations,” the scientific community has not been able to tease out what works and what doesn’t. However, a number of recent studies trying to get around different problems in different ways also conclude that lockdowns don’t work. 

What the new study from Northern Jutland shows is that an extreme form of lockdown didn’t work in one of the most law-abiding societies in the world. Why, then, should we expect lockdowns to be effective anywhere else?

Figure A2: Increase in infection in December