Did COVID-19 make it harder to get leave to the Supreme Court?

As many will know, Lenczner Slaght maintains an ongoing database of every application for leave to appeal to the Supreme Court of Canada that was decided from January 1, 2018 onward. We use this data to train a machine learning algorithm to predict the likelihood of cases getting leave, which we use for various purposes.

But value of the data goes beyond just predicting the probabilities of cases getting leave. This dataset can also help us answer questions more generally about what is going on at the Supreme Court. In this post, we provide our analysis of a question that many in the legal community have commented on and speculated about: did the COVID-19 pandemic make it harder to get leave to appeal to the Supreme Court of Canada?

To back up, there is no question that over the past decade, there has been a steady downward trend in both the raw number of cases that get leave to appeal as well as the percentage of successful leave applications. As the Supreme Court of Canada’s 2020 Year in Review shows, in the early 2010s, it was common for 10-13% of leave applications to be successful, representing over 50 cases per year. By contrast, in the last few years, the rate has dropped to 6-9%, representing no more than 42 cases per year. There has clearly been a downward trend in the number of cases to which the Supreme Court of Canada grants leave to appeal.

This downward trend raises concerns for the ongoing development of the law. The fewer leave applications that are granted, the fewer opportunities for the Supreme Court to make decisions or advance the law.

In this regard, some have raised concerns that COVID-19 would further restrict the work of the Court. Certainly, the number of cases argued and decisions being issued by the Court has decreased recently. Some have speculated that the COVID-19 pandemic might also be reducing the number of cases being granted leave as well. There are several plausible theories for why this could occur. One could surmise, for example, that there might be fewer cases in the pipeline that represent good cases for the Court to take; the theory here would be that the work of courts overall have slowed down during COVID-19, so there were fewer Court of Appeal decisions from which parties would decide to seek leave. Alternatively, one might speculate that the Supreme Court might have implicitly changed its standard for granting leave: perhaps the global pandemic has caused them to be choosier as to which cases are granted leave.

Our review and analysis of the data provides three concrete insights about leave applications to the Supreme Court during the pandemic.

1. The number of leave applications has dropped during the COVID-19 pandemic.

The Supreme Court of Canada’s 2020 Year in Review, which counts cases (with some exclusions) based on the dates the leave applications were filed, shows an approximately 12% drop in leave applications in 2020 compared to 2019. Our data tells a similar story when we look at the number of leave applications decisions: there was an approximately 17% drop in the number of leave applications decided in 2020 compared to 2019. This suggests a drop in leave applications being made during the pandemic.

2. There is no material change in the percentage of cases getting leave during the pandemic.

The Supreme Court of Canada’s 2020 Year in Review (which again, does not count certain cases) indicates that 7% of cases filed in 2019 were granted leave, compared with 6% of cases that were filed in 2020. This represents a very minor drop. By contrast, an analysis of our dataset, based on the year in which the leave applications were decided rather than filed, suggests a minor trend in the opposite direction. Our data shows that just under 6% of leave applications decided in 2019 were granted leave, while just over 7% of leave applications decided in 2020 were granted leave. In either case, the overall change from 2019 to 2020 is very small.

3. There is no evidence that it became either harder or easier to get leave to appeal to the Supreme Court of Canada during the COVID-19 pandemic.

This is perhaps the most interesting part of the analysis: our data allows us to test whether the COVID-19 pandemic has changed the likelihood of getting leave in any given case.

While the raw numbers and percentages tell part of the story, they don’t provide the full picture. After all, it may be that in some years, there are fewer interesting and important legal issues percolating through Courts of Appeal that are ripe for successful leave applications; this could result in fewer cases getting leave, even with the Supreme Court applying a consistent standard. Conversely, if there is an abnormal flood of interesting and important cases from which parties are seeking leave, yet leave rates remain the same, that could indicate that the Supreme Court has become choosier about which cases get leave.

So how do we test for this? In a nutshell, we used a statistical model that allows us to test whether cases decided during the COVID-19 pandemic have a different overall leave rate, while controlling for a number of factors that are correlated with a higher or lower likelihood of getting leave. (In technical terms, we ran two multivariate logistic regression models, using similar variables to the ones used in our weekly machine learning prediction model, as well as an additional dummy variable that reflects whether leave application was decided during the COVID-19 pandemic or not.)

Running this model allowed us to determine whether, accounting for other factors that influenced the likelihood of getting leave, the COVID-19 pandemic by itself made it harder or easier to get leave. Put differently, the question this allowed us to explore is whether the same case would have been less likely to get leave during the COVID-19 pandemic than before.

Deciding what cases might have been affected by the COVID-19 pandemic and which wouldn’t have been requires some judgment, so we ran the model across two different datasets. In our view, neither of these models is the right or wrong way to analyze the problem. Rather, they are different ways of analyzing the same issue.

In the first model, we assumed that every case in our dataset decided before March 1, 2020 was not affected by the COVID-19 pandemic, while every case decided after was. 

For the second model, we excluded all leave applications that were decided between March 1 and August 31, 2020. The theory for doing so is that if a case was already in the Court’s pipeline before COVID-19 but was decided during the pandemic, it’s unclear whether this would be a pre-pandemic case or a pandemic case. Because we know that the median time from leave application to decision is approximately six months, we simply removed from the dataset all cases that fall into the middle gray area. All cases decided before March 1, 2020 were coded as not affected by the COVID-19 pandemic, while every case decided after August 31, 2020 was coded as being affected by the pandemic.

The results? In both models, after controlling for other relevant factors, cases decided during the COVID-19 pandemic were neither statistically significantly more nor less likely to get leave than cases decided before the pandemic. Put simply, our analysis finds no support for the contention that the Supreme Court became either stingier or more liberal in granting leave during the COVID-19 pandemic.

In the end, while the total number of leave applications appears to be somewhat down due to the COVID-19 pandemic, we aren’t seeing a significant difference in either the percentage of cases getting leave or the likelihood of individual cases getting leave. Despite the global pandemic, when it comes to deciding leave applications, it seems to be business as usual at the Supreme Court of Canada.