A Nature paper, published today (Wednesday August 4, 2021), proves that the algorithm used by the Sortition Foundation in the StratifySelect open-source software, to select participants for citizens' assemblies by lottery, is the fairest selection algorithm possible.
Designing a selection process that results in a representative sample in the fairest possible way — to the greatest extent mathematically possible — is the main problem solved in Fair Algorithms for Selecting Citizens’ Assemblies, a paper by Professor Ariel Procaccia of the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), along with Bailey Flanigan, Paul Gölz, and Professor Anupam Gupta from Carnegie Mellon University and Dr. Brett Hennig from the Sortition Foundation.
The paper examines a typical two-stage assembly selection process. In the first stage, thousands of randomly-selected citizens are invited to participate. Then, the final assembly is chosen from those who responded affirmatively using a selection algorithm, i.e., a set of steps that makes random choices about who to add to the assembly.
Procaccia summarizes the challenges surmounted in their paper as follows:
“First, we asked: how do we even think about fairness in the context of panel selection, and then how do we formalise it in a way that means that everyone gets a fair chance? Then, once we established this mathematical framework for thinking about the problem, we needed to develop an algorithm that gives everyone as fair a chance as possible to participate, while at the same time guaranteeing representation.” Ultimately, Procaccia says, “Our goal was to develop a fairer and more principled algorithm to put the selection of these assemblies on firmer mathematical foundations.”
The challenge in satisfying these two requirements at once is that giving all volunteers exactly equal probabilities is generally impossible to do while also satisfying demographic quotas. This is because the pool of volunteers tends to be unrepresentative of the population as a whole, due to certain groups, such as those with more education, being more likely to volunteer to participate.
The selection algorithm, therefore, seeks to find a panel that satisfies quotas while giving potential participants as equal a chance as possible of being selected. It does this by computing a distribution over many panels, all of which satisfy the quota requirements, in such a way that the minimum probability of any volunteer appearing in the set of panels is as high as mathematically possible (this is one of many possible notions of “as equal as possible”). Then one panel is drawn randomly from this distribution.
This algorithm has already been used to select more than 40 citizens’ assemblies around the world, by the Sortition Foundation (in the UK and Australia) and other organisations in countries including Denmark, Germany, the U.S., Belgium, often supported by the Sortition Foundation. An implementation of the algorithm is free and open-source and available on Git-Hub.
See also:
[This page borrows liberally from the Harvard University press release of Leah Burrows.]