Marcello Di Bello and Collin O’Neill: “Profile Evidence, Fairness, and the Risks of Mistaken Convictions”. Précis by David Wasserman

32 Replies to “Marcello Di Bello and Collin O’Neill: “Profile Evidence, Fairness, and the Risks of Mistaken Convictions”. Précis by David Wasserman”

1. We are very grateful to PEA Soup for organizing this discussion, and especially to David Wasserman for his careful and lucid summary of our article and for the thoughtful challenges he raises. Since his summary is impeccable, we will try to respond to his challenges.

   —–
   The first challenge is about the source of the intuitive appeal of our account. The examples of profile evidence that we use to motivate our account (Ethnicity, Prior Burglary, and Bad Environment) are about groups that are stigmatized or disadvantaged. But we claim that the intuitive resistance to using profile evidence extends to other groups that are not subject to discrimination or otherwise disadvantaged, and even to groups that are not even socially salient. Warrior Gene is our example of this sort of profile evidence. David seems to agree that there is resistance to using Warrior Gene evidence and perhaps profile evidence about relatively advantaged groups, such as the groups that are more likely to commit insider trading, tax evasion, embezzlement, etc. But he suspects that this resistance arises only because the group is already stigmatized under another description or, interestingly, because the use of this evidence might itself stigmatize the group.

   If David is correct, why might this be a problem for us? One potential problem is that our account might be thought to overgeneralize since admitting profile evidence about groups that are not stigmatized or disadvantaged would infringe the right to equal protection as we have described it. (It’s not so clear that admitting profile evidence about advantaged groups would infringe their right to equal protection if they were antecedently at a lower risk of mistaken conviction because of their access to higher quality legal representation, etc., so we’ll set those groups aside.)

   We have a couple of replies. First, we think that resistance to profile evidence and stigmatization can come apart. Imagine that the Warrior Gene can only be tested for when someone becomes a defendant so that no one outside the courtroom can know whether they or anyone else has the gene. Imagine as well that information about the correlation between this gene and certain crimes never leaves the courtroom, so that even if people did know who had the gene, they wouldn’t know it had any link to crime. In that case, it is hard to see how the group could be stigmatized or even socially salient, yet there is still, we believe, intuitive resistance to the use of this evidence. David suggests that this group might still be socially salient and stigmatized under some other description, because of the aggressive behavior of many of its members. But it’s not obvious to us what that other description could be.

   Secondly, even if we’re wrong about this, we don’t think it would be a serious problem for our account (not that David is claiming that it is). It would be a serious problem for us if our account entailed that the use of routinely admitted evidence like eyewitness testimony, trace evidence, or motive evidence wronged defendants. But even if there is not a strong intuition that using profile evidence against members of a non-stigmatized group is objectionable, there is also not a strong intuition that using it against such a group is perfectly okay and that any account that entails otherwise should be rejected.

   —–
   David’s second challenge is to the use we make of the distinction between the risk of mistaken incrimination and the risk of mistaken conviction. Even though admitting eyewitness testimony might expose some defendants (e.g., those residing in high crime neighborhoods) to a higher risk of mistaken incrimination by this form of evidence, we argue that it would not expose them to a higher risk of mistaken conviction so long as the factfinders responded rationally and assigned testimony that incriminates these defendants less probative value. If this is right, then admitting profile evidence will infringe defendants’ right to equal protection even if factfinders respond rationally while admitting eyewitness testimony will not.

   David’s concern here is that while this response by factfinders may prevent people from high crime neighborhoods from being exposed to a higher risk of mistaken conviction after they have become defendants, they will still be exposed to a higher risk of becoming defendants in the first place. So, if eyewitness testimony is admitted, won’t this expose people from high crime neighborhoods who might become but are not yet defendants to a higher risk of mistaken conviction?

   We think it will not, so long as everyone responds rationally. Prosecutors will decide to make someone a defendant only if they expect the strength of the case against that person to exceed a certain threshold. Although the cases against people who live in high crime neighborhoods may be more likely to contain mistaken eyewitness identifications against them, prosecutors should not expect these cases to be stronger nor the factfinders more likely to convict if they also expect the factfinders to assess the probative value of this evidence appropriately.

   Another way to frame the concern might be that since the police will decide whether to arrest someone in part on the basis of eyewitness identifications, innocent people from high crime neighborhoods will be more likely to be arrested, and this, in turn, will make them more likely to become defendants and mistakenly convicted. However, if we assume that when police decide whether there is enough evidence for an arrest they will assess the probative value of that evidence correctly, then the police should be no more likely to arrest innocent people from high crime neighborhoods, other things equal.

   So given our idealization that the factfinders and police will assess the probative value of eyewitness testimony correctly, we think that the use of such evidence should not expose innocent people from high crime neighborhoods to a higher risk of mistaken conviction. Moreover, even if admitting this evidence would expose people from these neighborhoods to a higher risk of mistaken conviction by exposing them to a higher risk of becoming defendants, this would not infringe their rights to equal protection. This is a right that belongs to defendants–it is a right against being exposed to a higher risk of mistaken conviction than other defendants. If the concern here is that our account might overgeneralize and condemn the admission of eyewitness testimony, it is worth pointing out that admitting this testimony will not infringe the equal protection rights of people who reside in high crime neighborhoods if the only reason they are at a higher risk of mistaken conviction is that they are at a higher risk of becoming defendants. This point might prompt another question, namely, do people not also (or even only) possess a right against being exposed to a higher risk of mistaken conviction as people, not just as defendants? We have a few tentative thoughts about that, happy to discuss further.

   —–
   The third concern that David raises is about our idealization. He notes that “it is cold comfort to the former defendants to know they would not face an increased risk of mistaken conviction from ideally rational factfinders.” We make this idealization because we want to show that even if profile evidence were not prejudicial, its use would still be objectionable just because of the kind of evidence it is. It would be a serious problem with our account if it implied that defendants’ rights to equal protection would only be infringed if they were at a higher risk of mistaken conviction because factfinders were expected to respond rationally to the evidence, but not if they were at a higher risk because factfinders were expected to respond irrationally to the evidence. However, the right to equal protection is a right against lawmakers making decisions about the rules of evidence and other procedural rules that expose defendants to a higher risk of mistaken conviction from the lawmakers’ point of view. If lawmakers have reason to believe that the factfinders will not assess the probative value of eyewitness testimony against defendants from high crime neighborhoods correctly, then admitting such testimony could expose those defendants to a higher epistemic risk of mistaken conviction from the lawmakers’ point of view, and their rights to equal protection will be infringed. It’s worth pointing out that this isn’t necessarily a reason for lawmakers to ban this evidence–it would be better if they took steps to ensure that factfinders had the information they needed to assess the probative value correctly, and were disposed to do so. But unless this were done, the decision to admit this evidence would infringe some defendants’ rights to equal protection, in light of the fact that lawmakers had reason to believe factfinders would not respond appropriately to it.

   These are our replies to David’s challenges, as we’ve understood them. No doubt there is more to say. His precis concludes with some interesting questions about the scope of our account, which we will address next.

2. I am thrilled to be jumping into this discussion of Marcello Di Bello and Collin O’Neil’s terrific article. Like David, I think the article raises a novel argument and argues for it persuasively. Congratulations to you both.
   That said, I do have two worries that are related to those surfaced by David Wasserman. Perhaps my comments just build on his comments and extend the discussion.
   1. Is profile evidence really different from relying on eye-witness testimony? i.e. the uniqueness claim. Di Bello and and O’Neil distinguish eye witness evidence from profile evidence in part on the ground that the group of people picked out by the profile will be knowable in advance but the group of people picked out by an eye-witness description are only known after the eye witness provides a description. This is a somewhat artificial distinction and so one is drawn to wonder about its moral underpinnings. What I think animates this distinction (feel free to correct me if I have this wrong) is the idea that everyone would have a equal chance, ex ante, of fitting the eye witness description while everyone does not have an equal chance of fitting the profile. If that is right, I have two worries.
   First: perhaps everyone doesn’t have an equal chance of matching an eye-witness description because biases in perception make eye-witnesses more likely to report that a perpetrator was black than that he was white, for example? Second, why privilege a particular point in time for making the equal chances assessment? Ex ex ante – i.e. before the bad environments, bad genes, prior acts occur, we might say that we all have equal chances of matching those profiles just as we all have equal chances of fitting the description of an eye-witness. And just before trial, we don’t have an equal chance of being in the group described by either the profile or the eye witness. I am having trouble feeling the intuitive appeal of the particular ex ante spot identified by Di Bello and O’Neil.

   2. If equal protection really is morally relevant, why isn’t it tied to actual (rather than idealized) likelihood of mistaken conviction? I’m having trouble seeing the moral pull of a hypothetical equal chance of mistaken conviction, if actors were all rational.

   Thanks again for a thought-provoking article.

3. Thanks to David for his helpful summary and thoughts. And thanks to Marcello and Collin for their interesting and engaging essay.

   I had similar worries to those Deborah and David have articulated. A quick note, first, in response to Marcello and Collin’s comment above:

   > This point might prompt another question, namely, do people not also (or even only) possess a right against being exposed to a higher risk of mistaken conviction as people, not just as defendants? We have a few tentative thoughts about that, happy to discuss further.

   Evidence law can also affect who becomes a defendant: It can affect who is arrest and charged and, significantly, who decides to plea bargain rather than face trial. I gather most (98%?) of convictions in the US are obtained through plea bargain, rather than through trial. So if an [innocent] accused person is deciding whether to face trial or enter a plea, their risks of [mistaken] conviction can affect their decision. And if profiling evidence is allowed, it could incentivise plea bargaining. If profiling evidence were sufficiently inculpatory, it could almost impel charged individuals to plea bargaining.

   Perhaps, though, this consideration is best captured as an interest – they are a stakeholder – but not an actual right?

   Do those who plea bargain qualify as defendants on your view, and so have the relevant right?

   (I have more to add about the topic of the “ideally rationally fact finder” too. But I will post this first, since I have to head out again.)

   Thanks again!
   Georgi

4. This is a fantastic article, and I’m glad it’s been chosen for a discussion at PEA Soup. I have a worry related to one raised by Deborah Hellman, and about which I’ve corresponded recently with Marcello and Collin. (So, Marcello and Collin – don’t feel obliged to respond here; I just thought others might be interested in considering it.)

   But first let me say what I really like about the article:
   First, it doesn’t just address whether statistical evidence (in whatever form) can on its own suffice for conviction. This is the problem of ‘bare’ or ‘naked’ statistical evidence that’s been addressed extensively in the literature. Many theorists draw an analogy between cases of bare statistical evidence and the lottery paradox and conclude that, just as high probability given the evidence doesn’t suffice for knowledge or rational full belief, so it doesn’t suffice for conviction. And this motivates an interpretation of standards of proof (e.g., beyond reasonable doubt) which is based on knowledge or rational full belief rather than mere high probability.

   But this approach doesn’t entail that statistical evidence cannot combine with non-statistical evidence, in cases where neither would be sufficient on its own, to license conviction. It doesn’t condemn the admissibility of statistical evidence. (Note that there’s no strong intuition that, in a lottery case, you can’t know that ticket N is a loser on the basis of both the strong statistical evidence an some piece of non-statistical evidence, where the latter wouldn’t on its own yield knowledge or rational full belief.)

   Second, they target profile evidence, rather than statistical evidence in general. And I think some forms of statistical evidence (like profile evidence) are more intuitively problematic than some others (like DNA evidence in cold-hit cases).

   Okay, now here’s the worry. It’s related to what Deborah Hellman says here:
   “Second, why privilege a particular point in time for making the equal chances assessment? Ex ex ante – i.e. before the bad environments, bad genes, prior acts occur, we might say that we all have equal chances of matching those profiles just as we all have equal chances of fitting the description of an eye-witness. And just before trial, we don’t have an equal chance of being in the group described by either the profile or the eye witness. I am having trouble feeling the intuitive appeal of the particular ex ante spot identified by Di Bello and O’Neil.”

   I also have a worry about the proper informational perspective which is relevant for the equal chances assessment (i.e. which is relevant for determining violations of equal protection). My worry isn’t, though, about whether that the relevant perspective is one where everyone has an equal chance of matching the known profiles (e.g., an equal chance of being a member of a given race, an equal chance of coming from a bad environment, etc). Let’s grant that the hypothetical lawmakers (those deciding whether to make profile evidence admissible) know all that. My worry is whether the relevant perspective is one where the lawmakers know what profile evidence obtains, or not. That is, we can distinguish at least 3 possible informational perspectives:

   Ex post: The lawmakers know which individuals will face which charges and which evidence, and they also know what profile evidence obtains.

   Partial ex ante: The lawmakers know what profile evidence obtains, but they do not know which individuals will face which charges, nor do they know what non-profile evidence will obtain.

   Fully ex ante: The lawmakers do not know what profile evidence obtains, nor do they know which individuals will face which charges or what non-profile evidence will obtain.

   Marcello and Collin rely on what I call the ‘Partial ex ante’ perspective, and they need that for their argument. For if we apply equal protection relative to the ‘Ex post’ perspective, then all forms of evidence violate that right. And if we apply it relative to the ‘Fully ex ante’ perspective, then profile evidence doesn’t violate that right. (After all, if the lawmakers don’t know what profile evidence obtains, then even conditional on its being admissible, no identifiable individual has a higher risk of mistaken conviction than any other.)

   But I think that the ‘Fully ex ante’ perspective is the one relevant for the right to equal protection. This is because what profile evidence obtains will substantially depend on what legal system is established by the hypothetical lawmakers. What profile evidence obtains is therefore ‘endogenous’ with respect to the choice that the lawmakers are facing. This is because: (i) changes to the legal system affect incentives, which affect behaviour, and (ii) changes to evidence law affect convictions and arrests, on which profile evidence is often based.

   And I think that, in these hypothetical choice cases, the decision-makers should be blinded to facts whose truth depends on what decision they wind up making. (By analogy, the decision-makers in the Rawlsian original position shouldn’t be allowed to know facts about our rates of income inequality are, since rates of income inequality depend on what basic structure those decision-makers institute.)

   Now, one might think that what profile evidence obtains is unlikely to be substantially affected by the decision whether or not to make profile evidence admissible. But this is a tricky empirical question. And it also raises an issue of whether our hypothetical lawmakers are allowed to consider not only whether or not to make profile evidence admissible, but also to consider more large-scale changes to the legal system, or indeed to the basic structure of society. The more stuff they’re allowed to fiddle with, the greater the extent to which what profile evidence obtains will depend on what decision they make (and hence not knowable in advance).

   Anyway, there are various responses I think Marcello and Collin might make, but I’ll leave it here and see what others think.

5. As I intimated above, my main question is similar to that pursued by David above. I copy-and-paste five extracts, to orient people on the topic, and then I articulate my question:

   > Finally, the authors argue that no other form of evidence, statistical or otherwise, poses the same threat to equal protection, even if that evidence is more likely to be brought to bear against some innocent defendants than others. From the ex-ante perspective from which equal protection must be assessed, no group of innocent defendants faces a greater risk of mistaken conviction by a rational fact finder – as opposed to mistaken incrimination — from having, e.g., a common physical appearance or DNA profile similar to the perpetrator’s. Such individuals, who may be identifiable in advance, are more likely than others to be mistaken for the perpetrator of a crime. But they do not face an increased risk of mistaken conviction by a rational fact finder – the “rational” is critical — who will discount eyewitness or trace evidence by the increased likelihood of a false positive. [Wasserman, emphasis added]

   > It is cold comfort to the former defendants to know that they would not face an increased risk of mistaken conviction from ideally rationally fact finders. [Wasserman, emphasis added]

   > Even though admitting eyewitness testimony might expose some defendants (e.g., those residing in high crime neighborhoods) to a higher risk of mistaken incrimination by this form of evidence, we argue that it would not expose them to a higher risk of mistaken conviction so long as the factfinders responded rationally and assigned testimony that incriminates these defendants less probative value. [Di Bello and O’Neil, emphasis added]

   > We will argue that, although admitting eyewitness testimony and other kinds of evidence might well give rise to different risks of mistaken incrimination, this should not give rise to different risks of mistaken conviction if the probative value of the evidence is assessed correctly. (Di Bello and O’Neil, page 172, emphasis added)

   > In order to adjust their assessment of the probative value of the evidence and prevent inequalities in the risks of mistaken conviction from arising, fact finders will need to be informed about the risk of mistaken incrimination by that evidence. Equal protection, then, requires admitting various statistical generalizations at trial: error rates of laboratories that doDNAtesting, statistics about the reliability of eyewitnesses in different circumstances, and so on. If the fact finders are not given this information (and they often are not), admitting eyewitness testimony and other forms of evidence could violate equal protection. ((Di Bello and O’Neil, page 175)

   Consider a particular kind of evidence that seems like admissible trace evidence (or eyewitness testimony), but this evidence nonetheless means that some people face elevated risk of false conviction. Suppose that a large active criminal group know how to synthesise ginger hair, for example, and they leave ginger hair at many of their crime scenes. This misleading trace evidence suggests that the perpetrators have ginger hair. Ginger people now face an elevated risk of mistaken conviction from this trace evidence. (Perhaps the criminal group aim to frame a particular ginger family, or perhaps just incriminate ginger people in the city, to keep suspicion away from themselves.) The criminal investigators and scientists in the city have not yet discovered any method for synthesising hair, so it hasn’t occurred to them that the ginger hair could be fake.

   So, my question is, should this evidence be inadmissible? My question aims to probe what it means to “respond rationally to the evidence” / “correctly assess the probative value of the evidence”. The factfinders (broadly construed to include the police and forensic scientists) don’t yet know that the hair could be faked. So are they failing to be ideal, on this conception?

   > “as the factfinders responded rationally” / “if the probative value of the evidence is assessed correctly”.
   My question interrogates how ‘externalist’ the relevant norms/idealisation is. Here are three options from most internalist to most externalist. (a) The factfinders reason well with the evidence presented to them, whatever that is. (b) The factfinders are informed about the existing available evidence about the probativeness of the evidence, such as the low reliability of stranger eyewitness identification, especially cross-race, as according to current science. (c) This idealisation includes knowing that evidence is (for example) easy to manufacture to frame members of a particular group, even if the authorities haven’t yet figured this out. For the probative value to be assessed correctly/for the factfinders to respond rationally, this sometimes requires them to know things outwith current forensic science.

   Here is why the question might matter to Marcello and Collin’s view. On the one hand: Suppose the ginger hair should be admitted because the relevant norms concern what is known/available about the probative value of the evidence, and the possibility of this chicanery hasn’t been discovered by forensic scientists. Then perhaps generally evidence doesn’t violate equal protection, so long as scientists never figure out that it exposes certain groups to elevated risks of mistaken conviction. On the other hand: Suppose the ginger hair should not be admitted, because it is in fact (albeit unbeknownst to forensic science) exposing risk of mistaken conviction on particular groups. Then what evidence should be admitted depends on facts that aren’t yet known. It isn’t a matter of the factfinders being rational; instead it is whether they have all the relevant knowledge. That seems notably externalist for evidence law.

6. We wanted to respond to some of the questions that David Wasserman asked about our account at the end of his precis.

   —-
   (1) What implications does our account have for factfinding in other contexts, legal and non-legal, and for other types of problematic statistical evidence? Does the right to equal protection extend, suitably modified, to civil trials, to other legal and administrative proceedings, or to informal judgments and beliefs?

   We have just begun to give this some thought, and at this point, we’re not sure. We say a little about policing towards the end of our article. We do not address sentencing there, but we do have some tentative thoughts about sentencing that we can offer. Inequality in the risks of suffering some burden is most plausibly inconsistent with equality before the law when the burden is clearly an injustice. Obviously convicting (and punishing) the innocent is intrinsically unjust. We cannot, however, say whether a certain punishment for a guilty offender is intrinsically unjust without a theory of punishment. Still, for purposes of sentencing, judges sometimes rely on algorithms that assign offenders a score that indicates how likely they are to offend in the future, and our account might have some implications for the use of profile evidence in arriving at these “dangerous assessments.”

   One potential criticism of the use of dangerousness assessments is independent of the fact that they are based in part on profile evidence. On the assumption that only harmfulness and culpability are relevant to the seriousness of a crime, allowing assessments of future dangerousness to influence sentencing determinations will lead to violations of relative proportionality (a requirement that offenders who commit comparably serious crimes be given comparably severe punishments, and offenders who commit less serious crimes be punished less severely than those who committed more serious crimes–this formulation comes from Antony Duff) between individuals in sentencing.

   When we factor in that these dangerousness assessments are based on profile evidence, we can see that there will also be violations of what we might label relative proportionality between groups. Some offenders, such as offenders that are unmarried, that dropped out of high school, have family members with records, and so on, will be at a higher risk of being assessed as dangerous than others. Thus, among offenders who committed comparably serious crimes, those who have family members with records may be at higher risk than those who do not of receiving a sentence longer than the average sentence for a crime of a given level of seriousness. Even someone who is untroubled by violations of relative proportionality between individual offenders may be concerned about violations of relative proportionality between certain groups. Score adjustments might help with this problem. Of course if all of the inequalities between groups were eliminated by score adjustments, the algorithm would be rendered useless. However, these adjustments might be limited to the most worrying inequalities, e.g., those that disadvantage groups already suffering from disadvantage along other dimensions.

   But there is another application of an approach like ours here. Note that these dangerousness assessments can be mistaken, in the sense that someone who is not dangerous in some objective sense (e.g., who is highly unlikely, in some objective sense, to re-offend) can be mistakenly assessed as dangerous. Because these assessments rely on profile evidence, the risk of this error will be higher for some groups than others. For example, objectively non-dangerous offenders who have family members with records will be at higher risk of being mistakenly classified as dangerous than non-dangerous offenders that do not have family members with records. We would not claim that assessing someone who is not dangerous as dangerous is, like a mistaken conviction, intrinsically unjust. Nor, without presupposing a theory of just punishment, can we claim that a sentence that is lengthened on the basis of a mistaken dangerousness assessment is thereby rendered unjust. But we do think it is an especially troubling consequence of the reliance on profile evidence that non-dangerous offenders from some groups will receive longer sentences on average than non-dangerous offenders from other groups that have committed comparably serious crimes. Even assuming future dangerousness is relevant to sentencing, non-dangerous offenders that are, say, unmarried are not different in any relevant way from non-dangerous offenders that are married and that committed comparably serious crimes. Is it fair to expose these offenders to a higher risk than others of receiving a sentence longer than the average sentence for their crime? In any case, there may be ways to extend an account like ours to contexts like sentencing, but the extension is far from automatic.

   We think our account of equal protection might also have applications to larger debates about algorithmic fairness — see e.g. Deborah Hellman, Measuring Algorithmic Fairness, forthcoming in the Virginia Law Review — but we leave this for another discussion.

   —-
   (2) Is it an advantage of competing accounts of profile evidence that they apply to guilty as well as innocent defendants?

   That’s a good question. It’s true that competing accounts like the autonomy/individuality account apply to both guilty and innocent defendants, and ours does not. We lean towards thinking that this is not an advantage, for two reasons.

   One reason is just that whether this is an advantage will depend on whether guilty defendants are wronged by the use of profile evidence against them, and we are not confident that they are. Still, if they do have a complaint, our account could easily accommodate this by assigning guilty defendants a right against being exposed to higher risks of correct conviction than other guilty defendants.

   The second reason is that, although we are uncertain whether guilty defendants have a complaint, we are confident that if they do have a complaint, it is weaker than the complaint of innocent defendants. Our account could explain this difference by noting that innocent defendants’ right against higher risks of mistaken conviction is much stronger than guilty defendants’ distinct right against higher risks of correct conviction. Although the autonomy/individuality account could assign guilty defendants a complaint against profile evidence, it’s unclear that it could explain why innocent defendants seem to have a stronger complaint, since it assigns both innocent and guilty defendants exactly the same complaint against the use of profile evidence, namely, that its use is inconsistent with respect for their autonomy or individuality.

   —-
   Another question posed by David is whether it is desirable that an account that explains the resistance to profile evidence also explains the resistance to other kinds of statistical evidence, like that based on the kind of accidental association found in the gatecrasher case. We will try to provide an answer to this question in later posts.

   We are very thankful for your questions, David, and we hope our answers have shed some additional light on our account.

7. Hello Georgi. Thanks for drawing attention to larger issues in criminal justice, e.g. plea bargaining. Whether our account has something to say about these large issues is an important issue. We have some tentative thoughts about plea bargaining in relation to our equal risk account.

   You’re right that a surprisingly high percentage of convictions are now obtained through plea bargains. This is really something we ought to have addressed in the article, since one might wonder whether there is any value in worrying about trial procedure anymore, at least in the U.S. But as you point out, since trial procedures like the standard of proof, evidence law, etc., will influence the chances that an innocent defendant would lose if they went to trial, these procedures will also influence innocent defendants’ decisions about accepting a plea bargain or instead going to trial. Since admitting profile evidence will expose innocent defendants who fit incriminating profiles to a higher risk of mistaken conviction at trial, they will also be exposed to a higher risk of mistaken conviction based on a plea bargain. (In addition, in light of their higher risk of mistaken conviction were they to go to trial, one would also expect the outcome of any plea bargains they reach to be less attractive, in light of their weaker bargaining position.) Also, as you suggest, we do think that people in a plea bargaining situation count as defendants, and that the admissibility of profile evidence would infringe their right to equal protection, even if they never go to trial and profile evidence is never actually presented against them.

   And, just for clarity’s sake, we’ve just noticed you posted another question. We will address your latest question in a separate post.

8. Hello Deborah, many thanks for joining us, and for these perceptive comments.

   —-
   In response to your first question, we believe that one important difference between eyewitness testimony and profile evidence is that no type of innocent defendant is categorically immune from eyewitness misidentification (except the invisible!), whereas innocent defendants that do not fit an incriminating profile cannot be mistakenly incriminated by profile evidence. This fact about profile evidence lends itself to an argument that admitting it at trial would expose some types of defendants to a higher risk of mistaken conviction than others. Since the same thing is not true of eyewitness testimony, the same argument does not apply.

   Still, as you point out, perhaps a different argument could be made–even if no one is immune from eyewitness misidentification, it is plausible that some types of innocent defendants may be at higher risk of being misidentified than others, because of biases in perception. In response we claim that if factfinders lower the probative value of these identifications, in light of the higher risk of misidentification, admitting this evidence will not expose these types of defendants to a higher risk of mistaken conviction.

   Another worry you have has to do with our choice of the “ex ante” point of view. Brian Hedden has a similar worry in his post, so we will address this issue in a separate post.

   —-
   Your second question is why isn’t equal protection defined in terms of actual rather than idealized risks? We see a couple of ways of interpreting this question (but please let us know if we are misunderstanding your point here).

   For one thing, you might be asking, why are we making an idealization about rational fact-finders? Shouldn’t we consider how fact-finders actually operate and the extent to which defendants are actually subject to the risk of mistaken conviction? One possible answer to this question is that we make an idealization because we want to show that even if profile evidence were not prejudicial, its use would still be objectionable just because of the kind of evidence it is. We give a more extended answer to this sort of question when we answer David Wasserman’s third question above. We also just noticed that Georgi Gardiner has raised a similar worry about idealization. We will come back to the idealization point in another post.

   But here is another way of understanding your question. That is, why are risks of mistaken convictions understood as epistemic or evidence-relative risks and not as actual risks? If by “actual” we should mean “subjective,” our reason for not framing the right to equal protection in terms of subjective risks is that if lawmakers ignored or shielded themselves from information about the consequences of their decisions for innocent defendants, then even the most egregious decisions would not infringe equal protection so defined. Objective risks do not seem to be suitable either. Instead, we are inclined to think that equal protection is best understood in terms of epistemic or evidence-relative risks.

9. Hello Deborah and Brian. You both pose some related yet different questions about the “ex ante” point of view. Because of the overlap, we thought it’d be a good idea to address your concerns together in one post.

   Deborah points out that, from what she calls the “ex ex ante” point of view, everybody has an equal chance of fitting a certain incriminating profile. Whether there is inequality or not in the distribution of the risks of mistaken convictions will depend on how far upstream we locate that ex ante point of view. If we locate the ex ante point of view extremely far upstream– “ex ex ante”–then lawmakers’ decisions to admit profile evidence like Prior Burglary and Bad Environment would not expose anyone to a higher risk of mistaken conviction. Although only certain people fall under these profiles now, from an earlier point of view, anyone could have fallen under these profiles. If lawmakers take this “ex ex ante” point of view, no defendant would be at a predictably higher risk of mistaken conviction due to admitting profile evidence, and the argument in the paper would then collapse. This is a serious worry. Our argument needs an informationally richer point of view, what we call “ex ante”, in which lawmakers know about certain general facts, for example, that some people fit certain profiles correlated with crime and others don’t. But why take the “ex ante” point of view and not the “ex ex ante” point of view in assessing the risks of mistaken convictions for defendants?

   As a general principle, we think that risk assessment should be able, in some important way, to guide actions and decisions. This idea can be found, among others, in Oberdiak’s 2017 book “Imposing Risk: A Normative Framework” (Oxford UP). When one assesses risks as a guide to actions and decisions, one should take into account as much information as reasonably available. We believe that the “ex ante” point of view, as opposed to the “ex ex ante” point of view, captures as much information as reasonably available to lawmakers for making decisions about the design and maintenance of trial procedures. Assessing risks from the “ex ex ante” point of view would require pretending that lawmakers lack the information they actually have or should have, and this seems inappropriate.

   Deborah might ask, why not try to capture even more information — i.e. information about who was charged, for what crime, and on the basis of what evidence? The problem with taking this informationally richer point of view is that this would give no additional information that is either useful or reasonably obtainable for the purpose of guiding the decisions of lawmakers about the design of general principles of trial procedures. Knowing the details of what evidence is found at the crime scene or what witnesses were at a particular crime scene cannot enter into the design of trial procedures. Nor can this information be obtained as one is designing trial procedures. Patterns about how the evidence is found or collected would be relevant or obtainable, but not individual facts about how this or that evidence was found and against whom.

   An important addendum to this is that lawmakers can always revise the rules of trial procedure. Even if it would not be practicable or desirable for them to revise the rules on a case-by-case basis, the “ex ante” point of view must always be updated as new information comes in.

   Brian is also worried about the “ex ante” point of view and its role in our argument. What Brian calls the “fully ex ante” point of view — similar but different from what Deborah calls “ex ex ante” — is devoid of information about what profile evidence obtains. From that point of view, no defendant is at a predictably higher risk of mistaken conviction due to profile evidence, simply because lawmakers lack any knowledge about what profiles obtain and who are the people fitting the relevant profiles. As they assess risks of mistaken convictions for defendants, why should lawmakers take the “ex ante” point of view as we have defined it and not the “fully ex ante” point of view as Brian suggests?

   We think the “fully ex ante” would deprive the lawmakers of too much information. That would be information that could be useful for their risk assessment and also that they could reasonably obtain. It’s not clear why they should block it. It is not clear to us that lawmakers should make decisions about the design of the trial procedures behind a veil of ignorance.

   Brian points out that lawmakers’ decisions will affect what profile evidence becomes available and also what crime patterns emerge in society. This seems correct. We are willing to concede that lawmakers’ decisions might have a huge impact on crime patterns. After all, lawmakers decide what actions count as crimes in the first place and what behaviors count as criminal behaviors. And it is lawmakers and their definition of what counts as a crime that might affect research about crime correlates and profile evidence. We acknowledge there is a series of feedback effects and complex interactions here between lawmakers’ decisions and people’s behaviors. But the fact that lawmakers’ decisions might affect people’s behaviors is not a clear reason for lawmakers to discount information about crime patterns when they assess risk.

   We think it’d be interesting for the discussion if Brian could elaborate more on why he thinks that the fact that lawmakers’ decisions affect patters about crime and behavior suggests that lawmakers should not consider crime patterns when they make decisions about the trial system even if they could reasonably know that information. This question is prompted by a more general question about the information that policymakers should rely on when they design policies. Why should policymakers be blind to information that they could reasonably have access to on the grounds that the information is, in some important way, dependent on their own choices or perhaps the choices of other policymakers before them?

10. Thanks Marcello and Collin for the response. Let me just make three pointsL
    (1) Deborah’s question about the ‘ex ex ante’ perspective and not knowing who falls under what profile made me think of an objection which, while perhaps silly, might also get to something important. Suppose that some lawmakers decided to make profile evidence admissible beginning in 100 years’ time, when everyone who currently exists will be dead. While this would be a bad policy for all kinds of reasons, it doesn’t seem like it would violate the right to equal protection. That’s because they can’t refer de re to anyone who will be affected by the decision, and so in particular there are no individuals A and B such that (relative to the lawmakers’ evidence) A will have a higher probability of mistaken conviction than B if they make profile evidence admissible starting 100 years from now.

    (2) Marcello and Collin write, “An important addendum to this is that lawmakers can always revise the rules of trial procedure. Even if it would not be practicable or desirable for them to revise the rules on a case-by-case basis, the “ex ante” point of view must always be updated as new information comes in.” That may be true for the US system. But imagine we are starting a new society, and at the constitutional convention we write into the constitution that any and all profile evidence will be admissible. Being constitutionalized, it’s essentially unrevisable (perhaps changing it would require some supermajority, or a revolution). Now, encoding this decision in the constitution would probably be a bad idea, but would it violate anyone’s right to equal protection? (We can also imagine that, given that we’re starting a new society, we don’t yet know what profile evidence will obtain.)

    (3) Now to the central issue relating to my earlier worry: Marcello and Collin write, “We think it’d be interesting for the discussion if Brian could elaborate more on why he thinks that the fact that lawmakers’ decisions affect patters about crime and behavior suggests that lawmakers should not consider crime patterns when they make decisions about the trial system even if they could reasonably know that information.”

    I don’t have an objection to allowing the lawmakers access to knowledge about crime patterns up to the point of their decision. After all, that evidence about the past is fixed and doesn’t depend on what decision they make now. Suppose they’re meeting now, in January 2020, to decide on whether to make profile evidence admissible. And suppose that we (and they) know that, up to Dec 2019, members of group X have been N times more likely to engage in crime than members of group Y. Relative to this informational state, it isn’t clear that making profile evidence admissible beginning in Feb 2020 would violate anyone’s right to equal protection. For they don’t know that, if they make profile evidence admissible, members of group X will still be any more likely (let alone N times more likely) to engage in crime than members of group Y. For their decision to make profile evidence admissible might well change the behavior of the X-people and the Y-people.

    And the admissibility of the evidence that UP UNTIL Dec 2019 the members of group X were more likely than members of group Y to engage in crime won’t necessarily mean that a future (innocent) defendant from group X will be more likely to be falsely convicted than a future (innocent) defendant from group Y. After all, a reasonable factfinder would have to seriously discount the probative value of that ‘up until Dec 2019’ evidence, given that there was recently a major change in the legal system which likely affected the behavior of the X-people and the Y-people. The fact that the X-people had higher crime rates than the Y-people up until Dec 2019 doesn’t mean that some X-defendant in July 2020 was more likely to have engaged in crime than some Y-defendant (setting aside all non-profile evidence), since the big legal change which happened in Feb 2020 means a reasonable factfinder couldn’t expect the trend up until Dec 2019 to continue into 2020.

    Of course, as I mentioned, lots of statistical generalizations are likely to be robust with respect to relatively minor changes in the legal system (like the admissibility or inadmissibility of profile evidence). I assume males commit violent crimes at higher rates than females. And I assume that this has deep roots (in biology, psychology, culture, etc) and would be difficult to change. But this might only mean that the right to equal protection would be violated by making robust profile evidence (like that concerning male vs. female crime) admissible, and not that it would be violated by making less robust profile evidence (e.g., perhaps regarding crime rates among immigrants vs. the native-born) admissible, since the latter trends might well not continue to hold after a significant change to the legal system. Hope that makes sense!

11. I wanted to chime in about two things!

    The first is a worry about how the authors characterize equal protection. In their original response to Wasserman, they say: “the right to equal protection is a right against lawmakers making decisions about the rules of evidence and other procedural rules that expose defendants to a higher risk of mistaken conviction from the lawmakers’ point of view.”

    Saying that citizens have a right to equal protection only “relative to the lawmakers’ point of view” strikes me as problematic. Suppose that rational lawmakers could not have reasonably foreseen that rules of evidence would be biased but they turn out in the real world to function that way. As a result, some people, perhaps even members of vulnerable groups, face a higher risk of mistaken conviction. According to the authors’ definition of equal protection, defendants’ right to equal protection has not been violated. It has not been violated because rational lawmakers are not omniscient and couldn’t reasonably have been expected to know in advance that the rules would have these unfortunate effects. The fact would remain that some citizens have a higher risk of mistaken conviction under the law. However, these folks could not object to the rules on grounds that they violate equal protection…because equal protection has not been violated, as characterized by the authors. This strikes as a disturbing result. As a citizen, I want equal protection under the law, period. Why aren’t we entitled to that?

    I think this worry is intertwined with a bunch of the above worries (mentioned by Hellman, Wasserman, and others) about the authors’ heavy reliance on the “ex ante” perspective and the normative weight they give to it. But it has a slightly different twist. Not even the most rational lawmakers can predict the future (including future technologies, social dynamics, and psychological research). If so, the unforeseen/unforeseeable creates a gap in the right to equal protection. I would prefer to have a right to equal protection under the law, full stop. Am I missing something? Is having such an unconditional right impossible, according to the authors?

    Second, I want to ask for a clarification. In their response to Wasserman, the authors say: “If lawmakers have reason to believe that the fact finders will not assess the probative value of eyewitness testimony against defendants from high crime neighborhoods correctly, then admitting such testimony could expose those defendants to a higher epistemic risk of mistaken conviction from the lawmakers’ point of view, and their rights to equal protection will be infringed.”

    Am I reading this right? It sounds like a major concession (and deviation from the argument in the published paper) is being made, but then later I doubted it. Could you clarify? I would welcome the concession for the record. I don’t think it obviates all worries about the ex ante perspective. But it helps to make the argument more persuasive and consistent, in my view. If rational lawmakers know in advance that eyewitness testimony will be less reliable in cases of cross-race identification (as the authors suppose), they will also know that juries could harbor racial and other biases that make it hard for them to properly evaluate evidence. The same kind of sociological and psychological facts are needed to make both predictions. If human biases threaten justice in both cases, why wouldn’t lawmakers adopt rules that better protect vulnerable individuals from the relevant risks in both cases? The paper seems to say they won’t (174). However, the concession seems to revise the position in a more plausible direction.

12. Great discussion on a fascinating article. So glad to be a part of it. And thanks for the shout out to my paper:

    “We think our account of equal protection might also have applications to larger debates about algorithmic fairness — see e.g. Deborah Hellman, Measuring Algorithmic Fairness, forthcoming in the Virginia Law Review — but we leave this for another discussion.”

    I think it will as well. As that piece is not yet out, I will add a citation to your piece if the journal will let me at this point. I’m traveling today but will chime in again if I can. If not, thanks to everyone for a great discussion and to Marcello and Collin especially for a really great article! Glad to meet you, albeit virtually. Debbie

13. I’m delighted but somewhat daunted by the vigor and pace of the exchange. I’ll try to re-enter the unfolding discussion later; for now, I just want to address one of the authors’ initial responses that hasn’t yet been taken up: their hypothetical case of profile evidence generated in and never leaving the courtroom. They offer this case to argue that even in this contained environment, in which invidious stereotypes would not have the time or space to develop, profile evidence would still be objectionable. I had thought to offer a slightly less fanciful case in the precis but ran out of space: a newly discovered genetic profile admitted for the first (and perhaps only) time in court against a defendant. I saw this case as cutting AGAINST profile evidence, as suggesting that absent negative social salience, there would be only the slightest discomfort with admitting the profile evidence against this defendant – and that this residual resistance would be due to autonomy concerns – the defendant should be regarded as free to resist genetically-based psychological predispositions. I no longer believe (if I ever did) that it would be wrong to admit such evidence on that basis, just that it would explain the residual discomfort better than a concern for equal protection. Back this afternoon.

14. We want to respond to Georgi’s concern about idealization, a concern that was voiced by David and Deborah and that touches upon a worry about our notion of equal protection that Erin has also voiced in a more recent post today. But, to avoid confusion and even though there is some overlap between different comments, in this post we will focus on Georgi’s comments about idealization.

    To begin with, Georgi writes:

    “Consider a particular kind of evidence that seems like admissible trace evidence (or eyewitness testimony), but this evidence nonetheless means that some people face elevated risk of false conviction. Suppose that a large active criminal group know how to synthesise ginger hair, for example, and they leave ginger hair at many of their crime scenes. This misleading trace evidence suggests that the perpetrators have ginger hair. Ginger people now face an elevated risk of mistaken conviction from this trace evidence.”

    We feel the intuitive pull for concluding that people who have ginger hair are at higher risk of mistaken conviction than those with non-ginger hair, but closer scrutiny, we think, shows that this is not entirely true (and Georgi herself seems open to this later in her comments at least as a possible reading of our account). To make a determination about risk, we need to consider the ex-ante, evidence-relative point of view of lawmakers. Is it true, from their point of view, that ginger hair people are at higher risks of mistaken convictions? Presumably, lawmakers do not know, nor cannot be reasonably expected to know, that there is a fabrication of ginger hair evidence. From their point of view — which is the point of view from which risk should be assessed — there is no heightened risk of mistaken conviction.

    This might feel an unsatisfying result. Aren’t people with ginger hair at higher risk of mistaken conviction in some intuitive sense? They are, but relatively to a much more informed point of view that lawmakers typically don’t have. This raises similar issues to what was discussed in our answer to Brian and Deborah about the “ex ante” point of view and why this point of view and not another — say a very upstream point of view, i.e. “ex ex ante” or “fully ex ante”, or a more downstream point of view, i.e. what Brian calls “ex post” — is relevant for risk assessment and equal protection. We think that assuming that lawmakers have information about people with ginger hair being the target of misleading trace evidence in specific locations is akin to assuming that lawmakers have information about what eyewitnesses would show at particular crime scenes. That seems too much information for lawmakers to have access to, even from their reasonably informed point of view.

    Georgi puts a different spin on this line of questioning. She writes:

    “So, my question is, should this evidence be inadmissible? My question aims to probe what it means to “respond rationally to the evidence” / “correctly assess the probative value of the evidence”. The factfinders (broadly construed to include the police and forensic scientists) don’t yet know that the hair could be faked. So are they failing to be ideal, on this conception?”

    We do not think that lawmakers would fail to be reasonably informed if they did not know about the widespread fabrication of ginger hair evidence unless this was a well-know recurrent fact or careful research would have revealed that fact. But if the faking of ginger hair is a secret, known to virtually no one, we don’t think lawmakers cannot be faulted for not knowing that. We appeal to the notion of “reasonableness” here — a notoriously vague notion — and we are afraid we cannot do much better than that at this point. What the boundaries of “reasonableness” ultimately are, that requires more thought.

    Georgi raises a further point that cuts across large debates in epistemology:

    “My question interrogates how ‘externalist’ the relevant norms/idealisation is. Here are three options from most internalist to most externalist. (a) The factfinders reason well with the evidence presented to them, whatever that is. (b) The factfinders are informed about the existing available evidence about the probativeness of the evidence, such as the low reliability of stranger eyewitness identification, especially cross-race, as according to current science. (c) This idealisation includes knowing that evidence is (for example) easy to manufacture to frame members of a particular group, even if the authorities haven’t yet figured this out. For the probative value to be assessed correctly/for the factfinders to respond rationally, this sometimes requires them to know things outwith current forensic science.”

    This question prompts a distinction we do not make in the paper as clearly as we should have. There seem to be two idealization requirements in our paper. First, we require fact finders to be rational in the sense of considering all the evidence they have available according to the best practice of correct evidence assessment, say probability theory. If they are given statistics about the rate of mistaken identification for people in certain neighborhoods or cross-racial identification, they should use that information appropriately to weigh eyewitness identifications. We also assume fact finders to be free from biases. There is, however, no requirement on the fact-finders to collect evidence or information they are not explicitly given at trial unless this is common sense shared by anyone. They should simply assess rationally what they are given. That’s the idealization we impose on the fact finders.

    Second, we require lawmakers to be reasonably informed from a suitably defined ex ante point of view and from that point of view lawmakers can make determinations about risks of mistaken convictions for some people compared to others. Like fact finders, lawmakers are assumed to be rational. They are assumed to know about probability theory to assess risks coherently. Unlike fact finders, though, lawmakers are also assumed to be pretty diligent in collecting as much information as they reasonably can. Failure to collect reasonably available information or failure to assess risks rationally based on that information would be failures to live up to the idealization standards we are setting up for lawmakers.

    Using Georgi’s distinction, we think (b) should probably be the level of idealization we are aiming for, keeping the distinction between lawmakers and fact finders in mind. The idealization in (c) seems simply too demanding.

    We conclude with Georgi’s interesting dilemma:

    “On the one hand: Suppose the ginger hair should be admitted because the relevant norms concern what is known/available about the probative value of the evidence, and the possibility of this chicanery hasn’t been discovered by forensic scientists. Then perhaps generally evidence doesn’t violate equal protection, so long as scientists never figure out that it exposes certain groups to elevated risks of mistaken conviction. On the other hand: Suppose the ginger hair should not be admitted, because it is in fact (albeit unbeknownst to forensic science) exposing risk of mistaken conviction on particular groups. Then what evidence should be admitted depends on facts that aren’t yet known. It isn’t a matter of the factfinders being rational; instead, it is whether they have all the relevant knowledge. That seems notably externalist for evidence law.”

    We think ginger hair evidence should be admitted so long as, from the ex ante point of view of reasonably informed lawmakers, it does not subject defendants with ginger hair to higher risks of mistaken conviction. So we are inclined to go with the first option. We agree that the second option Georgi outlines is “notably externalist for evidence law” and thus problematic as an interpretation of our account. In this interesting case of the framed ginger people, although the lawmakers might have reason to believe that this is a (remote) possibility, they have no reason to think it is any more likely that gingers would be framed in this way than that non-gingers would be framed in this way (e.g., people with brown hair, or blond hair, or black hair, or people with other features). So the right to equal protection is not infringed.

    One additional thought. Suppose that for whatever reason only gingers are capable of being framed like this, the other hair colors can’t be synthesized. If the lawmakers know or should know this, then the admission of trace evidence consisting of hair could infringe equal protection. But there is, at least in principle, a fix available that does not require prohibiting the introduction of hair evidence. Ginger hair evidence could be still introduced as evidence, but its probative value as trace evidence should be discounted as a function of the heightened riks of mistaken incriminating by this evidence for certain people. Still, the fact that a fix is available in principle — and this is part of our idealization — does not mean that there is no infringement of equal protection. Equal protection will be infringed unless that fix is actually made and it works (or is reasonably believed by the lawmakers to work).

15. Hello Erin, thanks for chiming in and these important questions. Let’s start with your first worry:

    “Suppose that rational lawmakers could not have reasonably foreseen that rules of evidence would be biased but they turn out in the real world to function that way. As a result, some people, perhaps even members of vulnerable groups, face a higher risk of mistaken conviction. According to the authors’ definition of equal protection, defendants’ right to equal protection has not been violated. It has not been violated because rational lawmakers are not omniscient and couldn’t reasonably have been expected to know in advance that the rules would have these unfortunate effects. The fact would remain that some citizens have a higher risk of mistaken conviction under the law. However, these folks could not object to the rules on grounds that they violate equal protection…because equal protection has not been violated, as characterized by the authors. This strikes as a disturbing result.”

    We agree that it would be a serious problem for our account if equal protection would not be infringed in the situation you describe, where the lawmakers could not have foreseen those unequal effects when they made the rules, but now those effects have become apparent. However, when lawmakers are making the rules, they are not making those rules for all time. Maintaining those rules (i.e., not revising them) is also a decision that must be evaluated in light of equal protection. So when they originally made the rules, let’s say, this decision did not violate equal protection since those unequal effects were not reasonably foreseeable. But now that those effects have become apparent, a decision to maintain those rules in the light of the new information would violate equal protection. Lawmakers are always making decisions whether to revise or maintain the rules from an ex ante point of view, but that ex ante point of view will become better informed over time.

    Your question seems to underscore a possible difference between two informational points of view — that of lawmakers who have information that indicate no heightened risks for some people versus others, and the point of view of those who are assessing the justice system — perhaps legal scholars or activists — who have information that indicate an uneven distribution of risks in the system. Since lawmakers should have access to the best evidence and information reasonably available, we think that the case you describe is one in which lawmakers have failed to be reasonably informed. They should correct their informational deficiency and then take measures to address the risk inequalities in mistaken convictions. We concede however that, from some prior point in time, the lawmakers might not have had access to information that would indicate inequality in risk distribution in the justice system. So, from that older, less informed point of view, there would be no violation of equal protection. But lawmakers have a duty to keep themselves up to date and thus as soon it becomes clear, given new data and information, that there is a risk inequality in the system, equal protection would be violated and that violation should be addressed.

    Now the second question:

    “If rational lawmakers know in advance that eyewitness testimony will be less reliable in cases of cross-race identification (as the authors suppose), they will also know that juries could harbor racial and other biases that make it hard for them to properly evaluate evidence. The same kind of sociological and psychological facts are needed to make both predictions. If human biases threaten justice in both cases, why wouldn’t lawmakers adopt rules that better protect vulnerable individuals from the relevant risks in both cases? The paper seems to say they won’t (174).”

    We think lawmakers might be confronted with some difficult choices here. Should they strive to ensure that fact-finders are rational? In theory, this should be possible. Perhaps with sufficient training and guidance, or perhaps by selecting fact-finders on the basis of their ability to weigh the evidence properly, this ideal rational fact-finder can be realized. That would be one way. And this might be what transpired in parts of the paper in print. The other option is to simply give up on the hope of rational fact-finders and set up certain procedural safeguards that would protect vulnerable populations from biases. These two strategies are somewhat competing. What Erin feels as a concession on our part might be our realization that there are in fact two options here about how lawmakers should respond to the problem of biases and uneven distribution of the risk of mistaken convictions. In short, if the lawmakers have reason to believe that fact-finders will respond irrationally to evidence in a way that disadvantages a certain type of defendant, unless the lawmakers take steps to fix this problem (ban the evidence, improve the fact-finders, etc.), they will have infringed the equal protection rights of those defendants. We don’t think this is a deviation from the paper, but this is something we should have made clearer.

    There is a broader point here, and it is about the role of idealization in our account. We idealize the fact-finders only to show that the problem with profile evidence cannot be fixed by improving the fact-finders, whereas any equal protection problem with other forms of evidence can (at least in principle) be fixed by improving the fact-finders. In other words, we are using the principle of equal protection as a tool to differentiate profile evidence and other forms of evidence. Our claim is that even if fact-finders are fully rational, there would still be an uneven distribution of the risk of mistaken conviction targeting the profiled defendants. So the profile with a profile cannot be easily dismissed even under idealization. Instead, if the fact-finder were fully rational, then eyewitness would not pose a problem. This, we believe, is an important difference, at least theoretically. We are willing to concede, however, that in the real world the problem posed by biases in eyewitness testimony might be just as pressing as the problem posed by profile evidence — and it might be just as practically inescapable given fact-finders entrenched biases.

16. Thanks, Brian, for developing your earlier thoughts. That’s very helpful. Let’s get straight to your main point. You write:

    “I don’t have an objection to allowing the lawmakers access to knowledge about crime patterns up to the point of their decision. After all, that evidence about the past is fixed and doesn’t depend on what decision they make now. Suppose they’re meeting now, in January 2020, to decide on whether to make profile evidence admissible. And suppose that we (and they) know that, up to Dec 2019, members of group X have been N times more likely to engage in crime than members of group Y. Relative to this informational state, it isn’t clear that making profile evidence admissible beginning in Feb 2020 would violate anyone’s right to equal protection. For they don’t know that, if they make profile evidence admissible, members of group X will still be any more likely (let alone N times more likely) to engage in crime than members of group Y. For their decision to make profile evidence admissible might well change the behavior of the X-people and the Y-people.”

    This is interesting and helpful as a clarification of what you have in mind. Your thought seems to be that decisions by lawmakers will affect correlations between crimes and profiles. If up to a certain point in time, it was true that X people are more likely to do Y, this might no longer be the case moving forward given changes in the legal system. Let’s suppose that’s true. What this shows is that profile evidence about the correlation between X and Y can be no longer be trusted as good reliable evidence that is probative of guilt. So long as decisions by lawmakers have an impact on crime correlations, this seems to undermine the probative value of profile evidence itself (at least in a prospective sense). Let us know if we go this wrong, but this interpretation seems confirmed by what you say later:

    “The fact that the X-people had higher crime rates than the Y-people up until Dec 2019 doesn’t mean that some X-defendant in July 2020 was more likely to have engaged in crime than some Y-defendant (setting aside all non-profile evidence), since the big legal change which happened in Feb 2020 means a reasonable factfinder couldn’t expect the trend up until Dec 2019 to continue into 2020.”

    The big legal change, in other words, affects the probative value of profile evidence about the correlation between X and Y. And if the probative value of profile evidence is put into question — or at least for those forms of profile expressing correlations whose strength is put into question by changes in the legal system — then there is a more straightforward reason not to admit these forms of profile evidence, namely their lack of probative value. But, as we say in the paper, the puzzle we are trying to address assumes that profile evidence must be probative.

    Perhaps, one upshot of this discussion is that, in order for profile evidence to be probative of guilt, we should exclude “thin” profile evidence — i.e. evidence whose crime correlations are affected, in some important way, by lawmakers’ decisions.
    This, in turn, suggests we should just be concerned with “robust” profile evidence — evidence that is mostly impervious to changes in the legal system — and that evidence is objectionable on equal protection grounds. Instead, thin profile evidence — whose correlations are variable and depending on changes in the legal system — would seem to lack probative value. For that evidence, after all, it would seem questionable whether having feature X correlates with action Y looking prospectively. The correlation might have been true in the past, but it is unclear whether it still holds going forward.

    Perhaps, your thought is that what counts as a crime correlate — say, a bad environment or prior burglary — can vary over time. In some cases, it might be that those who fit profile X are correlated with crime, and in other cases, those who fit profile non-X might be. What counts as an incriminating profile might vary over time as society and the legal system change for various reasons.

    But if the crime correlations are so weak and transitory, we would say they have no probative value. If correlations hold some bite and exist over some periods of time and lawmakers can know that’s the case over these periods of time, lawmakers can say that some people will be at higher risks of mistaken convictions than others due to admitting those crime correlations. On the basis of that knowledge, lawmakers should act accordingly and ensure that equal protection is not violated.

17. Thanks Marcello and Collin,
    That response was quite helpful for me to clarify my thoughts. Perhaps my central worry shouldn’t really be framed in terms of whether the statistical facts depend on the lawmakers’ decision about whether to make profile evidence admissible. There is perhaps a more direct way of framing the worry.

    Suppose the lawmakers are making their decision at time t. There are lots of profile evidence facts that are unknown at time t, perhaps because they haven’t been studied, or perhaps because they haven’t yet materialized (because the statistical correlations concern times significantly after time t). The lawmakers’ decision to make these unknown-at-t profile evidence facts admissible doesn’t violate equal protection, because the lawmakers are in what I called the ‘fully ex ante’ situation, at least with respect to this subset of the profile evidence facts.

    What about profile evidence facts that are known at t? Some may be robust, such that the trends will persist even if the lawmakers at t make profile evidence admissible. This violates equal protection. Some may not be robust, in that the trends might disappear if the lawmakers make profile evidence admissible. This doesn’t violate equal protection, but those profile evidence facts won’t have probative value, so perhaps they shouldn’t be admitted for that reason.

    So we might get the conclusion that making admissible whatever profile evidence facts are unknown at t wouldn’t violate equal protection, whereas making admissible profile evidence facts known at t would either violate equal protection (for the robust facts) or be objectionable on other grounds (namely, that the non-robust facts will lack probative value).

    Now, it might be that it’s impracticable or otherwise objectionable to divvy up bits of profile evidence into those known-at-t and those unknown-at-t and to make only the latter (which don’t violate equal protection) admissible. So we might get the result that for societies like ours where lots of profile evidence facts are known, we should make all profile evidence inadmissible.

    But still, imagine a society where no profile evidence facts are yet known. Perhaps the society is just starting out (maybe we’re colonizing Mars, or something like that), or perhaps there is as yet no field of sociology or criminology which has studied correlations between group characteristics and crime (maybe it’s the early American republic, prior to the rise of the academy). It seems there wouldn’t be any violation of equal protection if they made all profile evidence admissible. And, if that’s put into the constitution, there’s no realistic possibility of revising it later.

    So I think there’s an equal protection-based argument for making profile evidence inadmissible in a society like ours where lots of profile facts are already known. But we don’t quite get an equal protection-based argument that the admissibility of profile evidence would be unjust in principle, for the equal protection argument doesn’t (it seems to me) condemn making profile evidence inadmissible in a society where the profile evidence facts are as yet unknown (whether because they haven’t yet been studies, or because they haven’t yet materialized). Is it a problem for your project if you don’t get the conclusion that admitting profile evidence is unjust in principle, as opposed to just in practice?

18. Thanks everyone for this discussion. It is very helpful. Thanks especially to Marcello and Collin for all their replies! I will write about four topics, in four separate successive posts. I think posting them separately will make for easier skim-reading. Apologies if this is misjudged.

    A. This stood out to me:
    > Is it an advantage of competing accounts of profile evidence that they apply to guilty as well as innocent defendants?

    > That’s a good question. It’s true that competing accounts like the autonomy/individuality account apply to both guilty and innocent defendants, and ours does not. We lean towards thinking that this is not an advantage, for two reasons. One reason is just that whether this is an advantage will depend on whether guilty defendants are wronged by the use of profile evidence against them, and we are not confident that they are. Still, if they do have a complaint, our account could easily accommodate this by assigning guilty defendants a right against being exposed to higher risks of correct conviction than other guilty defendants. (Di Bello and O’Neill, emphasis added)

    This seems like a slightly strange thing to have a right against. It also seems like this kind of higher risk wouldn’t be able to explain the perceived problem with profiling evidence. The reaction we have to profiling is that it is morally wrong; we have an aversion. But an increase in this kind of risk arguably can’t explain that sense of moral wrong.

    I raised a similar objection to Enoch, Spectre and Levi’s incentive-based explanation of the inadequacy of bare statistical evidence for affirmative verdicts:

    > Enoch et al.’s proposal, which concerns influencing one aspect of citizens’ incentive structures in some kinds of cases, does not provide adequate warrant for our response to the prospect of satisfying the burden of proof with purely statistical evidence. When we consider the cases described in section two, there seems something unjust about convicting, finding liable, arresting, searching, or detaining on purely statistical evidence. Plausibly those people are wronged. But Enoch at al.’s incentive-based account, which is rooted in practical policy considerations, is not the right kind of explanation to vindicate those moral reactions.

    Plausibly there is a disconnect between the explanandum – why is the practice morally wrong – and the explanation (incentivising behaviour, in the case of Enoch et al.).

    But this made me wonder whether generally increased risks relative to others are the kinds of things we have a right against, and whether they are the kind of thing that can explain what seems morally wrong. Here is the thought: Rights, wrongs, aversions, etc. might concern things like racial evidence being used, or individuals being falsely convicted. But they don’t concern increased risks of false conviction. (Increased risks, after all, might be small in magnitudes – both in overall magnitude of risk and the magnitude of the difference between the two numbers.) This worry, which I raise rather than endorse, would be a broader objection to Di Bello and O’Neil’s strategy.

19. B. Re. the dilemma
    > We think ginger hair evidence should be admitted so long as, from the ex ante point of view of reasonably informed lawmakers, it does not subject defendants with ginger hair to higher risks of mistaken conviction. So we are inclined to go with the first option.

    This makes sense. I did worry that it would vindicate maintaining widespread ignorance (such as ignorance about cross-race identification unreliability). This is because, on Di Bello and O’Neill’s view, so long as scientists don’t discover the disproportionate risks concomitant with the evidence, then law courts were permitted to use that evidence. But I think you can avoid this worry: It is unreasonable to curtail forensic science investigation for fear that it reveals the evidence is unfair in this way. So the overall view wouldn’t endorse preserving the ignorance of forensic science.

20. C. I very much enjoyed Brian’s comments. He writes:

    > But first let me say what I really like about the article:
    > First, it doesn’t just address whether statistical evidence (in whatever form) can on its own suffice for conviction. This is the problem of ‘bare’ or ‘naked’ statistical evidence that’s been addressed extensively in the literature. Many theorists draw an analogy between cases of bare statistical evidence and the lottery paradox and conclude that, just as high probability given the evidence doesn’t suffice for knowledge or rational full belief, so it doesn’t suffice for conviction. And this motivates an interpretation of standards of proof (e.g., beyond reasonable doubt) which is based on knowledge or rational full belief rather than mere high probability.
    > But this approach doesn’t entail that statistical evidence cannot combine with non-statistical evidence, in cases where neither would be sufficient on its own, to license conviction. It doesn’t condemn the admissibility of statistical evidence. (Note that there’s no strong intuition that, in a lottery case, you can’t know that ticket N is a loser on the basis of both the strong statistical evidence an some piece of non-statistical evidence, where the latter wouldn’t on its own yield knowledge or rational full belief.)
    > Second, they target profile evidence, rather than statistical evidence in general. And I think some forms of statistical evidence (like profile evidence) are more intuitively problematic than some others (like DNA evidence in cold-hit cases).

    I very much agree with Brian’s comments. The current debates about these kinds of problematic evidence in legal epistemology, and in the related moral encroachment debates, commonly refer to the problematic evidence as “numerical” or “statistical” or “base rate”. But often the actual evidence isn’t numerical at all. It correlates to perceived social generalisations, and correlates to base rates. But the actual evidence is association, stereotype and so on. An example from the moral encroachment debate:

    > The Cosmos Club. Historian John Hope Franklin hosts a party at his Washington D.C. social club, The Cosmos Club. As Franklin reports, ‘It was during our stroll through the club that a white woman called me out, presented me with her coat check, and ordered me to bring her coat. I patiently told her that if she would present her coat to a uniformed attendant, “and all of the club attendants were in uniform,” perhaps she could get her coat’. Almost every attendant at the Cosmos Club is black and few members of the club are black. This demographic distribution almost certainly led to the woman’s false belief that Franklin is an attendant

    The woman’s evidence here isn’t numerical or statistical, even though it is a kind of profiling. Instead it is her associations, interpretations, background knowledge, defaults, etc. Similarly, suppose S judges that A is gay. S’s judgement is based on A’s mannerisms, clothing, etc. This seems like profiling, but the evidence isn’t numerical. At best it is supported by, and correlates with, numbers. I mention all this because sometimes the literature conflates to the two. In doing so the literature might give too much credit to simple bias and association, by treating it as though it were backed by hard numerical facts and social science. It seems like a nice feature of Di Bello and O’Neill’s view that the explanation covers profiling, whether it is based on base rates and numbers or based on qualitative associations and stereotypes.

21. D. Finally, people who are interested in debates about legal proof and the ethics and epistemology of evidence law might find this resource useful. It’s the behemoth OBO annotated bibliography on Legal Epistemology:

    Behind a paywall:
    https://www.oxfordbibliographies.com/view/document/obo-9780195396577/obo-9780195396577-0390.xml?rskey=nAYbi7&result=187

    Not behind a paywall:
    https://drive.google.com/file/d/1oxwOSrmN1a9pHz9WPoSRgDb-SOOJEhPM/view

    (With apologies to those authors whose papers and books are missing from the OBO!!)

    Thanks again everyone!
    Georgi

22. Hi David, thanks for getting the ball rolling on this enjoyable discussion.

    In response to your point about Warrior Gene, if there is less discomfort about permitting this evidence to be used against defendants under these circumstances, that would cut against the equal protection account, since the fact that the evidence is wholly internal to the courtroom would not mitigate the equal protection complaint in the slightest.

    We might just have different intuitions about this. But perhaps there is something we could say to persuade you. We wonder how you would feel if we kept everything in the example the same, but instead of allowing this profile evidence to be used, lawmakers slightly lowered the standard of proof for these defendants because they are more costly to protect as a class. (Say, guilty people with this genetic predisposition tend to recidivate at higher rates than guilty people without it.) Again, no invidious stereotypes could develop outside the courtroom. In fact, not even the factfinders would need to be informed of the reason for the lower standard of proof. If you feel discomfort about the lower standard of proof, then equal protection would make sense of that discomfort. And once convinced of the right to equal protection, then even if using Warrior Gene would still not arouse much intuitive discomfort, you might be convinced that it should on the basis of our argument that its use would violate equal protection.

23. Thanks, Brian, for this very constructive discussion. We think you’re right that if the framers of this constitution wrote it so that it made profile evidence admissible and also made the constitution extremely difficult to amend, then the later introduction of profile evidence against innocent defendants would not infringe their rights to equal protection, as we’ve defined it. It’s a good question whether this is a problem for our account. It might be, since our intuition is that these defendants would still have a complaint against the use of this evidence against them, precisely because it exposed them to higher risks of mistaken conviction. Still, even if their rights to equal protection are not infringed, they may have related grounds for complaint against the framers. The framers knew that in time some categories of innocent defendants would be exposed to higher risks of mistaken conviction in virtue of their decision to write its admissibility into the constitution, yet they tied their hands so that when that time came neither they nor their successors could do anything about it. It seems like the defendants would have a complaint against that earlier decision of the framers to bind themselves and their successors unless the framers had an especially good reason to do so. This would not be an equal protection complaint since at the time the framers could do something about it, they were not yet exposed to higher risks, and by the time they were exposed to higher risks, the framers could no longer do anything about it. But it is still a complaint that derives from the fact that they have been exposed to higher risks.

24. Thanks Marcello and Collin,

    You write, “Still, even if their rights to equal protection are not infringed, they may have related grounds for complaint against the framers. The framers knew that in time some categories of innocent defendants would be exposed to higher risks of mistaken conviction in virtue of their decision to write its admissibility into the constitution, yet they tied their hands so that when that time came neither they nor their successors could do anything about it. It seems like the defendants would have a complaint against that earlier decision of the framers to bind themselves and their successors unless the framers had an especially good reason to do so.”

    This is a promising thought. But I wonder whether it doesn’t overgeneralize. The idea is that some future innocent defendants will have a complaint against the framers because they’ve been exposed to a higher risk (which here must be understood in terms of something like objective chance) of mistaken conviction than some others in virtue of their decision to constitutionalise the admissibility of profile evidence.

    But if future innocent defendants facing adverse profile evidence have such a complaint against those framers, why wouldn’t future innocent defendants facing eyewitness testimony of above-average strength also have a complaint against those who made eyewitness testimony admissible? In each case, from the ex ante perspective of the lawmakers, making the relevant type of evidence admissible didn’t give any future innocent defendant a higher probability of mistaken conviction than any other (since the lawmakers didn’t know what such evidence would materialize). And in each case, the future innocent defendants realise they’ve face a higher (objective) probability of mistaken conviction in virtue of the earlier decision to make that type of evidence admissible. But why do the innocent defendants facing adverse profile evidence have a complaint that those facing adverse eyewitness testimony do not have? Perhaps, as you suggest, what’s relevant is the fact that in the profile evidence case, we’re imagining that the framers have bound the hands of future lawmakers. But why does that yield a difference with respect to what complaints the two types of future innocent defendants have?

    It’s also worth noting that the framers might well have had especially good reason to constitutionalize the admissibility of profile evidence: They knew that the admissibility of profile evidence increases the accuracy of verdicts, which is a pro tanto good. And they knew that they wouldn’t be violating anyone’s equal protection rights by making it admissible (since they didn’t know what profile evidence would materialize). But they also knew that if they didn’t put it in the constitution and instead made their decision revisable, then future lawmakers couldn’t keep profile evidence admissible except by violating equal protection. So better to put it in the constitution so as to ensure that it remains admissible (thus increasing accuracy) but without themselves or any future lawmakers violating anyone’s rights.

25. First, a response to your suggestion about the Warrior Gene, then some parting thoughts. I would, of course, find it highly objectionable to lower the standard of proof for a given offense because of the higher cost of greater recidivism. But that’s because I think it’s wrong to consider the downstream costs of a false acquittal in setting the standard of proof, much as it is wrong to consider the magnitude of the punishment, or the future dangerousness of the defendant, in adjudicating guilt – considerations which jurors are sternly warned against. But not all costs are morally equivalent. Setting evidentiary standards always involves a tradeoff between the risks of false positives and false negatives; that’s what probative value is about. So, I think it’s reasonable to take the increased cost of mistaken acquittals into account in deciding whether to admit profile evidence. I agree with you that there may be reasons for bearing that cost (though we may disagree about the reasons), but unlike the cost of recidivism, it is a cost that should be considered.

    My parting thoughts concern equal protection as an ideal, not only for lawmakers and judicial factfinders, but for law enforcement as well. I’d initially raised the concern that the unequal risk of mistaken incrimination would inevitably result in an unequal risk of mistaken conviction, simply because there’s no risk of mistaken conviction for those who aren’t charged in the first place, and always some risk for those charged. In response, you extended their idealization from the agents who adjudicate the facts to those who develop the evidence — to borrow Yale Kamisar’s terms, from the judicial “mansion” to the police station “gatehouse.” I appreciate that the point of the idealization is to show that profile evidence differs from other differentially inculpating evidence in violating equal protection even under ideal circumstances. My question is whether other forms of evidence, staples of criminal prosecution, could avoid differential inculpation of identifiable groups of defendants even if the police conformed to their appropriate standard of rationality.

    The police, unlike judges and juries, don’t (just) assess the probative value of evidence, they develop evidence. Clearly, they should set about acquiring evidence they find reliable and should evaluate the probative value of the evidence rationally once they obtain it. For example, the police in high -crime jurisdictions should be as non-directive as possible in inviting crime victims and witnesses to look at “mug shots” or to see if they recognize any of the perpetrators in a drive around the neighborhood. They should avoid suggestiveness in their photo arrays and line-ups. But let’s stipulate that they accept a universal standard of presenting eyewitness evidence to the prosecutor for charging purposes only if they believe it to have a risk of false positive below some threshold value, the same threshold employed by their low-crime counterparts and police in every other jurisdiction. (The threshold for the police to take account of a given piece of evidence, or to pass it on to the prosecutor, arguably should be lower than the threshold for judicial factfinder to consider that evidence, but that’s not essential to my point.)

    The fact remains that in jurisdictions with a much higher per capita crime rate, the police will elicit fallible eyewitness evidence in many more cases, so the risk of falsely incriminating innocent individuals, even if the same for a single criminal investigation as innocent individuals in low-crime jurisdictions, will be higher cumulatively. That will also be so for such other evidentiary staples as accomplice evidence – it’s safer to feed the police some innocent chump to get them off your back than to finger the actual perpetrator, who does not suffer snitches. Of course, the police, being rational, will discount by this prospect, but they will inevitably be taken in by some good liars. And where the crime rate is much higher, innocent chumps will face a greater cumulative risk of being fingered by such convincing liars than innocent chumps in jurisdictions with much lower crime rates. The police in high-crime areas could raise their threshold for incriminating evidence to take account of this greater cumulative probability, but that would almost certainly cause a dramatic drop in their clearance rate, to the consternation of the innocent people living in high crime areas. It would be perverse to compensate for that cumulative risk by imposing a higher threshold for incriminating evidence in higher- than lower-crime areas.

    So, I don’t think that the risk of mistaken incrimination can be equalized even by assuming the complete rationality of police investigators. And however heavily factfinders discount eyewitness or accomplice testimony in light of the substantial risk of false incrimination, the risk of being taken to court to face such evidence will be greater for some groups of innocent individuals than others, which will in turn result in a higher risk of mistaken conviction.

    Maybe I’m missing something; it’s been known to happen, and I’m hurrying to get my final thought in before the discussion winds down. In any case, thanks for reviving an important debate with such an original proposal!

26. The fourth comment didn’t post yesterday. I think this is because I posted multiple comments in quick succession. Here it is:

    D. Finally, people who are interested in debates about legal proof and the ethics and epistemology of evidence law might find this resource useful. It’s the behemoth OBO annotated bibliography on Legal Epistemology:

    Behind a paywall:
    https://www.oxfordbibliographies.com/view/document/obo-9780195396577/obo-9780195396577-0390.xml?rskey=nAYbi7&result=187

    Not behind a paywall:
    https://drive.google.com/file/d/1oxwOSrmN1a9pHz9WPoSRgDb-SOOJEhPM/view

    Thanks again everyone!
    Georgi

27. Hi Georgi, thanks for this additional set of very thoughtful comments. We are going to mostly focus on your first comment in item A. This is the one that raises a potential problem for your account. But we agree with the observations you make in comments B and C. In particular, we agree it might be good to distinguish discussions about numerical evidence and evidence about associations (which need not be explicitly numerical yet still objectionable). Conflating the two seems problematic, as you point out.

    Now, onto your comment in item A. We agree that it is a little strange to think that guilty defendants would have a right against being exposed to higher risks of being (correctly) convicted than other guilty defendants. Guilty defendants do have rights against certain risks–e.g., they would be wronged if their verdict was decided on the basis of a coin toss, even if the verdict was correct. (Some people, notably Christopher Wellman, would deny that.) But it is less clear they have the same kind of comparative right against risks that innocent defendants do. And of course they have no right against being exposed to higher risks of conviction than innocent defendants.

    In the case of Enoch, Spectre, and Fisher, we think the point you made in your “Legal Burdens of Proof and Statistical Evidence” article is exactly right. Whatever duty there might be on the part of lawmakers to make evidence law in a way that preserves deterrence is not a duty that is directed to defendants or even to anyone in particular. Their deterrence account has considerable explanatory power, but it does leave something unexplained, namely, that allowing profile evidence to be used against defendants is not just bad policy, but wrongs defendants, at least the innocent ones.

    That is something we want to explain. Your worry is that while there is clearly a right against being falsely convicted and clearly a right against having generalizations about your race used as evidence against you, it is less obvious that there is a comparative right against being exposed to a higher risk of false conviction than other defendants. If your concern is just about the possibility of a right against even a slightly higher risk, we’re not sure about that either. We might want to say that if the risk is only a tiny bit higher than the risk to others, the difference is too trivial to infringe the right. Or we might say instead that while even a tiny difference would infringe the right, it would be easy to justify the infringement, on the grounds that the smaller the difference, the easier to justify infringement.

    But if you’re wondering about whether there could be a right against even a significantly higher risk, as we believe, perhaps it might help to persuade you to imagine that lawmakers lower the standard of proof against a certain class of defendants, on the grounds that they are more costly to protect. Wouldn’t these innocent defendants (who may have nothing to do with the reason their group is more costly to protect) have a complaint against the lower standard of proof, even if they are ultimately acquitted? We think the best explanation for this is those innocent defendants have a right against being exposed to a higher risk of mistaken conviction than other similar defendants.

    Finally, the fact that this comparative right against risks is not absolute might also make it somewhat easier to accept. Since equality before the law is very important in the context of criminal trials, we think it would take a lot to override this right, especially if the difference in risks is significant. But it could still be overridden.

28. Thanks, Brian for an interesting follow-up. Your comments deserve more consideration than we can give them right now, but here are some initial thoughts. We would characterize our earlier reply a bit differently. It’s not that today’s defendants would have a complaint against the framers because the framer’s past decision exposed them to a higher objective risk of mistaken conviction. Rather, the idea is that today’s defendants (e.g., Bad Environment) could have a complaint against the framers because the framer’s past decision caused them to be exposed today to a higher epistemic risk of mistaken conviction (from the ex ante point of view of the framer’s successors) and the framers knew their decision would subject some groups of defendants to a higher epistemic risk of mistaken conviction today (or at some future point), even though they did not at that time know which groups. This response is obviously still rough. But if this is correct, then it would not appear to overgeneralize to the framer’s decision to constitutionalize the admissibility of eyewitness testimony, since this kind of evidence will never expose, under idealized conditions, any groups of defendants to a higher risk of mistaken conviction (from the ex ante point of view of the framer’s successors), and the framers would have known this.

    Also, you make a good case for constitutionalizing the admissibility of profile evidence, but we still think it’s somewhat risky for the lawmakers at such an early stage to bind themselves to a form of evidence when they don’t know what it will be like or even whether it would be highly probative or not.

29. Hello David, thank you for these thoughtful replies.

    It’s true that both keeping the standard of proof the same for, say, warrior gene defendants and prohibiting the use of profile evidence against them would have significant costs in terms of mistaken acquittals. If there is a reason it would be wrong for lawmakers to take these costs into account when determining the standard of proof for such defendants that would not apply to the decision about profile evidence, then there would be an objection to lowering the standard of proof that would not extend to profile evidence. Of course this reason could not be equal protection since that reason to ignore costs applies to both the standard of proof and profile evidence.
    We doubt that there is such a reason–perhaps the standard of proof is special, but we think changing the jury voting rule would be similarly objectionable. But no doubt there are possibilities we haven’t considered.

    Regarding your other point, we agree that residents of areas with higher per capita crime will be subject to higher risks of mistaken incrimination by different forms of evidence, at least in part because these areas will be more heavily policed, which will increase the opportunities for the police to make mistakes. We also agree that these inequalities could be evened out by adjusting the evidentiary threshold for stops and arrests or by adjusting the probative value of the evidence so that a body of evidence that incriminates people from high crime areas is given less weight than an otherwise similar body of evidence against people from low crime areas in light of the higher risk of mistaken incrimination that people from high crime areas are subject to. You are right that these adjustments could lead to more crime and perhaps would not be worth it overall. This is a difficult issue and we’ve tried to sidestep it. As long as those unequal risks between people of being stopped, arrested, charged, etc. are not permitted to result in unequal risks of mistaken conviction between them as defendants, there is no violation of the right to equal protection as we’ve defined it.

30. We want to thank everyone who participated in this engaging and wide-ranging discussion, and a special thanks goes to David for kicking it off with his excellent precis. We are grateful for your interest in our article, and you have given us a lot to think about. We hope we have understood your questions and concerns correctly, and that our answers have shed some light on our article and on this general topic.

    We are happy to continue the conversation here until the organizers close the comments if you would like, and we are also happy to continue it via email if you’re interested. Thanks again everybody for such a careful reading of our paper, it has truly been a pleasure!

31. Thanks Marcello and Collin! I learned a lot from this discussion, and from your paper.

32. Pingback: Mini-Heap - Daily Nous

Comments are closed.