SIO266: Why Sam Harris Is Wrong about Police Killings


You're listening to serious inquiries on And welcome to serious inquiries. Only this is episode. Two hundred and sixty six. I'm thomas and back with me is lindsay ostrom and hide it really really good. Yeah everything in. My house is nice. That's all i know about. But i know we. You know that question should be stricken from the record in twenty twenty degrees. Every podcast has to do it. But i should just say salutations has great. You are on the show neutral statements. Okay actually accurate. So how do we explain what we're trying to do here. Well as i said. Back when lindsey. I started coming on the show. Well not i. I but started coming on the show. I dunno months ago. Now we had so many things we wanted to talk about So many ways. That i think her scientific expertise could could really help us breaks and things down and this is one of those things many listeners to the show likely used to be or maybe even still our listeners. To the sam harris podcast he used to be a big influence on me back down eight years ago six years ago until in my opinion he kinda lost it if you heard that episode where he tried to do it's called canley pull back from the brink and it was gonna save my thoughts on overall but the point is within an episode. And it's not just. This isn't just a reply. to sam harris. Because within the conservative kind of ecosystem there was a response to the george floyd protests to the black lives matter protests. That i know we're all we've all aged seventy years in the last year but do you remember when we had only aged about thirty five years in the last year in june. When we were all. I don't know in our late. Seventies that that those protests obviously we're huge deal and i in fact side note even set at the time. There's no way we're going to be able to maintain all this energy long enough to actually drive the reform that we need and it's not any of ours. There's too many things so anyway. Get uncovering all this. Onion layer of things here. There is some research that sam harris those people coming from kind of there are many people who are very anti black lives matter and one of the things that gets cited some of the stuff that gets cited Is this research that you've looked into. That has the maybe surprising conclusion. That wouldn't you know it. Actually black people aren't killed by police officers as much or when you adjust for certain variable so yes lindsay is here to maybe Tell us why those aren't accurate. Yeah because one thing that everybody agrees on regardless of you know whether in the they're in the same campus. Sam harris or support black lives matter. Everybody agrees that black people as a function of the population in the us are killed disproportionately by police officers. That's not something that anybody disagrees on But the issue that has come up in the way that The people the researchers that we're gonna be talking about today make this claim is that population isn't the appropriate benchmark. Be judging whether there's disproportionality disproportionality and how often black versus white people are being killed by police. It's a sam. Harris has talked about this and candace. Owens has talked about this ton of people of trotted out Findings from these articles that we're gonna be talking about today to support that claim in. It sounds very convincing. If you don't know what's wrong with the math that they did right. So that's the talking point. You hear a ton from a lot of anti conservative. Anti black lives matter people. it's while look sure. There may be disproportionate killings or whatever rest everyone say but black people responsible for x amount of the crime so therefore so that's how the claim goes every time. I wanna do a side note for the process of researching this. We both had to listen to a bunch of sam harris. I didn't. I don't want this to be a replying to sam hairs episode. However i have pages of notes replying to sam harris because it was so frustrating to listen to that. So what we're going to do is after this will do a like a patron only bonus episode where we more specifically address those sam harris. But this we're kind of hoping it'd just be more. You know not directly in response to him but responding to some of these claims because they're being made everywhere i mean. This is a fox news claim to this is this is a broad problem when it comes to The disagreements on police reform. Yeah exactly and and i really you know. I think it's important to be able to say in response to those things than just like well. That's probably not true like that. Sounds wrong Because it does sound wrong but But yeah. I think it's important to dig into so these these papers have been critiqued on very technical ways and so i think you're going to have to keep me honest here in terms. I'll try. yeah how clear. It is but But i think this'll be. I really hope this will be helpful for people all right. So yeah so. We're gonna talk about the claims of to vary widely cited studies that examine some version of these claims about. How actually you know black. People are not killed by police at higher rates and and if anything that white people were killed higher rates by police so the first article is by johnson trespass taylor and cesario it was published in twenty nine thousand. Nine hundred and the title. Here is officer characteristics and racial disparities in fatal officer involved shootings. It was cited forty five times. And it's only. It was only out for like a year before it was retracted. So that's a lot. That's retracted. Everybody you'd sky. I wasn't sure when you were gonna say that. Yeah that's a big deal that may be if people are still referencing. This retracted paper. That's indeed a problem Yeah so it was. It was cited forty five times alone just in the academic literature. That's that's massive for for only being out for year right But in the popular press this has been cited a ton right at personalities and things. Talk about this all the time that well side note this is what is so weird in kind of doing the thing we do. Here on a podcast. If you're on the left which is the right has this ecosystem that's very. I don't know what to say like centralized or something once they get on a thing once they're like. Oh this paper shows this. They all do it like they instantly all adopt that thing and so it goes from you know this obscure academic papers possibly to all. My god is being cited everywhere all the time. It's very weird. Yeah it is. It is and yeah. It's a problem. Yeah so this is one of the ones. That harris referenced explicitly in episode. Can we pull back from the brink And it's been criticized on at least two grounds and those are the two. We're gonna those to critiques. We're gonna talk about today. So the first. The most widely cited claim here that i want to criticize. Is that In this paper. They claimed that they found no evidence of anti black anti hispanic bias in fatal officer involved shootings and they claim that they found that white officers. Were not more likely to shoot minorities than non white officers to officer race to predict the race of the person being shot. That's the first claim that we want to talk about here. The second claim that we want to talk about is this claimed they made that. I'm going to quote here. They said that a person fatally shot by police was more than More than six times six point six seven times less likely to be black than white in a typical shooting in a typical officer. Involved shooting the sound like similar claims. But there's there we're going to critique them in different ways right okay. So those that's where we're going to we're going to try. To debunk here. So i wanna start with a little bit of information about the article itself and the way they got their data and things. Because i think that's important to critiquing this. So they opened up their article by talking about recent high-profile shootings of black americans. This was pre. George floyd and they suggest that it's it's very important very central to interpreting these cases as to whether black civilians are really overrepresented in fatal officer. Involved shootings And whether racial disparities are due to discrimination by white officers right saying these are the central things that people care about when they're talking about these these cases end like those seem reasonable claims. Right i mean. I think that those are probably central things that people are talking about. Our pr are minorities being discriminated against and is this because of of racism on the part of cops. Yeah that's it. Opens up a can of worms already. I wanted to say the same thing. Yeah yeah because. I think that i think that most people who are informed about this issue have concerns. That are way past the bad apples argument. It's it's definitely not just about a few racist cops. And i have noticed that with the conservative side of this argument and all these things. There's an obsession and obsession with the what were the actual end of individual motives of the person while they were doing the thick. Let's do it with trump. They do all there wasn't there. What if there's some other explanation for why he even had this argument with my family at one point you know. There's always something other than racism. That explains everything and you know to the black man who's being murdered by someone putting their knee on his neck. I don't doesn't get matters whether or not that individual officer is like yes. I am trying to be a racist right now. And that's the big problem so that's one of my biggest but that's kind of a macro level criticism on this conversation. I don't know how to apply exactly to these studies. I mean it might also be interesting to know if there's a individual bias problem you know. It's not as though we shouldn't be looking at that. But that if you if you even if you proved which you may well be able to. There's almost no individual officer bias in individual altercations. That would not at all solve the problem. No i totally agree and actually that is. That is one of my central critiques of this paper. Okay i mean it's it's not one of the statistical ones but they do this they do this move that i see a lot and i feel. Here's a little bit which is like all right. We're going to attack this one piece. That would be relevant to interpersonal racism. and then when we don't find support for it if we don't find support for it we're going to say okay so you know. The more important thing is just to try to get black people to stop being such criminals all the time. And that's that's not an appropriate move. I mean obviously it's not an appropriate move. But they do it they do it in a very subtle way and not too many So this is why. It's systemic and i know you probably hear that term a lot everybody and it really does capture it because it's not any one thing and that's what makes it so difficult. And what frustrates me about the. And i know we're gonna get more to it later. But in the sam harris conversation he makes reference passing reference to all income inequality inequality big problem. But it's like well. Yeah but that's all part of it like if yeah if you've got a subset of your population that is just dramatically less wealthy than the other set you know like. Do you not think that's going to cause problems which then might lead to over policing which then my you know lead. Moore's these altercations. which then has you know. Statistically it's gonna lead to more of them going wrong it does. It may not mean not that we can say for sure. Either way while you'll tell me what the data says but it may not mean that like individual motivated. Racism is the biggest issue. It may not mean but there's still a huge problem. Yeah absolutely i mean may say that he agrees below that. But you know another problem that i had with that episode. Is that like even if he does agree with all that his emphasis some completely different. Yeah okay. I don't don't get started on that. We're going to do it layer. We're gonna just absolutely tear that episode new one at a later date. But let's get back. Cutty sark. mary. All right cool. Okay okay. so so they are. They are talking about this broad problem and then they focus on this very specific claim rights of the central question. That sort of is his guiding. The research is what is the best way to really look at racial disparities in policing so we'd be looking at how many black versus white people are killed as a function of total population or issue. Or is there a more precise benchmark that we could use. Call this like the benchmarking problem. and so they you know. They acknowledged the past research that is used population as the benchmark showed that black people are killed at higher rates proportion personally to population But they're critique is that this calculation doesn't account for the number of encounters between police and civilians by racism so it could just be the counters higher for black civilians and that as a read of encounters rather than total population. Fatalities are equivalent by race. Or maybe you see the reverse or something now spoiler. They don't actually have information about police encounters in this paper this on the other one too they make this case about like what really what we need is encounters and they don't they don't have it well. I think that's a big problem here. Yeah with with with all sides of this even people who are trying to get the stats in aren't trying to just do racist conservative talking points. We don't have the data on on this. We might have decent data on killings. Because i know especially now like a lot of newspapers a lot of websites like tracking this and like yes trying to you know. Make sure none of those names fall through the gaps but in terms of just encounters. We we really do not have that data. No no we don't And especially not in a way that wouldn't be biased from the outset. Right right so we can come back to that. I've gotten there's a few notes i have about like. Here's what's a methods could look like the actually be better and getting us to that but anyway but yeah so. They don't have information about encounters. The fact that they are talking about encounters here is just a way of lake shoehorning in a claim later that black americans get killed more because they're violent criminals right. They're going to use a proxy. That is not encounters to predict the race of shooting victims. And it just. It doesn't work anymore. Yeah so they're making the point. That's the conclusion that you draw out. Racial disparities really depends on the back. Benchmark that you use and that's definitely true. sure it does. And so johnson colleagues suggests that one remedy to the benchmark problem is to use regression analyses to predict the race of the civilian that is shot in these officer involved shootings using various variables that could account for racial disparities so specifically in this study they predict within a database of all the civilians killed by police fire in two thousand fifteen. The race of the person shot using information about the race of the officer plus a few other characteristics About the civilian. Who was shot such as whether they were armed whether they were attacking police Again all of which are according to the police officers exactly much wrong. Isn't it every single time. We have video of shooting. That's just objectively terrible killing this subjectively terrible what it will go down as a according to polices well. He had what looked like a gun or he was on danger. And then never nothing ever happens to that story until there's protests until there's widespread attention on it and then all of a sudden we get the the video at the facts and things change. Yeah yeah exactly. I mean actually. We're gonna come back to those types of shootings in the second. Study that we talk about so that'll be fun right. So that's what they're trying to do. The trying to predict the race of civilians who are shots based on these different characteristics. So let me let me clarify. So when you're saying predict your so we already have that data but they're trying to like do a prediction and then match it up. Is that what you're saying or to see where discrepancies are. I'll grab the analytical strategy and a little bit later but they're using they're using regression analysis and you're essentially like trying to Calculate predicted values for some outcome on the basis of predictor variables. Okay yeah i'll come back to that. That's that's a we're going to have to talk too much about regression. Actually let's hasten that but Yeah okay so so they use this database of fail officer involved shootings. It was originally actually collected by the washington. Post and the guardian and they added to it So all fatal officer involved shootings in the us in two thousand fifteen. They limited their analysis to people who were shot that were white black or hispanic And across all of those cases there was something like nine hundred fatal shootings during two thousand fifteen in order to supplement missing data. That weren't there from the original database collected by the washington. Post and stuff They actually called six hundred eighty four police departments and requested info on the race sex in years of experience of the officers involved which i mean the one good thing i can say about. The study is it. That's a pain in the ass process and the ended up with the database. That i think you could actually do some interesting things with but anyway so they received information back from those departments Got complete data on sixty two percent of the shootings from that and then for the missed for the that was still missing. They went out to police or to like newspaper articles and things in court documents and added more to the data set so when all was said and done they had complete officer information for seventy two percent of the shootings in the set and partial information on the officers for ninety six percent of the shootings. So that's that's pretty good. That's a. That's a decently complete data set. So that's their data set. Before i go any further. I do want to underline one thing right so again. They started with this. Very broad context. People are getting shot by police. Why is that happening. And then it turns out that they're actually testing incredibly narrow question of whether officer characteristics impact. Racial disparities only among people who have been fatally shot. And the reason. I want to underline. This is because the that they ultimately draw. Do not describe what they are actually testing. This research is not looking at the risk of being fatally shot if you have an encounter with police or even the risk of dying if you're shot by police it's analyzing the racial makeup of a group of people who were fatally shot by police officers and that brings us to the first critique already even before we've seen the data so There was a response to this paper. Published by knocks mumble. Oh in two thousand twenty. And they pointed out that given that they're only investigating the selected pool of cases right incidents in which an officer shot someone and they died the possible inference. That you can draw here are super limited link you. You can't make a claim about whether white officers are more likely to shoot minority civilians. Have that information. You would need information about encounters in order to make that claim yeah right. That's that's essentially claim about rates And you can't make a rate unless you have the the denominator there right so the way that they have framed. Their conclusions is is not matching up with the data that they actually have game I'm actually going to quote their conclusion here. Just to to show that. I'm not making this up so This is the second paragraph of their discussion section. They say we did not find evidence for anti black anti hispanic disparity in police use of force across fatal shootings. And if anything we found anti white disparities when controlling for ray specific crime right. So if. I'm a journalist. And i read this statement. What do i for white people are more likely shot by police right. Yeah that's not names to be what they're saying yes. It clearly seems to be what they're saying. They made a response to this where they were like. Oh no this is just misinterpretation. Like no sorry dude like you are culpable for that misinterpretation right. This was incredibly unclear in the way that reporting it so knox in molo in their response. They said that in order to make the claim that you're making you need to have the number of encounters between officers and civilians the various races in other words. You need to compute a proportion and they just don't have that so this is. This is one of the central critiques. Here that's that motivated the authors to retract the article right after they were sort of hammered on this. I don't know if we mentioned this. They retracted the article. That seems seems like a big deal to me. It is a big deal. It is a it is a huge deal and side note. Yeah it's been cited after being retracted. Still right i mean people are still referencing it as though gospel no. I'm sure yeah. I mean i can't remember Sam harris a second episode when he had his guest on to talk about the stuff. I can't remember when that was released. But this this paper was retracted in july. I think if this year okay so so maybe it would have been after the first but before the second one but but anyway it could have been. Yeah anyway but yeah. I mean once you know about the research. If i've already read the article i'm not going to necessarily know. It was retracted. Yeah like you would only know if you download the article now because it says retracted along the all right so so that's the first critique right. They're they're making a clear can't possibly make based on the data all right so the second critique is about like the specific way. They went about their analyses. So yeah we're talking about regression for a minute so in a regression analysis. You're again you're trying to predict some outcome variable and they used a specific form of regression analysis allows you to predict the outcome of binary variable so in this case they were looking at race there as their binary variables so the person who was shot. Were they black or were they white and then they had other analysis where they were saying are the hispanic or white okay. Yeah and so what they did was Create these prediction models that would allow them to predict the race of the person who was shot on the basis of all these characteristics so again they had the age race and experience of the officers involved in the shooting shooting characteristics so they they had in their data set whether the civilian was armed Whether there was evidence of mental health issues recorded by police whether they were reported attack the police officer and so on and then they also had information on county characteristics so they had What proportion of the county was white black and hispanic what proportion of homicides committed in the county were committed by white black and hispanic offenders and so one that actually ends up being a central role in their conclusions. Even though they don't really play that. Up in the tyler intro. So because again they're going to make the case that black people get shot because they're criminals so According to them right they're they're results. Indicate that in fatal officer involved shootings the probability that the fatally shot civilian Was a minority. Member versus white did not increase as a function of how many white officers were involved in the shooting. And this is the the primary thing that they're using to claim that lake. You know certain indirectly. They're using this to claim the lake so it can't be individual racism on the part of officers because it was then more white officers should mean higher probability that the person being shot as a minority with there seems to be so much wrong with that. Yes there is. This goes in line with one of the counter arguments. You hear all the time. Which is that. Actually you know black officers and hispanic officers shoot people. The black people in spank people just as much or more. Well i think that that predates this paper. I mean that goes back a long way and you know. I've been hearing that argument for a long time and again it. There's two two problems with that one. Is that individual. Racism is not necessarily even what we're looking at here. That's not the biggest problem. It'd be a problem but not like the biggest issue. It's systemic not about any one individual bad apple to it assumes that black or hispanic officers couldn't display racism against black or hispanic suspects totally. Yes and i mean. I know that same. Here's probably doesn't like the implicit bias literature. Either but there is evidence from the psychological literature indicating that like all people regardless of their race seem to have the same implicit associations with certain social categories. Right so yeah do regardless of who you are. You have an association between young black males and So yeah totally agreed. Lot wrong with this especially making it. The focal analysis is just stupid. Okay so additionally they Purported to demonstrate that racial disparities in violent crime rates predicted the race of civilians killed in these fatal officer involved shootings so as the percentages at the percentage of homicides committed by black or hispanic civilians increased county so did the likelihood of the victims being shot being black or hispanic then finally third central claim was widely reported in a regression model that purported to represent the typical shooting and when i hear typical shooting i think oh so the most common shooting. Rice shootings A person fatally shot by. Police was six point six seven times less likely to be black than white. So i read this earlier right so now. We've got all of our provocative claims covered rates a claim on evidence of anti-minority bias. In these shootings if anything there may be an anti white bias Too no evidence that people shot by white officers and more likely to be minorities. And then we've got this purported evidence that violent crime rates by minorities is actually what predicts the likelihood of of the person being shot being a minority so those are three central claims right. These have been trotted out. A lot of people so just to recap so far. They looked at the super narrow claim. Can we predict whether a fatal shooting victim is white or racial minority. If we have information about the race of the officers involved and the crime rates of racial minority groups which doesn't actually have a lot of informational value. Here don't have. I wouldn't you like me. S j w black lives matter supporter to the stars woods. If you ask me like well do you think we could predict the race of a shooting victim based on the race of the cop. I'd be like no. I don't think that has anything to do with any of the claims am i. Am i crazy. You're not at all the issue that people are marching about. I don't think no. It's it's not so i i'm going to come back to the regression things in a minute because i think that the the term prediction is confusing. Your but it was designed to be like. They presented their results in this very confusing way. It was begging to be misinterpreted right And and then they conclude by implying. Strongly that like will based on these analyses. The real problem isn't racism like it's there's layers of a problems here so so let's let's come back to the the regression thing that been teasing so much. So second important. Critique of how these results were misleading is from Shinmachi and carlson as was another published response and reanalysis of the data in this in this article and its response is titled young non suicidal male victims of failure to force are thirteen times more likely to be black than white which seems to contradict the claims of the johnson paper and they start with this lake really nice clear explanation of what's actually being tested when you use these logistic regression models right. So what are you doing what are you. What are you trying to learn when you set up these models to try to predict the race of the of these victims right and their ultimate point where we're going with this. Is that the assumptions that you build into your regression models really matter in terms of the ultimate conclusions. You draw from your results these analysis so when you do regression you're essentially making point predictions based on various upsets of cases. So if i want to know whether the age of victim in a fatal officer involved shooting predicts the likelihood that that person is black or white The way that i would do that is to predict based on the observed cases in my set the probability of being black versus white for like each of the different ages. In my set right. So if you've got a significant relationship there essentially what that would mean is that you know people who are younger right or likely to be black victims of shootings that makes arrives to be black and white okay within this case. This is a hypothetical but yeah ok. Well it's kinda is but like they use this one of their variables right one of the things that the is they're actually in a relationship between like younger Interesting shouldn't that's where we're going. Yeah so if. I was doing this analysis. I i can use that regression line to estimate how likely i am if i'm a victim of one of these shootings to be black versus white okay. H twenty versus thirty five versus fifty. And so on right using my observations when you use multiple predictors in regression analysis. And that's what they did in this study like some of their models had like fifteen predictors in them. You're essentially just like expanding the way that you define line so so instead of just predicting the probability of being black versus white with age you're also predicting it with the black murder rate in the county officer rate and so on right so i'm setting all of us up thoroughly in order to say this. They created a regression model in the original paper. The johnson paper That they described as predicting the probability of black versus white victims in the typical shooting. And what that actually means is that they're plugging in like an average or middle value for each one of their predictors. In order to come up with a predicted probability of whether the victim was white or black. Does that make sense. I'm not entirely sure. I understand that. Okay all right. So i'm gonna. I'm gonna describe the typical shooting model and then see if if it does make more sense. Okay so again klay from the original that we're responding to here is that there was No evidence of anti-black or anti spanish disparities across all fatal shootings and this was based on the results of aggression model typical shooting which suggested that a person fatally shot by. Police was six point six seven times less likely to be black than white so what was not obvious. In the original paper about that specific claim that specific analysis and what shinmachi carlson made explicit and their response were the specific characteristics that were being specified in order predict that value at that in that quote unquote typical shooting all right so the typical shooting had the following characteristics. They were specifying a county. That had four times more white than black citizens so fifty percent white twelve percent black. That is typical in in the sense. That's probably representative of a lot of the country. Right yeah but also like the probability of being a white victim of a shooting is likely to go up. Yeah fortunately i've got a question. Why do you have to use a typical shooting. Don't you have all the data. can't you just use maybe. I'm not a statistician here. Now the they did they did more models right Yeah it's gonna take more time than it's worth to get into one of the models that they set out okay but but this was a. This was a claim that they highlighted. And then they made the claim that like even when we played around with all these variables. And the values of all these variables. It didn't change the ultimate conclusion. We still found that white people were more likely to get shot than black people so typical was like base rate but then they played around other things and they were like now. It didn't change the results okay. I still don't totally get why you have to use a quote unquote typical shooting. Like i feel like if you have the data set. Don't you just use the data set and then try to see what matches up like what you do I think like what you're trying to do. Regression is like sort of get a summary statistic here of like whether there's a there's an effect overall and so i think that that's the point sort of saying like in the typical shooting right in the in our database right. What i think they're they're point is like what typically is true is that white people are more likely to get shot but okay do they have any information on whether or not a typical shooting is something. That's actually likely to happen or could there data be. Here's a bunch of shootings from overwhelmingly black areas. And here's a bunch of shootings from maybe overwhelmingly white areas because we are such a segregated society and then they like average that out to be a quote unquote typical shooting. Oh sorry i misunderstood. That is exactly the point that we're making at least i think so so Yeah let me let me tell you about these characteristics and then yes. That's exactly the point. That i wanna make okay. Is that this. Typical shootings not typical. Gotcha all right. So in the in the typical shooting right quote unquote. These are the characters that had so again county had Four times more white than black citizens and that is important for regional. come back to the county had Equivalent rates of homicide per perpetrated by white black and hispanic citizens. The average age that was plugged into this regression model was thirty seven years old and it was specified that white and black victims were equally likely to have mental health. Problems be suicidal. Be armed or attack officer and so the odds ratio that was estimated for this group quote unquote typical group For being black versus white was an estimate for that group of shootings. Which i agree is probably not typical. Yeah and there's a lot of assumptions being smuggled in their eight lake the higher number of white citizens which will inflate the likelihood there will inflate the estimate that you have of how likely someone is to be white versus black all right just puncture population. The older age rate is important here because we know that young black men are most stereotypes being violent criminals. But they were portraying this in the results as though you know. This result for the typical shooting meant that in most cases whites are more likely to be shot than blacks. That's what they were sort of pitching the conclusion usually the way that you would do. This is is create an average point based on the characteristics in your data and remember this is like all of the shootings for the united states. right so this should be representative of all the shootings in the united states. At least in two thousand fifteen yeah did they only use one year. The yeah i mean this. Is this a labor intensive data set so to do this over a larger number of cases would be gotcha but yet so yeah. Usually what you do in regression is you. Would you would use the characteristics of your data set sort of center your variables on the mean right and then as you're controlling for things right. You set certain things to be at the meeting or at a middle value depending on what kind of variable you're using so they did. Yeah i think and it's it's not necessarily to do that by and large but for the specific claims that they're making right. They need to be super clear about what this means and just saying that white people are six times more likely to be the victims in in these shooting shooting dating data set than than black people. Typical shootings vastly distorts. What they're actually showing in this regression model so just to sort of underline their critique and carlson laying this out created a second model where they re specified the variables. Okay yeah and this allowed them to generate odds ratios for being black versus white in these incidents for a different subset of shootings so in this case they were predicting the likelihood of being black versus white for shootings in which the civilian is young age. Twenty the civilian is unharmed. The civilian showed no signs of mental health. Problems civilian was not. There's no evidence that civilian was trying to commit suicide by cop. That was another thing in their data. Seven and in which the county had an equal proportion of black and white citizens. Okay and that again. That's important because it sort of holds. The population is something. That's not going to be affecting okay. The prediction of the rate right Of course in johnson's original allosaurus didn't do that. Just problematic all right so in jamaican carlson's estimates using this new model. They found that the the shooting victims were thirteen times. More likely to be black than white in this other. Subset of cases. Wow yeah almost like maybe. The other model was misleading. Almost like that. Yeah so yes this is. This is the second way in which the conclusions people were. Super misleading again. This is one of the claims that was the most widely cited being really distorted sam harris quoted that specific finding specifically civically. That is specific enough. And yeah it's it's widely misinterpreted. Also right. I'm done with a study. But also i downloaded their data's at from the publisher's website because they make their data publicly available and it was nonsense. There were there were arizona. There were variables that mislabeled. I looked at their supplementary. Really data. yeah. I looked at their supplementary datasheets which had like descriptive. Statistics correlation tables and stuff. The correlation tables were were mislabeled. They it looks to me like there were problems in like the the assumptions the regression models that they are running that they did not address in the paper and so just like after looking at the stuff myself. I'm like this is just sloppy research. This is not like aside from all the conceptual problems here like. They're just they weren't being careful. And if you're if you're doing research on something that has this big of an impact. I can't think of something more irresponsible than to be this sloppy. Yeah it's a high stakes thing. Yeah and this is not. I mean this is not like okay. They're doing this for fox news or one american network like this is trying to be actual science right. Yeah maybe that's why it got retracted. I think so. I think that's I mean maybe they felt some some trepidation in the wake of this. I hope so. Oh okay so second. Study this overlapping author list. So this was cesario johnson and terrell published in two thousand nineteen so two of those offers authors. Run the other papers as well And the title was. Is there evidence of racial disparity in police use of deadly force an analysis of offer officer involved fatal shootings in two thousand fifteen in two thousand sixteen so they had really similar methods. They were actually looking at a similar data sets it was those cases that were pulled by the washington post and the guardian thanks But they looked at it across two years. Two thousand fifteen two thousand sixteen and they were. They were tried to answer a very similar question but they did a few things differently so they were looking at. They were again using crime rates to benchmark here so they're trying to predict the race of victims of fatal officer. Involved shootings crime rate. Did we get. I don't even think we got to the point in the last paper. Where you 'cause like the whole point of the last paper was they don't have they don't have data on police interactions. How did they estimate that in the last paper. I don't know if we got to that question. No they didn't i so they are. They are sort of sneakily using the crime rates as a proxy for that. They're saying yes they're saying if the crime rates are higher than that's a good proxy for officer involved shootings excuse me for for interactions with police officers And that you can therefore use that to benchmark the fatalities in these shootings. I know in politics. We tried to get the second paper. But now i'm realizing there's there's quite a lot of questions i had from the first one that we didn't get him. Which is if you say okay. Let's compare you know. White people being shot versus black people being shot. And then you say well i bet there are more interactions setting aside that the fact of the matter of the interactions. Because i think that's a different thing but if you don't know that data and you say we'll let's use murder rate as a proxy. I don't really think that has much at all to do with interactions. Nope i mean a typical murder. You know like in in every okay. I'll say this in anecdotally again. But i think this is part of the day that we're talking about in every shooting police shooting. I've seen that's a viral video thing. None of those anything to do with murder. I don't know that a one of them had anything to do with well. This guy's a suspect in a murder right there's traffic stops. There's selling loose cigarettes. There's mighty use the counterfeit Twenty i can't recall a single one. Maybe there's one summer. I can't recall a single in the had anything to do with murderer. And so when and this was specifically responding. I think sam harris hughes murder rate as a proxy for like. Well if they're responsible for this percentage of the murders then there must be by. It's like okay setting aside the fact that. I do think there is more interaction with police in the black community. And that's a problem a separate problem. You can't really use murder rate to have anything to do with that. I don't think no no. You're you're exactly right. So here's how they make the case for that and this is true in both papers so we are still talking about the second paper okay. So the way they make the case for this is to say that the vast majority of fatal officer encounters happened in the context of them responding to violent crimes and they make the claim that well. It's like five percent or less die outside that context and so this has to be approximately. No they're wrong. Oh they're all right. Oh yeah there definitely wrong. You're definitely right right. I was just guessing. I'll tell you okay. I'll tell you right now. You're a careful sci-fi person. So i assume you're not just saying that based on nothing. Yeah no no. I'm not yeah. So so here's here's the here's the thing right and i'll go. I'll talk about the the ryan. Evers response here so ryan and ever responded the second paper right again. They're doing very similar things trying to predict race here. They break things up by again like the us. Crime rate is a proxy to predict the race. And they're also they do one other additional i study. Which is the they look at these analyses separately for like different types of police encounters. So they have like one cat one analysis. They do is just like all encounters in which people were fatally shot. Then they have a second analysis where they look at all encounters in which police fatally shot when the person is unarmed and not aggressing and then they have a third category. That is all encounters in which the person was shot and there was an and the reason that they were shot is because the police mistook an object that they were holding a weapon right which is an. Yeah according to them. Yeah yeah yeah so in. Each of this speaks exactly the point. You were just making right for all of those three categories of shootings though they they they benchmark it on the violent crime rates. How does that make any seven. Doesn't make sense. It doesn't make sense. So the ryan numbers published a response to this and they make this this claim just purely on logical grounds that again cesario data insufficient test their central claims here. So they say right that they benchmarked all benchmarked all fatal officer involved shootings using only police encounters regarding filing or using only violent crime rates. I should say right and ryan and others argued that instead right nor do a proper comparison. They either have compared all fatal officer. Involved shootings to all police encounters full stop. Yeah or fatal officer involved shootings in the perpetration of violent crimes to police. Yeah okay yeah and just not ending them all together and comparing it to violent crime or something like that. Exactly right yeah. It does and they use analogy that. I really liked to sort of just show how this is. This is a problem with the math right so they were like if you wanna determine whether there's a gender bias in nsf winners in engineering you should compare the gender ratio of nsf winners and engineering to the gender ratio of applicants with an engineering right. What you should absolutely not do. I'm still quoting is compare the gender ratio of nsf winners overall to the gender ratio of applicants only within engineering which just illustrate would show a massive anti male bias. Right because an engineering got more men than women. So you'd think that there's going to be more mail. Applicants than women. Look winters overall. That's probably going to be closer to fifty fifty in terms of men and women winning because women are better represented in other disciplines that would get. Nsf awards right so if you make that inappropriate comparison then it's gonna look like hey ten. More men than women are applying equally men and women are getting it matter discriminated against so they continue quote this incorrect approach is the method. This asarco uses in a paper. He compares all of those who shot who are shot by police to only violent criminals and so see sorry method could only be correct if and only if every person shot by police was a violent criminal which is not true right right. Yeah that's the point of making okay. I like yeah people response. It was wonderful out of ten. Yeah i know know. We're out of time. But i'm gonna i'm gonna i'm gonna the take a minute here. Finish off the second paper so So yeah i'm gonna give you all these papers but the the response that we definitely don't have time to get to essentially takes that argument that that response that we just talked about and then they they prove it mathematically well they come up with a better analytical strategy they demonstrate in a hypothetical world in which there really is anti-black bias. Obviously it's not earth but maybe it's mars right in a hypothetical world in which there absolutely is anti-black bias. The strategy that these original authors use this asari used analyze data would not show it and actually show an anti white bias is what they claim to show. Wow so they're saying it's kind of poison from the beginning. There's no like even if even if they were right about their claims there their the way they modeled it and showed us. The results wouldn't even show us. That wouldn't show it wouldn't show it. And and then on top of that they go back to the original data set they re analyze it with the different benchmark for them. Yeah they do. They use the different benchmarking method that they have already. You know sort of showed mathematical proofs would yield unbiased estimate and they find surprise surprise. This is important. They find in there in the reanalysis. that there does seem to be disproportionately more black people killed in the types of shootings where they are unarmed and not addressing. Yeah but there is equivalency through. Doesn't seem to be racial disparity in the category where the person is addressing an armed. Okay so no racial bias there right and the reason. I think this is important right. Is it the fact that you don't get racial bias in unambiguous situations but you do get it in ambiguous situations accents. Yeah it's exactly consistent with what we know from the psychological literature about hot. How bias inter. Personally right when it's when it's unambiguous right. This person is attacking me doesn't matter whether they're black and white black or white right But if you have an ambiguous situation and just to talk about a different situation like if you're evaluating a black or white candidate for a job and the black person is clearly more qualified than the white person going to get hired but if it's ambiguous to which one is better that's where bias tends to operate if it's happening happening. Inner personally right by the way can be us. All of this data though is still according to police reports. Isn't it or it's still like self reported so it's it's take all of it with a grain of salt because the even the ambiguous or you know the unambiguous who knows what really happened there. Yes you're absolutely right about that. And that's a that's a super important point to make they're focusing on one tiny piece of it. They're doing their math wrong to being sloppy. It's bad and so was sorry. This all still the first paper or debunking. This is the second paper your. That's the second. Okay so this one. So when's this going to get retracted. Yeah we'll see. Gosh what's some of the same people. Right they should know how to retract. It should get practicing. They know which forms to fill out or whatever the heck they have to do which website to go to. Yes i mean. I think that that's a good point to go out on. You know like there are so many. There are so many other places at which there is bias in policing and we have lots of good data on that. We know that black people get sick more often. We know that they get searched more often. We know that there are all of these ways in which there is bias operating on every level and again these these researchers are just sort of pushing all that to the side and being like. Yeah but you know how many cops are a appropriate. I have so much yelling to do. I'm telling you got so much yelling to do about this. We're going to do it on a on a bonus episode where we yell at sam harrison other people making these terrible arguments in a memorial morally superior way When in fact they're actually cherry picking the data and they're not on solid footing it is. It's infuriating absolutely infuriating. So we've got to do that was wow. I think you did a great job lindsay in explaining it without the use of you get visual aids or anything. you don't get a laser pointer. You don't get slides. i thought about making powerpoint. I really appreciate you doing your best to To help us. Non science people a science. Noobs understand the trickery. That's going on here. That's so helpful. That's an excellent breakdown Hey if you see. These claims being made somewhere. Please share this. You know 'cause like it's it's it's maddening. it's everywhere. I i'm surprised i haven't heard about this before you know a good response this now. It sounds like people are issuing studies that reply to these other studies or whatever you know papers and of these papers so that's great. You could also link that but help get the word out viral that this is crap that these studies are not that these papers one of which has been retracted. The other is not on solid footing. And maybe hopefully in my opinion should be retracted. Soon to be retracted. How infuriating but Excellent breakdown lindsey. Thank you so much. And let's do some yelling on a patriot only episode. Thank you so much for listening to serious inquiries. Only if you like the show the absolute best thing you can do is support us at patriotair dot com slash serious pod. The second best thing you can do is please. Share it on social media or by word of mouth speaking of social media. Follow me on twitter at serious pot. If you'd like be facebook friends. Send me a request at facebook. Dot com slash thomas podcast. If you're not up for that level of intimacy that's fine. Maybe you wanna join the discussion at facebook dot com slash groups slash s. I o members. Thank you again for listening. We'll see you next time

Coming up next