Class Is In Session

Your instructor for Stats 101 is Professor Yggy.

Suppose I invent a magical device that can be pointed at a Muslim and say with 90% accuracy whether or not he’s an al-Qaeda operative. Well, if I start waving it around and it starts beeping on one guy, what should we conclude about him? A terrifyingly large number of people are going to say “there’s a ninety percent chance he’s with al-Qaeda! Let’s panic!” In fact, that’s not the case. There are a billion Muslims in the world. A test with 90 percent accuracy is going to mistakenly classify about 100 million of them as al-Qaeda operatives. And al-Qaeda actually has fewer than 10,000 people working for it. I’m going to get something like 10,000 false positives for every actual terrorist I find.

Meanwhile, applying the test to people is going to have severe consequences. The public doesn’t understand this correctly and is going to be put into a wholly unwarranted state of panic about the prevalence of terrorists. People will, of course, demand that those flagged by my machine be subjected to extra-heightened scrutiny. It’s easy to imagine lots of innocent people being mistakenly killed or subjected to discrimination or shunning. And that sense of beseigement and unfair treatment would ultimately heighten tensions between the world’s Muslims and the West, while wasting massive quantities of law enforcement resources chasing basically worthless leads.

For the record, he's talking about Bayes' Law, which basically is used to show that the more rare the event, the more accurate the test for that event has to be.

In other words, in the above example from Yggy, the Magic Terrorist Test gives false alarms 10% of the time (0.1). The actual odds of a Muslim being an AQ terrorist is 10,000 out of a billion, or .00001. Our test is 10,000 times more likely (0.1 / 0.0001) to find a false positive than a real terrorist, making the Magic Terrorist Test effectively useless.

Now, the real question is "How bad of a press situation are 10,000 false positives going to be?" If it's profiling Muslims, it's horrible. If it's 10,000 tornado watches on the other hand, well...that's socially acceptable.

But now pretend you're not sure how accurate your Magic Terrorist Test really is. Because of that, you really have no choice but to test every Muslim. You're not sure what your test means because you don't know how accurate it is. Therefore the only way to be sure...yep, you got it: assume all Muslims are terrorists. You'd have 999,990,000 false positives, but you'd win your AQ hunt. Now how socially acceptable are your 999,990,000 false positives?

Kinda getting the point as to how ridiculous profiling Muslims really is?