Debunking Denialism

Fighting pseudoscience and quackery with reason and evidence.

Category Archives: Misuse of Statistics

New Medium Primer: Three Things You Need To Know About Multiple Testing

Three things to know about multiple testing

Anti-science activists often abuse multiple testing. Here is some basic information that you need to know in order to get up to speed on the issue.

Read the full primer here: Three Things You Need To Know About Multiple Testing. Remember to hit the heart at the bottom of the primer to help other people find it.

Read more of this post

New Swedish Crime Report: Almost All Crime Decreasing or Unchanged

Crime trends

The Swedish National Council for Crime Prevention has recently published a new report on crime trends in Sweden. It features an introductory chapter and then separate chapters devoted to the different major crime categories being measured in Sweden, from lethal violence and physical assault to drug crimes and drunk driving. Each chapter exhaustively identifies data sources and synthesizes their information while being careful to consider both strengths and limitations of different data sets. The report can be read in full by going here and a cached version of the document is available here.

Many politicians and media activists want to instill fear, uncertainty and anger into people in order to gain attention and votes. They often make the claim that crime is on the rise and that only they have the willingness and power to take this alleged problem seriously. In reality, crime is generally decreasing across Europe and the United States over time.

The general crime trend found in this most recent report is clear: almost all crime categories are either unchanged over time or decreasing and this trend can be seen in many countries in western Europe. Some crimes like bank robberies or car thefts have substantial collapses (decreasing between ~75%-80%). Other crimes like lethal violence have had a long-term decline since the 1990s whereas others like robberies are more or less constant over time. The one key exception is fraud. The proportion of the total population that have been the victim of fraud have increase from 3% to 3.5% of the population during the years 2005-2015. About 80% of frauds are either Internet-based or involve the use of debit or credit cards. The clearest reason for this is likely the rise of Internet crime since so many more people are now using the Internet to buy things or make transactions.

Read more of this post

How Anti-Immigration Activists Misuse Lethal Violence Statistics

Lethal violence in Sweden

The Internet is a wonderful thing. It has put the combined knowledge mass of the human species in a format that is easily accessible by billions of people hungry for scientific and historical facts about the world. The Internet, however, has also brought with it the possibility of spreading misinformation and nonsense at a rate that was never before possible. Someone can post an incendiary fake news story that inspire fear and anger about some real or imagined event during breakfast and before the evening has set in, the story has been shared hundreds of thousands of times on social media, provoked and misled millions of people and sometimes even made it into the mainstream media.

Sweden has recently become a target of various political outbursts designed to spread fear, anger and misinformation about refugees and immigrants. In reality, Sweden is a country that has not declared a single war since 1814 and is one of the best countries in the world to live in based on dozens of different metrics such as safety, education, health care, happiness and so on. In the dark and damp places of the Internet, however, Sweden is wrongly portrayed as a hellhole where murder and rape are out of control, criminal gangs have taken permanent control over several dozen areas and the radical feminist government and the media are actively covering it all up.

The reality, of course, is entirely different. Anti-immigration activists abuse rape statistics, the rape definition used in Swedish law has expanded multiple times since the late 1990s, the propensity to report rape has doubled in recent years and the police records each individual case as a separate police report. Two reports published by the Swedish federal police has shown that although there are social problems in especially vulnerable areas, but the idea that they are somehow no-go zones is a propaganda myth and the police works there every single day.

Read more of this post

More Deadly Doctor Gambits: Mortality Rates and Doctors on Strike

a deadly doctor?

Mainstream medicine and medical doctors are increasingly under attack. Homeopaths, naturopaths and other groups pretend to have real medical knowledge and subject vulnerable and sick people to fake “treatments” that has never been scientifically tested or has been tested and shown not to work.

Concerned parents think that a few hours spent reading blogs written by medically unqualified individuals without any credible sources gives them more knowledge and diagnostic skills than medical doctors. Cranks on the Internet who are literally making stuff up about avocados, sun-staring and bleach as a cure for all diseases amass an audience of millions for their nonsense assertions that fly in the face of rational thinking and published scientific evidence.

Even more sinister, quacks attempt to portray medical doctors as a dangerous threat to the health and life of patients. This is often done by deploying the so-called deadly doctor gambit (originally discussed by C0nc0rdance) by falsely claiming that doctors are more dangerous than guns. In reality, this argument falls apart when you realize that there are 33 million hospitalizations per year in the U. S. so the relative risk from guns is much higher than doctors. As an analogy, there are more people who die from car accidents than from motor cycle accidents, but since there are many more people driving cars than motor cycles, motor cycles are more dangerous per capita drivers.

Thus, many of the hateful alternative medicine proponents that would say almost anything to make doctors seem evil really just ignorantly misunderstand and abuse basic statistical considerations.

Read more of this post

At Least 60% of Reported Shootings in Malmö Not Actual Shootings

Malmo gun violence

International news media seems to think that the Swedish city of Malmö is being overrun by gun violence. In reality, the definition of a shooting differs between Malmö and the other two main cities. In Malmö, it does not have to be an actual gun that was fired and there is no requirement for forensic or eyewitness testimony. The geographical areas also differ, with Malmö only covering the main city and the urban area Arlöv. Once researchers looked through the data and counted the number of actual shootings, the figures dropped by 60-75%. The Malmö police does offer some justifications of their classification scheme, but in the end, organizations that gather statistics have an intellectual responsibility to ensure that their data are not easily abuse by being clear with definitions and what can and cannot be inferred from their data.

A recent analysis by Swedish Council for Crime Prevention was covered in the newspaper Sydsvenskan. The sad reality is that fear propaganda gets front page news, whereas a careful statistical analysis gets only a small notice in most papers.

What is a shooting event?

Perhaps surprisingly, the definition of a shooting differs drastically between the three largest cities in Sweden (Stockholm, Göteborg and Malmö). In Stockholm and Göteborg, a shooting requires two criteria: (1) the discharge of a gunpowder-loaded projectile and (2) corroborating forensic evidence or eyewitness (or earwitness) testimony. In Malmö, a lot of other things are also classified as a shooting, such as firing of airsoft guns, slingshots, damages to windows that look like a bullet hole and even damages done by stone chips. Thus, the figures cannot be naively compared because they do not measure the same things.

Read more of this post

Apparently, NHST Defenders Could Get Even More Ridiculous

Häggström and NHST, again

Looks like Häggström has decided to re-join the crucial discussion of p values and NHST again, despite refusing to continue after our last encounter because he claimed (without evidence) that my writings were a “self-parody”. This is reminiscent of childish and narcissistic posters on Internet forums who writes a post about how they are leaving the forum because of this or that perceived insult, yet stays around to continue posting. Tiresome and pathetic, especially since he apparently considers a link to the ASA position statement on Twitter to be equivalent to “spewing Twitter bile”. Talk about being easily offended to even the smallest amount of (fair) criticism.

Häggström recently managed to get a paper of his defense of NHST published in the journal Educational and Psychological Measurement. Perhaps “managed” is not quite the correct word, as it is a journal with a very low impact factor of 1.154 and is either in the middle or the bottom half of journals in mathematical psychology (8 out of 13), educational psychology (30 out of 50) and interdisciplinary applications of mathematics (46 out of 99). Perhaps a low quality psychology journal is the only place Häggström can get his rabid defense of NHST published? Well, that and a paper from a conference held in Poland. Not exactly impressive stuff.

Ironically, at the very same day he wrote his blog post about his new “paper”, the prestigious American Statistical Association published a position statement severely criticizing NHST. A previous article on this blog discusses several aspects of it in greater detail. Häggström claims that he agrees with the ASA, yet his paper in EPM attempts to refute NHST critics, both those he call “strongly anti-NHST” and those he labels “weakly anti-NHST”.

Some of the problems with new NHST defense by Häggström

There are too many errors and problems in his paper to recount in this space, but we can look closer at a couple of them:

(1) Häggström presents the NHST situation as a debate, thereby committing the fallacy of false balance.

There is no debate about NHST. The vast majority of papers published discussing NHST are very critical and there have been hundreds and hundreds of such papers published in the past 20 years. Today, there are hardly any papers published defending NHST and those that do defend NHST are few and far between. This shows that Häggström does not have a sufficient command of the NHST literature which is, as we shall see, a recurring theme. It also demonstrates that he most likely deliberately deploys a pseudoscientific debating methods against his opponents called false balance. This is because he, as a self-identified scientific skeptic with substantial experience from the fight against climate change denialists, knows full well that it is socially effective to attempt to undermine the scientific consensus position by portraying it as if there were a debate with two equally legitimate sides. It is not.

Read more of this post

American Statistical Association Seek “Post p < 0.05" Era

American Statistical Association

The edifice of null hypothesis significance testing (NHST) is shaken to its core once more. On March 6th, the American Statistical Association (ASA) revealed to the world that they’d had enough. For the first time in its history since being founded in 1839, they published a position statement and issued recommendations on a statistical issue. This issue was, of course, p values and statistical significance. The position statement came in the form of a paper in one of their journals called American Statistician, together with a press release on the ASA website. The executive director of ASA, Ron Wasserstein, also gave an interview with Alison McCook at the website Retraction Watch and the Nature website has a news item about it.

What was the central point of the position statement?

The press release (p. 1) summed it up quite nicely:

“The p-value was never intended to be a substitute for scientific reasoning,” said Ron Wasserstein, the ASA’s executive director. “Well-reasoned statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold. The ASA statement is intended to steer research into a ‘post p <0.05 era.'"

In other words, ASA acknowledges that p values was not supposed to be the central way to evaluate research results, that basing conclusions on p values and especially if the results are statistically significant or not cannot be considered well-reasoned and finally, that the scientific community should move in a direction that severely de-emphasize p values and statistical significance. Coming from a world-renowned statistical association, this is a stunning indictment of the mindless NHST ritual.

The final paragraph of the preamble to the position statement (p. 6) also points out that this criticism of NHST is not new:

Let’s be clear. Nothing in the ASA statement is new. Statisticians and others have been sounding the alarm about these matters for decades, to little avail. We hoped that a statement from the world’s largest professional association of statisticians would open a fresh discussion and draw renewed and vigorous attention to changing the practice of science with regards to the use of statistical inference.

ASA seems to share the sentiment among many critics of NHST, namely that there are several valid objections to NHST and that these have been raised as very serious problems for many decades with very little progress.

Read more of this post

How Anti-Immigration Activists Misuse Rape Statistics

Nationella Trygghetsundersökningen

The Internet has brought an enormous mass of knowledge to the fingertips of everyone with a computer, smartphone or tablet. Never before have so many individuals been so close to true scientific facts about the world, from fun facts about animals to the latest crime statistics. Large communities with blogs, forums and social media groups have grown up around a wide variety of special interests and it has become a powerful tool for communication, cooperation and the advancement of human knowledge.

However, this has also led to the creation of ideologically isolated Internet communities, where faulty claims and misunderstandings of statistics and empirical evidence gets repeated in an endless echo chamber and all refutations are either ignored, misrepresented or subjected to ideologically driven rejection, often with stale references to supposed “political correctness”, as if that was a statistically mature rebuttal.

This article will show that according to crime victim surveys, the actual rate of sex crimes has been more or less unchanged in Sweden between 2005 and 2014, despite the fact that immigration has increased during the same time period. Instead, the increasing rates of reported rapes are influenced by expansion of the legal rape definition, an increase in the tendency to report rapes, police efforts to classify each individual rape as a separate crime and their tendency to classify any sex crime that could potentially be rape as rape. It will also demonstrate that reported rates between countries such as Sweden and Denmark cannot be naively compared to do the large difference in legal rape definition and police registration methods.

Read more of this post

%d bloggers like this: