Swedish Study Did Not Show 75% Refugee Children Lied About Their Age

Partial data from a Swedish study on medical age assessment of unaccompanied minors using dental X-rays and MRI scans of the knee-joint superficially appeared to claim that most minors lied about their age. This led to a gleeful celebration among anti-immigration activists who frothed at the mouth on social media and even fooled some mainstream media. However, the sample was not representative and exclusively consisted of individuals the migration authorities already suspected being above 18. The real surprise is that almost 1 in 4 of the cases where the migration authorities would have dismissed as not demonstrated to be below 18 years of age might be below 18 according to the imaging techniques proclaimed as the supposed golden standard. In reality, they have substantial scientific, ethical and legal problems.

Age assessment is often a crucial aspect of some parts of society, from unaccompanied minors seeking asylum and criminal cases where the proceedings and punishments sometimes depend on the age of the criminal to football leagues and other sports. In the asylum process, a minor has the right to special treatment and protection according to the United Nations Convention on the Rights of the Child. They are also rarely deported unless there is some organized reception in the country in question.

Determining if a person is above or below 18 is difficult. There are many different methods that can be used to gain some insight, from physical exams by doctors, psychosocial methods based on interviews, careful fact-checking and collaboration with other countries and medical age assessment. The latter involve the use of X-rays or MRI scans to study wisdom teeth, the wrist, the knee-joint or the collar-bone. Administrative employees at migration agencies that do not necessarily have a scientific background or expertise base their decision about age on these and other sources. In some cases, medical age assessment is given substantial weight in the administrative process while some experts think that it should be a complement to many other sources of evidence instead.

What are the scientific, ethical and legal problems with medical age assessment?

There are several problems with medical age assessment in the asylum process.

Medical age assessment attempts to estimate chronological age (how old you actually are) by measuring biological age (how old you appear to be). Because these are different things, it is not always an accurate method. Most studies done on the accuracy of these methods show error bars that are 2-3 years in either direction, which means that it is difficult to use those methods to produce a hairline decision about whether a person is above or below 18. These estimations also require a standard by which to judge the dental or bone growth against, and it is not uncommon that these are based on white middle class children from the 1930s from the United States. It is unclear how realistic such a standard is when dealing with individuals from different populations and different environments. Some evidence suggests that the time at which the growth zones close can vary by a couple of years depending on the sample. Due to these and other issues, many researchers prefer to use the phrase medical age assessment instead of medical age determination.

The ethical problems with medical age assessment comes from the fact that a medical procedure (X-ray or MRI scan) is being performed without any medical justification whatsoever. It is purely due to an administrative reason. In medical ethics, medical procedures should only be carried out if the benefits outweigh the risks and this is not obviously the case here. The risks include unnecessary radiation and the usage of inappropriate (or pseudoscientific at worst) methods that might produce an erroneous result too often. All methods have an error rate, but one has to think critically about the methods and also requirements for accuracy in the legal system.

The legal problems stem from the fact that the Swedish constitution Regeringsformen protects people from “forced bodily intrusion” (2 kap. §6 RF) and there is currently no specific law that would allow this. Thus, it if medical age assessment is done in a way that is not entirely voluntary, it may likely be illegal.

More information and selected references can be found here, here, here, and here.

Follow Debunking Denialism on Facebook or Twitter for new updates.

What is the National Board of Forensic Medicine?

The Swedish government decided (cache) during 2016 to start using medical age assessment in the asylum process. To circumvent some of the legal and ethical problems, they decided to make it voluntary and offered as an option if the Swedish Migration Agency concluded that the asylum seeker had not made their status as a legal minor likely. In Sweden, the unaccompanied minor has the burden of proof to show that they are below 18, so there is no innocent until proven guilty or benefit of the doubt in this context. The Swedish government decided to let the National Board of Forensic Medicine handle it.

The National Board of Forensic Medicine deals with forensic genetics, chemistry, medicine and psychiatry. In particular, they perform psychiatric assessments of charged criminals, forensic autopsies, paternity testing and tests for drug crimes or drunk driving. They are now also responsible for medical age assessment.

Somewhat alarmingly, the Swedish government decided to go ahead with the medical age assessments even before the research (cache) will be conducted to test in their approach works, highlighting the political nature of this project. This is to some degree understandable since Sweden received (cache) over 35 000 people identifying as unaccompanied minors and 160 000 people in total during 2015 and so politicians “need to do something”, However, putting research to the side and going ahead with something that can impact the lives of thousands of people is an approach that is not without problems.

How was the present study done?

The basic approach (cache) of the medical age assessment carried out by the Board of Forensic Medicine is based on dental X-rays of the wisdom teeth and MRI scans of the knee joints. Two blinded dentists independently analyze the dental X-rays and make a decision of they appear fully developed or not. In a similar way, two blinded and independent radiologists make judgement about the growth zones of the knee-joint. Both dentists need to conclude that wisdom teeth are done growing for the decision to be reached and the corresponding needs to happen for the radiologists.

Based on the results of these analyses, the National Board of Forensic Medicine makes an automated medical age assessment based on a “standardized matrix” and a forensic pathologist writes a likelihood statement about the age of the examined person. In most forensic certificates issued by the Board of Forensic Medicine, five decisions are possible, but the two strongest decisions (demonstrates and suggests strongly) has been judged to not be possible with current methods. So in practice, only three decisions (cache) are possible for medical age assessment:

– Assessment of performed examination suggests that the person examined is 18 years old or older (it is substantially more likely that the person is above 18 than below).
– Assessment of performed examination possibly suggests that the person examined is 18 years old or older (it is more likely that the person is above 18 than below).
– Assessment of performed examination possibly suggests that the person examined is below 18 years old (it is more likely that the person is below 18 than above).

Notice that there is no decision possible that is more certain for the below 18 years old conclusion than “possibly suggests”. It is estimated that about 10% of people who are near 18 years of age but has a fully developed wisdom teeth and knee-joint will be wrongly categorized, but it is unclear if this takes into account all of the scientific limitations.

What were the results?

National Board of Forensic Medicine recently released preliminary results (cache) for 581 performed medical age assessments.

– 442 (76.08%) were judged as “suggests that the person examined is 18 years old or older”
– 5 (0.86%) were judged as “possibly suggests that the person examined is 18 years old or older”.
– 134 (23.06%) cases where judges as “possibly suggests that the person examined is below 18 years old”.

However, there is one fatal problem with this study. Or rather, there is one fatal problem with the way the study has been interpreted in the mainstream, alternative and social media.

The study used non-representative sample of people who were already suspected of being above 18

So far, 4200 such medical age assessments have been ordered by the Swedish Migration Agency in cases where they have concluded that the age of the asylum seeker claiming to be below 18 is unclear. In practice, these are cases where the migration agency suspects that the persons are not minors or where their burden of proof has not been met.

This is the crucial fact that has been left out of most media coverage of this study. It is also predictably absent in virtually all Twitter chatter in anti-immigration circles. The sample was not representative of all persons claiming to be unaccompanied minors. It was a sample of persons claiming to be unaccompanied minors for which the Swedish Migration Agency concluded that the age of the asylum seeker is unclear. In other words, it is a sample of people for which the migration authorities already suspect are above 18.

Thus, these results cannot be generalized to the population of all persons stating that they are unaccompanied minors in their asylum seekers. However, anti-immigration activists on the Internet have almost completely ignored this essential feature of the study, likely because it would cut through their convoluted distortions of the data.

The real miscarriage of justice?

Droves of anti-immigration activists on the Internet has latched onto this press release and eagerly claimed while frothing at the mouth that “3 out of 4 refugee children lied about their age”, that “75% of unaccompanied minors in Sweden are grown men claiming to be children” or “political correctness gone mad!”. An expected response, since many anti-immigration activists have a documented ignorance about rape statistics and lethal violence statistics.

In reality, the real surprise here is that as many as 134 out of 581 (~23.06%) individuals tested (who the Swedish Migration Agency suspects being above 18 or where they consider the issue unclear) get the verdict “performed examination possibly suggests that the examined person is below 18 years of age”, which is the only possible verdict below 18 years of age.

This means that, for the people who push medical age assessment using X-ray of wisdom teeth and MRI scans of the knee-joint as a golden standard, the Swedish Migration Agency had an empirically false positive rate of at least 23.06% or about 1 in 4. Since we know that medical age assessment has substantial uncertainties for this kind of application, the problem is potentially even more complicated. This is probably not compatible with rule of law and could possibly end up being a miscarriage of justice.

There are certainly people who lie about their age in the asylum process to get preferential legal treatment and a higher chance at getting a permanent resident permit. However, misrepresenting preliminary data that is based on a biased and unrepresentative sample makes things worse, not better. It is also possible that some people were below 18 when they came to Sweden, but has passed 18 after delayed treatment by the immigration system.

How was this presented in the media?

So how did the mainstream media in Sweden handle this preliminary study that used an unrepresentative and biased sample?

The tabloid newspaper Aftonbladet that is often rightly berated for being clickbait junk actually got this story right. They interviewed (cache) process manager Daniel Salehi at the Swedish Migration Agency who stated that he is not surprised by the results because of this biased selection of cases where the agency thinks the evidence is insufficient and points out that the results would probably be very different if the study had been done on the entire population. They also published a column (cache) by Lena Mellin that highlighted how anti-immigration activists instantly attempted to abuse this study.

The competing tabloid Expressed were much less critical. In their primary article (cache) about the preliminary results, the only place where the non-representative aspect is explained to the reader is near the far end of the article where they interview the migration policy spokesperson Johanna Jönsson for the opposition Centre Party. It is perhaps a bit surprising that a politician have to explain science and statistics to a journalist. A column (cache) written by political reporter K-G Bergström is an instant train wreck, where he uncritically swallows the results, ignore that they are not representative and generously generalize to the entire population of unaccompanied minors. Not a single shred of critical thinking.

The Swedish public television also botched the reporting on their news website. In their article (cache) about it, they do not even mention the fact that it is a non-representative sample. At all. Not in the headline and not in the article itself.

Far-right “alternative news” websites either did not mention the fact that the sample was non-representative or insisted that this fact actually meant that the proportion of people claiming to be unaccompanied minors were even higher.

Conclusion

Mainstream media continues to have a weak understanding of scientific research and statistical analyses, and there has likely been a shift in the media climate over the last couple of years as anti-immigration sentiments have become more normalized. Yet again, the mainstream media failed in its intellectual responsibility as the Fourth Estate. This highlights how warped reality becomes when facts are crowded away by clickbait journalism or anti-immigration ideology.