New Study Demonstrates Weak Correlations Between Social Media and Youth Mental Health are Merely a Byproduct of Other Issues
Also: More evidence for why meta-analyses need to stop using bivariate correlations.
In a recent peer-reviewed paper I confirm what many people have been saying: that any weak correlations between time spent on social media and youth mental health are due to “third” variables. In other words, youth who are stressed by their real lives may turn to social media a bit more a compensatory mechanism rather than social media causing those mental health problems.
I analyzed a sample of thousands of youth in the UK in the BrainWaves dataset (and a heartfelt thank you to the BrainWaves folks for giving me access). This included data on hours per day spent on social media as well as several outcomes related to mental health (depression and anxiety, mental wellbeing, quality of life, self-esteem, social phobia1 as well as friendships and other activities).
The dataset also had variables related to neuroticism/emotional stability, resilience, school connectedness and social belonging. These allowed me to control for trait personality factors as well as positive social connections inside and outside of school.
In bivariate correlations (not controlling for those other things), social media time was weakly correlated in a negative way with most outcomes, with the exception of friendships2. The mean effect size across outcomes was r = .154 which is pretty consistent with “noisy” results, meaning we can’t be fully confident such correlations are real as opposed to due to methodological noise (depressed people tend to overestimate their social media time, for instance).
To this in perspective, we can compare the predictive value of such a weak correlation to the impact of IQ scores on standardized testing outcomes. Correlations between IQ scores and standardized academic achievement scores tend to be in the range of about r = .6 to a high of .8 or so. Let’s assume that the real score is somewhere in the r = .7 range which would mean that IQ scores account for about 49% of the variance in educational achievement on standardized exams. By contrast, social media time would account for a measly 2.4% in the variance in youth mental health, assuming that correlation isn’t just noise (which it may very well be).
In a graph3 of r = .49, you can see a pretty clear trend, though certainly imperfect, with higher IQ scores predicting higher educational achievement. This is pretty useful to know.
By contrast, r = .1544, there’s much less predictive trend, barely different from a random scatter of scores.
But the interesting thing is, controlling for other variables, those effect sizes drop pretty much to zero with the average effect size now being r = .011. Only activities was the exception to this with r = -.2335. By contrast the trait neuroticism/resilience as well as school/real-life connectedness were more consistent predictors of mental health. This says several things.
First, this is consistent with genetic studies that suggest genetic factors control both trait neuroticism, but also a tendency to use more social media. Controlling for genetic factors in studies removes the predictive value of social media time. Although this study is not a genetics study, it is consistent with this general observation.
Second, this points out the critical value of including theoretically relevant controls in correlational analysis. All such studies should include some measure related to trait neuroticism, school stress or connectedness and family stress or connectedness (or adverse childhood events) where possible. A more interesting question is whether sex should be considered a control or moderator variable and, honestly, it’s potentially both. In this study, sex was not a significant factor, aside from anxiety and friendships (girls had slightly more of both) suggesting that narratives about girls being more vulnerable may not be well-founded.
Third, this study also highlights how meta-analyses often mislead us. Unfortunately, despite both good statistical and theoretical reasons for abandoning bivariate correlations and using standardized regression coefficients in meta-analyses, too many meta-analyses continue to naively rely on bivariate correlations. Doing so is, undoubtedly, a major source of misinformation. Few studies today simply rely on bivariate analyses, with multivariate analyses being typical. It is strange then that so many meta-analyses continue to rely on and crudely interpret bivariate correlations, leading to gross overconfidence in research hypotheses even when individual multivariate studies largely contradict them. This is likely due to both bad training and the lack of critical thinking typical to situations where people benefit from not understanding something. Meta-analyses of correlational studies in this and most other realms should cease using bivariate correlations and use standardized regression coefficients instead6.
Increasingly we can be sure that “correlation doesn’t equal causation” was the correct warning for social media effects studies and that, particularly with fairly standard control variables in place, we can’t even detect correlations.
Across studies, these things actually correlate very highly with each other suggesting they are all basically the same thing rather than distinct categories the way we tend to think of them, but more on this in a future post.
This is consistent with other research which has found, contrary to the popular narrative, the social media does not appear to negatively impact real-life friendships.
I asked ChatGPT to produce these simulated scatterplots.
Note, I mistyped .154 as .156 when I asked for the scatterplot. Because I am Scottish and cheap, and have only the “free” ChatGPT I soon ran out of images, mostly futilely trying to get a good image for this essay. A scatterplot of .154 would look pretty identical to .156 so I left it here. But just to explain that minor discrepancy.
I excluded this from the mean mentioned just above.
Of course, I’m well aware it’s evitable some ninny will use the bivariate correlations from this study in some future meta despite this study explicitly saying not to do that.