p24. Teens are about twice as likely to state that Instagram improves their mental health than to state that it harms it.
p28. Teens who were already unsatisfied with their mental health tend to think Instagram harms their mental health more than teens who are satisfied with their mental health. Still, teens who are not satisfied with their mental health (in the USA) are more likely to think Instagram is a positive influence on mental health than a negative one. This is not true in the UK.
If you're the kind of person who hates when statistics are abused for a narrative, this should be something you dislike. The "damning internal documents" actually show that 80% of teens have a neutral or positive reaction to Instagram, and teens are two times more likely to have a positive reaction than a negative one.
Zuckerberg's comments about Instagram having an overall positive effect on teen mental health look a lot less like an obvious lie from this angle.
My personal opinion/guess: Instagram and social media does deliver harmful content to some people and can have negative effects on people's mental health. However, in the discussion we've been having this week, every possible positive effect has been discounted to 0 value. That's not really fair, doesn't reflect the actual results of the leaked studies, and certainly doesn't match my anecdotal experience using social media (including fbook, reddit, hn and insta).
Of course... It has been taught in the most basic courses to design questionnaires since... forever... There is a whole science around that. Not only for mental health. Carefully identifying the best proxies and looking at the internal consistency of multiple answers and/or events/metrics is the intricate part of the design of the questionnaire. A simple/dumb example; you do not ask: "do you constantly read Hacker News?" but "have you heard <name some prevalent news in HN from today> of these news?"
Did you have a look at the slide deck? I don't design survey's for a living, but the questions seem carefully chosen (often included as a footnote on slides with graphs).
I just did; I was replying to your exact question in the comment. Questions like "Instagram can make me laugh when I am feeling down" do not look like the kind of questions that a good independently designed survey would do; in general, the would tend to be more of the kind that focus on a concrete event and their surroundings - it may look like they loose the perspective, you are looking at the only bad time that person had, but, done across a lot of people, it is more informative (would statistically represent the most likely state for the population, even if it is totally wrong for some individuals).
It's not showing me the 'reply' button on your comment.
That seems like fair criticism. It's not my area of expertise. I'm just here to post the source document because in light of what the study actually claims, the media reaction has been ridiculous.
Tangent: The “reply” link is hidden in some cases (fast replies) to avoid flamewars. If you insist on replying, clicking the date stamp to go the comment’s permalink will show a reply box.
You could use diagnostic tests, such as those described in the DSM. This is what psychologists and doctors do in real life to determine whether someone is depressed.
answer to your question then some editorializing on this thread:
answer: yes there are better ways to independently measure mental health than one self-report variable.
Generally in the social sciences you can measure things in a number of broad ways. Generally we refer to a 'construct' - a theoretical description of some phenomenon or set of concepts that is difficult or impossible to measure directly. Basically, it is a definition and the hard part is that you can't directly touch the thing so sometimes these 'variables' have definitions that get argued about in the margins - c.f., motivation, how do you define categorize and measure motivation?
Effectively, we are dealing with ways of evidencing things that are inferred.
There are a couple paradigms of measurement (more than the two I'll mention):
Observation - using some protocol or clear and as objective as possible tool for having a (typically) trained external individual. This can be either an overarching conclusion or observation of specific actions or symptoms:
Self report - relying on the person reporting things. This can be the constructs themselves (not great) or it can be more concrete markers of things (better but still used with caution. The issue here is sometimes lying, sometimes social norming, sometimes self-awareness - all represent sources of error.
Asking self report questions directly isn't inherently bad, but for something like mental health which is a latent construct, a single direct self report question is not going to get you accepted into any serious journal. You might ask 10 or 15 indirect, symptoms based questions, that is called a scale. For example you would ask: "how often over the last two weeks have you felt little interest or pleasure in doing things?" (from the PHQ-9, which implements the DSM-IV clinical criteria from depression).
personal opinion:
The question that the grandparent mentions is "In general, how has instagram affected the way you feel about your mental health?" Asking teenagers that is just...not great research praxis. The reason asking that is not great rather than bad is because it was part of a battery of data that they collected, all of which has to be interpreted together.
For a lot of reasons, likert scale questions like this are bad, particular bad with children (who will tell you what they think you want to here), and increasingly bad in a society where 'please rate my customer service a 10 because a 9 is failing' is the norm. In fact, most respected likert-like scales still see a bias in all answers above the norm - a distributional skew towards the 'good' option whether its agree or better or whatever the scale is. The results of this question alone are not nearly as positive as they are implied to be.
The next slides begin the absolute death for the claim about slide 24 ...
Slide 25 - explains that the positives come from memes and comedians. That drop to 'conversations you have' and 'comments you receive' absolutely wipes out the idea that instagram is a positive. The sorting of this group of categories makes me honestly just sad.
Slide 26 - "While the overall effects of instagram are positive [ed: that is not how I interpret this data], the effects are determined by the moment." I would encourage you to read the quotes.
Slide 27 - "Teens blame instagram for increase in the rates of anxiety and depression among teens" Take a look at the text of this one as well...kids aren't stupid
Slide 28 - "Teens who struggle with mental health say instagram makes it worse"
What collectively this means is - it amplifies. For developing children who already have unstable hormonal and neurological responses, that is like society giving an arsonist a lighter and me claiming its not my fault because I only gave him accelerant. The teens understand this and the data backs them up.
If your consultants bother to categorize the harm your company is doing pulling out one question and oversimplifying the results isn't an effective counter argument.
Not only is there a better way than a single self-report variable - this variable was just plucked completely out of context and a claim made about it that is unsupported.
Does Instagram, knowing this information, have any obligations when making choices that affect teen's mental health? Would making choices which negatively affect teen's mental health be an ethical action? One such choice could be segmenting user's by mental health for the purpose of advertisement. Is such a segmentation ethical? What if doing so increases Instagram's profit (say for example teens who struggle are more susceptible to ads), does that change your answer? Should that change Instagram's answer?
My point is that we are capable and obliged to dive deeply into these questions. 22% of teens[p24] of a total 22M teens(DAU)[1] are negatively affected by Instagram. This is 4.1 million people - not a small number. By stopping short and simply saying teens are twice as likely to benefit than suffer ignores the 4.1 million teens who are suffering. I'm not sure they take much solace that 8.2 million feel better off.
You're right and Facebook should be taking action to reduce the negative effects their products are having on teens.
I have no idea if any of the recommendations at the end of the study were actually put into action.
To me a charitable interpretation of the tone of this presentation, which focuses on highlighting negative outcomes rather than celebrating positive outcomes, was to identify areas for improvement. Again, interpreting charitably, this means the company is interested in identifying and reducing negative impacts.
Again, I don't know if they ever made any of these changes. I would like to see more transparency from the company.
Instagram certainly has an ethical obligation to act on this and improve the situation.
At the time that I left Instagram in early 2020, there were more people working on improving wellbeing, particularly for teens, than working on trying to improve engagement. I wasn't specifically tasked to work on well-being, but like all engineers working on recommendations I was required to work with wellbeing and completely address all well-being concerns before shipping anything.
Yes. I felt I was swimming against the currents and wanted this opinion to exist in separate space from the rest of the opinions I express on this site.
https://s.wsj.net/public/resources/documents/teen-mental-hea...
p24. Teens are about twice as likely to state that Instagram improves their mental health than to state that it harms it.
p28. Teens who were already unsatisfied with their mental health tend to think Instagram harms their mental health more than teens who are satisfied with their mental health. Still, teens who are not satisfied with their mental health (in the USA) are more likely to think Instagram is a positive influence on mental health than a negative one. This is not true in the UK.
If you're the kind of person who hates when statistics are abused for a narrative, this should be something you dislike. The "damning internal documents" actually show that 80% of teens have a neutral or positive reaction to Instagram, and teens are two times more likely to have a positive reaction than a negative one.
Zuckerberg's comments about Instagram having an overall positive effect on teen mental health look a lot less like an obvious lie from this angle.
My personal opinion/guess: Instagram and social media does deliver harmful content to some people and can have negative effects on people's mental health. However, in the discussion we've been having this week, every possible positive effect has been discounted to 0 value. That's not really fair, doesn't reflect the actual results of the leaked studies, and certainly doesn't match my anecdotal experience using social media (including fbook, reddit, hn and insta).