I think it's a left over from the eighties, as a response to the hippie areas romantic view of such drugs. I also believe that is why, for example, LSD is so stigmatized.
Imagine being a parent, never done LSD, not having a single clue what it can do and seeing your child becoming crazy, different, telling you things you can't comprehend and never had an desire to do so.
I can see why it is frightening to many 'normal' people who are just living their lives, wanting to have a house, family, job, holidays and perhaps partially overwhelmed by all of that already anyway.
I brought weed brownies to a birthday party for me and also for good friends to try if they wanted to. They live in 'rural' areas. Its common for them to drink alcohol, a lot. Drugs? no never no interest don't care.
While i already had ~5 year of experience with weed, they didn't even see it once. why would they? They have alcohol, it works for them, its common, easy and fun.
They had a really really hard time of even trying a brownie. They didn't even get why i would smoke weed if the effects are shallow. They didn't knew that you will not see halluzinations.
i think i'm over curious. This leads me to not being happy about what i have and looking for new. They are under curious and they are happy with that they have. Is that a problem? No its just different balanced.
I'd say a small dose of mushrooms (or LSD, but less experience there) is actually more likely to be a positive experience than weed edibles. Most people I know who have had bad experiences with drugs had it with edibles (or even just weed). Of course, due to the culture, weed is considered more "normal" in most countries than psychedelics, which is somewhat ironic in my experience.
To be clear, not implying that weed can't be wonderful or that you can't have a bad trip on psychedelics, just speaking from what I've experienced, seen and heard from others.
A psychedelic is in my opinion not an easy drug at all.
I don't think it is a normal experience for anyone, for the first time, to realize that what you see or what you believe can be influenced that easily.
While halos around light with lsd is nice to watch, its still a crazy thing. I never had visual impairment on that level from weed or alcohol.
Or when you see moving patterns on a wall, it does break one sence you as a human have a lot of trust for.
Our neighbor is an old lady who had cancer and apparently smells sometimes things which are not here. She doesn't want to be crazy and doesn't even accept that it can easily be nerve damage or whatever which makes her sometimes smell things which are not here.
Instead she thinks we and neighbors above her are trying to get her out of her flat. We are living in a very stable modern building, there is no realistic benefit of anyone to boot her out and walls are so thick, there is no noise issue at all. She still called the policies at least twice, had an expert here and was standing in our door twice angry and pissed of because we are creating that smell. It was 2 am in the morning and i was asleep.
Getting tired, not being able to walk well anymore, loosing concentration or forgetting things are apparently easy to understand and to accept as a human.