The popular psilocybin mushroom types are all saprophytic, they grow on decomposing plant matter or dung. Truffle fungi are mycorrhizal, they form complex symbioses with trees, exchanging water and nutrients with them.
Not only does that make them harder to grow, it also makes them a lot more chill about producing fruiting bodies (a.k.a the actual mushroom caps that we eat). Saprophytic fungi need to move on once the matter they grow on runs out of nutrients. Mycorrhizal fungi can live comfortably for decades with their host tree and only produce fruiting bodies when the conditions are optimal.
Some fungi have very complex relationships with other species before they will produce fruiting bodies and it isn't easy to figure out exactly what they need to thrive.
