No. That's generally because once something gets popular and jumps to the front page, it gets a huge boost in visibility, especially from people who weren't looking at the niche subreddit it comes from.

A lot of those people aren't interested in that content, so it will suddenly get an influx of downvotes.

Thanks for the reply.

I'm glad there is an explanation based on user behavior for this phenomenon because admin level vote tampering is such a tired theory.