I'm not sure pointer concepts are very difficult, pointers might confuse people that didn't grow up with assembly language, but they really aren't very hard to understand and they are also less and less relevant. Over the years the abstraction level increases, it's far better to teach people to continue computer science to the next 21st century level, not go over stuff from the 70's again that they will likely never use. The breakthroughs in CS coming very likely won't be written in C (or assembly for that matter).
You cannot write effective code in Java, Python, or C++ without understanding the difference between references and values. You can certainly stick to higher levels of abstractions - most functional languages do a good job of that, especially ones like Haskell that emphasize immutability and value-only semantics - but pointer concepts are absolutely relevant to the industry-standard languages.
I kind of feel like as long as we have imperative programming languages, pointer concepts will be useful. I mean, that's basically what references are.
