To answer these questions one would need to begin with rigorous definitions of 'semantics', 'context', 'memory', 'complex' etc. What we know is that attention models are heavily trained associative memories. Anecdotal perceptions are deceiving, as humans we are often prone to see human/animal-like behavior where it doesnt exist (e.g. cgi/games).