Hello, Southern Italian here. I've never heard any bashing against North African olives or oil. Common sense says that olive is a Mediterranean tree and no particular Mediterranean region is better than all the others.
On the other side, the issue raised against non-EU olives is that they are not subject to EU quality regulations, and those regulations are considered quite strict.
Oil quality depends mostly on how it is produced; setting aside the possibility of contaminated product, olives from different places can only have a different taste.
