Sure, and in theory that should be the prime example of "free market" at work - merchant doesn't want to accept visa/mastercard? They will lose customers then, if that's acceptable to them then that's their choice.
This would make sense if (and only if) all other things are equal. In reality it is slightly more complicated than that.
The second-largest chain, Jumbo, will happily accept my AMEX credit card. In many cases, the groceries are even (slightly) cheaper there. That's why it has been my preferred place to do the groceries for years. But I find my self goes to ah much more often than Jumbo recently, simply because I moved and there is one right across the street.
Same if they pass on the card costs to the customers though - card user doesn't want to pay the true cost of using their card, they can choose to shop elsewhere.
