True, given any reasonable well-behaved notion of "set", there is a category Set of all sets. (Its properties will depend on the specific notion of "set" chosen.)
But this observation isn't really relevant to issues of formalization. Mathematics can be formalized in lots of systems, set theory being one, type theory being another, and category theory (more specifically ETCS) providing yet another alternative.
All of these foundations are interesting, enjoy unique features, and are interrelated. In particular, ETCS and set theory are very similar from the point of view of foundations: Basically, ordinary set theory and ETCS are two different flavors of set theory: ordinary set theory rests upon a global membership predicate ∈, while ETCS rests upon the notion of a morphism.
Closest to mathematical practice is probably type theory. (But keep in mind that the points above still stand, and that most mathematicians work informally, not knowing any details about set theory, type theory or ETCS.)
