It's true that the retina and optic nerve do a lot of processing, but that doesn't make them part of the brain. The spinal cord does its own processing too (including reflexes), but it's not part of the brain either.
According to your logic, my USB-connected webcam is part of my computer's CPU (or at least motherboard). It isn't, even though it does a lot of processing on its own and probably has a CPU of its own inside.
Reflexes are simply a hardwired circuits because waiting for high order thought processing up to the brain in response to certain stimuli (pain) will end up you being rather dead or badly injured.
The retina has the same layered architecture of a scaffold of neurons and support cells as your cerebral cortex.
According to your logic, my USB-connected webcam is part of my computer's CPU (or at least motherboard). It isn't, even though it does a lot of processing on its own and probably has a CPU of its own inside.