The force will be perpendicular to the surface light reflects from, not perpendicular to the light arriving at the surface. Therefore if the sail was mounted at a 45 degree angle to the Sun, there would be a force that reduces angular momentum and puts the object in a spiral towards the Sun.