A late fee is a tool to encourage payment, if they ignore it and pay even just the invoice amount earlier than they otherwise would have you could say it has done its job.
As long as they pay the invoice, most people at that point will waive the late fee (providing it's not a very large sum of money as would be the case with the suggested 10% per week). I know I've done this in the past when a client has called me up after weeks of nagging and apologised for the late payment and the lack of a fee.
In addition, the late fee can come as a shock to the reluctant payer and spur an email expressing said shock. This is a great time to offer a waiver of the late fee if they pay by 5.
As long as they pay the invoice, most people at that point will waive the late fee (providing it's not a very large sum of money as would be the case with the suggested 10% per week). I know I've done this in the past when a client has called me up after weeks of nagging and apologised for the late payment and the lack of a fee.