As long as you're just "responding to HTTP requests", there's nothing to worry about and the GDPR does not apply.
It's when you start collecting personal data on EU residents, send their personal data to third parties for analytics/targeted advertising, and so on, that things get interesting.
I can't think of any web server that doesn't log ip addresses by default, and I think it's been established that satisfies the GDPR threshold test for personal data. So while what you say is true, I think you're being a little bit deceptive when you say 'As long as you're just "responding to HTTP requests"' because all practical and established means of doing that violate the GDPR by default.
Ip used for technical reasons such as logging access are not concerned by the gdpr per se. The same goes to KYC informations.
The gdpr is actually a well written piece of legislation which should worry you only if you do shady stuff.
The only edge case that I know are not well addressed concerns the status of encrypted data (would be deleting private keys considered to be deleting them? This question is important on blockchain data storage)
If you log for security purposes that is a "legitimate interest" which would allow you to keep doing that, provided:
- You make a note that this data is being logged.
- You state for how long this is logged (6 months is reasonable), and justify that time frame.
- You state who else has access to these logs.
- You state what steps you have taken to try to minimize unauthorized access to these logs.
- In a register (these statements should be delivered on request of a law supervisor) you also provide your personal details, which users are affected by this data processing, and your goal (which should be something along the lines of: "fraud prevention and intrusion mitigation" to have legitimate interest. Expect big companies with law firms to push this "security interest"-angle hard, as they try to justify their data processing).
Pretty reasonable, no? It would be nice if the large web logging softwares provide standard options to automatically limit disclosure of PII web logs.
You already described far more work than I'm willing to do for the small web site I happen to host. If there's a simple geoblocking switch I'd much rather flip the switch and block Europe than continuously worry that I didn't dot every 'i' and cross every 't' to make some obscure European regulator happy.
Post process the bloody logfile on a regular basis. You can legitimately capture IPs etc for a while, provided that it is for a good reason - fixing a problem, or gathering aggregate trends but not for direct marketing reasons against individuals who have not given consent. So, your logrotate.d/ might get a few post/pre rotate scripts, if it really bothers you.
I run a small UK based IT firm. So far I've turned down some of the logging on my HA Proxy instances and stopped logging IPs and user agents in general and a few other things. If I need to do some diags then I'll turn them on again. That's on the long term stored logs (due to backups). So far, my backups are smaller 8)
I do keep very detailed logs with IPs (actually full packet capture) in the ES cluster for IDS purposes but those are turned over (deleted) within a few hours. Less detailed logs last a lot longer.
It's when you start collecting personal data on EU residents, send their personal data to third parties for analytics/targeted advertising, and so on, that things get interesting.