It is shit. Duplicate stories are arbitrarily allowed to be submitted (exact same article/url), while sometimes it actually works and prevents duplicates.
Oftentimes you see submissions for an interesting article languish in the 'new' section, but then a user re-submits the exact same thing ('duplicate detection' algo at work, lol) and it's on the front page in minutes. Whether that user is a 'power user', besties with dang, lucky, or ???..
> The algos is presumably a straight up string compare with some url decoding
Well, except it fails in some cases to straight up compare two strings that are identical, so there must be some other 'secret sauce' in the algorithm that makes it unnecessarily complicated and error-prone.
I suspect it's probably some mix of re-submitting affecting timestamps - I am not sure, which was why I asked. it's not terribly important.
I think the problem to be solved however is hard, and i don't know of any obvious solutions, so perhaps "shit" is probably too harsh.