I have to plug Hypothes.is [1] who are doing this in an open source and distributed fashion. They have quite a team and a lot of grant money to make it happen.
Hey whoever's building this -- please give me a shout. christianpbrink at gmail. If you haven't built it yet, maybe you don't need to, as I already have and may be interested in a partnership.
[1] http://hypothes.is/