> [BitTorrent] isn't designed to sequentially stream data
We’re working on improving the algorithm to switch back to a rarest-first strategy when there is not a high-priority need for specific pieces. In other words, when sufficient video is buffered, there’s no need to deviate from the normal piece selection algorithm.
But the fact is that with the speed of today’s internet connections, the user is going to finish fully downloading the torrent in a fraction of the time it takes to view it, so they will still spend more time seeding than downloading.
In practice, the only time that the rarest-first algorithm is important is on poorly-seeded torrents, or in the first few hours of a torrent being published when the ratio of seeders to leechers is really bad. I plan to keep improving the piece selection algorithm so that WebTorrent can be a good citizen.
Also: you should note that not all WebTorrent users stream sequentially. That's just one option for downloading the data.
Also: It's noteworthy that BitTorrent Inc.'s official torrent client (as well as the largest player by marketshare), uTorrent, offers sequential downloading, as well as selective file downloading. And the BitTorrent network remains very healthy.
> why they didn't take this opportunity to design a protocol that has more suitable trade-offs than BitTorrent
BitTorrent is the most successful, most widely-deployed P2P protocol in existence. It works really well. My goal with WebTorrent was to bring BitTorrent to the web in a way that interoperates with the existing torrent network.
Re-inventing the protocol would have made WebTorrent fundamentally incompatible with existing clients and prevented adoption. The way we've done it is better. The wire protocol is exactly the same, but there's now a new way to connect to peers: WebRTC, in addition to the existing TCP and uTP.
Also, re-inventing the protocol is a huge rabbit hole. There was already a lot of risk when I started the project -- will WebRTC get adopted by all the browser vendors? Will data channel stabilize and be performant? Is JavaScript fast enough to re-package MP4 videos on-the-fly for streaming playback with the MediaSource API? My thinking was: Why add inventing a new wire protocol and several algorithms to the table?
Thanks for your thoughtful comment. Hope you'll give WebTorrent and our new desktop app, WebTorrent Desktop a try!
>Also: It's noteworthy that BitTorrent Inc.'s official torrent client (as well as the largest player by marketshare), uTorrent, offers sequential downloading
Its worth mentioning that the option is in a hidden menu
That sounds like you prioritized implementing streaming first over being a good citizen.
> Also: It's noteworthy that BitTorrent Inc.'s official torrent client (as well as the largest player by marketshare), uTorrent, offers sequential downloading, a
To my knowledge that is only available if the swarm condition allows and is not purely sequential. But that is second-hand knowledge, so I may be wrong. But either way, the default is rarest-first.
There's a difference between an industry-wide proof of concept and a personal one. If I was making a text editor, I would begin by focusing on making a proof of concept that I could accomplish the features that I wanted in the way that I wanted to do it -- it wouldn't help much to say "emacs has that feature so that's fine".
These long term details are implemented by all established bittorrent clients. I would bet that version 0.1alpha of many of them did not, but were rather in a state of "holy moly this works! I should go show HN".
Webtorrent is 2 years old and seems to have several active contributors, do you really think the "0.1 prototype" argument applies here?
Not to mention we're not talking about some optional, nice-to-have feature here, we're talking about a core aspect of bittorrent which gives it robustness.
Yep! I didn't address your other argument because I accept it and there's nothing about it I disagree with. I don't disagree with any of what you just said, either.
I just wanted to point out that "streaming over web-torrents" is the feature being demo'd here, which means that (a) it's a new feature (I assume?) to this project / these developers, and (b) it's clearly something they feel is a nice-to-have feature, because they not only chose to spend time making it, but also announced to HN when they had a working PoC. If people never posted something to HN until they were "100% complete", I think this place would be a lot less interesting than it is.
WebTorrent supports both sequential and rarest-first downloads. Just pass the right option to the webtorrent library. The client.add() method takes a 'strategy' option that can be set to 'sequential' or 'rarest'.
Feel free to open a GitHub issue if you have suggestions for how we can do better.
> [BitTorrent] isn't designed to sequentially stream data
We’re working on improving the algorithm to switch back to a rarest-first strategy when there is not a high-priority need for specific pieces. In other words, when sufficient video is buffered, there’s no need to deviate from the normal piece selection algorithm.
But the fact is that with the speed of today’s internet connections, the user is going to finish fully downloading the torrent in a fraction of the time it takes to view it, so they will still spend more time seeding than downloading.
In practice, the only time that the rarest-first algorithm is important is on poorly-seeded torrents, or in the first few hours of a torrent being published when the ratio of seeders to leechers is really bad. I plan to keep improving the piece selection algorithm so that WebTorrent can be a good citizen.
Also: you should note that not all WebTorrent users stream sequentially. That's just one option for downloading the data.
Also: It's noteworthy that BitTorrent Inc.'s official torrent client (as well as the largest player by marketshare), uTorrent, offers sequential downloading, as well as selective file downloading. And the BitTorrent network remains very healthy.
> why they didn't take this opportunity to design a protocol that has more suitable trade-offs than BitTorrent
BitTorrent is the most successful, most widely-deployed P2P protocol in existence. It works really well. My goal with WebTorrent was to bring BitTorrent to the web in a way that interoperates with the existing torrent network.
Re-inventing the protocol would have made WebTorrent fundamentally incompatible with existing clients and prevented adoption. The way we've done it is better. The wire protocol is exactly the same, but there's now a new way to connect to peers: WebRTC, in addition to the existing TCP and uTP.
Also, re-inventing the protocol is a huge rabbit hole. There was already a lot of risk when I started the project -- will WebRTC get adopted by all the browser vendors? Will data channel stabilize and be performant? Is JavaScript fast enough to re-package MP4 videos on-the-fly for streaming playback with the MediaSource API? My thinking was: Why add inventing a new wire protocol and several algorithms to the table?
Thanks for your thoughtful comment. Hope you'll give WebTorrent and our new desktop app, WebTorrent Desktop a try!