It is easy to make a torrent client, but very hard to make a good torrent client. A very good, or let's say "perfect", one, needs to support multiple transport protocols (TCP, "uTP" aka UDP, "WebTorrent" aka WebRTC), multiple discovery mechanisms (DHT, PEX, HTTP trackers, WebSocket trackers), multiple torrent formats (v1, v2, hybrid), should use the network optimally (max the speed without overloading the network - IIRC some clients measure average packet latency and if it starts going up, put some backpressure), resolve magnet URLs, set up port forwarding, reconfigure firewalls, offer API for *arr stack, be a good netizen (report stats correctly, send packets within the specs, do not spam - otherwise other clients will blocklist your in their code or config), implement many BEPs (mutable torrents are cool), be able to recover from interrupted state based on only the data that's on disk, have configurable downloading order (people want to start playing videos before they finish downloading, so you may want to e.g. download header and footer of each file first), and ideally detect duplicates between torrents (cross-seeding). And then there will be people throwing 2TB+ torrents at it (e.g. TLMC) to benchmark it, and saying your client is "literally unusable" if it doesn't handle it.
So, building a "perfect" torrent client from the ground up is a daunting task. But the "good" news is that nobody built such a "perfect" client just yet, so if you have some spare months of your time, you can take a shot at it. Or even better yet, open the issue tracker for one of popular clients or libraries, and add one of the missing features from the list above.
Yeah. I was working on [0] for a while (I might get back to it when I have more time) and implemented a chunk of the protocol with an eye for a more scalable libtorrent replacement, and it's not entirely trivial. Even things like the piece picker get trickier at scale.
My client anacrolix/torrent has most of that, including streaming, hybrid, v2, all trackers and protocols.
I don't do much port forwarding, it's too fiddly. I think I provide upnp out of the box and few other variants that were contributed.
I also have all the DHT extensions, like mutable torrents and get/put etc.
Large torrents do pop up occasionally, but it's been a long time since someone found a performance issue there but it used to happen every few months. Lots of optimisation.
NB: especially after https://github.com/Luminarys/synapse died and Transmission "botched" its 4.x rewrite (currently on 3 with some backported patches).
I know your library does a lot, I have been your GH Sponsor for months ;) Thanks for reminding me about it, I wanted to up the amount. And to thank you for working on such cool stuff and making the ecosystem better.
I missed the upnp support, thanks for correcting my beliefs.
Which client with WebUI would you recommend the most? I tried using exatorrent and distribyted, but was running into bugs.
Also, good disk I/O scheduler needed, or you can thrash even stripped NVMe disks, not to mention spinning rust.
Good multi threading without any global locks is a must or your UI/remote control become unresponsive on big number of active torrents.
Also, it needs daemon mode, which can be compiled & installed without any GUI libs (like QT, GTK, Waynald, libx11, anything like this) and fast responsive remote GUI for all platforms which allows to manage thousands of torrents simultaneously (I'm looking at you, <any-existing-torrent-client>). It could be Web UI, but for now I didn't se usable one. Only thing that semi-works is Transmission + Transmission-Remote-GUI written in Pascal. But it is effectively abadonware and works worse and worse with each Transmission update.
And be scriptable to allow automate change of torrent on tracker, for example (you need to detect it by "Torrent not registered" tracker-specific error answer and re-download torrent building URL from, for example, "comment" field of torrent, again in tracker-specific way).
I am using qBittorrent, and from the top of my head:
- WebTorrent and WebSocket patch for qBittorrent is ready but not merged (waiting on libtorrent's decision),
- cross-seeding support is poor (a separate "cross-seed" binary can be used to set up hardlinks to fool qBittorrent into cross-seeding, but it cannot detect duplicates on its own)
- when it comes to torrent management, there is no way to group torrents into groups with common settings (important if you use multiple private trackers) - people recommend having multiple installations of qBittorrent side by side
- when it comes to reconfiguring NAT and firewall, qBittorrent supports UPNP IGD protoocl, but I am not sure about NAT-PMP and PCP
- I have never seen qBittorrent connect to a single IPv6 peer - so I don't know if the support is there
- download order - you can choose "download in order" or "download rarest first". I dont think "download in order" downloads footers, so mp4 files won't work (IIRC mp4 store metadata in footer, mkv in header)
> - when it comes to torrent management, there is no way to group torrents into groups with common settings (important if you use multiple private trackers) - people recommend having multiple installations of qBittorrent side by side
Probably easiest to do this via docker-qbittorrent-nox.
> - I have never seen qBittorrent connect to a single IPv6 peer - so I don't know if the support is there
The Linux ISOs (not a euphemism) I'm seeding probably get about a third of their peer connections via IPv6.
See my other comments, but my client fully supports Webtorrent and webtrackers.
Not much support for port forwarding.
IPv6 definitely works.
Download order is handled by providing readers directly into torrent data and using that for prioritization. So basically request what you need when you need it. No arbitrary list of algorithms.
Personally, "being as small as uTorrent used to be", but clearly that's not a deal-breaker. (Then again, neither is like two thirds of the stuff GP mentioned!)
To be fair, circa 2010 all torrent clients got "good enough", so my wishlist above is just a wishlist (because torrents are cool and I like to see progress). I use (and sponsor) qBittorrent myself and have no plans to change.
Tixati is simple in the same way Deluge is simple. Because of this they both lack a lot of the quality of life things out of the box that qBittorrent provides.
Alternatively, a minimal client with few features that can seed large numbers of torrents without crashing or freezing. Settings for how many it may seed at a time, prioritize that what needs seeding, ignore overly seeded things. Options to seed only blocks no one has. Remotely controlled.
I had a super seeder implementation that seeded over 400k torrents simultaneously using some fancy callbacks at certain points in the protocol.
If this is actually a common use case, I can resurrect it into a usable form for the public if there's interest or funding. https://github.com/anacrolix/torrent
> So, building a "perfect" torrent client from the ground up is a daunting task.
Don’t you think you’re exaggerating a bit? It’s not daunting by any stretch. The feature set you described is fairly straightforward and something even a beginner developer could tackle without too much hassle.
I’m honestly kind of tired of seeing people act like doing anything these days is some impossible feat.
Try it yourself, from scratch, and see how quickly you will finish something that will rival and surpass, say, qBittorrent.
None of the tasks is breaking new ground or is creatively difficult. But there's a metric ton of those tasks in order to make a truly good torrent client.
There's only a handful of actually complete ones. A few it's surprising they even work, they're crazy complicated, and weird, but once they're stable they need very little maintenance. They're all also written in very different styles due to a mix of C/C++ and very high concurrency.
But shouldn’t be hard to do with modern facilities (async runtimes, streaming libraries etc.) I don’t enjoy many clients yet I use them.
It’s definitely a good problem with which to test a programming language. Many fail on parsing, or inefficient network interfaces, inefficient file interfaces, sha sum unusable without intrinsics, etc.
So, building a "perfect" torrent client from the ground up is a daunting task. But the "good" news is that nobody built such a "perfect" client just yet, so if you have some spare months of your time, you can take a shot at it. Or even better yet, open the issue tracker for one of popular clients or libraries, and add one of the missing features from the list above.