Hacker News new | past | comments | ask | show | jobs | submit | Nyan's comments login

You should at least be benchmarking against par2cmdline-turbo instead (stock par2cmdline isn't exactly performance-oriented). Also, you need to list the parameters used as they significantly impact performance of PAR2.

Your benchmark also doesn't list the redundancy %, as well as how resilient it is against corruption.

One thing I note is that both ISA-L and zfec use GF8, whilst PAR2 uses GF16. The latter is around twice as slow to compute, but allows for significantly more blocks/shards.


Got it, I'll add par2cmdline-turbo (I didn't know it existed) to the list along with those key details. Likewise, I'll also go and describe the benefits and downsides of each tool in more detail. I'll get around to it when I release the next version soon that fixes some of the problems described in this thread.


Thanks for doing that.

> par2cmdline[-turbo] encode: par2 c -r25 test

That command is rather unfair to PAR2 - you should add `-b48 -n2 -u` to make the comparison fairer.

PAR2 ain't exactly fast, particularly compared to GF8 focused formats, but the numbers you initially gave seemed wildly off, so I suspected the comparison wasn't really fair.

Ideally you should also be including the version of each tool used.


I've updated the benchmarks to include those flags. I've also specified the versions of all software involved.

It seems par2 is significantly faster with those options set than without, as in by an order of magnitude, it seems par2 struggles greatly with the large number of blocks that it sets by default. Thank you for telling me.


Nice work - thanks for your efforts!

Yeah, the compute complexity for Reed Solomon is generally O(size_of_input * number_of_recovery_blocks)

If you don't specify the number of blocks, par2cmdline defaults to 2000, so at 25%, it's generating 500 parity blocks, which is obviously much slower than what you're generating with the other tools.

Having said that, PAR2 is generally aimed at use cases with hundreds/thousands of parity blocks, so it's going to be at a disadvantage if you're generating less than 10 - which your benchmark shows.


We've already got JPEG-XL as a good replacement format, but some feel that jeopardizing its adoption is a sensible cause.


Why not use fpnge?


It's a Java system, so not quite so simple. Maybe it's worthwhile to create some Java bindings? Recent JDKs make it feasible to swap out the underlying zlib implementation, so swapping out zlib-madler with zlib-cloudflare or zlib-ng might provide the best cost/benefit.


Someone made this: https://github.com/manticore-projects/fpng-java

Replacing zlib might give you a few percentage points' worth of difference, whilst fpnge would likely be several times faster.


Very cool, thanks for the pointer! We might be able to run an internal test to check performance vs. a zlib replacement, but I think that AGPL license is going to be a showstopper for anything else...


Many older formats, ZIP included, didn't standardise on an encoding, so they just use whatever the local 8-bit encoding happens to be. And if your system charset is different from the ZIP creator's, you could get messed up filenames.

This is just another cost of sticking to legacy formats.


Hopefully that slowly stops being the standard: https://github.com/animetosho/Nyuu/wiki/Stop-RAR-Uploads


Probably not, if the releases come from scene groups they follow the standard.


If you search up guides on how to upload to Usenet, you'll often find most of the recommend creating split RARs.

Uploaders who just transfer scene releases without bothering to extract will of course stick to RAR, but there's a lot of content these days that isn't from scene groups.


Your experience may be slightly out of date.

Fansubbers these days are a tiny shadow of that they used to be. The vast majority of anime piracy these days consists of ripping official streams, which include subtitles.

Of course, if you're into older shows, you'll find many to be missing on the mainstream streaming platforms, and will often need to resort to piracy and fansubs to get that. But as anime becomes more mainstream, those interested in older stuff become a smaller slice of the market.

Also, due to more mainstream adoption, those interested in "cleaner rips, better encoding, etc" are dwindling. Whilst these people are still out there, you'll find the vast majority of people are perfectly happy with low quality streaming sites (also worth pointing out that these streams still offer vastly better video quality than what you typically got in the 90s).

Enforcement wise, I doubt they're targeting the hardcore fans - they're going after the mainstream, which is a big slice of the market.


Your experiences are absolutely out of date.

For example, the recent (6th season) of JoJo, are all sorts of references that the fansubbers can make from the original material, but the corporate subbers aren't legally able to.

In fact, the corpo-subs are considered degraded and garbage, compared to a fansub. There aren't 'sensibilities' to maintain, nor is there corporate rights etc to hold up. And frankly, we make better!

If you want a comparison between corporate and fansubs, look no further than the book and movie of Ready Player One. That's a fair comparison.


To be fair, this has always been pretty evident. Fansubs quality are higher because they embrace the anime-watchers culture, rather than being a translation in the traditional way. Starts with -chun -kun and friends, but over time there are many more words that should not be translated.

And the fansubbers subs quality for some are incredible, integrating the sub in the anime itself (text over a sign and similar things, sometimes different coloring to easily identify characters and such.

Very passionate about anime, but lately struggling to get hooked, I have 2 children that are young, so what I can watch without traumatizing them is limited (had to watch Made in Abyss late at night)


If you think honorifics matter, this means only one thing: you don't speak Japanese.

They don't matter and are not what you're missing by reading a translation. (Similarly, it's not possible that a word "should not be translated". Words are just part of sentences.)

All the good old fansub translators have moved on simply by being old enough to need real jobs, but a lot of them are professional anime translators now.


Of course I don't speak japanese, I'm talking about fansubs, here, the purpose of those is to digest the media without knowing the language.

I definitely do not speak japanese, which is why it's important to me, some language joke literally don't transmit. In "Shakugan no Shana" there is literally a scene where the protagonist drops honorific and Shana talks about it. That section would need to be cut, or the joke would just not come through because I would have no idea what an honorific is. "Senpai" also has no direct translation in my mother language and it's a figure that doesn't really exist, there is an episode of Full Metal Panic that's around a character that's important to Kaname and it wouldn't come through the "why" if without knowing the meaning of the word. But the word just doesn't exist in Italian, so it was translated with a made-up sound "senpaia" in the official dub. Result was that nobody (we were teenagers at the time) had any idea of what it meant or why this was important.

There are various examples spread everywhere for this.

Teaching some words that do not translate is important, to convey some cultural aspects of japan that give context to the anime.

Which incidentally is the reason why I don't watch dubbed animes, because the voice acting is different. This might sound like a joke, but the high pitched screams of some female characters in anime just don't translate to my mother language and indeed when dubbed, the scream was mild at best.


Generally the translator will find a way to work around the lack of a direct equivalent, trying to convey as much of the meaning as possible.

Now you may think that these workarounds cause some meaning loss, and you'd be correct. However, the problem with this line of thinking is that this isn't unique to honorifics - there's a tonne of other stuff in the Japanese language which has no English equivalent (I don't know Italian, so can't say much on that front), and non-Japanese speaking anime fans are often unaware of these because fansubs never introduced those concepts (for example, purpose of -desu/-masu which can be used to show respect, somewhat like honorifics).

The reality is that a lot of nuance gets lost in translation, with or without honorifics; unfortunately, if nuance is important to you, there's no substitute to learning the language. As such, people who think honorifics are highly important generally give me the impression of blissful ignorance (i.e. don't know what they don't know).

Of course, you're welcome to have your own preferences regarding translations. But having some hybrid of Japanese/English subtitle to portray slightly more nuance (whilst still missing out on a lot), can seem weird (e.g. why is this particular aspect exposed, but not something else?) and isn't necessarily objectively "better" than a translation that just sticks to English.


Yeah, a translation is always inaccurate because it's not in the original language. This is unavoidable, but the fact that you can hear some of it is missing does not mean that part is in any way what's changed the most in the translation.

As a related issue, Japanese has a lot of English words (wasei-eigo) and they /all/ mean completely different things than in English. But people wanting subtitles to sound like what's spoken causes translators to leave them in even though it makes the meaning wrong.

(When someone buys a "juice" from a vending machine they're buying a soft drink. Don't even ask what "feminist" means.)


Unfortunately I disagree. Funny enough on Shakugan no Shana, there is a character that says -desu a lot and that not being translated and also explained to me when it first appeared, provided the ground to figure out a bit of the background. She also "expands" -desu in various ways, becoming essentially

I watched many animes in japanese with fansubs, as well as with translation, there is just too much lost and the professional subtitles applied did not provide enough context, like at all (in dvds you could watch italian dubs with subtitles).

Of course it could be a problem with the studio, but the experience has been pretty consistent.

The panorama might also have changed, who knows.

Oh, one glaring example is watching animes with subs on netflix was a terrible experience both in visual quality and subs, while the exact same anime from fansubs are really high quality, with amazing subs (all the stuff I described earlier), it just doesn't work.


> Funny enough on Shakugan no Shana, there is a character that says -desu a lot and that not being translated and also explained to me when it first appeared, provided the ground to figure out a bit of the background.

That's a joke. You shouldn't literally translate jokes because they won't funny; explaining jokes ruins them. The purpose of watching a TV show is to be entertained, not for you to learn grammar quirks only used by anime girls, so they should put new jokes there. You have not learned any useful information about Japan or their language by observing someone says desu a lot.

An amusing case when translating for people like this is you often have to translate honorifics into other honorifics, because they don't actually know all of them but just get upset when they don't see the few they do know. So -dono -me might get left out or shinobi/bushi changed to ninja/samurai.


No, -desu was explained at the beginning of the episode, you have a whole 20 minutes (or more than 1 episode, I can't remember) to get to that point, so the joke worked perfectly fine as it was originally


GP is talking about the numerical popularity of fansubs versus corpo-subs, not their quality. Fansubs have always had freedom and going-above-and-beyond as their strong point. They've just lost their relative share of the audience (barring the occasional exception). Doesn't mean some subbers won't continue to put out labors of love.

Also, newer people getting into the medium are mostly only used to consuming media via official streaming sites, so there's simply less awareness of fansubs versus the 2000s when fansubs were the only way to get Naruto online.


Another example: Just a few weeks ago somebody released a restoration (painstakingly recreated frame-for-frame from bluray source) of Nadia: The Nautilus Story. This is a substantially cut-down version of Nadia: The Secret of Blue Water that removes a ton of filler as well as many widely hated episodes (island arc). Originally it existed only on Laserdisc, with very poor quality audio and video and no subtitles. A high quality copy of this never existed until last month. No corporation ever would have done this.


That sounds amazing! There are a bunch of animes that are unwatchable if you don't remove all the fillers. Or in case of naruto, all fillers and flashbacks, lol


Nadia: The Secret of Blue Water is one of my all time favorite anime. While I think that many criticisms of the island arc are valid, I don't think it should detract anyone from watching it from start to finish and I wouldn't recommend skipping it, either.

While I'm happy that someone went to the trouble of recreating a different cut of the show based on official work, I don't know that it's really even necessary, just as I rarely find those sorts of efforts (which happen somewhat frequently in anime, where they condense tv shows into movies) to be very good.


Yeah, I'm not sure I would recommend somebody watch The Nautilus Story first, unless maybe they couldn't find the time or motivation to watch the full thing. Nonetheless I think The Nautilus Story is pretty cool for what it is, and perhaps even ideal for somebody who wants to rewatch Nadia but doesn't want to rewatch all of it.

For reference, The Nautilus Story clocks in at 5h30m, while the full show is something like 20 hours. So TNS is a substantial streamlining, removing a lot more than just the island episodes. But this might be what some people want.


disclaimer: lapsed typesetter/scanlator/vn patcher

Your and the parent's statements are actually congruent: they're not saying ripsubs aren't shit, which they are, but that they're good enough to minimize the relevance of fansubbing to the mass market, which they are.

I mean, just look at the output of prolific fansubbing groups from ~2010 to ~2020. At the start of that window, groups like FFF/Underwater/gg/commie/Vivid (and many more I'm forgetting) would race to release subs for airing eps and would frequently have them out within a day or so of the rips. Now, like... maybe three, generously? shows a season get that treatment, and a lot of the people from the scene have moved on.

I believe fansubbing still has immense value, but effort is obviously shifting towards making really high quality releases of older stuff rather than racing to sub airing shows (not to imply that the former didn't happen beforehand; it did, but was a smaller slice of the overall scene). And even though I never worked on airing shows, I get it: why kill yourself to release a better version ASAP if 99% of schmucks will just grab Horrible instead?

e: s/high-tier/prolific before someone goes in on me for gg/commie


Within a day is understating it a bit; in gg’s heyday we got some episodes out within hours of them airing and occasionally beat out the public raws.

I think for most of us fansubbing stopped being about the anime at some point along the way - I personally kept subbing for a few years after I last watched a show, and then continued to develop Aegisub for a few more years after that - but basically everyone got into it because we wanted to make it so that shows could be watched. The first few things I worked on would simply not have subs at all if I didn’t step up and do the work, and I can’t imagine I every would have gotten into subbing if that wasn’t the case.

There’s still room for better-than-the-official-release subs, but the simple fact is that most fansubs never were that, and most viewers never wanted that in the first place. There’s no longer a reason for kinda slow subs translated by a college student who’s taken a few years of Japanese classes to exist, and that’s what most of the scene was.


>I get it: why kill yourself to release a better version ASAP if 99% of schmucks will just grab Horrible instead?

Yeah the racing was silly. The subs themselves weren't necessarily that much better either, which is why I think rips like Horriblesubs won out in the end for most people. I loved fansubs and the community around it (and the drama).

Though I think the scene dying is less of a mix of people moving on and more the internet changing (and the medium subjectively getting worse). There's more shows out now than ever before, but relatively less interest for most of them.


> Fansubbers these days are a tiny shadow of that they used to be. The vast majority of anime piracy these days consists of ripping official streams, which include subtitles.

Yep, like another commenter says, the scene got Crunchyrolled 2010-2012ish and many subbers retired.

It's funny (frankly outright amazing) how Crunchyroll, originally a major pirate streaming site, managed to cross the chasm by striking legal agreements with content owners, and for a large amount of content to boot. It might have been possible only for a small window (before the real rise of global online media platforms), during which the owners wanted to gain the global online audience and didn't have as much familiarity with building online international distribution channels. Can't imagine moves like that happening any more in our climate.


The truly ironic thing is how the cycle is now complete. Crunchyroll is now owned by Sony. Sony also owns Aniplex, a major anime distributor, and A1 Pictures, its animation subsidiary that produces a lot of popular anime. The former pirate site is now owned by the company that produces the work!


Just according to k--plan. A Japanese media conglomerate wanted vertical integration with a global distribution channel, so they went out and got it. They would probably have done it earlier; they just never had a suitable paid platform to acquire before (and possibly tried to build their own unsuccessfully).


Fansubbers have been making a very large resurgence lately because of "localization" being introduced to anime, which has been inserting everything from out of date memes, politics, and very poor translations.

The translations can be so poor at times, it may invert the meaning of the joke or phrase.


I follow the latest stuff but haven't seen anything I'd consider to be "a very large resurgence".

Personal opinion: whilst bad official translations certainly exist, most of the time, they're fine. There's usually stuff that could be improved (for example, songs often don't get included, typesetting etc), but as far as portraying the meaning, they generally work. And this is likely a key reason why fansubs died - the official subs are often in the "good enough" category.


A possibility might be that the bigger sites have paid more attention to opsec, and are harder to take down, whilst smaller players may be easier to intimidate.

Current Nyaa (nyaa.si) has been targeted [https://torrentfreak.com/mpa-lawyers-are-trying-to-shut-down...], but efforts haven't been successful so far.

> I guess because these are commercial?

Highly unlikely - it's not like rights holders get anything regardless of commercial status. Also, Nyaa runs ads on their NSFW site, so it's commercial in that respect.


> It's jumped several domains in the past

One thing to note is that there's actually two Nyaa sites, so it's important to not confuse the two.

The original jumped several domains, and closed down in May 2017 (last domain was nyaa.se). The "current Nyaa", nyaa.si, is a clone site that launched shortly after, and hasn't changed any domains.

With the original, I recall the domains jumped were linked to various legislative changes in Europe, with the shutdown in 2017 likely due to similar reasons.


Which one is nyaa-dot-eu?


From memory, the original Nyaa was nyaatorrents.org -> nyaa.eu -> nyaa.se


Very useful. In fact, it speeds up a single instance (i.e. not taking advantage of SIMD) of MD5 by 20%: https://github.com/animetosho/md5-optimisation#x86-avx512-vl...


Faster PNG en/decoders exist though, it's probably just libpng being the most commonly used, but not extensively optimised.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: