Hacker News new | past | comments | ask | show | jobs | submit login

I can’t help but notice you have not explained how to handle progress reporting in gnu parallel.

Also do you really use parallel from software trying to parallelise its internal workload? Note that in tfa the workers are an implementation detail of a wider program.




> you have not explained how to handle progress reporting in gnu parallel.

`parallel --eta` provides progress reporting over all tasks based upon observed completions.

> in tfa the workers are an implementation detail of a wider program.

TFA could write the workers as standalone processes then subprocess out to GNU Parallel.

At that point, one might not even want the Python outer process in this article. I often write little shell wrappers that establish non-trivial GNU Parallel or Make invocations under the hood. Generally, leaning on common workhorses for multiprocessing seems more sensible than rolling my own one-off.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: