Hacker News new | past | comments | ask | show | jobs | submit login
Imagor: A fast, Docker-ready image processing server written in Go (github.com/cshum)
174 points by thunderbong on Dec 10, 2021 | hide | past | favorite | 27 comments



Author here. Apologize for the late response, as I only realized it was on HN a few days after. Thanks for posting it!

A bit of our background: we have been a long time user of Thumbor, with millions of images serving under it and applications built around the Thumbor syntax. However Thumbor is not quite in active development anymore. Support is getting stale, and its stable version is still using Python 2.

So we started to look to alternatives. We wanted to keep the Thumbor syntax as it is a pretty good one, and we want no disruption to our existing image urls being created, and applications that built around it. So we wanted a drop-in replacement. We studied closely with Imaginary and Imgproxy which both are made with Go and libvips. Imgproxy is a pretty solid one, but we are not happy it is charging for premium, even for common features like Chained pipelines (a.k.a filters). Imaginary is very popular, but the feature parity gap is pretty large between Imaginary and Thumbor, with ongoing memory issues and the author seems not interested in maintaining it recently. And most importantly, neither of both sides are interested in adopting Thumbor's URL syntax. And building a proxy around them would be a half baked solution, as their high level APIs are unlikely meet the Thumbor's feature parity. And so we decided to do the hard work and here it is.

We have already implemented all features we need and it is working great for us in production. The S3 Loader/Storage is battle tested, but the file store is not though. We do acknowledge it has a long way to go to meet the Thumbor's feature parity. But this is just our first step and of course we wanna keep pushing it.


I don't see any documentation around libvps that claims it's safe to run on untrusted inputs, so you might want to run this thing in gVisor or a dedicated VM if that's your plan.

(I also looked for fuzz tests up the chain; libvips -> govips -> imagor, and didn't find any.)


For what it's worth, libvips is part of OSS fuzz

https://github.com/google/oss-fuzz/tree/a21768ce6a5056d27f82...

I thought they mostly fuzz the loaders, which are the most critical, apparently the found bugs present a picture of a pretty wide coverage:

https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=Lib...

As of June 2021, OSS-Fuzz has found over 30,000 bugs in 500 open source projects.


Ah, very nice! I should have checked there.


quite. inside docker, on a VPS with only other instances of that container running, uploading to an S3 bucket using IAM write-only privs, and the instance gets automatically cycled every hour.


Load it up as an ECR lambda ? Write file, return data to caller.. lambda terminates... Put the lambda on a reasonably short max execution time


Only every hour? I see you like to take risks.


There is also battle-tested imaginary tool [1], offering similar functionality and much more, also using libvips

[1] https://github.com/h2non/imaginary


"image processing" in the context of docker is very confusing. This should be "Docker-ready graphics processing server"


Agreed.


Great to see more players in this space, especially re-using the thumbor syntax. Curious how much of a feature-parity is there between Imagor and thumbor though? (particularly with some of the plugins)

Context/disclaimer: I've been maintaining the docker image for thumbor[0] (not maintaining thumbor itself, although I work with the thumbor team occasionally). Thumbor works great for us at my company, but it's still python 2.x based. There's some work to transition to python 3.x, but it seems[1] like there are not enough resources or drive to push this forward? and thumbor 7 does not (yet) have feature parity with thumbor 6.

I'm yet to find something that has feature-parity with thumbor though... But it definitely feels like high time for a some new blood in this space. Especially compared against some of the cloud options out there, like Cloudflare Images, especially since they seem to be also lacking some crucial features[2].

[0] https://github.com/MinimalCompact/thumbor/

[1] https://github.com/MinimalCompact/thumbor/issues/67

[2] https://news.ycombinator.com/item?id=29474743


libvips is great! The performance is so good you can probably process your images on the fly. Can't recommend it enough.


It would be great to see a cache layer in front of S3, in my experience fetching the original image from S3 will be slower than the actual resizing (regardless of the library used) and that happens at least once.

So ideally something like:

1. upload to s3

2. manipulate image to a friendly size (ie. 2048x2048)

3. write the image to an LRU cache (ie. memcached)

with steps 2 and 3 on a separate go-routine

Besides that, really cool project :)


Thumbor (with nginx-proxy caching) does that[0] but thumbor itself is python-based.

[0] https://github.com/MinimalCompact/thumbor/#minimal-compact-t...


Isn’t that cloudfront?


Typically Cloudfront would sit in front of this service and either serve the requested resource from cache or forward the request to the origin (Imagor) if missing from cache.


Or you’d use a cloudfront lambda and combine both?


Wouldn't you then be processing images on multiple CDN nodes, instead of just doing it once and serving from multiple CDN nodes?


What would you all use if you had to deploy something like this today? I see here at least thumbor and imaginary mentioned as alternatives, which would you choose and why?


imgproxy probably the best one I’ve used. Built in go, open source (license for some features/commercial use), also uses vips.


imgproxy looks amazing, but some features (e.g. pipeline) are locked behind Pro as far as I can tell? (and I guess Pro isn't open-source, or not free).

disclaimer: I maintain the docker images for thumbor[0]

[0] https://github.com/MinimalCompact/thumbor/


Thank you!


So how does it compare to Imaginary?

It looks Imagor can save/cache the resulting files in S3. If so that’s a great benefit since Imaginary does not support it and you have to use a CDN.

Do I understand it correctly that Imagor can transform the image from URL and store the results to S3 and next time the same image is requested it is not transformed again but returned from S3?


How does it compare with imgproxy? Seems to be the same (libvips wrapper as docker container) but less polish?


Replying so I can follow this thread. I’m using imgproxy now and pretty happy, but willing to try something else if it’s faster.


"ready to be containerized using Docker" is one of the stupidest software brags I've seen all year. Virtually any Linux software at all is "ready to be run in a container". It's a giant red flag saying "I don't really get containers but this phrase should get a ton of GitHub stars and maybe I can turn that into getting hired by a FAANG"


Perhaps it means something like: the hard work has already been put in, you can simply run it in docker. That's not a trivial Dockerfile.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: