As much as I'd love to use CloudFlare's uniformly fast network for package update, I'd never use this simply because I already depend on them for too much stuff.
It's not like letting them gather logs on my computers updating would be terrible somehow, but if it's so easy to avoid donating more logs and more dependence on them, why not do it for the sake of preserving diversity?
Growing companies often exhibit a long term pattern that usually seems innocent at first manifestation:
- Offer something for free
- Become hugely popular at offering that free thing
- Alternatives slowly dwindle in popularity and/or begin to die off
- User 'choice' increasingly becomes concentrated in that single option
- Company introduces surprising policy change, but users can switch to an alternative if they don't like it
- Company introduces differentiating functionality setting its option apart from the alternatives.
- Users grow additional dependence on the new functionality, alternatives look increasingly obsolete
- Company becomes increasingly aggressive at introducing changes, as they now own the primary alternative
It's hard to imagine CloudFlare are somehow going to monopolize package management :) But nonetheless, free services like this generally signal the beginning of this pattern
At some point I'm sure e.g. DejaNews seemed like an innocent dependency, and that attempting to look far ahead into the future would not have revealed its eventual sale to Google and the loss of the only comprehensive archive of the early Internet community in existence.
I think this is still a good service. It is just a mirror. It is a way of giving back to these open source operating systems and cutting their costs. Thus embrace it but be smart about it so that no one is trapped.
Dejanews isn't the same. That was an archive of a dying technology. This isn't the same case. Usenet was a dying technology, ultimately killed by http.
Cloudflaree has always done the free thing to raise it's popularity. You are right that as companies get bigger it can be harder to keep revenue growing and they slowly start to cross lines they didn't before though. If that company has built reliance on it's free tools which have reduced diversity, when they start to cross lines it can be sort of a trap. This usually isn't intentional now but it is just how things work... Years from now revenue growth might require crossing a line...
Companies sponsoring free mirrors has a long history without anything going wrong though. The key is to keep using mirror lists, not a mirror. Anyone who tries to "differentiate" should be removed from those lists.
The difference with their DNS and CDN services is that all the information gets funneled through them instead of being one of many used in a round-robin manner.
Another service I worry about in this regard is let's encrypt. They passed 50% marketshare 2 years ago. And many systems have special casing support only for let's encrypt.
Please parter to make mirrors for Launchpad PPAs. They're so painfully slow around the globe and there is tons of open source projects that depend on PPAs...
I recently moved to Vietnam, and I sometimes get awfully slow connection speeds to the mirrors nearby my city, to a point that switching to US mirrors is faster. (ftp.[kr|sg].debian.org)
I also live in Vietnam for 3-6 months every year so it's part of the reason I posted it. PPA is real pain since as far as I understand they're not mirrored at all or at least they haven't been before.
In my experience mirrors in Japan work best in Vietnam for whatever reason.
How is Docker hub from Vietnam nowadays? I lived in Cambodia from 2014-2016, and it was nearly impossible to pull Docker images which have large single layers.
Would be great to have a Cloudflare solution for this.
Off-topic. I visited Cambodia for 3 weeks in 2017. It's a beautiful country and I enjoyed every minute. My mind has pondered over living there long-term -- it seems so affordable.
How was your experience living there for a while, especially as a developer?
I surely miss the developer/tech atmosphere that was there in Berlin, Frankfurt, etc.
The life is quite simple, and the coffee culture is based around chatting with friends for hours, instead of sitting in a cafe and getting work done on a MacBook Pro or a Thinkpad X1 carbon.
If you live like a local, which is not to say any lower living standards, it can be pretty cheap. At around 2x the living cost of a local, one can live comfortably here. I pay around third the rent which was once near Berlin, and now spend about half on food if I eat out.
Internet is cheap but unreliable. I sometimes have to use my 4G mobile hotspot (1GB/day at ~€4 a month).
I was born and raised in an Asian country, so there isn't much of a culture difference to speak of. However, Vietnamese is pretty difficult to learn, and even after six months here, I still cannot get the tones right and speak more than 50 words. Indonesia in that regard is amazing.
If you are thinking to move to south East Asia, I can highly recommend Indonesia. They have countless mountains to climb, a good hiking/riding groups to easily join, similar living cost to Vietnam, and a much easier language to learn. Bahasa Indonesia is understood pretty much everywhere, and quite easy to learn. Internet is censored in both countries to some extent (Reddit doesn't work in Indonesia with a DNS blackhole that you can easily bypass), and has worse Internet in Indonesia in general though.
There are many startups based in Indonesia (such as Gojek, Traveloka, Tiket.com), some of which are owned by SoftBank a few layers up. I feel it's more easier to I best/startup in Indonesia too.
Where exactly in Indonesia have you lived? Bali is like two times more expensive compared to Vietnam.
Also internet connection was generally much worse for me in Indonesia due to their island nature I guess. In Vietnam it's super easy to setup two 50-100Mbps FTTB lines for something like $80 / month (just in case one of ISP go down).
As the other commenter hinted, Siem Reap is a great place for expats, and that’s where I lived as well. The population is much more poor than the surrounding countries, but they are all so incredibly friendly and happy despite all this. You do need to make an effort really going into the country side to see the “real” Cambodia, I strongly advise against using only the cities as your reference, as they are generally very “tourist/expat savvy”, by lack of a different definition.
I did freelance work for a few European businesses, mostly C++ and devops stuff, and for a long time I felt I was living the life. What drew me back, however, was ambition: before taking the sabbatical, I had successfully founded and “exited” (more like an acqui-hire with some pocket money) a startup, and I was craving for more. It was very difficult to find people that I could consider my peers in Siem Reap, and this left some void. This may also have to do with my nature as an introvert, they may have been out there, but most developers out there actually are web developers, which isn’t my cup of tea.
All in all, however, the whole experience of being completely detached from my life back home for an extended period of time has made me a much more stable person, and I would do it again in a blink of an eye. As crazy as it sounds, I can now better appreciate the “rat race” and its good parts, and know that I also miss being challenged in that area.
Feel free to reach out to me directly if you feel like it.
I was actually in the Puok district outside Siem Reap for the 3 weeks. Mostly stayed in the rural side living with locals. I also visited the city for the weekends and it seemed like a great place to settle.
I also don't tend to go out of my way to meet peers in my local city. I guess I just prefer larger conferences for networking.
Thanks for the input! Maybe one day I'll also get acqui-hired and move there :)
I haven't lived in Cambodia, but what's I dislike about it is that it's feels generally like it more poor than Vietnam in quite ugly way: there are more people begging on the streets; including children. Might be something changed in recent years, but I have no idea.
In Vietnam / Thailand you just generally don't see this except in few tourist areas that you can easily avoid.
It seemed to me that Siem Reap was the logical place for expat coworking spaces but the internet there was really bad back in 2008. Not sure how things have changed.
Internet I general seems to be pretty bad here in Hanoi. I do not really work with bigger images (~100mb alpine images tops), and it takes ~5-10 minutes in evenings which appears to be the peak time on my current VNPT 40mbps line.
How is this supposed to work? (I use Arch BTW) supposed I'm in a country X, does Cloudflare basically caching the request to server that are in country X? But I'm already close to that server and I'm always cautious with mirrors that are in country X, often they were out of date :(
This is my issue with it as well - it's basically their geo-sniffing magic routing you towards the closest officially listed mirrors for that country (glorified proxy); it makes the mistake of presuming that all those mirrors listed in the Arch country-mirrorlist are created equal in terms of reliability and performance, but they're not (we have a whole web page just to track out of date mirrors).
I'd guess if this gains traction, the caching on Cloudflare's side will usually make up for this, and if it hits the actual mirror behind it, the load on that one should be lower than before. It reads like they will try to proxy multiple local mirrors so time will tell how well they will pick amongst them. I'm switching for now to see how it will perform over a longer period.
The difficulty is if the mirror itself is behind, Cloudflare's cache for that specific endpoint will be correct (functional, working) but out of date compared to another mirror.
Example: around the end of June this year the mirrors.kernel.org infra was failing to sync (DNS issues of some sort) the latest content, so your upgrade would appear to have no new packages because the stale content was a valid repo, when in fact there were many updates and other mirrors were in sync.
I'm not aware of the technical how, but the Arch infrastructure seems to measure this successfully; a hook into this data might be a great way for the two to interoperate (so that Cloudflare always serves the freshest cache/mirrors). https://www.archlinux.org/mirrors/status/
> How is this supposed to work? (I use Arch BTW) supposed I'm in a country X, does Cloudflare basically caching the request to server that are in country X? But I'm already close to that server and I'm always cautious with mirrors that are in country X, often they were out of date :(
According to what I read in their main page https://cloudflaremirrors.com/archlinux, they seem to be caching the packages from mirrors and serving it via their anycast network.
Repeatedly poke the maintainers of the mirror and the core developers alternately until everyone gets mad and does something about it. :P
Alternatively (/ realistically), perhaps CF would prefer the value-add of rerouting where there are newer packages instead of the singular approach of geo-routing. Definitely let them know.
I'm seeing multiple comments in this subthread, I'd apply the 2nd line above to everyone affected, if a bunch of people let CF know maybe they'll figure something out.
I saw the same. My best initial guess is that this is somehow caused by cloudflare either setting the content-type header to text/plain or not setting the accept-ranges header. Or maybe both?
Clear your partial downloads using sudo pacman -Scc, another comment says that it doesn't support byte ranges so it can neither display progress, nor resume partial downloads
It would be really nice if web proxies supported this as a generic mechanism, which could be applied to Linux packages, ISOs, git commits, docker layers and more. I wrote up a very half-baked proposal for this kind of thing a while back: https://rwmj.wordpress.com/2013/09/09/half-baked-idea-conten...
Then you could set cloudflare as a proxy (or transparent proxy even?) and it could inject the objects without having to visit the endpoints.
Ideally soon repos will be mirrored to ipfs nodes and that will automatically pull the hash-addressed content from the closest proxy, which may or may not be cloudflare. The protocol is already there. Some initial solutions are already there too: https://github.com/victorb/arch-mirror
When I saw the title I though they were proxying npmjs or docker hub.
The install process of editing a file is a bit painful. Means few people will do it. Somehow would be nice to have this offered in the is as a built in optional feature.
I hope this is expanded to other package repositories (Maven, Docker, npm, etc). I'd really like to make it easier for people at edge locations to build my code.
I know, but lets be real for a moment:
Cloudflaremirror will probably be the fastest. This will cause people to configure it and because people are generally lazy they will not configure other mirrors.
Being a proxy and cache is their business. All it takes to set things up that way is then a little bit of configuration that points to the actual origin.
I think if they would be an official mirror there would be a fair amount of more work being involved. Eg you can’t just randomly wipe the cache anymore and need to have higher persistence guarantees, since you are not supposed to refetch from other mirrors all the time.
They have Cloudflare Workers (with limited disk space), but act as a caching proxy is still the main business of Cloudflare. So i think acting as a proxy make sense for Cloudflare.
It's not like letting them gather logs on my computers updating would be terrible somehow, but if it's so easy to avoid donating more logs and more dependence on them, why not do it for the sake of preserving diversity?
Growing companies often exhibit a long term pattern that usually seems innocent at first manifestation:
- Offer something for free
- Become hugely popular at offering that free thing
- Alternatives slowly dwindle in popularity and/or begin to die off
- User 'choice' increasingly becomes concentrated in that single option
- Company introduces surprising policy change, but users can switch to an alternative if they don't like it
- Company introduces differentiating functionality setting its option apart from the alternatives.
- Users grow additional dependence on the new functionality, alternatives look increasingly obsolete
- Company becomes increasingly aggressive at introducing changes, as they now own the primary alternative
It's hard to imagine CloudFlare are somehow going to monopolize package management :) But nonetheless, free services like this generally signal the beginning of this pattern
At some point I'm sure e.g. DejaNews seemed like an innocent dependency, and that attempting to look far ahead into the future would not have revealed its eventual sale to Google and the loss of the only comprehensive archive of the early Internet community in existence.