> bored developers

Bored? Git's a distributed version control system, so no excuses. Get back to work!

But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

Go has a mirror of our GitHub project via this thing "Maintner" I wrote (running at http://maintner.golang.org/) that syncs GitHub in realtime to a log of mutations. (As well as syncing Gerrit and all its comments etc).

So then we can slurp all of our GitHub & Gerrit history into RAM (takes about 5 seconds and 500 MB) via https://godoc.org/golang.org/x/build/maintner/godata#Get and walk it in-memory and do stuff with in. (runs our realtime bots, alternate web UIs on planes, alternate search, stats, etc.)

> Maintner is short for "Maintainer"...the name of the daemon that serves the maintner data to other tools is "maintnerd".

Nice work! But as is custom on HN, I'll bikeshed on the name instead of delving into the contents of the tool. Why shorten the word by just 2 letters? Is there something special about the tooling that makes 8-letter projects more desirable than 10-letter projects, or is it linked to the removal of Artificial Intelligence from the process? I'd personally mis-type that name all the time.

I'd prefer it just for searchability. Good luck finding anything online called "maintainer".

I think he was excgarating the issue a bit. Your logic is sound

> excgarating

I commit to re-use this as often as possible!

http://www.urbandictionary.com/define.php?term=Excgarated

An inspiring new word invented by a redditor on March 23, 2014. The redditor was actually trying to spell exaggerated. This word was misspelled so horribly that when it was googled the only thing that came up was a link to the comment itself.

Glad someone caught it :)

ahh, a googlewhack

Should have called it Matenerd, shorter and much better name

I think it's a Go joke. Just be glad it's not called "m"!

Oops, instructions not clear, accidentally created another suite of build tooling called "gb"

> alternate web UIs on planes

Could you expand on this? Sounds interesting. You mean like an offline UI powered by the in-memory mutation list?

I think they just meant lighter. They load quite a bit and on really high latency and slow connections it's painful to use.

> have to be fully reliant on one service

My inner cynic says that that's probably the reason why they don't provide this service.

I keep meaning to dig into Fossil (SQLite's VCS/Project Management system), but I have no faith I could convince a team to use it.

Another model to look at is Trac, which had pretty extensive integration with SVN and integrated (ie, cross-linked) issue tracking and wiki, and stored all the data and change history in a svn repository.

> I keep meaning to dig into Fossil, but I have no faith I could convince a team to use it.

Developer or Fossil and SQLite here: I agree. In my experience, you'd have better luck convincing the team to switch from vi to emacs. For all its many and well-documented faults, the Git/GitHub paradigm is what people want to use because it is what they are familiar with.

All the same, I intend to keep right on using Fossil, thank you very much!

So here is the idea I've been thinking of lately: What if Fossil were ported or enhanced to use Git's low-level file-formats so that unmodified git clients could seamlessly push and pull against the (enhanced) Fossil server. Call the new system "Fit" (Fossil+Git). Using Fit, you could stand up a GitHub replacement for an individual project in 5 minutes using nothing more than a 2-line CGI script. Git fan-boys could continue to use their preferred interface, while others who prefer a more rational and user-friendly design could use the Fossil-like "fit" command. Everybody could share code, and everybody would have a nice web-based interface with which to collaborate with tickets and wiki and all the other cool (and to my mind essential) stuff that Fossil provides. And nobody who already knows git would be forced to learn a new command-line interface.

I'd be all over writing the code for "Fit", except that I'm already over-extended. Anybody who thinks this is a good idea and would like to pitch in and collaborate, please contact me privately. Thanks.

The pushback will come in two forms.

First, a backend that works exactly like git is very different from one that works almost like git. You'll be blamed for other people's problems.

Second, you'll have to have a way to deal with commit history the way git does (arbitrarily, and able to be rewritten at any time)

Otherwise the first push -f or rebase breaks the whole thing.

Reading from and writing to a git backend (that is, making fit a client too) might be the safer option. Sort of like git-p4 in reverse.

That sounds like a great idea. By leveraging Fossil's existing UI and getting it to work with Git, it might really gain a lot of traction, and become a viable alternative to roll-your-own-github services like GitLab Community Edition.

To be attractive to github users it would have to have a better issue tracker and a better patch review system. Given that sqlite uses a mailing list for both I'm guessing it doesn't have that.

This'd be a great idea, but I don't have the skillset.

My guess is their sales pitch would be for you to run your own instance of Github Enterprise or doing something fancy with Web Hooks. At work we did the former for a while before switching to Gitlab. We had less downtime than Github (not that they have much) but since the ops is on you in that situation, YMMV.

I'm of the impression that issues and PRs are all accessible via an API. What more should they do?

> What more should they do?

How about storing issues and PRs in the actual git repository? They should index them for the UI, sure, but the source of truth should be a branch of the git repo just like it is with gh-pages. It should be possible to file a new issue by committing a markdown file to the correct branch and pushing it to Github. Their hub command line tool and their own client could facilitate adding all the correct metadata. This would allow people to work while Github.com is offline and synchronize everything once the service outage is over.

If you have access to a distributed database that is already the best available for manual conflict resolution, why wouldn't you want to use it to store this kind of data? A Github outage is like a partition when you think about it from a distributed db perspective, so treat it like an individual node died and still allow reads/writes to the rest of the cluster that can be merged back once the node comes back online.

> How about storing issues and PRs in the actual git repository?

Access to the git repository is regulated; issues and PRs aren’t. Anybody can fill an issue/PR on your repo, but only you (and your team) can modify the repo. You’d need to also store all the comments on all issues and PRs, even closed/rejected ones. In some repositories, that’d be huge.

Of course the issue data in the repo should be considered read only.

And the amount of actual data needed to store issues is peanuts anyway.

> And the amount of actual data needed to store issues is peanuts anyway.

It’s not. Take something like github.com/Homebrew/homebrew-core. There are 15k closed PRs there. There have been 20 new ones today. It’s not rare to have 10-20 comments per PRs; some of them even go over 200. Add CI status, actual PR contents (git patches), comments reactions, edits, labels, milestones, assignees, reviews, projects.

IMHO having a tool to fetch issues locally is a good idea; storing them in the repo is not.

From the perspective of someone mid code review during the outage, it's worked as well as it could too. They preserve comment drafts client side that haven't posted to the server yet across page reloads. While it's a little frustrating to have to submit something 3-5x, they're definitely doing something for this use case already.

Also as others pointed out, just because the GitHub app is down doesn't always mean the GitHub git server itself is down.

You might wanna check out Fossil, it's version control system with integrated bug tracking and wiki. It doesn't really work for all projects, though.

[0] http://fossil-scm.org/index.html/doc/trunk/www/index.wiki

GitLab.com has repository mirroring https://docs.gitlab.com/ee/workflow/repository_mirroring.htm... (this will soon be a paid account feature)

It doesn't mirror issues and PRs but it can do a one time import of them https://docs.gitlab.com/ee/workflow/importing/import_project...

Shame that data isn't store in a side repository where it could be replicated like the repository.

It would be clever to store issues and PRs on a special branch in the repository. I can't currently think of why this wouldn't work (apart from business-wise, it would reduce lock-in considerably).

Well, it's unlikely to be feasible due to Git's on disk structure.

With git, every commit has an sha checksum of the commit contents & metadata. And each commit's metadata points to the previous commit.

So, if a maintainer wanted to fix a typo or do any other kind of correction/update to an old issue you'd need to rewrite every commit on disk since that one with the updated chain of checksums. eg from the fixed comment, to the head of the special issue/pr branch.

That alone would probably make syncing with the repo's a real pain for anyone working on medium to larger sized projects. ;)

Your objection is predicated on the assumption that issues would need to be edited "for all time". That's not true for anything else in git, I'm not sure why that would be true for issues. I mean, if you had an "issues/" directory in your repo, and files named "issue1" "issue2" and so on, you would just edit the issue whenever. If someone made a branch for the issue, then they'd want to merge your edit into their branch ASAP - which is good, because it indicates that they read and understood the change. And this merge would be particularly painless since changes to "issues/" are absolutely not going to conflict with anything in "src/".

Ahhh, I see what you mean. Yeah, that would probably work. :)

I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch.

GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.

DISCLAIMER: I work for GitLab. We have a feature to replicate GitLab to different locations (EEP). We call it "Geo", which stands for "Geographical Replication". Part of the "Geo" effort is Disaster Recovery which is under heavily development. We want with Disaster Recovery to be able to reliably promote any Secondary node to a Primary, so if your US datacenter melts down under a nuclear war, you can start working with your copy in EU, India, China, etc.

If your US datacenter melts down under a nuclear war, I can guarantee that none of your devs will give a damn about contributing to the code base.

You might want to modify your scenarios.

As someone interested in these scenarios, where could I find out more information about what would happen post-nuclear explosion?

A lot of this is out in the open

There are active simulations going on: http://www.sciencerecorder.com/article.php?n=scientists-cond...

There are preparedness drills: http://www.independent.co.uk/news/world/americas/us-politics...

There's plenty of policy analysis: https://www.cato.org/publications/policy-analysis/social-eco...

Older .mil reports: http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA080907

There are entire books taking apart what happens in specific fields: https://www.ncbi.nlm.nih.gov/books/NBK219154/

Or you could just read disaster porn^W^W post-apocalyptic sci fi.

So, a lot depends on what kinds of info you want to find. But overall, I'd say it's not worth reading, because life's not going to be much fun. Even if you're prepared.

In that event, how much would your "guarantee" be worth? b^)

it depends on your SLA ;)

(only partially serious here) I'm not sure recovery from a nuclear event that wipes out US data centers is economically feasible. If an event were to effectively wipe out 20% of the worlds economy I think having my github issues online is the least of my worries.

If it's not Git based then I'm not interested in it.

I see no reason to have a RDBMS for issues, merge requests, pull requests, etc. All of these could be modeled in multiple ways using a Git database. Issues are files in a repo called issue1.md, issue2.md ... or issues are branches on a repo ... or something else.

Yeah I know I'll be on vacation for a long while!

Hope it will be on the moon or in a nuclear shelter. Otherwise all that fallout may ruin your trip.

Reminds me of https://en.wikipedia.org/wiki/On_the_Beach_(novel)

Contentless, illogical mince.

GitHub clearly contributes a fair bit to open source - not only their own projects, but the free hosting for open source projects. You have no basis for this accusation, and frankly we are all stupider for having read it.

Well I made 2 claims..

1 Claim was that Github is not a open source company, and they are not. Their core business is not open source, yes they have some side projects that are, but so does MS, I do not believe anyone is going to say MS is a "open source company"

the second claim I made is they are a threat to free software, I used free software here for a reason, free software and open source are different.

GitHub may support some Open Source things, they do NOT support free software.

Tom Preston-Werners infamous post on open sourcing only some things (http://tom.preston-werner.com/2011/11/22/open-source-everyth...) Is a key piece of evidence to prove this.

Git hubs general actions over the years show they do not support free software at all, and have limited support for Open Source software.

They are classic Free Software leaches, using and abusing free software not for the ethical reasons around free software but to advance their profit center

See I am support free software for ethical reasons, not monetary reasons.

That is with out even getting into the MASSIVE issue that is having all of these open source projects on a SINGLE platform. Unless you think github is "too big to fail" which I can assure you it is not

Terribly sorry for this rubbish reply, but here you go: http://i.imgur.com/ueRIgtq.png

I love it, and it's perfect for when you want to say something witty like FUD or D&D, but can't justify it. Now with CIM, I can!

What? Github employs members of the Rails core team including Aaron Patterson. They literally pay for open source software development.

I'm sure they employ direct maintainers/contributors to other open source projects as well.

Are they paying him to help make an open source alternative to github? If not, then it's not really relevant.

They are actively working against open source tooling for software development by developing features only for their closed platform.

Gitlab is a much better example of a company that fully embraces open source.

GitLab does not fully embrace Open Source.

GitLab Enterprise Edition does not have an open source license. Since GitLab.com runs GitLab EE, their two (presumably) primary sources of revenue (EE licenses and GitLab.com paid accounts) come from non-open source software.

But: it's still fair to call GitLab as a company way more Open Source than GitHub. All of their development happens out in the open, the vast majority of their codebase is Open Source, and the source code for GitLab EE is even made available.

This is a spectrum, not black and white.

And you believe that makes them an "Open Source Company"

To me in order to be a Open Source company your primary product must itself be Open Source.

RedHat as an example... RHEL is open source

GitLab.. GitLab is open source

These are open source companies

Hiring a few devs for work on some side projects that are open source does not make one an Open Source Company

If they open GitHub Core then they can claim to be a open source company

Aaron Patterson, who I'll continue to use as an example, works full time on Rails while being paid by Github. Rails is not a side project for him.

Also as mentioned elsewhere in this thread, Github has published plenty of open source software. Electron, Atom, resque, and updates to git itself.

It is a side project for the company, as are all the other projects you mentioned

As I clearly stated, GitHub can not be a Open Source company simply because they hired some devs to work on Open Source

Do you consider Microsoft to be a Open Source Company?

The Atom editor is widely used and was a major inspiration for VS Code. Electron, which is a GitHub project is also used for countless other projects.

"major inspiration for VS Code"

Which is a Microsoft product, and Microsoft is the inventor of EEE. Ergo, GitHub is complicit in an EEE attempt. Q.E.D. Case closed.

/s

So anyone who releases open-source code, that Microsoft might use in another open-source project is complicit in EEE? wow.

My apologies; I should have made the "/s" - which denotes sarcasm - much more apparent.

Yup. GitHub also released/contributed to a lot of other open-source Ruby stuff, like Resque.

> if they could find away to make Git only work with Github they would in a heart bet.

Sorry, but this is unfair :) I remember the early days of Gitlab, before they added all the insanely cool features they have now, when they were just a Github opensource copycat. My thought at the time was : "wow, github is super cool to let them go. At least the almost exact design copy could be a legal problem". Clearly, we would have heard about Github vs Gitlab back then, if Github wanted to lock people in.

Well, if we "heard about GitHub vs. Gitlab" more people outside HN might hear about GitLab. Might be more dangerous, than the chances of a win shutting GitLab down.

So you fault them for being evil geniuses by not acting evilly?

More likely they got sound legal advice and realized they could not actually win in court, or their investors killed that idea as a waste of money

You do not hear about many VC funded startups initiating lawsuits for a reason. That is the realm of established companies.

Just because it's A common motivation and strategy, doesn't mean it's THEIR motivation and strategy.

A more likely strategy/motivation is that the product is you. AKA: farming the users.

Create a desirable pasture, and the animals farm themselves.

It is partly the MS EEE model and partly the Adobe model, where they give away free or low cost services to get indivuals hooked, often at young age, then when they are employed at larger firms they push the firms to adopt that software internally.

Get a bunch of Open Source Developers to do your marketing at their "real jobs" so they can sell the Enterprise Version.

Since GitHub is a private company it is Unknown (at least I can not find the info) if they are profitable or not, or what their revenue numbers even are so it is unclear if that is a successful plan or not.

GitHub could very well run into the same problems as SoundCloud.

"git on the block chain"

At GitLab.com we're not seeing an increase in usage: https://www.dropbox.com/s/eiz2h7tdnownvfl/Screenshot%202017-...

From http://monitor.gitlab.net/dashboard/db/haproxy-stats?refresh...

But of course it is hard to mirror your repo to GitLab.com when GitHub.com is down, so maybe other sites are a better measure.

That's our chart of backend sessions / sec for GitLab.com [1]. There's also a chart for frontend sessions [2], please keep in mind that there's a lot of CI / bot sessions on these dashboards. In addition to that there's much other interesting info on our HAProxy Grafana dashboard [3]. In case anyone's interested in even more metrics, there's a bunch of interesting dashboards on our Grafana instance [4].

[1] - http://monitor.gitlab.net/dashboard/db/haproxy-stats?refresh...

[2] - http://monitor.gitlab.net/dashboard/db/haproxy-stats?refresh...

[3] - http://monitor.gitlab.net/dashboard/db/haproxy-stats

[4] - http://monitor.gitlab.net/?orgId=1

Its not like your editor is down.

My team was working on merging pull requests when this happened.

We might actually be at fault! That would be kinda cool.

If only you were using a decentralized VCS, you wouldn't have this single point of failure…

The problem isn't the VCS part so much as all the code review/project management features of Github.

I seem to remember there was a decentralized VCS which included issue tracking besides commits, but it never was mainstream.

Yep, that was the one.

There's also plenty of DIY distributed issue tracker tools to keep issue tracking inside a git repo (often as yaml+markdown). I'm still surprised a good one hasn't stepped up with direct GitHub issue/project tool integration+sync, but that should certainly be a possibility.

Fossil SCM

the linux kernel development works just fine w/o github.

They use self-hosted git repo and bugzilla.

It's not exactly less of a point of failure.

I think the spf there is Linus' email account, right?

The VCS is decentralized but they made the process centralized.

but github is honestly so nice. I'll admit in a very subjective way. whats your favorite decentralized vcs?

The joke is that git is decentralized–it's just GitHub that's a single point of failure.

We have Gitlab Enterprise and it has trouble scaling.

Sorry to hear you're having trouble scaling GitLab. We have many organizations running GitLab and successfully scaling to 10's of thousands of users, and GitLab.com which is the largest GitLab installation. As a GitLab Enterprise customer our support team is happy to help you review your scaling problems and resolve them. Please submit a support request at https://support.gitlab.com.

O crap I meant to say GitHUB Enterprise. That's what has trouble. We also have Gitlab Enterprise and that's fine. Sadly for reasons internal we are mostly stuck with GitHub E.

I'm sorry to hear that. GitLab Enterprise Edition should scale fine to 100,000's of users. Are you in touch with our support about this?

thats why i asked what his favorite was. just curious to see what the feature differences would be

But reviewing, issues, PRs, etc. are. A lot of the planning parts are down.

> implying developers spend most of their time in editors.

Well the parent was implying they spend most of their time in Github.

I don't see that implication. I personally spend most of my time on GitHub.

I wouldn't be surprised if this became an issue in the future, with Atom's popularity on the rise…

Uuuh ... Shouldn't they be, you know, developing? Or has GitHub turned into Reddit when I wasn't looking?

what are you doing on GH all day? you are supposed to _write_ code.