This is very interesting to me. It also makes me think about the natural rise of monopolies and monoculture in tech. GitHub has really been extending to "eating the world" as of late. Recently in terms of their package registry that must have folks like Artifactory and Nexus a bit shaken, and now this, which is bad news for folks like CircleCI (and I say this as a CircleCI user).
As a developer, in the short term I love this. Fewer things I need to cobble together and worry about how to integrate. I mean, it's already the case that if GitHub goes down that my CircleCI jobs won't work, so having one company to yell at and monitor alone is a plus.
But long term it makes the competitive ecosystem much less robust. And as a startup employee, makes me feel how disrupting established platform competitors gets that much more difficult - even if you have a better product, it's hard to fight against the "platform" as they have more integrated points of value.
A monopoly is not inherently bad. They often benefit users with increased efficiency and productivity. They're only bad when progress stagnates and/or prices rise.
In this case, Github is actively competing and adding features that helps all of their users. If that means that some other companies lose market share then it's just a sign that the value proposition has changed. I see no problem with this, it's how the market works.
> A monopoly is not inherently bad. They often benefit users with increased efficiency and productivity. They're only bad when progress stagnates and/or prices rise.
My argument is every single monopoly eventually becomes 'bad' and makes progress stagnate and prices rise.
Just look at Google and Amazon. Up until I'd say late-ish 00s, nearly everyone loved Google and Amazon. I remember when Google search, GMail, Maps, street view, etc. first came out. I loved them, and I felt they were so much better than the competition. Same thing with Amazon - their selection was huge, great delivery and customer service.
I still use Google and Amazon for so much, but it's usually with reluctance and distaste. Check out Google search results these days. The ENTIRE first page about half way past the fold is ads, or AMP carousels, or shit I just want to get out of my way. But I still use them because with their heft and scale no one does long tail searches as comprehensively.
I hate to point this out on HN but it does seem relevant. If you dislike Google's use of ads, why not block them?
I think you've acknowledged that Google and Amazon provide the best services of their respective classes. Can we find a way to maintain that high standard without the crud that comes with it?
I'm still not convinced someone couldn't build a better gmail.
The web app is bloated and slow. The ads. The awkward design. Privacy issues. There's a bunch of benefits someone could offer. Fastmail tried but I still think it could be better and not based in Australia where privacy has been eroded.
The barrier to entry for doing categorization, spam detection, search, autosuggestions, and even complex JS frontends is significantly lower than it's ever been. You could even offload some of the above challenges to the client with prepackaged ML models. The spam filtering stuff could even be open sourced and community run like Adblockers. Abuse could still be handled server side.
Now that you mention it maps hasn't really improved since streetview. Search is worse. Gmail hasn't really changed much. I guess after it's launched new features are too difficult to add for google.
Recommend taking a look at this blog post: https://www.justinobeirne.com/google-maps-moat the author has updated a couple times. The changes are subtle and are amazing for your benefit. Not seeing the change is them getting better at easing you into the new functionality, not them stagnating.
Building new stuff is always more attractive to engineers than maintaining and incrementally improving existing stuff. Google suffers from this problem somewhat worse than most companies, where its best engineers jump to new projects and the existing products languish because it's not attractive to work on.
The Apple Maps 'street view' equivalent (coming in iOS 13) has a transition animation (for moving down the street) that absolutely knocks the socks off Google's. It was surprising to see just how strikingly better it is. I don't think Google changed it in that time.
In what real scenario do you see a monopoly not doing exactly what monopolies do? Do they have some moral code they can't break? Do they hate money? What exactly is it that would make them immune to human tendencies?
GitHub has gained near-monopoly status because it has provided a superior user experience at a great price. If either of those value propositions start to crumble,
git remote add ...
Exploitative monopolies almost always have help from the state: The Patent Office, Interstate Commerce Commission, Federal Reserve, FDA, etc either grant monopoly or cartel privilege to their customers. Monopolies achieved on the market cannot be sustained without continuing to satisfy customer demands.
The point I think is that as GitHub adds more and more integrations (git + issues + fork graph + CI/CI + ...) then it's not so simple to move to another service. The vertical is more convenient until you realize you're an Oracle customer all over again.
With CI, I wonder if someone could build a tool which had some sort of DSL to define your CI pipeline, and then translated that to configuration files for multiple CI providers (Travis CI, CircleCI, GitHub Actions, Jenkins, etc.) If you write all your CI pipelines in that DSL, then moving to another provider would be just telling the tool to generate output for another backend.
(One issue is that such a DSL could only provide lowest common denominator functionality... or else, suppose CI providers 1 and 2 offer feature X but 3 and 4 don't, then if you use feature X, you can switch between 1 and 2, but the tool would give an "unsupported feature" error if you tried to generate output for 3 or 4.)
Having migrated from Travis to CircleCI for most of my active projects, I’d say their models are so vastly different it’s pretty much impossible to have an AST of sort that can be translated to both except for the simplest and most well-encapsulated cases (at least not before Travis supposedly improved Docker support, by which point I was already gone).
Basically everyone I talk to despises Facebook, which gets pretty much no help from the state, but they still use it because there are no real alternatives.
My argument is that technological platforms are pretty special when it comes to monopoly power because their lock in/switching costs are so high that they can last long after they've stagnated in terms of innovation.
There are tons of alternatives just none that your friends are on.
Surprised no one has taken facebook's launch strategy. Only allow it in one school. Then only ivy league colleges. Then all colleges. Then everyone. Also allow you to import in your email contacts on signup.
That strategy created press / word of mouth and kept it youthful until they were ready for the big launch.
Microsoft is an even better example of a massive company continually competing. Their dev/tech products and tools have gotten magnitudes better in recent years.
Huh? MS is the perfect example of how monopolies cause things to stagnate. It was only when new technology paradigms allowed for the growth of companies like Google and Amazon (wrt AWS) did MS turn around.
IE stagnated for years after MS killed Netscape. No company has really ever challenged Windows on desktop PCs. It was only when new platforms like the Internet and mobile computing came along that provided space for upstarts did MS start competing again.
Sure, but that's the example of monopolies facing competition when they turn bad. Sometimes that takes a new platform shift, other times its just a better product, but either way if there's new value then the monopoly breaks down.
I think if anything, the problem today is companies that could be challengers just end up selling to the very incumbent they're competing against.
In the instances you’ve described where “monopolies” haven’t dominated the market, they were not actually monopolies because they didn’t have total ownership of the market.
A company can be a tech giant and still not be classed as a monopoly. Which are the kind of examples you’re think of.
For those that lived through the Windows 9x desktop monopoly in the 90s, the Internet Explorer monopoly, etc, they will see Microsoft as a prime example of how monopolies are terrible.
In fact the few areas where you actually see Microsoft improve things are in areas where they have experience but don’t actually have a monopoly (eg IDEs).
this is where the Valve vs Epic argument gets interesting as Valve has consistently been pro-consumer, so i have no problem with them being a monopoly.
Epic appears to be firmly pro-developer, which no matter what Tim Sweeney says isn't currently translating into an improved consumer experience.
As a .NET developer, this just smells like trying to rope in more people to Azure DevOps and then the Azure cloud. This is Microsoft's endgame. Everything they do is to win marketshare in the cloud.
I don't think it's an evil plan by any means, though, and their competition has been nothing but good for the ecosystem. Remember, they were evil about Windows because so many people didn't have any other real choice. They don't have a monopoly over the cloud, so they can't be bullies (and management is totally different now, too, probably as a result).
However on my very first impressions, the tools they're rolling out kind of look like Azure DevOps under the hood. There's a good chance they'll leave the github brand alone though and just make it better.
Personally, I've been happy working in the MS stack. They really want to earn the love and mindshare of developers.
I also see this as a play for cloud marketshare, but if Github can nail the “simple yet good enough” UI standard for this CI service, it’ll win out hands-down over Azure, which would evolve to become the more niche product that does more for SME, enterprise and larger teams, perhaps. Win/win, especially if they’ve a migration strategy between the services. I just hope the free Azure DevOps plan (for open source) doesn’t go away now that this exists. The more cloud companies can do to lower the barrier of entry, the better. :)
CircleCI is fine for the private repositories of work projects, but not for public libraries.
In trying to migrate a fairly popular open source library I maintain to CircleCI, from TravisCI, I bumped in the following two problems:
1. setting up a build matrix with various configurations (JVM version, compiler version, coverage on or off, etc) is a pain in the ass (possible, but a pain nonetheless), a configuration that TravisCI has nailed
2. basic functionality, like triggering a notification on Gitter via a simple HTTP hook when the build has finished, does not work
I do love CircleCI for our work project, as it doesn't need a build matrix or notifications to Gitter and thus was fairly easy to setup.
But I also love to see some competition for my public projects.
We’ve had to build a lot on. Like support for passing state across jobs in a workflow, or branching, or triggering a workflow with parameters, or triggering a workflow from a different branch, or etc. CCI went to unnecessary lengths to not support obvious use cases, or that’s how it feels. :(
I've been there and spent hours if not days shaping builds and using workarounds/hacks to fit the esoteric manner in which CI services need your builds to run (setting up the build pipeline as a series of containers but without easily accessible persistent storage between the containers seems to be the current popular model - which is the case because it's very easy for a service provider to provide and manage the compute this way but pushes all the work onto you instead to get it working).
This problem is what I'm trying to solve with https://boxci.dev for myself and hopefully others too. The idea is it'll do all the management stuff that a CI service should do, but let you run the actual builds however and wherever you like, on your own machines, using an open-source cli. Check it out, perhaps you'll find it useful. It's launching very soon!
I think you’re conflating monopoly and vertical integration. GitHub is not abusing its power as the market leader just yet. What it is doing is what Apple has been for years: make its core product and other products work better together causing users who use one to want to use all the others.
Honestly, I think it's a good thing in general. Like you've said, being able to just do things here and cut out all of that wasted time spent connecting disparate services... I mean assuming they pull this off well, it will absolutely make things better for me personally.
That all being said, I'm not sure how the long run comes into play here? It's bad if we extrapolate a long run prediction where no one else innovates or creates something in this space. But in reality, someone's going to be annoyed by it, someone's going to start up a different solution, and whether it works or not...it'll be there. Whether it's a small startup or a big enterprise challenging - if there's market gains to be had from adding this, it'll get added.
Don't get me wrong, you're on point with a lot of things in tech. Monoculture and monopolies are becoming super "in-your-face" considering just how much control the FAANG companies have in influencing tech and the like. However, I do find it odd that a single feature release by a larger company results, more often in not, in much of HN community losing their minds over prospective world domination. (not you op, but just in general...granted similar outrage happens to small open source stuff just as much so maybe this point is moot).
As John Maynard Keynes put it, "In the long run we are all dead." But in the short run, this just seems like a super useful feature added to a super useful product that will just make things easier for all developers.
This is interesting to people viewing the events in isolation.
These happen routinely on a daily basis. And being in a strategically foundational position makes such moves straightforward.
In the developer tooling market, GitHub is foundational, obviously. So it naturally will expand, and can also easily do so.
And it was these moments I lament Google's inability to buy GitHub. All these stuff are just repeating Google has had for a decade, they would have been the perfect marriage but Google refuses to pay big money...
I think its seriously such a hard problem to solve in tech. Its so easy and beneficial even for customers if one company does everything. Github can (and does!) offer the most open, accessible, and integrated APIs possible for CI/CD to plug into. But, when Github builds it, its just another tab in your PR, which is much more convenient.
We use Jira for ticketing and Github for code storage, like many companies. We've had SOC2 auditors tell us that they'd prefer to see us on Bitbucket just because the integration with Jira is more seamless; it guarantees a much deeper audit trail to go from RFC in Confluence -> Ticket in Jira -> Code in Bitbucket -> Deploy in whateverthefuck atlassian does for CI/CD. They didn't dock us, certainly.
But they're not wrong! I agree with them entirely! There's a part of me that hates that monopolization of our stack into one company, but there's another part of me that's like "that auditability and integration is so nice, and we literally will never see it if we're using ten different SaaS providers for each thing". How do you rectify those competing viewpoints? I don't know if you can.
CircleCI invented many of their own problems. No API to trigger workflow runs and the only trigger “API” is pushing to a git branch. A horrible job-centric UI when all you really care about are workflows. Workflows that don’t let you express conditionality and assume all errors are fatal. A broken data model that confuses the same workflow on different branches with different workflows (no good way to figure out how long your PR workflow takes on average because each PR is it’s own workflow, for some reason). No good state management or data passing options. Hopefully they will sort those things out, but it feels like they went out of their way to build a thing with a crazy data model and triggers that are tightly coupled to github.
I would say that monocultures, monopolies, vertical integration, etc. are all really good at making a local maxima, but at the expense of making it harder to find things that might be better still. And... sincerely, they are good for local maxima. They have the mindshare, they have the money, they have everything they need to succeed, today. And then tomorrow will come and they'll be stuck. But that doesn't mean that they can't be good, just that costs have to be weighed.
Yes. Gitlab can run CI from an external repo, and continuously replicate from another repo. And one time import all code and issues from GitHub.
(Caveat: import is broken. Worked for a few tiny repos, silently failed for our most important one, failing to import issues. Gitlab support just say "watch the open source issue tracker" , which is not really acceptable but at least more open than GitHub)
This is bad news for the CI providers that depend on GitHub, in particular CircleCI. Luckily for them (or maybe they saw this coming) they recently raised a series D https://circleci.com/blog/we-raised-a-56m-series-d-what-s-ne... and are already looking to add support for more platforms. It is hard to depend on a marketplace when it starts competing with you, from planning (Waffle.io), to dependency scanning (Gemnasium acquired by us), to CI (Travis CI layoff where especially sad).
It is interesting that a lot of the things GitHub is shipping is already part of Azure DevOps https://docs.microsoft.com/en-us/azure/architecture/example-... The overlap between Azure DevOps and GitHub seems to increase instead of to be reduced. I wonder what the integration story is and what will happen to Azure DevOps.
It's a horrible trend. CI should not be tied to version control. I mean we all have to deal with it now, but I'd much rather have my CI agnostic and not have config files for it checked into the repo.
I've browsed through the article you linked to, one of the subtitles was "Realizing the future of DevOps is a single application". Also a horrible idea: I think it locks developers into a certain workflow which is hard to escape. You have an issue with your setup you can't figure out - happened to me with Gitlab CI - sorry, you're out of luck. Every application is different, DevOps processes is something to be carefully crafted for each particular case with many considerations: large/small company, platform, development cycle, people preferred workflow etc. What I like to do is to have small well tested parts constitute my devops. It's a bad idea to adopt something just because everyone is doing this.
To sum it up, code should be separate from testing, deployment etc. On our team, I make sure developers don't have to think about devops. They know how to deploy and test and they know the workflow and commands. But that's about it.
I'm an operations guy, but I think I have a different perspective. The developers I work with don't have to think about CI/CD, but the configuration still lives in the repo, I'm just a contributer to that repo like they are.
Having CI configuration separate from the code sounds like a nightmare when a code change requires CI configurations to be updated. A new version of code requires a new dependency for instance, there needs to be a way to tie the CI configuration change with a commit that introduced that dependency. That comes automatically when they're in the same repo.
Having CI configuration inside the codebase also sounds like a nightmare when changes to the CI or deployment environment require configuration changes or when multiple CI/deployment environments exist.
For example as a use case: Software has dozens of tagged releases; organization moves from deploying on AWS to deploying in a Kubernetes cluster (requiring at least one change to the deployment configuration). Now, to deploy any of the old tagged releases, every release now has to be updated with the new configuration. This gets messy because there are two different orthogonal sets of versions involved. First, the code being developed has versions and second, the environments for testing, integration, and deployment also change over time and have versions to be controlled.
Even more broadly, consider multiple organizations using the same software package. They will each almost certainly have their own CI infrastructure, so there is no one "CI configuration" that could ever be checked into the repository along with the code without each user having to maintain their own forks/patchsets of the repo with all the pain that entails.
> organization moves from deploying on AWS to deploying in a Kubernetes cluster
I had (and still have) high hopes for circleci's orbs to help with this use case. Unfortunately, orbs are private - which makes it a no-go for us.
But, in my dream world, we have bits of the deployed configuration that can be imported from else where - and this is built right into the CI system.
In practice, for my org, the code and configuration for the CI comes from both the "infra" repo as well as the "application" repo. The configuration itself is stored in the app repo, but then there's a call `python deploy_to_kubernetes.py <args>`. The `deploy_to_xxx.py` script would be in the "infra" repo.
It also depends on your workflow - do you change the common deploy infrastructure more often, or do you change the application specific deploy infra more often.
Yeah, writing code to deploy code is sometimes fun, but sometimes nasty.
You can create a separate repo with your own CI config that pulls in the code you want to test; and thus ignore the code's CI config file. When something breaks, you'd then need to determine in which repo something changed: in the CI config repo, or the code repo. And then, you have CI events attached to PRs in the CI config repository.
IMHO it makes sense to have CI config version controlled in the same repo as the code. Unless there's a good tool for bisecting across multiple repos and subrepos?
I hear you. I thought about it and I think you need to reframe the problem. If a change in the application breaks your CI, it means you need to adjust the CI so it doesn't break when such changes are introduced. In my experience, these kinds of things happen very rarely.
In my experience maintaining an active project at a large enterprise, these kinds of things happen nearly daily. Sometimes I wake up and our EMEA team has merged in a change that requires a CI change as well, and they are able to self-service those through their PRs.
I'll give you a counter example: whenever I change CI workflow for something that has nothing to do with the repo - like a new deployment scheme to staging - I have to go and ask people to merge/rebase from master into their branches or they won't be able to deploy. It happens pretty often and I'd rather avoid this.
To fix this problem you can set up you CI server to merge into the base branch automatically before running the code (in fact you should probably do this by default for other reasons).
This way your devs won't have to merge, they can just rerun their tests, which should be the same workflow as if your CI config is separate from your codebase.
I 100% agree and am building https://boxci.dev to solve this problem for myself and I hope others too.
It's a CI service that lets you run your builds however you want, on any machine you want (cloud vm; in house server; your laptop) using an open source cli that just wraps any shell script/command and streams the logs to a service to give you all the useful stuff like build history, team account management, etc.
In other words, how you configure and run your builds will be up to you - scripts in the same repo, in another repo, in no repo - whatever you want, since you could just git clone or otherwise copy the source from wherever it is - there's no 1-1 relationship between the source repo and the CI, unless you want that. It'll be launching very soon :-)
Looking forward to it, very interested. However, I'm going to be honest with you - it'd be a hard decision to rely on a service rather than an open-source software. Like, you guys can go bust any time and while the runner is indeed open-source, as I imagine, the management console for it won't be. And I understand that's what people are paying for, but, perhaps, consider licencing it, rather than running from your own server and charging for the service.
This is a really good point and something I'd been thinking about, thanks. I'll have think about the licensing option - it's a good idea and something I'd not considered.
My initial thought was a guarantee that should the company not work out, the management console software will be completey open sourced. Obviously this would just be relying on trust though, which I can see could be an issue.
You might appreciate the sourcehut approach (which I am the author of). Each build job is decoupled from git and can be submitted ad-hoc with zero or more git repos declared in the manifest. Your manifest is machine-editable too so you can add extra build steps or tweak various options before submitting it in specific cases. postmarketOS does this, for example, to automate builds of all of their packages. SourceHut itself does this to fill the demand made by this trend, too, by pulling build manifests out of your git repo and tweaking the sources to check out specific commits, then submitting it over the API.
Having all the tooling being integrated makes it a lot easier to offer good feature interaction. Right now the atomisation of "software project management features" means that your project is run over 10 servers all held together by flaky webhooks, and requests that should be instant instead take 30 seconds because everything has to be remotely queried.
Though really the thing that should happen in this case is that we should be seeing more tooling that can be run on a local machine that has a deeper understanding of what a "project" is. Git is great for version control. But stuff like issue tracking and CI also exist. So if there was some meta-tool that could tie all that together.
A bonus: if you make some simple-ish CLI tool that ties all this together, the "Github as the controller of everything" risk goes down because it would become easier for other people to spin up all-encompasing services.
A tool like this would do to project hosting what Microsoft's language server tooling has done to building IDEs. A mostly unified meta-model would mean that we wouldn't spend our times rewriting things to store issue lists.
Technically, version control lends itself naturally as part of the now well-accepted infrastructure-as-code mantra.
Operationally, version control is the one that developers interfacing most primarily, shifting the interactions to that interface would be benefitical to users.
Of course, DevOps as a skill set is becoming less and less relevant, given the increasingly integrated toolings that interfacing directly with developers, that's for sure.
So, it's a very interesting situation. On one hand, I agree with you: when I start a new project, I am the master of it, I take care of everything and I need to make sure deployment and testing are implemented as early as possible. But this needs to scale. When we hired more people, I quickly realized no one wanted to deal with CI. They wanted it to just work and I wanted people to work on features and bugs, not fighting CI. So you can call me a devops guy by accident (turns out, that's a huge chunk of what I do as a CTO - remove obstacles by implementing and managing devops).
I think my ideal devops situation is the one where you start with a simple deployment script from your local machine when it's a one man show and then scale it to a large organization not by switching to a large piece of devops software, but by gradually adding pieces that you need as your team grows and requirements change. Exactly what happened to us and I think our devops workflow is really great and I'm very proud of it.
I'm interested to know how many different teams use your CI system, as well as how many different platforms (operating system, distro, programming language) you support.
Some companies have tons of old and new projects with very heterogeneous technologies in use. Imagine 50+ teams, several different programming languages, and things being deployed to different "hardware" (bare metal, cloud VMs, kubernetes, etc). It just seems like a lot of work to manage CI configs for all those different teams/cases, handle "support" requests from different teams, fix issues, and so forth. Hence, why the "easy way" out is to have each team manage CI configuration themselves as much as possible, to spread the maintenance cost across many capable developers.
I don't think it's that bad for CircleCI. CircleCI's focus is CI/CD and it's highly unlikely GitHub is going to do it as well as them. It's hardly commoditized technology. Their current customers are already heavily integrated with them and their orbs offering is further solidifying that relationship. Also, GitHub is expanding the CI/CD market with Actions so competitors in this space are likely to benefit.
Furthermore:
- waffle.io was acquired and shutdown by the acquiree
- TravisCI was sold to a private equity firm and lost their way
> I don't think it's that bad for CircleCI. CircleCI's focus is CI/CD and it's highly unlikely GitHub is going to do it as well as them.
I don't think it's bad for entrenched CircleCI users, sure, but I do think it's bad for prospective CircleCI users, who, are likely using GitHub already, so why would they not use something tightly integrated?
If you don't think GitHub is going to try and increase adoption of this tool then why did they build it?
> it's highly unlikely GitHub is going to do it as well as them.
They don't need to. They only need to be able to run tasks in sequence when triggered by some event, and the rest just builds itself.
An integrated solution that bundles issue tracking, CICD pipelines, and package/container repository always beats spreading each feature throughout multiple separate service providers.
> An integrated solution that bundles issue tracking, CICD pipelines, and package/container repository always beats spreading each feature throughout multiple separate service providers.
I agree but with GitHub actions, it's also possible for CircleCI to build a tight integration. When it comes down to it, each CI system has its idiosyncracies so choosing a CI system isn't as simple as how tightly integrated it is (e.g. how flexible it is with Docker, or the way it builds containers, or running multi-container setups)
I think a good strategy for Microsoft would be to reuse as much as possible CI/CD code from github in Azure devops. Azure devops probably doesn’t need to be as flexible as long as it is robust and just works. Github will probably be the place where experiments can happen.
I'm in the process of replacing TeamCity with Azure Pipelines (not my choice). It's ok but I also doubt it's best in class, TeamCity is far better IMO (more flexible, more informations/actions on one screen, less clicks, better logs display, better tests results display, live tests results, etc). I wish they had an hosted offer, and yaml configuration.
Can't find the discussion now on GitLab, but I recall comments claiming that Azure Pipelines is very flexible, and GitLab CI's new task DAG should be modeled after that.
You can bodge your own solution if you use your own build agent.
For hosted build agents, this is about to change, as a new caching task is in preview. I've tried it, and it's unstable at the moment, so I'd recommend just waiting until it's out of preview.
Sure! This is how Heroku works. Although, if you also mean change management (issues, code review, browsing, etc), that's a much bigger lift, especially if your intention is to outdo GitHub.
Just a warning for those who haven't ventured into actions yet, I would have to say so far I've found the experience very, very average. Even just doing something simple like posting a release notification to Slack seems to end up with me having the action triggered a random number of times causing multiple messages being posted to slack. The whole experience feels amateur and clunky.
There are issues open to look into it but no fix in sight yet. While this announcement sounds useful, don't throw away your current CI/CD tooling which is probably a lot nicer to use.
Lastly I really dislike how pretty much any really useful actions are created and maintained by single people. There are just some actions I'd want to see be supported by GitHub, I don't wan to have to handover things like Slack access keys to a non-trusted third party to post messages.
Every time I try to use actions I'm surprised it was launched in it's current obscure, unpolished state.
This feature release has been a dumpster fire from my perspective. I love GitHub and what they are trying to do, but I read over the Actions documentation 2x and I still can't tell if my use case is supported: I just want to see if master builds on a net core app without spinning up or otherwise utilizing some other cloud instance.
Also, I am not even sure what the appropriate syntax to use is with all the mixed messaging and examples (YAML or the other thing? Which do I use!?).
Regardless of which variant of syntax I attempted, the actions UI told me there was some generic error and that nothing was to be done. One additional problem I noticed is that if you have a protected master branch, you are going to be forced to get code reviews from your team every single time you try to iterate on the workflow script. There is no apparent way to test or validate actions without committing directly to master and seeing what the result may be.
All around, a complete mess in my estimation. I will be sticking with Jenkins for the foreseeable future. This GH feature is apparently not designed for people who care about straightforward solutions to simple problems:
That is all I want to do, Microsoft. Can you handle that? I feel like there should just be a simple toggle switch in my GH repo for this concern, considering Microsoft is now responsible for that entire vertical stack.
Good point. This could very easily lead to the same problems plaguing the NPM ecosystem — an action repo could be sold to a malicious actor, giving them full access to your entire codebase!
As a Gitlab user that's not strongly committed to the platform, this looks like a pretty interesting option.
Anyone have thoughts on how this compares to e.g. Google Cloud Builder in terms of functionality? Being integrated into the GH backend seems like a big perk, rather than having to use webhooks for everything.
One thing that's great about Gitlab is the Gitlab server/runner split, where you can run workers in your own VPC, but still use their hosted offering. This makes it easier to keep your deploy secrets (aka the keys to the kingdom) locked down, as they never leave your cloud provider.
People will think twice when investing the time & money in develop an app for the github marketplace as if it succeed, Github could just built the app inside their system. They bought PullPanda which was good for his creator because I think it was less expensive that build that themselves.
> People will think twice when investing the time & money in develop an app for the github marketplace as if it succeed, Github could just built the app inside their system.
That's always a concern with platforms with attached marketplaces (heck, it's even potentially an issue with building apps for OS’s, even if they don't have an attached marketplace—ask Netscape.)
macOS pricing makes sense given that Windows and Linux runners are probably standard VMs on some Azure host, while macOS runners are on their own rack-ified set of Mac Minis. (What MacStadium does)
I'm not sure if for large open source projects, these machines will be nearly enough to run the CI jobs.
The price for Linux seems quite steep when you compare with for example what you pay with GCP.
It will be interesting to see the Github security teams catching those "public" repos doing nasty stuff like mining crypto - even with hard timeouts on each job it will be cool to see how this plays out!
I didn't mean about the costs for large OS projects but on the requirements. As an example, in my experience, I couldn't build the `semantic` project (https://github.com/github/semantic) with those requirements.
> - macOS: $0.08 per min (yeah that's not a typo, it is copied straight from the page, macOS is mad expensive)
Roughly inline with what other CI providers charge for MacOS per minute. Circle CI cheapest plan is 0.0498 per minute, but you need to pay USD$249 per month to get up to 5000 minutes. To make it that cheap you need to use all of your minutes up.
2000 minutes -free- on macOS is unheard of. Is there any other providers do anything like that? Finally I can run my iOS Unit Tests for free without a farm of on-prem Macs!
Hopefully Gitlab will learn that a bullet list of features can only take them so far. If (probably when) Github matches their feature set, the most stable product automatically wins. ("breadth over depth" as a strategy assumes a false dichotomy)
I honestly don't know how anyone uses gitlab with this nasty but that they've been ignoring for years. Had to move off of them and haven't looked back.
I can't help but feel a little worried about this. GitHub's UI has become a lot less coherent as they rapidly develop new functionality. The simplicity/focus was one of the main reasons I preferred it to GitLab. Blurring the line between the two makes it easier to imagine switching, not harder.
Huh, I find the opposite to be true. GitHub is a breeze to use and browse, while GitLab is a convoluted mess. Once GitHub made private repos free I dropped GitLab and never looked back. (Granted I'm speaking of general Git repo features, I personally have no use for CI at the moment).
What have they “always” supported exactly? I’ve used GitHub for a long time, but I use it entirely as a git server and sometimes an issue tracker. That said, I’m pretty sure they didn’t support anything that even remotely resembles CI/CD back when I started using it.
I was using GitHub actions to do build, test and push to production for a while now, with the older syntax (the one without yaml). I'm not sure exactly what changed now, and the messaging is indeed confusing
I wonder what they'll offer for Github Enterprise. Especially for organisations that are using it on prem and would prefer not to use Azure for builds.
Thanks for your interest in GitHub Actions.
You can set a property `continue-on-error: true` for each step and the runner will ignore a failed result and continue the workflow. For more details on workflow configuration please see https://help.github.com/en/articles/workflow-syntax-for-gith...
Workflow runs can be triggered via a `repository_dispatch` event with a custom payload. Using this model you could create a tool to allow for manual triggering. However, we do expect to provide a more integrated experience for triggering manual runs with custom inputs.
More capabilities for actions to post artifacts and reports as part of our experience is absolutely on our radar.
We are working to bring as many new users into the beta as quickly as we can and we look forward to your feedback.
Really interesting. My project just migrated to Azure DevOps and I noticed that their configuration file looks really similar. My suspicion is that Azure DevOps is backing GitHub CI/CD.
Hey there, I’m the Director of Engineering at GitHub with the team working on Actions. We really wanted to get an extremely flexible and powerful SDLC automation platform into our users hands as fast as possible. To do that we borrowed some code and infra from Pipelines to be able to offer a rock-solid CI/CD to every developer on GitHub sooner, and to be able to scale out to 40 million developers. With hosted runners, you’ll be able to run your Actions builds in any cloud, or even on your own hardware; if you don’t want to set that up, builds run on VMs and containers in Azure by default. You can use Actions to deploy anywhere and we have AWS, Azure, and Google Cloud actions to enable you to get started right away. If you want to read up a bit more about our virtual environments as well as what powers our runners check, we have some of that info in our docs here: https://help.github.com/en/articles/virtual-environments-for....
I feel like everyone who uses MS these days and still doesn't carry over distrust cultivated in the 90s knows that the problem isn't going to be lock in. But I'm sure MS's end game is to make you want to use Azure more, which is fair game.
It’s not backed by Azure DevOps, the infrastructure is different and maintained by GitHub. However, the underlying technology is based on the Azure Pipelines technology (a sort of fork).
Not sure I understand. From my understanding Azure Pipelines is a service offered as part of Azure DevOps. Does this make the offerings of Azure Pipelines redundant?
Azure Pipelines remains available and it offers extra features that won’t be in GH Actions - like support for repos outside of GitHub (BitBucket, or Git anywhere)... it also supports SVN and TFVC.
Pipelines is also tightly integrated with Azure and has a richer CD experience (eg the K8s integration)
In fact there is no stable caching (it's being worked on) and that is a major problem with the system. [1] What pushed us over the edge to take the plunge was out of the box support for MacOS builds.
Free CI/CD for repos public and private is one thing that makes Gitlab really attractive to me. If Github's is simple/sane enough I would probably consider that also.
I've been using GitHub actions for my personal projects for the past few months. The documentation was not the best and the UX was not polished at all, so the experience was painful sometimes. A lot of my problems will be solved with the features announced today, except I'm not sure if they're also adding a way to cache e.g. dependencies between jobs.
While writing your own actions was painful (at least for me), reusing actions that other people wrote worked like magic. I think the reusability aspect is going to be huge when Actions get more and more popular.
I do not understand why Microsoft finances this. I understand the hosting as a specialized social platform, but GitHub Actions vs. Azure DevOps Pipelines are a duplicated effort. Better integration for GitHub for the Azure DevOps suite would solve that in a similar way without duplication of capabilities.
All roads lead to Azure. Github's user base is far bigger than Azure's user base, and Microsoft wants to eventually get as many people as possible buying IaaS (or equivalents).
The next logical step after Github Actions for CI/CD is to offer Github as a place to run the production code too. You would be buying Azure with fewer steps. It's pretty arbitrary that we develop code in one place while then going off and using a separate suite of tools and processes to run it. Having a suite for everything in one place could be very appealing to the market.
Yes, MS is intent on grabbing that cloud market as best as they can. It's phenomenal how many technologies they're coming up with to do that. They want you to use their tools, love them, and naturally start using Azure on your next project.
If they added a button that was essentially just zeit now it would be revolutionary. Especially if they had a generous free tier. Imagine one click hosting for blogs, etc. Now THAT would certainly hook people in
Exactly. First you are using Azure through Github actions and do all the leg work to develop Docker images, etc. to test your code. Then you start off hosting on a small free plan. Fast forward a bit and you're paying Azure $$$ because why learn all the IaaS esoterica when you can just click a button on Github.
> github's infrastructure can't be compared to actual paid-for cloud hosting.
I think the idea in this thread is that since github is owned by MS, and they have Azure, that won't be an accurate statement for very long. Github doesn't have to build that infrastructure, they just need to competently integrate into MS existing infrastructure.
I can't believe I didn't piece this together sooner but I agree with this threads premise. I'm already hosting my source on Github so why not build it there? And if I'm building it there then why not deploy it there? Rather than manage a complicated pipeline, I just `git push` and everything else just works ... all the way to massive scale.
As someone who has worked on hand-built github to AWS pipelines ... I can actually see this being the killer feature Azure needs to actually win a large market share.
I think (hope) they intentionally avoid tight integration with Microsoft branded stuff. Even at the slightest indication of such a thing happening, I bet it would lead to an exodus and outrage among developers, however irrational that might be.
This has been played out for decades before and since MSFT's containment to the desktop OS market, but also with most other capital-rich tech companies.
The brand is already MSFT Github, it just hasn't changed the site masthead. MSFT's core strategy and future goals aren't opaque. There's only 2 paths that maximize profit. Either MS bundles it up into a MS toolchain alongside hobbled alternate toolchain support, or abandon Github in parts until it's decommissioned (no need to dilute your brand). Maybe I won't be alive to see it, but I left for Gitlab because I believe this is how Github will maybe survive.
It's because there's no unified grand vision driving this and the GitHub team is an independent silo. Yes there will be duplicated efforts, but at least the GitHub login system doesn't get replaced with Microsoft Account login.
- Market segmentation, the people on Azure DevOps are not likely to use GitHub Actions and vice versa.
- Long term plan might be to bring both of them closer together.
- Build good will (marketing).
Competing against yourself adds expense, but also gives you more ways to win. It can be better to capture more of the market and then converge products rather than letting someone else have the market. Or, if there are different specific needs in different niches in what is descriptively the same product class, don't converge and win in two markets that a naive analysis would have considered parts of the same market and lost part of by not meeting the different needs.
> Better integration for GitHub for the Azure DevOps suite
One small improvement I would really appreciate is a reduction in the number of clicks required to reach the real build status page. Clicking "Details" on a status row for a travis build takes me directly to the summary on travis, whereas for azure it leads to a redundant summary page hosted on github first, and requires another click to actually reach azure.
I've been on the beta for a while, and while it was understandably limited, I really enjoy it and have done a lot with it. I'm excited about the announcement because it appears that a lot more can be done now.
However, it's not clear what happens to existing actions and workflows. Do they just stop working? Can actions still be made from a dockerfile and entrypoint script?
Already used the "migrate-actions" binary on a few projects, and while I don't have the new version enabled (you'll get a notification when the repo is available for an upgrade), this is better in terms of simplicity since you don't have to manage action references or the "resolves" field (when you were manually editing the hcl).
The only downgrade is that it doesn't look like you can run actions in parallel within the same "job"; so you can't have eg. "cd project1 -> npm i" and "cd project2 -> npm i" running at the same time and then have a third action that can use the output/filesytem of both of those commmands. Now the "job" will run one, wait for completion, then run the other, then you can have an action that uses the changes of those.
I also find the ability to process the logs with some other platform like Kibana to be a top feature; I suppose there will be apps on top of this soon enough to meet the demand.
Lots of comments here about what this means for CircleCI. Found it interesting that CircleCI's CEO has a testimonial in the "What our community is saying" at https://github.com/features/actions:
“CircleCI has been building a CI/CD platform since 2011, and GitHub has been a great partner. GitHub Actions is further validation that CI/CD is critical for the success of every software team. We believe that developers thrive in open, connected ecosystems, and we look forward to working with GitHub to lead the evolution of CI/CD.”
Jim Rose
CEO of CircleCI
I work for Codefresh a CI/CD solution for Kubernetes/Helm/Docker.
One of our main goals when creating Codefresh was to make plugins that are not tied to Codefresh itself. As a result we followed the obvious model with plugins where they are just docker images and nothing else.
We are very glad to see that Github actions follows the same model. This means that we instantly get all Github actions as possible Codefresh plugins (and the opposite).
I would be really happy if other CI companies follow suit so that eventually a central CI/CD plugin repository can be created for all competing solutions.
Woahhhhh, MacOS support? That's enough for me to switch from CircleCI right now. $40/mo to Circle is just a lot to do CI/CD for some React Native apps, with CD only getting kicked off once a week or so.
This "sounds" interesting, and I'd absolutely love to manage ci/cd in github. Can anyone point me to some clear docs or articles on its usage? The blog post and developer guide on actions reads like some hybrid of a marketing page and a plain index of terms. Most of the examples on workflows/actions documentation and the like are in HCL which is being deprecated.
THIS. I have yet to even be able to get "echo 'hello world'" running in a container. I want to love this but I cant figure out how to use the damn thing. I don't see any action in the visual builder to "run a shell command" or even access a "Linux container".
I scanned through, but I couldn't see how GitHub Actions supports deployment to 3rd party platforms like PyPI.
Travis CI has a CLI that allows you to encrypt your PyPI password and stick it into the Yaml file (I think it works by encrypting your password and then uploading the decryption key to your Travis account). Will GitHub Actions have something similar, somehow?
Hi I'm an engineer on Actions, and a few months ago I probably would have said the same thing, but I've come around to it (and then some).
We spent a ton of time working through the syntax changes in HCL, YAML, and other languages. What we found was that while HCL and other languages are amazing for expressing pure configuration, software development lifecycle and continuous integration workflows are a blend of configuration and pseudo-scripting, and we were only able to come up with a clean syntax for this in YAML.
For example, although HCL supports heredocs, we felt that they suffered from readability problems versus multi-line strings in YAML when you're just trying to read through a workflow file and see what it does.
There are many, many other reasons we felt that YAML was actually the right choice, and I'm optimistic others will come around to it in the same way I have.
From what I remember, we did look at it some, but I can't remember many specifics, unfortunately. Looking at it now, I think that it wouldn't give us some of the flexibility that we want—for example, TOML arrays can only contain values of a single type, and for the sake of brevity, there were places in our configuration where we wanted to allow you to, for example, pass either a shorthand string or an object.
When you are coming from Azure DevOps (previously Visual Studio Team Services) this is godsend. VSTS was only configurable via a GUI that produced an unreadable configuration file with hundreds of lines with stupidly long variable names. Then taunted this UX disaster by reminding you to comment the changes you made before saving. So each CI config had like 50 changes at the beginning of its history where you tested the configuration by making minimal changes.
Eventually they started using an in-repo config file like GitLab but only supported about 20% of the GUI features. It's still abysmal. I wouldn't mind when Azure DevOps (Server) were to be replaced by GitHub (Enterprise).
Same as what you have to use in gitlab, for what that's worth. But it is interesting that their existing "main.workflow" file is in a different syntax.
Ahh, now I see, support for the main.workflow syntax will be removed, and you must migrate:
That’s disappointing. The original Actions was amazing and much of that was the thoughtful main.workflow file. This feels like a regression. I’m not a yaml hater like many but main.workflow was clean, almost pleasant to use. It felt like it actually came from understanding developers.
What would you like instead? Personally, for those type of things I never thought yaml was limiting in any way, but would be keen to hear what better options you think are out there. I like the simplicity and readability of yaml, as long as there's not too much logic to be expressed.
I think the future is that all configuration is code. Code in a proper functional/imperative language (preferably the one you wrote the app code in), rather than a special declarative language. The attraction of toml is minimalistic and simple definitions initially - but there is always too much logic for these languages. If not immediately, then eventually.
"Real" programming languages have good compilers, good error messages, debuggers, IDE's with syntax highlighters and so on. json/yaml/toml/xml/whatever always feels like you are poking something and then trying it, checking for errors from typos and starting over. This feels quite backwards if you are used to developing with a good strict compiler and debugger.
There are projects like pulumi that provide API's for multiple languages and hook into almost any service (e.g. Azure, AWS)
Yeah, I know pulumi, and I'm working on something similar myself. Personally, in my experience, I don't think there's always too much logic, far from it.
A lot of times you really only have a list of tasks that are executed one by one, and the nature of that doesn't really change. I think it's overkill to introduce a new wrapping layer in those cases (and that is what Pulumi really is, it mostly just wraps Terraform if I understand right). But yes, sometimes it's better to have 'real' code at your disposal. It's just a matter to know when's when. Like always :-)
Also, debugging those yaml problems or real code doesn't really seem to make a difference for me. It's both equally annoying, and since there is a 'declarative' layer involved most of the time anyway, just a bit deeper down, debugging the 'real' issues (not typos and such -- btw. check out yamllint if you don't know it) is not any easier with wrappers like Pulumi. But maybe we have just worked on different type of complexities in our pasts. That usually explains different preferences for tooling. Just saying that this is not a thing we should/can generalize.
Sadly my company's account hasn't been granted access yet :( We are currently evaluating a switch from our current CI/CD platform and this would have been perfect. Can't wait for November general availability!
When I clicked the link to sign up for the beta, it let me select my personal account or the one organization account that I own. So based on that I strongly suspect that the answer is yes.
Considering CircleCI has many more years in operation, are there features and use cases it supports that GitHub Actions doesn't? Are there ones that GitHub Actions already supports that CircleCI doesn't?
GitHub is way late to the game here. I personally have started moving away from GitHub. They don't even offer an open source version, yet try to claim to be some gift for open source. GitLab is better, but their architecture is a bit more complicated than it should be. I really like sites that use Gitea, but they haven't integrated CI/CD like GitLab, nor have they proven to scale with their own hosted solution. When I think about the point of these solutions though, there is a lot of opportunity here. Heck, Git is decentralized and distributed by itself. Really people just want a place to manage their issues and MRs, and I hope someone will decentralize that with a native app and then add additional features that integrate into multiple providers with ease.
That’s awesome! I’ve always been a big fan of GitLab CI but I’ll have to take a look at this and see how it stacks up. Competition has proven to be good for the market of Git hosting platforms.
That’s pretty huge. I wonder if it’s possible to have the results committed to the repository. To be able to display a badge in the readme. Like 87% test passing.
Hi, Coveralls CTO here, the pass/fail status of any commit is shown as a green check / red x in the commit list and at the top of the repo details page as the "most recent commit".
As for a % badge, you can use our new Coveralls GitHub Action to post your coverage data to https://coveralls.io, then add the badge to your readme:
They just raised more funding in a Series D[1]. CircleCI isn't the only player in the space and GitHub entering the playing field doesn't change that either. There's enough space for a few competitors and having competition will probably push everyone to build a better product.
As a developer, in the short term I love this. Fewer things I need to cobble together and worry about how to integrate. I mean, it's already the case that if GitHub goes down that my CircleCI jobs won't work, so having one company to yell at and monitor alone is a plus.
But long term it makes the competitive ecosystem much less robust. And as a startup employee, makes me feel how disrupting established platform competitors gets that much more difficult - even if you have a better product, it's hard to fight against the "platform" as they have more integrated points of value.