I read through the getting started guide and I have a hard time seeing how this differs from an existing workflow I use with just pure git.
Basically, having a stack of patches as commits in my local branch. Edits to older commits saved via fixup commits, and occasionally merging in the fixups and/or rearranging the commits with rebase when its convenient to have a different commit on top of the stack or when I'm pushing a patch out from the bottom of the stack.
Probably the main value-add of StGit versus the git-only workflow described is that StGit makes operations on its stack of patches fast and fluid.
Reordering patches with StGit is trivial with `stg push` and `stg pop`. Arbitrary patches can be combined with `stg squash`. A modification in the work tree can be incorporated into the top-most patch with `stg refresh` or an arbitrary patch in the stack with `stg refresh --patch <patchname>`.
In other words, StGit provides a command vocabulary for performing these kind of patch-stack manipulations in a first-class manner.
But as you note, everything StGit does can be accomplished with plain git--albeit with more or less difficulty depending on the particular stack manipulation goal.
A sibling comment mentions interactive rebase. I had never thought about it this way but if stacked got gives you a command vocabulary / imperative interface to adjusting your local commits, I would say `git rebase -i` gives you a declarative interface.
I prefer the latter; it's easier for me to prepare a bunch of modifications in an emacs buffer / vim tab and then have git go and do them all at once rather than try to maintain the state of the commits in my head and plan out the commands one at a time to achieve the same result.
Good point that StGit's docs could benefit from a compare/contrast with interactive rebase workflows. Thanks!
> slightly nicer interface to interactive rebase
More accurate to say StGit is an alternative interface that does some of the same things as interactive rebase (and more). For me, part of StGit's value proposition is that it is not interactive. A single imperative StGit command may accomplish as much as an interactive rebase session.
Workflows using `git rebase --autosquash` feel more similar to StGit because of their non-interactivity.
Describing a patch queue development workflow as a "slightly nicer interface to interactive rebase" does it a tremendous disservice. It's like saying "a database is a slightly nicer interface to a filesystem" — technically true, but a disservice.
I tried StGit recently, having used rebases for a while now. In practice, StGit operations are much more intuitive and easy to learn than rebasing. Conflicts are much easier to track down and resolve compared to rebases. Personally, I find rebase operations so confusing that I often do multiple rebases on the same branch, a few operations at a time. It's easier for me to track down rebase conflicts that way.
Rebases and merges use diffs behind the scenes, although git stores the commits as snapshots. There is always a conflict of the two mental models while rebasing. With StGit, it's explicitly just patches that you can inspect anytime. I assume that this is the reason why StGit operations make more sense than rebasing.
Another important fact is that you don't need to wait till a feature-branch is complete to edit the history. With StGit, you can craft the ideal commit history along the way while you develop. I treat a stack of patches as multiple staging areas - each with specific purpose/feature and a predefined commit message. That way, you can develop code hunks in whatever order that makes sense, and then check them into (refresh the patch) the appropriate patch. It's even possible to edit or reorder these patches as we progress. Finally, the patches can be committed when the series of features are complete.
You miss the important point here. With rebases, crafting the history is a separate step that can be done only after you've made a few commits. Whether you do it continuously or as one batch makes no difference. With StGit, crafting the history can start at the very beginning of the feature development. It goes hand in hand with the development and can be refactored any time.
I'm afraid I still don't understand the workflow. I continually rebase my branch(es) on top of upstream.
At any time, I can choose to consolidate a few of my WIP commits into a more (logically) cohesive commit and keep that around for further rebasing.
I must be missing something here, but I'm not sure what.
(If it helps understanding, we work with Gerrit which basically expects a 'list of patches' on top of the branch you want to commit your changes to. So that's already part of our mentality, but we just use plain git.)
This is mostly what I do as well; I make a branch, make a ton of little tiny commits, and then squash and merge them on GitHub with a better commit message. Best part is the series of commits are preserved on the PR in GitHub if I ever want to go back and look at the process.
I actually think the value add with stacks is a bit different. It makes pieces of code far too large to review in one PR manageable.
To compare: I often use the workflow you described. But large PRs, I might rebase, and then rewrite my entire history as a clean series of commits, each incremental and easily reviewable.
Most often, the commits while working and the commits at the end look different, but the final presentation of a series of clean commits makes life easy for reviewers.
This is my way as well, but in part it's because we use gerrit instead of git(hub|lab) which explicitly allows for this workflow of separately reviewed patches that build atop each other. I'm curious, if you are using git(hub|lab), how does it work with their workflow?
It's okay on github, if somewhat implicit. You can click the "commits" tab of a PR, and then clicking any individual commit brings up the usual github diff view (like you'd see for the whole PR). It's not immediately obvious but you can leave comments in that view, unlike the other view for a single commit on branch history.
But I find most people on github end up reviewing the whole PR as one unless we all are on the same page ahead of time. So despite supporting the functionality it's not all that highly used compared to, say, the way individual patches in a set might be reviewed on the Linux mailing lists during later revisions. I feel like on github it's more common to see asks to break up PRs (maybe just as a result of the projects I've seen? both approaches have valid usecases imo, and there are good reasons to break up PRs which can be done in single commits anyway).
I still feel it's good hygiene even if reviewers don't look :)
I used to use kiln from fogcreek and found that it's code review feature was much better suited for commit by commit style reviewing than either GitHub/gitlab.
On GitHub it tends to push you towards smaller and smaller PRs, which is generally good, however I kinda miss the "full context" of having a larger change on a single PR with well thought out commits
It seems like Phabricator supports this type of workflow, and I really wish there was wider popularity so I could push for it at work. The lab|hub style PR ends up with people falling into giant PRs that are difficult to review.
Same. With magit's "instant fixup" it's quite easy to maintain a stack of patches on your branch. I didn't realise this was considered anything special really.
I’d say that highly depends on your habits and how your brain works.
Some people prefer to focus on one topic at a time, have their current branch open, and choose not to give in to opportunistic edits if unrelated to the task at hand.
Others prefer to work their way through a piece of code for example from top to bottom, making opportunistic edits along the way, even if unrelated to the task. Not having to switch branches can be convenient in that case.
I’m 100% in the latter team.
Near the end of a coding session, my `git diff` often shows three or four completely unrelated sets of changes. Some might call that a rather unproductive and annoying way to work. But I really love it that way. My brain just _craves_ to fix that typo or convert that tab to spaces right away, even when I’m supposed to fix a bug right now. I love dealing with those things right away so I get them out of my mind and don’t have to put them on a mental stack — even if that means a little more chaos later when it’s time to bin my edits into different commits.
Arguably, I _could_ switch branches for such edits. But frankly, I’m used to opportunistic editing, and sticking with that approach feels just super convenient.
This is me. Commits closer to upstream (older in the branch) are usually more stable, easier to merge, probably already out for review. The newest commits at the tip of the branch are the crazy WIP changes I'm considering, and not ready for prime-time.
> Why would I use Stacked Git rather than Git’s fixup/interactive rebase dance?
Yeah, dance is good analogy. Lot of training, one little mistake and it is ruined.
For example, if you need to introduce a change in commit "one thing" which would require merge conflict resolving in "other thing", with stgit it would be much easier than with fixup/autosquash.
Or another example. You need to put away "other thing" for a while, and work with "one thing" for an hour, and then apply/fix "other thing". Another set of git tricks needed.
> But I’m not sure I’d take on a new tool dependency just for that.
If you are happy with git, nobody forces you to.
By the way, you don't have to use stgit all the time. I usually just use git, but when a change in a branch requires changing of previous commit, I simply do: stg init; stg uncommit; stg pop. It's easy to convert local repo state to and from stgit.
The main reason I've heard of people using tools like this is so people can review a commit at a time (a lot of review tools don't handle this well) and CI runs on each commit.
Generally, it sounds like people are simulated Phab with these tools which is trying to simulate the email patch workflow.
We use gitlab and review a commit at a time and do CI runs on each commit in the MR branch. Just plain git with the fixup/interactive rebase dance. Gitlab is not optimized for this workflow, but it’s liveable.
Gitlab-runner can execute any shell commands, so it’s easy to just loop through all the commits and run bazel test.
Gitlab has improved lately and it’s now easier to review each commit in a MR, but you can sense that most of the PM attention is on the single commit/squash everything MR workflow.
With the kind of git workflow you describe, bringing StGit into your tooling/workflow may not be worthwhile.
I'd say if you wanted to do these kinds of stack manipulations a little bit faster or with a little less cognitive overhead (e.g. named patches vs SHA1 hashes), then investing in StGit might pay off.
I generally would agree that using StGit isn't going to be a slam-dunk win for someone who already has a capable workflow using interactive git rebase.
Can you explain what your tool does to someone who knows what an interactive rebase is? I don't get much more from your blurb than "we reimplemented interactive rebase for some reason", but I'm sure you do something different or unique...
if the commit name is more readily at hand, just name the commit "fixup! whatever". --fixup just does this for you since references are shorter and easier to type (it doesn't have to be a hash, it can be --fixup HEAD~2 or something).
Cool. I've been using this for many years. Before that, I used quilt[1], and before that, Andrew Morton's patch scripts[2].
It's a great workflow that makes you look like a genius who rarely makes mistakes. Absolutely vital if you need to maintain a set of patches against a bunch of branched source code (e.g. maintaining an open source driver with variants upstream (linus's plus stable kernels) and also for various distro kernels.) This was my main use case for years, but I still use it even for things that don't fit that pattern, just because it's convenient to do so.
Sure everything that stgit does can be done with git by itself (duh, that's what stgit is doing under the hood), but I find it's a lot easier with stgit.
@jpgrayson (since I see you lurking around here) Thanks for maintaining stgit! I've been a recent convert and it's a great workflow improvement. I used to be a big `git rebase -i` user and stgit works so much more fluidly and fits great with my mental model of how I want git to behave. Thanks!
I'd love to add functionality that mimics `git rebase -i`. That is, you would open an editor and be able to select which patches you want on your stack as well as possibly designate patches as 'squash' or 'fix' from your editor. Think of it as `stg sink`, but able to operate on multiple patches at once.
Prior art: this script[1] already performs a re-ordering of commits but in a pretty hacky way. I'd like to productize it!
I'd love to have this new `stg rebase --interactive` be part of the main repo to enjoy the benefits of the existing test suite. My question for you is around how to include the new command with the rest of the tools. Would you want it to integrate with the existing rebase command (`stg rebase --interactive`) or is it something more appropriate for `contrib` (so a new independent command like `stg-rebase-interactive`)?
I think I get the gist of what this interactive script does and how you propose to extend it. A PR for such a script to go into StGit's contrib directory would be non-controversial. Maybe a little higher bar to have this capability as a first-class StGit command.
Thank you for the great tool! I gave it my first try today, and immediately fell in love with it! It's a shame I didn't know about it sooner - could have been a great improvement to my workflow.
> Would you mind expanding upon this? I'm really curious.
Not GP, but I share that sentiment. Let me add my two cents. Here are some of the things that confuse me while doing interactive rebases in git:
1. What happens when I delete a commit? Won't the subsequent commits still hold on to its changes, since git uses snapshots? (Nope, that's not how it works)
2. Why does a rebase conflict happen when we try to squash or reorder a few commits? Yes, we can see the original commits, but we have wrack our brains to understand what trips the algorithm.
Git internally uses snapshots to store commits, but uses diffs for rebasing, merging etc. Knowledge of that fact reduces the confusion a bit, but doesn't relieve the cognitive overload. With StGit, they are clearly just patches that you can inspect any time. The reason for any conflicts or results are immediately apparent when you see the patch - much the same way you know why a program misbehaves when you see the source code.
> Have you tried Pijul too?
I have tried Pijul, but didn't stick with it. It didn't feel mature enough yet. However, I sure am going to keep trying until it feels right. What excites me about Pijul is that the author stresses the advantage of patches over snapshots when it comes to merging and rebasing (both are the same in pijul). If the combination of git and stgit is any indication, pijul is likely to be way more powerful. I am excited about its future!
Your comment helps! I've noticed in my own local workflow that I've slowly been gravitating towards similar conclusions that you've made.
Pijul excites me too in this regard! I'm trying to use it concurrently with git in local projects, but perhaps stgit would be a good in-between solution.
May be my hints can be helpful. I have much less cognitive load developing multiple features or task switching.
# coding some-task
# critical bug approaches
$ stg refresh; stg pop # save current changes and pop patch
$ stg new -m 'fix of critical bug' # start working on new patch
# fix fix fix
$ stg refresh; git review # commit fix
$ stg push # bring some-task to the table again
# continue to coding some-task
It can look similar to branch model. But devil in details. `stg pull` allows to not think about rebases. pop/push/sink/float allows to toss around patches. I started to think in terms of atomic features and not branches and commits instead.
I think this maps to my model when working on things too. How often I do fixups and rebase is a lot, and I effectively treat the different commits as independent patches, but having a parent (as Pijul allows).
I have also several git worktree's to facilitate working on multiple things at once though, so I don't necessarily need to switch away in my current repo, cause I just go to one of my 4 worktrees that I always have. As I explain this out loud though, it does sound like a very stopgap solution!
Preface: I love Stacked Git and have convinced many co-workers to use it and they all love it. It's really great, especially when used with Phabricator!
---
A question for the maintainers...I was enjoying the command `stg publish` to allow myself to keep refreshing patches locally but every so often push to a branch that others non-stg users were pulling and pushing to. The command was removed in 1.0.
I'm curious why it was deprecated and removed and if there is a good replacement flow for working with non-stg users on branches I can't force push to.
I removed it because it was challenging to test and maintain, its semantics are somewhat complex, and it wasn't clear that it provided value to anyone.
If you wouldn't mind making your case on the issue tracker [1], I will reconsider the fate of `stg publish`.
I'm not using it any more, but for a while I used stg publish as a way of working with a stack of patches but also providing other people with a "this branch doesn't rebase, you can pull it" interface. It was also nice when working continuously with the same stack of patches (periodically rebasing them onto a moving upstream, cleaning them up, etc) to have the publish view's commits as a record of "this is the delta since the last publish". But it's been years since I had a need for the functionality...
These two commands ultimately do different things. `stg publish` was in the business of not force pushing and only committing the 'delta' between what lived on a branch and the currently applied patches, so that you can `git push [no -f]` to a branch.
Definitely cumbersome to use, but it had fair reasons to be cumbersome.
I have been using quilt a lot a few years ago. We have been working on some embedded systems,and were receiving the base firmware sources of questionable quality from some shady Chinese manufacturer as a zip (or sometimes rar) archive on a flash drive sent by mail. We had to maintain our own "overlay" of changes applied on top of it. The discovery of quilt back then was such a relief!
There are so many skeptic comments in the thread. They are like: why we need cars then we have horses? StGit is not a trivial wrapper around simple git commands. It changes a workflow completely and brings great value into a process.
I've been using StGit for more than 10 years. And I want to thank authors to free me from `git rebase` and `git stash` and branch switching burden. StGit is a so very fun part of my everyday job!
I discovered StGit just a few days back and tried it recently. I found stgit operations much easier to understand and learn (less than an hour), compared to regular rebase. Another important advantage that StGit adds is that we can craft a nice commit history while we are coding. Don't need to wait till the branch is completed to edit it reasonably. For me, StGit is a sure addition to my workflow. My only wish is for StGit integration with magit. The emacs package that exists now is too outdated.
Are tools like git-absorb safe/reliable? Git-absorb is a port of hg absorb:
"Essentially, when your working directory has uncommitted changes on top of draft changesets, you can run `hg absorb` and the uncommitted modifications are automagically folded ("absorbed") into the appropriate draft ancestor changesets. This is essentially doing `hg histedit` + "roll" actions without having to make a commit or manually make history modification rules."
I haven't (yet) wrapped my head around the algorithm. I get that an algorithm can "recollate" a series of commits in a way that yields no commit conflicts, but that's not the same as rearranging and combining commits into a sequence of semantically coherent atomic commits.
I use git-absorb a lot, it's been my daily driver since I discovered it.
It is very conservative in what it does: if it doesn't know what to do, it does nothing. And in any case, each of the changes it introduces is a separate commit, so you can reorder or amend them however you like.
Also, what a shameless plug I am, have a look at git-crecord to interactively commit files line by line.
Can someone who has used this comment on whether learning the newish terminology and workflow pays off, vs doing what I do now in situations which this seems tailored for: multiple branches, cherry-picking here and there and/or interactive rebasing when features have been tested and are ok to put into master, likewise when one feature needs changes from another branch?
I use it all day every day, and I really like it for the workflows I have, which are basically contributing to open source projects that use the "send patches by email for review, features should be sent as patch series with each commit doing one thing for ease of code review" workflow. If the thing you want to send off for code review is not "a commit" but "a series of maybe 10 to 30 carefully crafted commits", stgit just makes it very easy to create that series of commits and work with it, putting code changes into the right patches, reshuffling the patch order, etc, as you go along. It's possible to do all that with interactive rebase, but I never got the hang of that and find the stgit interface much more intuitive. In particular, the 'git rebase' manual suggests that the typical rebase workflow is "create a lot of commits, and some fixup commits, and then occasionally tidy up with interactive rebase". The stgit workflow is much more "keep your patchseries tidy as you go along" because it's trivially easy to say "actually this little change should go in the patch 15 commits before the one I'm currently at", or "actually let me unstack these last 5 commits and spend the next day creating patches that logically belong before them, and then tomorrow push those last 5 commits back on again". I use separate branches for different topics/features, but basically every branch is an stgit patchseries.
I originally used mercurial and its queues (mq) extension and then found StGit when switching to an organization using git for version control. Thus I got used to the patch-stack model prior to really being exposed to git's interactive rebase.
Some days I wonder if an interactive rebase oriented workflow would be good enough for me, but manipulating the patch-stack with StGit has become so second-nature that even the small amount of extra friction of `git rebase -i` sends me right back to StGit.
Funny to hear that you learned patch queues with mq. mq was my favorite part of mercurial and it's how I learned to use patch queues too.
Thank you for maintaining StGit — I really enjoy it as a tool! topgit is fine, but quite heavy. I don't use stg much at work with squash merge workflows, but I love it for personal projects or when I need to work on a series of commits with intricate change sequencing.
Interesting. I usually manage this workflow via rebasing my commits, re-ordering commits (can be moderately painful depending on merge conflicts), then squashing fixups together.
Although I imagine you'd still run into problems when you push a commit you've popped and there's a merge conflict.
The feature description sounds to me as a subset of what git's own packfile format is. [1]
Git packs, however, don't crash on rebases or commit changes, and can be merged whereas a stack of patches would likely fail in the beginning and then is inefficient to work with if other patches rely on those changes.
Maybe someone can explain why a stack of patches might be better if you need to use git anyways?
That sounds extremely useful, indeed. What are the caveats/corner cases? I assume it does a lot of debasing behind the scenes, what happens in case if conflicts?
You are correct that conflicts can occur when pushing or reordering patches. These conflicts are resolved with the regular git mechanisms. I.e. conflict markers in files, use `git add` to mark conflicts as resolved, and then `stg refresh` to update the patch with conflicts resolved.
I had the strong feeling that this can be very helpful, but cannot wrap my head around why.
Can someone explain why this is useful? This means I can maintain a set of "private" changes to a branch, say, better log messages, have this applied to the working tree, do non related changes in a feature, and commit my work without having to deal with the log message's lines?
How does stg deal with periodical merges from remote/master into your current branch to keep things in sync? I find that with rebase, having master merges dramatically increases the complexity of rebasing, particularly if the code you are working on has been touched by others on a different branch.
There's no magic in resolving merge conflicts, stgit helps only to some degree.
For example, stgit allows you to not fix all conflicts at once. You have a stack, and you may have first commit in a stack rebased, have a nice usable repo state, and return to rebasing of the second commit tomorrow.
Obviously `stg push` and `git push` are different commands, but to your point, yes, they have very different meanings and the overlap is somewhat unfortunate.
StGit uses push/pop in the sense of pushing and popping patches to/from the patch stack. An alternative such as apply/unapply would be valid and could be aliased, but this ship sailed 16 years ago (StGit has been around almost as long as Git) and so is unlikely to change.
Gerrit solves review part of a patch-oriented workflow. But leaves commit juggling to end-user. It helps a little to manage such patches in workging directory. StGit helps to organize the workflow locally.
I use StGit with Gerrit. My workflow has nice property: single StGit patch maps to Gerrit review directly. Managing multiple simultaneous reviews is a breeze.
Basically, having a stack of patches as commits in my local branch. Edits to older commits saved via fixup commits, and occasionally merging in the fixups and/or rearranging the commits with rebase when its convenient to have a different commit on top of the stack or when I'm pushing a patch out from the bottom of the stack.