Studying the design document [1], one sentence makes it clear why this project is needed and simply using the Dropbox client does not suffice:
> We can perform a compare-and-swap operation in Dropbox by using the "update" write mode with a specific revision number.
Since a Git repository basically consists of a hash-addressed content object store alongside a mutable list of references (mapping branch names to hashes of latest commits), you need a compare-and-swap operation on the reference list to update the branch reference when you push new commits. The Dropbox client by design does not do a compare-and-swap whenever a file is updated, but the Dropbox API supports it.
I'm not sure how this is "needed" just because it is possible. Setting up a small headless git server takes 2 minutes without sharing your code with every intelligence agency on the planet.
Could be alluding to the appointment of Condoleezza Rice to the Dropbox board, and implication that Dropbox just gives them access versus requiring pesky NSLs or warrants (given Rice's defense of those NSA programs).
If it's for your own use, you can skip the part about creating a `git` user, and host the files in eg `/home/$USER/repositories` instead. The repository setup instructions remain the same, just the path differs, and the `authorized_keys` file to add keys to will be in `/home/$USER` instead of `/home/git`.
It's so simple, I think it's worth showing just how simple. For a single user who has already SSH access to some server, and has installed Git on that server with apt/yum/whatever:
I deployed Gogs at home and am loving it for things I don't want anyone else to have access to (mostly hobby projects with some secret keys in them), and Gitlab is just so fantastic nowadays that I'm adding projects to it instead of Github.
I wish they would fix their Github syncing (they can currently do it, but don't tell you their public SSH key so you can add it to Github as a pushing/pulling key).
Thanks for the 'fantastic'!. We’re always trying to enable our users to integrate with more services/tools. I would love to have your input on this specific topic at https://gitlab.com/gitlab-com/support-forum/issues/695
I agree that Gogs works very well for small repositories.
Just a heads-up to anyone planning to use Gogs for large repos: Gogs is basically unusable for large repositories( such as the git repository itself and the linux kernel). The response times through the web UI can extend into a few minutes on small servers( I tried it with a 2 cpu and 2GB RAM server on DigitalOcean) and even on extremely powerful servers, the response times for large repositories is almost the same( I tried it on a 8 cpu and 16GB AM server). The reason for this is that Gogs lacks a cache system for git repos. You can follow the issue here[1].
It's not documented anywhere, so it's possible they use different keys for different users, but I was able to figure out they use this key for my account:
The OP seems to be interested in a SaaS (alternative for GitHub). For the people that think you always have to install GitLab, instead of running your own GitLab server you can also use GitLab.com with unlimited projects and collaborators, see https://about.gitlab.com/gitlab-com/
Has Gitlab done any work on the high availability front? I remember reading a while back that gitlab.com was hosted on 2 servers, or something like that.
but if that's the case, I would be extremely nervous about the integrity and stability of their platform. I've had too many bad experiences with NFS mounts failing and general instability that I don't want to deploy it anymore.
We're currently using 20 Azure instances to host GitLab.com. Git data is stored on an NFS server though I can't recall the exact setup from the top of my head.
I've never tried it, but couldn't you theoretically make another github account using your email (assuming you use gmail) with a "+whatever" in it, and use that to create an infinite number of new github accounts? I believe gmail redirects you+something@gmail.com to you@gmail.com.
Of course, just using Gitlab is the more sensible option :)
[EDIT] - as lfowles pointed out, github doesn't private repos on free-tier accounts, so this is pointless.
I'm pretty much using a single
private Git repository where I store all of the things I don't want to open source. When I'm the only one using that repository, I don't see the issue with doing so.
http://visualstudio.com Team Services is also pretty nice. Free unlimited hosted git repos for 5 users, also includes free build/test/load test/release/CI management, kanban boards, team chat rooms. eclipse/xcode/xyz integration, etc, etc. If you've got a small team, not bad.
> Why shouldn't I keep a bare Git repository in a Dropbox shared folder, use it as a folder-based Git remote, and sync it with the desktop client?
> There seem to be some articles on the Internet suggesting that this is a good idea. It's not. Using the desktop client to sync a bare Git repository is not safe. Concurrent changes or delays in syncing can result in a corrupted Git repository.
How dangerous would this be in practice? Ie, what are the chances of corrupting a git repo with this method with a small team, and what exactly must happen to cause it?
I use this for some OSS projects I help maintain (symlinks in Dropbox, actual dirs on my GOPATH, etc.). The use-case for me is syncing between my laptop & desktop without having to remember to git pull everything before a flight/trip.
There are occasional conflicted copies (once every few months), but it's rare and I don't recall encountering one in quite some time (months).
Tips:
* Don't work on things concurrently
* It's brittle (like any sync service) if you're editing things when that folder is still updating. This is especially so when it comes to git.
* Repositories with lots of small files take a long time to sync.
I use this method by myself, and even then I had conflicts where Dropbox would fail to sync for some reason and then I would have git tree conflicts. But this not a big problem, since Dropbox saves both versions of conflicting files and you can just delete the conflicting copies, and then merge changes normally using git.
Even if it were possible to corrupt the repository, corruption would result in obvious failures rather than subtle data loss because of git's use of cryptographic hashes for everything, and the fact that local copies are full repositories makes it easy to nuke the Dropbox repository and recreate it if you need to.
I didn't downvote you, but I don't really agree this is not an issue. The article itself shows that the git repo can get into a confused state which you have to fix, even if the fix is simple. The experiment itself doesn't emulate common causes of Dropbox pain. Firstly Dropbox stores and syncs files in a way that gits number of files and layout of files is not kind to, causing thrashing and high cpu when people commit. Different clients have different ways of locking a git repo. Issues occur when two clients make changes and are online at different times. I'm not saying everyone will hit these issues, but these are all issues that someone has run into.
Important thing to note — this may not be the best solution if your goal is to always have an up to date local (dropbox synced) version of a git repo. The docs discourage that:
> If you're using the Dropbox client to sync files, it's a good idea to use selective sync and disable syncing of the folder containing the repository to avoid any unexpected conflicts, just in case.
Also, in case this wasn't obvious, this is basically a "shim" to use dropbox as a git server (as opposed to say mercurial or SVN). No actual git server is being run and if I understand correctly, some useful features from a normal git server are not implemented.
> git-remote-dropbox is a Git remote helper.
> git-remote-dropbox stores all objects as loose objects - it does not pack objects. This means that we do not perform delta compression. In addition, we do not perform garbage collection of dangling objects.
> As far as I know, git-remote-dropbox is the only safe way to host a Git repository on Dropbox.
I've another hackish method that's good enough for a single developer, plus it's encrypted.
I've got a truecrypt file container that resides in Dropbox. I mount it as read-write when I want to push. On the rest of my machines, its mounted as read-only. Even if the container is 100GB, it takes 10 seconds to sync daily commits - because of block encryption & syncing. This won't work with Google / MS / Amazon drive etc. because they upload the entire container on each incremental change.
This feels dangerous. While I'm not sure, I don't think changes to a truecrypted filesystem are guaranteed to be atomic. I'm pretty sure there's no guarantee that the filesystem is consistent every time Dropbox starts syncing. You probably end up with a partially correct file system on the ro-side, which might end in kernel panics if you're unlucky. Also it's not two-way as the git-remote-dropbox solution offers.
Uh, this is also dangerous to your security. The truecrypt docs contain very large warnings about this[1]: someone who can see your disk at multiple versions may be able to drastically weaken your security. If you had keys in those truecrypt files, you should rotate them; if you had secrets, consider them compromised.
Truecrypt uses XTS mode, which is Not Great -- it only exists because someone was trying to hammer ciphers into fitting nicely with fixed sized disk sectors, and it makes a number of serious compromises to do so. [2] has a good discussion of this. You do not want to combine XTS with sharing multiple versions of the ciphertext.
tl;dr you've seen the penguin pictures? As [2] cleverly says, you're pretty much doing that to yourself by sharing multiple versions of the file.
There's also significantly more powerful attacks that can be mounted by an adversary that can feed you corrupted blocks which will allow them to permanently compromise your key, or perform other hijinks like manipulating your content without setting off warning bells. Dropbox, or someone that compromises dropbox, is in that position, by nature of the service they're providing. These issues are rooted in the fact that XTS is not an authenticated cipher -- this leads to such an endemic category of subtle problems that it's been nicknamed "The Cryptographic Doom Principle" [3].
So yeah. If your life depends on it... don't store a truecrypt volume in dropbox.
> IMPORTANT: If you store the backup volume in any location that
> an adversary can repeatedly access (for example, on a device kept
> in a bank’s safe deposit box), you should repeat all of the above
> steps (including the step 1) each time you want to back up the
> volume
... where "step 1" is "Create a new TrueCrypt volume".
Reading through the code (20 second skim) it does not appear that this remote would reveal any information to the OP such as your dropbox access token or code being pushed.
I find it interesting that Dropbox isn't building these types of utilities themselves. This looks pretty cool and also seems like a really good way to up-sell Dropbox storage later.
Well, they sort of did build it. Anish was an intern at Dropbox when he wrote it. I'm not sure if he started it as a personal side project, but it was finished and released during a Dropbox Hack Week.
Shameless plug: I've implemented something similar in the past, but with the addition of client-side encryption. I've been using it for some time without problems.
At a first glance at the design docs here, the present project looks more sophisticated in terms of syncing. I'm just using local file operations and hoping for dropbox to do its stuff :)
I use Syncthing to sync around my personal git repo between my machines. All pushes and pulls are instant. I don't have conflicts because I'm only working on one machine at a time. `git-remote-syncthing` would presumably be even better, allowing multiple simultaneous users.
What I meant to say is that I generally use git via the GitHub Desktop app, not the command line. I don't think it is possible to point GitHub Desktop to a Dropbox repo. Or is it?
I love svn, but appreciate some git things as well. Particularly the way you can do a couple of commits before pushing; I don't think there's any equivalent in Subversion?
If you understand how git works, you don't need a tool to do this, and especially not a python tool. Don't misunderstand me, I love no language more than Python. But putting a directory under Dropbox supervision is the easiest thing you can do. That doesn't change because it's name is ".git/".
Are you sure you've correctly identified the "this" in question? This enables using Dropbox as a shared remote repository, which is more than putting the directory under Dropbox.
> We can perform a compare-and-swap operation in Dropbox by using the "update" write mode with a specific revision number.
Since a Git repository basically consists of a hash-addressed content object store alongside a mutable list of references (mapping branch names to hashes of latest commits), you need a compare-and-swap operation on the reference list to update the branch reference when you push new commits. The Dropbox client by design does not do a compare-and-swap whenever a file is updated, but the Dropbox API supports it.
[1] https://github.com/anishathalye/git-remote-dropbox/blob/mast...