It has 12 "how" sections of code snippets, each with 12 language options. That means 144 code snippets need to be correct for this page to be 100% correct! And we don't auto-generate any of these snippets, since auto-generating idiomatic code is a rabbit hole that's consumed many a good developer.
So we, like Etsy, have come up with a separation of concerns between How and Why that works for us. The text on the page is maintained by a tech writer. The code on the page is a series of blocks that look like this:
These snippets can be maintained with all the wonderful tools that exist in the world for maintaining code! We don't need to think of them as just docs. So that "snippets-android" repo has continuous build, a robot that updates dependencies, PRs/Issues for third-party contributions, etc.
What we've found is that this system allows us to maintain many thousands of code snippets with only a handful of people watching over them.
I can't upvote this enough. It also means it is easier for all internal people (not just tech writers or engineers) to make modifications (such as fixing typos, grammar, or adding missing context) safely and without needing to set up complicated tooling/tests or annoy people beyond review diffs+approval.
I noticed the same system in the Google Cloud python samples repository. It's actually quite useful to have the full code available and to easily see which parts are displayed in docs.
Feature request, though: include a url in the source code that takes you to the snippet's usages in documentation. Something like // [START https://codesamples.google/firebase/set_document ] or just the url as a separate comment.
I think the part about this that is truly successful, is the use of slack significantly lowers the transaction cost of adding the documentation. In a wiki, where there is a certain level of quality expected, you might make the decision to just move on with your life once you find the answer you need, as you've already wasted enough time. With this, however, it's so cheap to leave a breadcrumb it's just polite to do so.
> the use of slack significantly lowers the transaction cost of adding the documentation. In a wiki, where there is a certain level of quality expected ...
The wikis I've used also have slow editing interfaces because they have separate read and edit modes. Does anyone know of a modeless wiki, which probably means one that allows you to edit-in-place?
The problem with modes is that to edit something on a sizeable page you must,
1. Load the page in read mode
2. Read/search to locate the spot on the page where your edit goes, read the context around it, and plan the edit.
3. Load the page in edit mode
4. Read/search to locate the spot on the page where your edit goes
5. Type your edit
6. Reload the page in preview mode (a form of read mode)
7. Read/search to locate your edit and verify it
8-?. Repeat steps 5-7 until your edit is correct.
N. Reload the page in read mode.
Compare to a modeless wiki:
1. Load the page in read mode
2. Read/search to locate the spot on the page where your edit goes, read the context around it, and plan the edit.
Have a look at Nuclino (https://www.nuclino.com), essentially a real-time wiki for teams. We're eliminating modes the way you describe in order to speed up the workflow. It supports edit-in-place, changes are automatically saved and you can collaborate in real-time (like in Google Docs), but you also have core wiki features like easy linking and consistent styles.
How easy and efficient are other wiki functions, such as creating hyperlinks between documents, creating new documents, and a simple, structured style system? Last time I looked at it for these purposes, Google Docs failed in those areas.
That's sad to read, since Wiki was originally invented to create a simple user-editable document, to solve the problem of "just move on with your life once you find the answer you need, as you've already wasted enough time"
This is just the next iteration following the same pattern down its logical path. Assuredly at some point in the future, we'll find another faster and more convenient way to convert real-time answers into documentation, and we'll seize it.
Why isn't the how-docs tested? I don't work in ops or anything like that, but it seems to me that it'd be the best way to ensure everything stays up to date.
In Rust (and many other languages), when you document a function, you usually add a code snippet that explains how to use this function. That code snippet will be treated as a unit test and ran when testing the functionality. In this way, documentation is always up to date (assuming the tests are ran).
Why can't tool documentation be the same? The tool documentation usually contains a sample tool invocation. All we need to do is run the tool in a controlled environment, and ensure that the invocation runs successfully. When a tool gets updated, run the "documentation tests" to ensure all the commands in the docs still run successfully. If they don't, then update the docs. Surely, there are tools out there that already do this? Does anyone know?
It's designed so that you can write tests that are the basis of the "how to" documentation - with a small script and some jinja2 you can generate readable documentation in the form of markdown from the test:
It's also designed so that the testing framework can be programmed to rewrite the test automatically for simple code changes (e.g. changing the wording of that exception).
This makes building tests much quicker and much, much less tedious and perfectly in sync docs almost becomes a side effect.
Essentially "how to" documentation and example driven behavioral tests are the same thing for all projects. It's just that most people don't realize and don't treat them as such.
In the specific example given here of "dbconnect", I'd consider a different approach: incorporating the "how" knowledge into the CLI itself. For example, running the command without any arguments could trigger an interactive guide that asks the user questions.
"What do you want to do?"
o Get a list of databases
o Connect to a database
"Do you want to connect to dev or prod?"
o dev
o prod
o I don't know (this option could link the user to documentation on the different environments; maybe the user is a student on his or her first internship and has never even heard of the concept)
"What's the shard number?"
"Which side?"
The CLI could output the equivalent, non-interactive command before performing the requested action, making it a teaching opportunity as well. Maybe the "I don't know" option or this approach in general is overkill, but I generally think tools should be smarter. You don't need as much documentation if your tools are inherently capable of guiding you to the right answer.
That's quite a bit more work than adding a reacmoji to 100 chars which were typed into Slack anyway. We all want better tools with fantastically helpful interfaces, but we also want functioning MVPs yesterday. I like the Etsy approach because it fits into our existing JIT dev/doc workflow.
1. I'd use a library or framework that handles the interactive parts, which should make it fast to implement. We have tools to help make other tools.
2. A tool's popularity/usage could be a way to judge when it's worth the effort to add this sort of system.
3. With the Etsy approach, I would also be wary of the cost of interruptions. Getting constant Slack notifications is not conducive to getting work done. You could silence them, but then what if someone can't move forward without getting an answer to their question?
4. There is a nice balance to be found between wasting time documenting everything possible upfront and not documenting anything until someone asks a question.
5. Personally, when I find documentation that needs to be updated, it's usually as simple as editing a wiki or editing Markdown and making a pull request in GitHub's interface. To me, that's not much effort.
Seems like a lot of overhead and more things to maintain. On top of that it doesn't teach people how to use the tool. Kinda like 600 hours of coding saved me 30 hours of documentation.
This feels incredibly broken to me. The 'how' should not change often, because it enforces rapid change on anything which depends upon it.
Any system inevitably acquires dependencies. Maybe it's fine to break those dependencies if you're building some public website where you can always have the latest version of the code running, but if you've got customers who depend on having a stable build which was initially branched three years ago and you still need to build, run and test that, you damn well better have a consistent and stable mechanism for dealing with that codebase which spans multiple years. Your ops and QA people will hate you otherwise.
Our own approach has been to pull out the coarse-grained operations and have an 'environment' script which examines the version it's working with in order to spit out relevant details. The 'porcelain' scripts are never allowed to change their API in any breaking fashion, and their behaviour is adapted based on the environment's output.
DDD: Documentation-driven development. With apologies to Fred Brooks, write documentation first, then write tests for the functionality and syntax described in the documentation, then the code. I would speculate that doing straight docs instead of something like Cucumber would be more productive.
One of the big picture value propositions of technical writers IMO is that we're shepherds of organizational knowledge. For example, if someone sends an email to me asking how to get a particular task done in Chrome DevTools (which I write the docs for), I will ask their question on Stack Overflow, provide an answer for it, and then send the link back to the asker. It's a similar amount of work, but I'm making the knowledge public, rather than one-off.
What's really innovative about Etsy's idea IMO is that they have found a way to make tribal knowledge shared on an ephemeral platform like Slack more permanent. My big issue with real-time chat up to this point has been that all the beautiful knowledge that you share on the platform isn't recorded in a permanent way. I acknowledge that a lot of people prefer the real-time messaging format, so it's really cool that Etsy has figured out a way to get the best of both worlds.
This is a really interesting idea and I feel like it has the potential to make the common usages much easier to ramp up on, but I'm skeptical because I run into conceptual completeness problems of the "how do I think about this?" sort much more than "how do I do this?" problems.
Blog posts like this are always optimistic and celebratory. Sometimes it's warranted, sometimes it isn't. I'd love to hear how people on the ground at Etsy actually feel about it.
Our documentation is split into around 2 parts due to this.
We got runbooks/ - these are hard about how. And they are prone to some degree of decay, I won't lie about that. Thus, runbooks/ is encouraged to be changed, extended, modified, clarified. runbooks/ isn't anything pretty, runbooks/ is the oil- and coffee stained notebook with more loose paper than bound paper somewhere on top of a machine and we're all going to die if you lose runbooks/.
And on the other hand, we have doc/. doc/ doesn't change without a lot of discussion, and doc/ defines hard standards. doc/ won't generally help you with an immediate problem. doc/ explains why things are defined the way they are - or if this is just an arbitrary naming convention.
I personally feel that the problem with most documentation is that it always focuses on the how/what. That will always decay and be wrong. The code is the only truth in the end.
The thing I suggest here is that if you are going to document the function, it should only be the WHY. The how can change and the documentation doesn't really decay. If you are making a new function you have to start with the why and if you can't then you shouldn't be adding a new interface. This also helps build a better picture of how the pieces interface together.
> I run into conceptual completeness problems of the "how do I think about this?" sort much more than "how do I do this?" problems.
While I agree with you, the latter problem is particularly frustrating to encounter.
Solving the conceptual problems feel more like an exploration or a learning experience, and I rarely feel like time spent solving those problems is wasted even if it's very meandering.
Solving the "how do I do this?" problem rarely feels like more than a very isolated learning experience, and more often it feels like banging your head against a wall. Doubly-so when the documentation is obsolete and leads you in the wrong direction.
Learning the basics of media compression is interesting and worth spending time on. But I'm going to hate myself if I have to spend 6 hours figuring out how to convert a file using ffmpeg.
Sounds like we're talking about different levels of conceptual abstraction.
Here's an example: I need to generate a report where I need to cross-reference data from A and B. I need to know where A is going, where B is going, how to modify the schema of the events going into B to add a specific piece of data, where we can process inputs from A, how to do the join, and how to add the report into the dashboard.
On the one hand, I could ask the surface level question "how do I generate the report for ____?" but really there's a whole lot to unwrap, and what I need is the information to do the unwrapping. I don't need a course on statistics, I just need to understand our specific architecture.
I think I see what you mean, though maybe I'm misunderstanding still.
The answer (from my experience at Etsy, anyway) to questions like:
> how to modify the schema of the events going into B to add a specific piece of data,
was "Run schema_modifier", the docs for which were frequently obsolete. Figuring out how to "run schema_modifier" was usually a very frustrating experience in that particular instance, whereas unwrapping the actual problem was usually less frustrating.
I'm surprised no one has yet mentioned how well automated functional tests can document the how. I constantly go back to my automated tests to understand how to do something I've since forgotten but that the code supports.
Discoverability is often a major documentation shortcoming. They solved task-oriented discovery of 'how' snippets. Someone can search based on what they want to do, and not based on a specific command.
I also prefer the 'documentation is code' approach, but it doesn't solve the same problem.
I've been fascinated recently with out-of-line documentation, I was inspired by this project: https://github.com/adobe/hyde
A more robust implementation would parse the code to object-code, then emit a document(s) for the developer to fill in.
The killer feature of such a system is that non-code changes (whitespace, etc.) would not trigger a need to re-document, but seemingly benign changes that affect distant parts of code (or a -O4 being added to the Makefile) would trigger a round of documentation
well, automatically ensuring the documentation is correct is hard/impossible, yes, but it's simple to check if code change X impacts Y usage, and Y usage is documented, yet code change X has no accompanying documentation changes, so code change X is 'incomplete' due to lack of accompanying documentation..
Internal documentation has a handful of patterns. There's the cookbook patern, which corresponds to a "how-to." There's the "readme" pattern, which is meant to conceptually get you started on a project. There's the "architecture" pattern, which is a conceptual design and talks about the rejected alternatives - this is not used everywhere but is a key deliverable in both outsourced and waterfall applications. There is the "small chunk of information that is hard to organize" pattern that has multiple solutions, be it FAQs or Etsy's bot.
Immutable documentation, at its essence, isn't new. Wikis are in many ways immutable documentation due to the diff. The interesting thing here is trying to organize the "small bits of hard to organize" information around slack. More often, however, the real problem is that we need to properly organize this documentation at the point the organization becomes more evident.
For a tool, that should be obvious: --help should be obvious. It's not true in all cases.
> Internal documentation has a handful of patterns.
This is a great way to articulate something I've been trying to talk to people about for a while - is this something you've thought of for yourself, or do you know of anywhere that talks about these patterns?
I've noticed the concept of _information architecture_ is being used in technical communication more frequently these days, borrowed from graphic design. Which covers a bit more ground then patterns, including topics like search, labeling, and taxonomy.
Sure, getting everyone to maintain docs with their features is useful. But given enough time, you'll end up with so many "close but not really close" concepts in documents it's very hard to stay on top of without someone putting some organization into play.
Old school tech departments used to have librarians for documentation. I still think it would be valuable for departments, but it's one of those positions for which it is hard to quantify until you get much larger as an organization.
I have some associated notes I've been thinking about for this. I haven't seen anyone articulate it, and I vaguely have an idea for a startup based on an opinionated guide to documentation, so I've held off on publishing until I know I'm ready to release it into the wild.
It took me a while to work out why this doesn't feel like a full solution.
I can imagine it working for simple one-line commands and their switches.
I can also imagine it failing to work for any application that requires multi-line code blocks and/or more detailed how-to examples.
For example - as a hypothetical, imagine you change your API from Oauth to Oauth2, and you have to provide examples of new standard code, perhaps in different languages.
That's very much a how-doc and not a why-doc problem, so supposedly it doesn't belong in the Wiki.
But I really can't see the changes being explainable in a fyi.
Of course you can say "The new code is in repository X instead of X-1." But if there are other questions and fyi's around Oauth2 - and there will be - there's no guarantee that important fact is going to be at the top of the read list.
For complex breaking changes that read list is going to get very long.
Simpler changes are going to have the same issues on a smaller scale.
So this isn't really immutable documentation - it's more of a database-hosted command cheat sheet.
That's a useful thing in itself, but I don't think it's a complete solution for documentation.
Never refactoring your interfaces means that you can never improve the design as you gain new information. It means you can never change to something more appropriate if the underlying use cases change. It means that you can't easily consolidate common functionality in a different place as new similar interfaces are constructed. In other words, you are locking yourself in to a brittle system that will degrade into a mess of spaghetti that nobody can understand.
There are, of course, also downsides to refactoring. No simplistic rule of thumb will tell you your best approach.
Or use a new name for the new interface whilst not breaking your old one. If your api is private then break it all you want, but if it's public and you want a large grateful user base then you should only grow. See rich hickeys talk "speculation" for a much better explanation of this.
I believe in all(?) public windows COM interfaces, methods can only be added, not changed or removed. So it’s at least possible, modulo any concerns about the design or cruft of the windows api surface
Supporting and maintaining old endpoint versions is expensive. Especially if the old endpoint shares a back end data store with the new endpoint. You can get locked into a bad data model and unable to move forward.
If you can do it then great, but it's just not always technically feasible or cost effective.
Yeah, you can sometimes build an adaptor to expose the old API on top of your new API, but other times you have to actually maintain 2 parallel implementations. It's always a hard choice. Or at least it should be if you are thinking it through all the way :-)
A lot of times I find that developers lack experience of how legacy applications get the way they do. Either they have only ever worked on green field applications before (a surprisingly large contingent!) or they assume that the crappy legacy code was created because the previous developers were stupid. The reality is that most legacy systems were made with very reasonable decisions at the time, but that the world shifted under them. The mistake was locking themselves into their original decision and making it very expensive to change. The decision to never deprecate old APIs is one of those decisions that makes change expensive. Such decisions should always be made looking at the trade offs in cost. How expensive would it be if we changed this API? How expensive would it be if we don't change this API? Usually the second is far more expensive in the long run, but you have to take it on a case by case basis.
If you are only ever adding and never refactoring, aren't you already locked in? I guess supporting one piece of technical debt is probably going to be cheaper than having to support a few legacy versions on top of the current version. However, the legacy version can at least be end-of-lifed eventually, whereas you would have to support all legacy features for the life of the product with the 'only add' method.
Maybe the docs are binary and code aware, so the doc system can pull up the binary's code in source control and find all the possible doc strings. Developers can't add/change flags without also updating these.
Maybe the docs are history/execution aware so they can list the top N ways the binary is executed.
Maybe the docs are also team aware so they list the top N ways the binaries have been executed by your team.
So you say, "Here at CoolTechCompany, we built a tool to help control our widgets. Please see the `cli widget` command. And everything kinda works automatically.
I feel like searching for the solution may become time-consuming because you have to search the right set of keywords to find the answer, which may or may not be an intuitive set
This kind of falls into the same problem as wikis though. It requires people to A. remember it's there and B. be actively monitoring it for questions. We have one at my location and I constantly forget it's a thing.
With something like this tool it puts it right in a primary communication tool.
yikes, not at the enterprise price point. that's well over six figures a year; might as well hire a technical writer to sit with all your team members.
Genuine question, is the internal search any good? I can’t remember ever going straight to stackoverflow so I’m wondering how many people actually use Stack Overflow’s search compared to finding the most relevant answer via google’s index? This would be lost for an enterprise version.
Consider a page like this: https://firebase.google.com/docs/firestore/manage-data/add-d...
It has 12 "how" sections of code snippets, each with 12 language options. That means 144 code snippets need to be correct for this page to be 100% correct! And we don't auto-generate any of these snippets, since auto-generating idiomatic code is a rabbit hole that's consumed many a good developer.
So we, like Etsy, have come up with a separation of concerns between How and Why that works for us. The text on the page is maintained by a tech writer. The code on the page is a series of blocks that look like this:
{% includecode snippet="android_firestore_write_doc" %}
We have a system that resolves these, at publish time, to match snippets on GitHub. For example: https://github.com/firebase/snippets-android/blob/5f94890d7b...
These snippets can be maintained with all the wonderful tools that exist in the world for maintaining code! We don't need to think of them as just docs. So that "snippets-android" repo has continuous build, a robot that updates dependencies, PRs/Issues for third-party contributions, etc.
What we've found is that this system allows us to maintain many thousands of code snippets with only a handful of people watching over them.