Hacker News new | past | comments | ask | show | jobs | submit login
The Tools I Use to Write Books (thorstenball.com)
472 points by misternugget on Sept 5, 2018 | hide | past | favorite | 97 comments



The most important quote here for me:

"What kept me from [writing my own tool] was a calm voice in my head telling me that I’m here to write a book, not a preprocessor. [...] Now I have written two books and zero tools, which I consider a success."


It's a crazy important quote and everyone should really keep it in mind when they set out to do something.

Many people have problems focusing on the task they set out to do. It's a type of procrastination. Insert that famous gif from Malcolm in the Middle.

This is relevant to startups as well (and business in general): Being able to decide whether something is in scope of a project or not. You are encountering a roadblock, how do you react? Do you set out to solve it perfectly or do you work around it (or even, do you give up)?

The quote you pasted is the perfect way to put it into words. Another common theme I hear about is people asking: "How do I become a successful [profession here]?" (eg. "streamer", "youtuber", "musician", "programmer", "writer", ...). The answer is always the same: "Start".


I think there is a time in a programmer's life when writing tools to make yourself more effective is finally worth it.

For me, that time started about two years ago. At 31, a full 25 years since first learning how to program[0] I finally felt like I could be trusted to make the right calls on when to invest the time to abstract or create generation or code parsing tools; and when to do things the old fashion way.

After two years of spending about 20 to 40% of my time on tooling I'd wager I'm about 10x faster than I was at 30 and about 20x faster than I was at 25.

The problem now isn't writing the code. The problem now is that the code is so fast that I don't have the same amount of passive time to think about features / strategy. I have to intentionally set aside time for it and I find it unnatural. Also, I know too much about cybersecurity which is good for my end users, but bad for my feature productivity.

[0] Poorly. My mum taught me how to code QBASIC. I altered video games.


> I think there is a time in a programmer's life when writing tools to make yourself more effective is finally worth it.

Indeed yak shaving should only be done when one has the breathing room to do so. It's just good common sense, except there are exceptions. Slack was a yak shaving effort. The most valuable things I've shipped have all been yak shaving efforts. I guess I'm just better at building tools than anything else. Perhaps I enjoy it more. shrug

To each their own, but if you're trying to ship a product instead of a bald yak you're going to have to do it the ugly way, with tools you could fix, etc...


Shipping is a feature. Your product must have it.

That recursively applies to each tool used to make the product as well.


Shipping is a feature. Your product must have it.

I think I’m going to frame that and put it on the wall. Did you come up with that or did you hear it somewhere?



This worldview does not take into account network effects, which consolidate the aggregate investment of many people onto a few bad projects. For example Common Lisp is a more effective language than Javascript and always the wrong choice. Corollary: ClojureScript is ecosystem-compatible with Javascript, and TBD if it is a long run better choice!


What do you mean by, "I know too much about cybersecurity which is good for my end users, but bad for my feature productivity"?


One thing that pervasively infects every decision is the principle of least privilege. Basically it’s the idea that someone/something should only have the minimum access necessary to do what they/it is supposed to do. It applies to infrastructure access control, network communication, application access control, everything. Once it becomes a default part your thinking you sometimes have to make conscious tradeoffs so you don’t spend forever perfecting things and not delivering any value. Ideally a lot of that thought and effort is common or reusable though.


This is most easily explained by examples.

Before:

I'll get my clients to give me access to an S3 bucket that I can push their data to.

Now:

I'll make sure I get my clients to give me a password or public key to encrypt their data before I push it to the S3 bucket because many people fuck up security on those things.

To my end user, it's obviously more secure, but it isn't lining my pocketbook and most end users don't care about security, even if they should. Same thing with security headers or anything, really. Security slows you down at some point.


> It's a crazy important quote and everyone should really keep it in mind when they set out to do something.

I don't agree at all. It seems to me that the most interesting advances of mankind have happened precisely when people were trying to do something else, and had to invent new tools. The tools turn out to be more important than the final result because they can be used by other people.


> when people were trying to do something else, and had to invent new tools.

(Author here.) My point was not that you shouldn't invent tools if you need them. By all means, go ahead. My heart beats for creating tools (especially tools to create other tools...).

What I wanted to express was this: if you want to write _a book_, first write the book and only then worry about the tools. If you then notice that you need a custom toolchain to get the book in exactly the shape you need it to be, do it.

I've had people tell me that they would love to write a book, but, well, the existing tools are not sufficient enough. And I think that's the wrong approach to writing a book: write something and you'll quickly realize that the tooling is not your biggest hurdle.


> What I wanted to express was this: if you want to write _a book_, first write the book and only then worry about the tools. If you then notice that you need a custom toolchain to get the book in exactly the shape you need it to be, do it.

I agree 100% with this. Following the case of Knuth/TAoCP/TeX, in fact the first edition of TAoCP was not written in TeX, he made TeX to improve his already written book.


> I've had people tell me that they would love to write a book, but, well, the existing tools are not sufficient enough.

That’s hilarious. Humans were hand writing books on paper and giving them to other humans to hand write copies. A good craftsman never blames his tools.


> Humans were hand writing books on paper and giving them to other humans to hand write copies.

Not very many were, and a big reason for that was the available tooling.

> A good craftsman never blames his tools

Someone who writes a book by blames the available tooling support for the low quality of its content is a craftsman (but, per the saying, not a good one) blaming his tools.

A person who chooses not to write a book at all because the anticipated opportunity cost given the existing tooling support and that person’s personal utility function is not.


You keep wording what I'm trying to say far better than I ever could. I'm not surprised you wrote a book :)


Yes! I use Scrivener. It exists because the developer needed a better tool to write novels. I end up too disorganized trying to write with other tools like word processors or flat files.

I managed to write a 25,000 word story in a word processor once, but it was such a jumble that I just closed the file and forgot about writing for a few years. I know now there are ways to organize with folders and spreadsheets, but now I do most of my writing on the phone. Scrivener's binder makes that easy on a small screen.


Thank you. I've thought about making this exact thing multiple times. Word docs and dropbox just weren't cutting it for my organizing needs. Combined with the scrapple product oh man. The only thing they are missing is a wiki feature for character/show bibles


"Most interesting" seems like a pretty subjective metric. The advances that come to my mind are things like agriculture, calculus, the scientific method, Newton's theories, representative democracy, post-modernism... those all resulted from decades of effort by groups of people who focused on that task specifically.

Tools are super valuable; it's arguable that many of the things I list above are sort of tools. I just don't think that a tool that is quickly slapped together in the service of another project is likely to be a game-changer.

In fact, I'd say that one of the keys to growing your personal impact is to find the best work that's already been done, and use it to further your goal. Even Newton said he was standing on the shoulders of giants.


Calculus of all things was not an end by itself, but a set of secret tricks by Newton to prove results about the orbits of planets (that then he translated into a more cumbersome geometrical language)

Then there are the three branches of modern mathematics, according to V.I. Arnold:

1) modern hydrodynamics and PDE, are just guys procrastinating while they are paid by the manufacturers of nuclear submarines

2) celestial mechanics, control theory and ODE, are just guys procrastinating while they are paid by the manufacturers of rockets and missiles

3) number theory, are just guys procrastinating while they are paid by the CIA to develop cryptographic tools

agriculture : this kid is just playing with small seeds instead of collecting nuts for winter

computers : just tools to speed-up the important computations of nuclear reactors

TeX : the guy just had the write a damn book about algorithms but spent 10 years procrastinating

Bach : the guy just wanted to glorify God

I stand by my point that nearly all important advances of mankind are tools developed to solve a long forgotten goal.


The idea that calculus was some sort of side project for Newton both diminishes the level of focused effort he put into it, and ignores the contributions of Leibniz and the many other mathematicians whose work was known and incorporated by Newton and Leibniz.

If we just open up the idea of "tool" wide enough, everything becomes a tool. Writing a book isn't actually a goal, it's just a tool for acquiring money and fame. Calculating the orbit of planets was just a tool for Newton to establish social credit and historical significance. Agriculture is just a tool for eating, but eating is just a tool for living, but living is just a tool for procreating, but procreating is just a tool for perpetuating genes.

Why are we perpetuating genes? Really, no one knows. Solve that problem, I guess, and finally we'll have the one project enriquto can't easily dismiss as tool development.


"I stand by my point that nearly all important advances of mankind are tools developed to solve a long forgotten goal."

That is indeed true.

Of course, almost all of the goals I have accomplished for myself have been accomplished by actively engaging in pursuing that goal rather than pursuing some side channel, while the side projects I've created have generally been a lot less useful to myself and humanity than, say Bach. :D


CERN springs to mind, they have sprung a few decent spin-offs.


For many years a toolmaker was considered an extremely important member of a software development team.


[citation needed] -- but addressing your point without a citation, even if what you say is true, it doesn't change anything at an individual level.

If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.

If you're trying to write that tool, and want to write that tool, then power to you, but don't lie to yourself and tell yourself it's "for the book". Just admit you're switching focus.

A close friend of mine has been trying to create a game for .. a decade or so. Every time he starts, there's always something. Instead of using, say, Unity, he tried his hand at writing a game engine. A physics engine. A programming language. Because none of the tools are "quite right". Of course, none of those get completed either because they're a shitton of work. And his work didn't change the world, because it's never completed. At the end of the day, much like many in the software engineering world, he's a brilliant mind busy optimizing ads. And he doesn't have a working game engine, a working programming language, or a game.

I don't want to sound demeaning, because it's still very good experience to acquire. And in general I don't want to tell people "don't write your own a programming language". I think these things are super cool. But then if that's what you set out to do, make that your project instead, it'll have a much higher chance to actually complete, instead of sitting in an incomplete state, in a hidden unfinished repo with nobody to share it with.

The common thread between all of mankind's most interesting advances: They're finished. How do you finish a project? Well... start.


> [citation needed] -- but addressing your point without a citation, even if what you say is true, it doesn't change anything at an individual level.

> If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.

Not the person you were replying to, but their point certainly rings true to me, from all the history I've read or otherwise heard of over the years. They could just as easily ask you for citations.

"if what you say is true, it doesn't change anything at an individual level" -- yes it does. It's usually individuals that come up with the important tools and inventions. Those development are very significant because they make systemic advances.

"There's an infinitesimal chance that the tool you're writing instead of your book will change the world" - it doesn't matter how small the odds are for any one individual.

There's two points here. One is that the effects can be cumulative -- even small advances and bits of knowledge gained contribute, over a longer period of time, to large benefits.

The other is that the historical reality is that tools that were developed as side effects of other goals have made a very significant overall effect, and this can't be denied by a blinkered focus on the probabilities concerning individual cases.


I'd say it's a balance. I try to lean towards staying focused on my goal while keeping an eye out for potentially more important detours.


this is how a lot of cool shit got done in history though, not focusing on the task necessarily


On the other hand, I'm glad Knuth didn't have such calm voice in his head 40 years ago...


Digital typesetting ("phototypesetting") at the time sucked rather hard. troff was kinda the state-of-the-art then.


And the point is that troff is actually an excellent tool with very beautiful typesetting. The fact that Knuth didn't consider it good enough says a lot about him and his ambitious level of exigence.


Actually, troff was simply not an option for Knuth, for least a couple of reasons.

For one thing, at the time, Unix and all its associated tools were really not prevalent outside of Bell Labs and a few places — troff would not have been available to run on the computer systems Knuth had available to him in 1977. (The ones at SAIL, which were PDP-10 machines running either TOPS-10 or WAITS, I forget what I read.) He didn't have the source code of troff either; all he had was published descriptions: and those he did use, e.g. the paper by Kernighan and Cherry describing `eqn` came out in 1975 and he used ideas from it while designing TeX's syntax for math.

For another thing, troff was designed to control line printers and other such devices that were typically attached to computers; it simply wasn't designed to drive a digital typesetter of the kind Knuth was thinking of (Alphatype CRS). So he'd need a new program anyway. In fact, he says that until he saw Patrick Winston's book on AI in 1977 (which was one of the early books produced on a digital typesetter, i.e. where the actual technology for getting ink onto the page was based on 0s and 1s, instead of being based on either pieces of lead (hot metal typesetting) or light and lenses (phototypesetting)), he had simply not associated output from computers with “real books”. Things like troff were designed to control the simplistic devices that came with computers, that could move by discrete (large!) amounts on the paper (e.g. some could move by full “lines” only), and strike down specific symbols (from a limited font repertoire that came with the device) onto the page. This wouldn't have seemed relevant to Knuth, who primarily wanted to create his own digital fonts (as he did with METAFONT), and TeX was among other things simply a way to use those fonts.

So, regardless of whether one considers troff to have very beautiful typesetting or not, it simply wouldn't have seemed applicable to the problem at the time, to be rejected as "not good enough" or not. Now of course with output going to generic page-description formats like PDF (even TeX's DVI was not Knuth's idea—he was thinking TeX would just write code for the device directly—and he had to be convinced to do it by Fuchs: https://tug.org/interviews/fuchs.html), and high-resolution printers (fairly high DPI, even if not 5000+ as with the CRS then) available to typical consumers, we don't appreciate all this. :-)


I think your main point is correct, that phototypesetting at the time was really bad and that's what drove Knuth to write the tools. (He had been happy to publish with Addison Wesley because of the quality of their typesetting, and now if they were going to move to such an inferior process, it was very upsetting.) The details are off though: phototypesetting is precisely not digital typesetting — in phototypesetting, the decision of where ink goes on the page is controlled via light and lenses and a fixed set of fonts that have been made (physically) for them; it is not controlled by arbitrary 0s and 1s, which was the appeal of digital typesetting. About troff, see my other comment in this thread.


Donald Knuth seems to have got it right. On the way to writing a book, he wrote all the tooling including a typesetting system, a programming paradigm, a font, a font authoring tool, and so on.

https://yakshav.es/the-patron-saint-of-yakshaves/


And then he wrote a book about the tooling!


And turned the source code of the tooling into a book!

https://www.amazon.com/Computers-Typesetting-B-TeX-Program/d...


That is also the reason why people are obsessed with the best IDE or framework or even keyboard layout.


I do think obsessing about trivial things is harmful, but finding an editor you really like working with is very valuable. I was exposed to vim and had a really hard time working with it, but I saw the value that lay beneath the arcane surface. I knew how much of a pain it was for me to copy/paste/rename/switch between windows when editing; it took me out of my flow. Becoming minimally proficient (I don't use any of vim's really advanced features) was a huge boost in comfort, translating to increased productivity for my attention-deficit brain. This feeling was compounded even more when I started using a tiling window manager: now my hands never needed to leave my keyboard during a programming session unless I was looking something up (I played around with vimperator or the like for a while, never 100% clicked for me).

Of course, that's the ideal. I can't use my preferred setup when I'm working on our legacy Windows codebase. I also won't pretend that I'm immune to chasing the new shiny thing to distract me from doing real work, it happens with some frequency. I just think that editors (and perhaps a good tiling window manager) are the exception(s) to the rule.


I am glad that you brought this up. I tend to spend a good chunk of time cleaning up and optimizing my .emacs right before I start coding. Absolutely useless waste of time!


I settled on something similar for my needs, and I'm really happy with it.

1. Book is entirely written in LaTeX, every chapter in it's own file, master doc with everything combined. I really like having different gutters for left and right pages, ultimate control over formatting, fancy things like Lettrine, header layout and design etc. When I want to do something across the entire doc I write a new command and can use it anywhere.

I Create perfectly formatted print-ready pdf to go to printers directly from this.

2. Run LaTeX file through pandoc to create an ePub.

A little known trick that is now supported by pandoc is conditionals in the LaTeX doc, so when formatting gets hairy I declare commands like this in the master LaTeX document:

  \iftoggle{ebook}{
    \newcommand{\textbreak}{\newline\hrule\newline}
  }{
    \newcommand{\textbreak}{\begin{center}\LARGE{$\Psi\quad\Psi\quad\Psi$}\end{center}}
  }
So I can use \textbreak{} anywhere I want, and I know it will be formatted perfectly in both output formats.

3. Un-pack the ePub and tune some of the html/css to my personal taste, then pack it back together. I run it through kindlegen to make sure it's valid, and I have an epub/mobi ready to upload to stores.

I like this workflow so much I'm using it again for my next book


I've never been able to get a good LaTeX -> epub pipeline working. There are too many issues with things like tables and splitting code listings across pages -- epubs are just optimized for fiction. I find it's better just to ship a PDF.


I agree epub are painful when you have complicated layouts like tables and code listings.. it's just how the format is.

My book was a non-fiction novel, so that made life much easier, I only had some tricky image layouts to deal with.


I'm writing a sci-fi book at the moment. Somehow I've come much further than any of my other book writing attempts, including one that I actually (self)-published.

I use Sublime Text, sometimes in distraction free mode, but usually not because I often want to Google sciencey things.

I have a single file called "story.txt" that has all the important notes - the names, ages and visual descriptions of the main characters, a detailed (historical) timeline, and chapter outlines I wrote before I started when I knew roughly what the overall plot was going to be. The chapter numbering has changed drastically but the overall outline has stayed pretty true to the original idea.

I really like this system. Keep it simple and focus on writing!


I've written a couple of technical books, and then I've helped my wife grow her business as a USA Today best selling romance author.

The biggest challenge here, to me, isn't writing the book or figuring out what tools to use, but it's marketing. Email lists, FB ads, etc. are really what drive the success or failure of your book. Nathan Barry's writing on this in Authority is my bible for that.

Also, if we're talking tools, shout out to Leanpub. Love that platform, especially for developer books!

EDIT: Nathan Barry doesn't specifically talk about FB Ads, so in terms of a marketing bible, I guess courses and private FB groups have helped us the most. This guy is actually the main goto for fiction authors: https://selfpublishingformula.com/


> The biggest challenge here, to me, isn't writing the book or figuring out what tools to use, but it's marketing. Email lists, FB ads, etc. are really what drive the success or failure of your book.

I've looked into creating a mailing list with MailChimp and/or its competitors, since I think it'd work great for my target niche. The thing that stopped me is that you need to provide your real name and location. Though I understand the reasoning, I'm publishing under a pseudonymous identity and do not wish to tie my real identity to it. Bummer.


Have you checked out the self-hosted https://sendy.co?


We rent a box at the UPS store, and you can use a pen name.


I interviewed a romance author recently about her strategy for going from 30 to 30,000 sales, it was pretty interesting[1]. TLDR is build community with your readers to turn them into evangelizing super fans. Pretty much the same audience building strategies you use for early startups.

Maybe you’ll get some ideas from the podcast :)

[1]: https://howauthorswork.com/8ae47f49


Thanks! Yea, she's had a ton of awesome milestones, including multiple top 100 books on Amazon. The challenge now is figuring out the repeatable playbook for consistent high launches.


Any specific thing from Nathan Barry?


Build up -- spend at least a month building a list of people interested in your book when it releases, tease them with sample chapters every few weeks. Imagine releasing and emailing 1k-5k people who want the book, getting a bunch of them to purchase, and having your rank on whatever platform you're on going up, along with some heavy ads.


Here's a direct link to Nathan Barry's book "Authority" https://nathanbarry.com/authority/


Thanks!


If anyone is curious this open source tool will convert markdown (with support for images) into pdf, epub, mobi and html:

https://github.com/softcover/softcover

It was written by the same guy who created the original Rails tutorial. You'll be able to generate books that look and feel like this https://www.railstutorial.org/book.

I've released a couple of PDFs using it, and it's pretty decent to work with. There's room for improvement but I haven't come across anything better.


Softcover was used for all the Learn Enough books as well. In fact, Learn Enough Ruby just came out today: https://news.ycombinator.com/item?id=17919998


Are you going to update softcover to include that theme by default?


For project-based tutorials/books, which are centred around building a program step-by-step by following a series of human-readable diffs, I think a tool is very much needed.

For https://viewsourcecode.org/snaptoken/kilo/02.enteringRawMode... , the code is kept in a git repo where each commit is a step in the tutorial. This way steps can be modified, reordered, or split into more steps using git interactive rebase (git rebase -i).

I'm working on a tool which abstracts away git rebase, and gives you a nicer interface for manipulating the series of steps in your tutorial. For example, you can run `leg 3` to checkout step 3, make edits to it, and then run `leg amend` to apply the changes, resolving conflicts as the changes make their way through the rest of the tutorial's steps.

The tool is still in very early development, but here's a quick tutorial to get a better idea of how it works: https://github.com/snaptoken/leg/blob/master/TUTORIAL.md

Also, I'm playing around with a "literate diff" file format that might be of interest: https://github.com/snaptoken/tgc-tutorial/blob/master/doc/02...


> I'm working on a tool which abstracts away git rebase

This is 99% of what I need! In the best case it's just files/folders I can edit manually or with the help of a CLI.

> The tool is still in very early development, but here's a quick tutorial to get a better idea of how it works: https://github.com/snaptoken/leg/blob/master/TUTORIAL.md

Now, if those diffs are rendered nicely and the splitting up of a step into multiples (or: explain separate parts of one step) works, I think that really is 99% of what I need. The 1% being the grouping of steps into a chapter and some other small things probably


From the article:

"But, most importantly, the code is also duplicated: one version lives in a Markdown file and one (or more) lives in the code folder that comes with the book. If I want to update a snippet of code presented in the book, I have to manually update every copy of it. Yes, cumbersome."

I had a similar problem in writing a book about gnuplot: keeping code samples, code in the text, and the generated plots synchronized. I decided doing it manually was too tedious and error-prone, so I wrote some Pandoc filters to do it for me: https://lee-phillips.org/panflute-gnuplot/


Nicely done.


I'm pretty sure that orgmode + Babel would get him a lot closer to what he wants: you could have chunks of code that edit using the natural editor mode and can join them together in an external file (tangle) in any way you want. You can even post-process right from the editor when tangling.

Orgmode Babel: https://orgmode.org/worg/org-contrib/babel/


In my experience, Org mode and babel shine at

- "literate programming" where the output is actually commented code and the eventual documentation

- "reproducible science" articles, where small independent snippets are executed, even piped into each other, to produce "results" in the form of tables, charts, ...

The author seems to need to highlight differences, which may not fit without hacking its own features, which he appeared to be reluctant to.


I use pandoc-flavored Markdown (it's a specific superset), sprinkled occasionally with LaTeX to do things like force page breaks in specific places.

I should write a blog post like this. My use case is different, since I write role-playing game books.


I use a plaintext editor to write xhtml+css, and 7Zip to go directly to epub.

I made a blank epub template with 50 empty chapters in it, so when starting a new book, I copy that and write new IDs and strings to the metadata, and if I finish it, I delete the unused chapters and remove their entries from the metadata files. It's easier to delete empties than add new chapters, and doing the metadata by hand ensures there's no auto-generated cruft in my file.

For illustrations I use Paint or GIMP or Inkscape, but the template doesn't have more than just a cover image in it.

After testing various ebook readers, there's no way I would recommend xhtml+css zipped to epub for anything technical. The only reader that actually renders text they way I expect, right out of the box, is Edge. Epub is for novels, not textbooks, papers, or manuals.


I use Scrivener on Mac for putting together various thoughts for game design docs. Adding references for the various things I've come across to assist with any hobby game dev is awesome, particularly when attempting historical accuracy.


I'm surprised not to see any mention of AsciiDoc, I know that it is what some publishers use.

https://en.wikipedia.org/wiki/AsciiDoc


Yes, O'Reilly. I have another comment in the thread about using asciidoc with a solution they call snippets that more or less solves the OP problem.


I wrote a free e-book tutorial for the Kitura web framework called Kitura Until Dawn [0] using an almost identical workflow: per-chapter Markdown files, pandoc, and kindlegen. Pandoc is an amazing tool. In addition to tweaking some Latex stuff to get the output I wanted, I also use a simple Lua script to rewrite internal links so that they don't break. I export to several different downloadable formats, and host an HTML version via GitBook[1] - though I may self-host at some point.

I have the entire book in a GitHub repository [2], including the export script [3], which may be of interest to those wishing to write a book a similar way.

(Pardon the new account, but my "normal" HN account is more closely tied to my IRL identity than my Nocturnal identity.)

[0]: https://learnkitura.com

[1]: https://nocturnal.gitbook.io/kitura-until-dawn/

[2]: https://github.com/NocturnalSolutions/KituraBook

[3]: https://github.com/NocturnalSolutions/KituraBook/blob/master...


I wrote a book for O'Reilly and they have something called O'Reilly Snippets which allows you to embed snippets from another file inside your asciidoc files. It sort-of solves the problem noted in the OP.

The problem I found with it was that I would add the snippet, then later in the chapter, I would refactor the code to better fit the story, and that worked, as long as you only needed one copy of the file. But, it didn't work if you introduced a basic version of the code and then later elaborated and expanded that code and wanted it to reside in the same file. Additionally, you have to keep the code in the same repository, which I didn't want to do since I published companion repositories (which couldn't contain the sources for the book, obviously) and I didn't want to maintain and synchronize two copies of a file.

So I wrote a little preprocessor that ingested a subtle variation on the snippets O'Reilly pioneered. Normal snippets work. But also, you could specify a file and a SHA hash, and if it saw that, it would pull the file at the commit (just using git show). Then I could update the file inside the repository as the code progressed, and still have all the sample code. And my repository always had working code, no matter what branch or point in time I had checked out.

It's here: https://github.com/xrd/oreilly-snippets/blob/master/README.m...



I find TeXmacs to be one of the few most amazing and useful pieces of software in existence.


I've recently been researching this for my own needs, and this author's md -> pandoc -> etc chain is pretty much where I settled. Reading this article, I was hoping to find a way to point code blocks at a git repo (local, remote, w/e) so that as code changes, the book follows (I do mean during the formatting phase - not that an ebook would follow a remote repo to get code.) I did not find that solution in this article.

Has anyone come up with a method for doing what I want? Maybe the ```lang code blocks in md could take parameters like repo, file, line range; then an update to pandoc would fix it all right up. Thoughts?


You should take a look at embedmd, which allows you to include files (or just portions of a file) in a Markdown file: https://github.com/campoy/embedmd

As I wrote in the post, I didn't use it because it doesn't allow me to include a portion of something and then comment out a part of that, but today I got thinking: if it could include a portion of a function and also the relevant (semantic?) context, it would be 99% of what I need. But the "relevant context" is the tricky part, I guess. I like what git diff does, when it shows the signature of a changed function (does it simply use the first of the previous lines that are less intended?), but I guess it's hard to get really right...

edit: GitChapter is also a really interesting project. If it didn't use a real git repository, but something like folders that resemble single commits, I think I'd use it. https://github.com/chrissound/GitChapter/


You could probably hack something together with pp and python?


Interesting, I tried to use https://github.com/jostylr/litpro-commonmark to do literate programming when writing a tutorial but was completely unable to go from their specific markdown flavor to a simple markdown file void of all extra macros and annotations that I could just transform to HTML. i didn't know about "pp" which looks like what I need to use.


I used Libre Office to write a few short eBooks. I found the "flow" of writing in a word processor better for me and it avoided the thought of writing my own tools for the job. I've published step-by-step instructions for the document setup that worked best for me.

https://www.amazon.com/dp/B00A4112ZM


Recently came across this article where it shows how easy it's to write a book using `bookdown` (an R package that uses R + markdown) and publishing on Amazong https://blog.datascienceheroes.com/how-to-self-publish-a-boo...


The fact that Thorsten manually synchronizes the snippets in the text and external source files seems like crazy talk to me. But the fact that he's cranked out two really good books on languages while I'm still slowly lumbering through my one [1] is proof that his system works for him.

Personally, I'm too error-prone to pull that off. I change the code snippets frequently after chapters are done (often in response to bug reports from readers) and having to manually update the snippets in the text would drive me bonkers.

Also, I want to rigorously, automatically validate that the code in the book is correct and contains everything you need.

To do that, I wrote a Python script [2] that weaves the code and the Markdown together. In the Markdown files, you use a marker like (here in the chapter "Strings"):

    ^code memory-include-object (2 before, 2 after)
This means "insert the code snippet labeled "memory-include-object" here. Include 2 lines of contextual code (which is shown grayed out) before, and 2 after. In the code, the marker looks like:

    //> Strings memory-include-object
    #include "object.h"

    //< Strings memory-include-object
The "//>" begins the snippet and the "//<" ends it.

The build script reads the "^code" markers and finds the snippet of code in the sources that correspond to it, as well as any desired surrounding lines. But it can also do a lot of other useful things with the snippet markers:

* If some lines of code are marked as being part of a chapter, but that chapter doesn't have a "^code" marker to include them, it reports an error. This way I can't accidentally drop code on the floor.

* It can generate a separate copy of the entire codebase that only includes the snippets that are in a given chapter (and the preceding ones). In other words it can output "what does all the code look like by the time the reader reaches the end of chapter ___" for every chapter.

I output the code for each chapter and then compile that and run all of the tests. This way, I can ensure that by the end of each chapter, the program supports what it's supposed to support by then. I've found countless bugs doing this -- places where I inadvertently put code in a later chapter that is actually needed earlier.

However, this build script is not a generic "build a book" system. It's a bespoke program that only supports this one specific book. That helps keep the complexity down and makes it much easier to put little hacks in when I run into weird edge cases my specific book has. I think that's a reasonable trade-off to keep me producing a high-quality book without too much yak shaving.

One of the days, I'll write a full blog post going through this in more detail.

Of course, none of this is a criticism of Thorsten's system. The simple most important skill in writing a book is figuring how to get those pages done. It doesn't matter if that means waking up at three in the morning and writing on parchment with a red pencil. Whatever works for you, works.

[1]: http://www.craftinginterpreters.com/

[2]: https://github.com/munificent/craftinginterpreters/tree/mast...


Interesting that the language is called Monkey.

Garmin's own programming language for device apps is called Monkey C.

https://developer.garmin.com/connect-iq/programmers-guide/mo...


There's also Monkey X, a dialect of BASIC: https://en.wikipedia.org/wiki/Monkey_X

It was previously called Monkey but then changed to Monkey X, which allowed me to use Monkey. Fun fact: I originally wanted to call it Tiger, because they're my favorite animals, but not two days after I wrote that name down, I opened Appel's Modern Compiler Implementation in ML and discovered that the language implemented in the book is also called Tiger. Thus, a Tiger became a Monkey.


I tried md, but go back to rst again. I tried many things, but all have some disadvantages. Outline of tools tried and considered on my blog at https://nocomplexity.com/the-next-version/


> After nearly two years, I’m still a happy customer and the only two features I’d love to have are more payment methods and pricing per country, so I can set a lower price for readers in India, for example.

I think KDP allows setting different pricing for different countries.


There was an Ask HN recently on what tools to use to write a textbook - https://news.ycombinator.com/item?id=17781569


I would love to see his Latex template for this as well as a few samples of markdown.

I wonder how well this method would work for something like an illustrated children’s book?


Hmm why not asciidoc or basically any other doc markup language? Most provide additional support for books.


I love AsciiDoc and I use it wherever I can. I'd bet the reason for using Markdown has more to do with mindshare than anything else. Plus, Pandoc is a great tool for converting Markdown to anything else.

What I really like about AsciiDoc is that it's right near Markdown in terms of simplicity, but with a bit more beef behind it to handle more complex things like referencing figures. If you don't need them, you can stick with the basic syntax, but the power is there when you want it.


Those tend to be pretty markup heavy, which can be annoying and distracting when actually cranking out prose.

Markdown is limited, but it's very lightweight and doesn't break your flow. It's a piece of cake to type a couple of "*" when you want to italicize something.

Also, the limitations are a benefit because they keep you from getting distracted by the hundred different ways you could structure or format a paragraph.


>, are written in GitHub Flavored Markdown (GFM) [...] code presented in the books is contained in the Markdown files in the form of fenced code blocks.

I understand the appeal of markdown but I avoid it for writing coding tutorials because it doesn't have an easy way to annotate code with graphics like arrows and floating callouts. Here are some visual examples of code enhanced by arrows: [1][2][3]

Also, all types of text files besides programming syntax can also be explained with graphical annotations such as configuration files or log output files. E.g.[4]

Therefore, I use MS Word because it makes annotation easy. (LaTeX can also do annotation but that has its own workflow issues such as the typical programmer not knowing it which means the team can't easily update the documentation.)

IMO, annotation is important because if we performed eye-tracking and heatmap tests with people reading code tutorials, we'd see that a significant amount of time is wasted with the eyes constantly shifting back and forth between the code and the subsequent paragraphs explaining that code that's located a few inches down the page. I also think this is one reason why live coding demonstrations can sometimes be more effective than reading a book because the presenter is wiggling the mouse (or laser pointer) around the exact spot of the syntax while he's talking.

This can be somewhat mitigated with adding inline code comments in the example but often, it still doesn't point to the exact point in the line that needs to be studied. Code comments also don't easily illustrate (because it has no spatial cues) how separate lines are related to each other.

One workaround of markdown's limitation would be to rasterize the code's text into an image and then use Photoshop to annotate. However, with MS Word, that cumbersome step isn't required.

[1] https://introcs.cs.princeton.edu/java/11cheatsheet/

[2] https://www.oreilly.com/library/view/head-first-python/97814...

[3] https://i.stack.imgur.com/D8JeF.png

[4] https://csharpcorner-mindcrackerinc.netdna-ssl.com/UploadFil...


How do people feel about Gumroad vs. Leanpub these days?


Is he writing them in GitHub web interface?


screw writing an entire book in markdown. You'd have better luck writing it in troff for chrissake.


Not much to see here despite some self promotion. It's mostly not about writing but preparing for (self) publishing. In short: Markdown, pandoc and kindlegen.

Missing is everything about the writing process, structuring content and editing.


Did you not read the title?


I take umbrage with calling these technical manuals / instructional texts "books" due to the conflagration of the commonality of the term. Stephen King writes books. There's no shame in calling a piece of text by its correct name.


What about college textbooks? In your mind, is long form narrative writing a requirement for something to be called a book?


Where did you acquire the idea that "book" only refers to long-form fiction?

Every dictionary I just consulted considers the primary definition to be the physical artifact: pages bound together. Some have secondary definitions that refer to any long written work. I found none that excluded non-fiction texts from it.

Are you claiming that phone books are not books? Recipe books are not books? Have you ever once heard a human say, "Look up their number in the phone text?"




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: