As I understand it, Jupyter Book takes one or more Jupyter Notebooks and converts them into a static "book" output (looks like it primarily targets HTML output but with LaTex/PDF possible)

Yes that's about right :) The book can contain a mixture if Notebooks and normal markdown files, and also text representations of notebook files (see https://jupyterbook.org/content-types/myst-notebooks.html) The initial focus has been HTML, but now there is work ongoing to create a nice LaTeX/PDF "theme"

I mean even for a single notebook the display looks pretty nice. Might consider using for single notebook books just to avoid dealing with the hassle of themes and jupyter extensions.