IMO there are always going to be tensions (as well as differing personal preferences) for surface syntax and authoring environment.
What I think may emerge though is a better shared understanding of “what’s needed” as well as a great (ideally libre) library for implementing those ingredients consistently and interoperably — e.g., a DOM/CSSOM for these graphs as well as maybe an adaptagrams- or cairo-like library for the common layout and drawing capabilities.
(For example, have you noticed how many general-purpose drawing tools now have “align and distribute” capabilities that all work similarly? AIUI, many of those tools are either directly using or are inspired by Kim Marriot, PJ Stuckey, Tim Dwyer, & colleagues’ work in this area?)
This looks like a really nice attempt to improve the comfort and capability of styling drawings of small graphs that I’m excited to explore; thanks for building and sharing it!
Total control is good for small charts and for especially important charts but for complex uses, the number of clicks and drags required to perfectly position connectors really rapidly degrades the editing experience, especially if you’re still working out the contents of what needs to be depicted.
As a result, for either high-speed or larger scale usage, my experience is that text (in combination with constraint-respecting automatic layout options that suit one’s personal aesthetic) really starts to shine.
yEd is freeware (not free software) but I have not found a better program that matches accessibility (Java-based so runs cross-platform) and sophistication in automated graph layouts.
In my experience, typing dot requires a lot of quoting and brace-matching, often in editors without the best affordances for those operations, with so-so error reporting.
The result is that it’s very challenging to type syntactically correct dot at “line-speed”, e.g., if one is taking notes.
(This is one of several reasons that I think there is definitely still room for improvement in notation for graph-drawing, as I have begun exploring in my own work.)
That's interesting. It's not hard to see the problem. HTML labels don't help, either. How do other languages solve this? Like, a shell HERE document that has one fixed string as a terminator? The graphviz lexer https://gitlab.com/graphviz/graphviz/-/blob/main/lib/cgraph/... seems hackable but disabling downstream processing of escapes elsewhere would be necessary too. Possibly this would mean bypassing some of the processing in the function make_label in https://gitlab.com/graphviz/graphviz/-/blob/main/lib/common/...
It's a disappointment or even failure of our current language scanning and parsing tools or, really, the way we used them, that this is not an easy exercise.
One potential solution direction, which you can try out via my own incomplete drawing toy [1] is to treat punctuation characters like SP (“ “), COMMA (“,”), and SEMICOLON (“;”) as markers for the product operations of a family of monoids that allow you to specify more and more complicated sequences without requiring the typist to “move the cursor left” to add a matching character.
This way, simple lists can be specified via juxtaposition:
a b c
And then more complex lists
thing 1, thing 2, thing 3
and still more complex lists like
A complex thing; with data, and more data
can be specified in a way that is potentially still human-legible and easily editable.
Combined with ~instant feedback while typing and, ideally, a “brushing” system to allow selection of parts of the textual model via the linked drawing, I am hopeful that this can be solved resiliently, at least for the most common use cases.
(Part of why I am excited about OP’s work here though is that while I have done a fair bit in my own project on drawing a related kind of diagrams, I have myself only begun thinking about how to make the resulting drawings nicely stylable/themeable.)
You might find it helpful to distinguish between visualizing the design of the system being implemented by your software, visualizing protocols being implemented by your software, visualizing the design of your software implementation itself, and visualizing important implementation details at runtime, e.g. for debugging, profiling, and operations.
For visualizing system designs, you should take a look at STAMP, e.g., via “Engineering A Safer World” + the resources at mit.edu/psas + on YouTube.
(Multiple tools, both commercial and libre, exist and are being developed to make these diagrams, although for what it’s worth, I mostly hear about people making them using draw.io, Google Drawings, on physical paper/whiteboards, or occasionally with specialized tooling.
I have also recently published a project in this area, https://github.com/mstone/depict, which I believe is well on its way toward addressing some unmet needs here.)
For visualizing protocols, things like sequence diagrams, data flow diagrams, DRAKON flow charts, value stream maps, and occasional more specialized objects like CPSA “cryptographic protocol shapes” / strand space skeletons are where I start depending on the flavor of what’s needed.
For visualizing the design of implementations themselves, I have not yet seen anything that I feel obliged to recommend; rather, here, I suggest investing in adding illustrations to your existing documentation in whatever way is easiest for you to use to clarify whatever subtleties you need to clarify for your audience.
(Here I tend to look at things like ASCII-art, SQLite’s railroad diagrams (now made with pikchr, AIUI), and sequence diagrams, as mentioned by other commenters, as helpful examples to start with.)
Finally, for implementing debugging/profiling/operational illustrations, there is a such a rich set of examples to turn to — whether from the very specialized (custom process model video rendering pipelines in robotics) to TensorBoard for TensorFlow to general-purpose tools like browser performance debugging suites, flame charts, or Go’s built-in profile graphing tools - that rather than learn any particular such tools, I’d instead suggest trying to get comfortable with the building blocks underlying these systems, which include contemporary GUI/web apps, custom drawing and animation tools like SVG, pretty printers, and Grammar-of-Graphics systems like vega-lite.
(Note: although it may seem superficially extraneous to your question, the reason I also suggest thinking about debugging visualizations in this context is because IMO, to work, they ~necessarily encode a visual model of the design of your implementation since it is the design of the implementation that provides the vocabulary and relationships that have to be understood and navigated in order to successfully debug/optimize/monitor any given running instance of whatever system you are building.)
After several years of exploring, my current gotos are:
* Minion for IP for scheduling + edge crossing minimization.
* cvxpy (wrapping ECOS, OSQP, and SCS by default) for convex optimization for making nice geometry.
* Z3 and STP for SAT/SMT for program analysis.
All are FLOSS, which is my main criterion in many situations.
Beyond that, I like minion for its focus on only providing efficiently implementable primitives, cvxpy for the awesome lectures and documentation the folks behind it have produced, and z3 + stp for their use in tools I like, such as klee.
Not OP, but I work as a data scientist consultant, so various industries applications require different types of solvers e.g. mining operations optimization, branch location optimization, etc. would often each require a different solver depending on the problem space.
Not OP either, but there are many different aspects to solvers that make them useful in different situations.
For example, I would perhaps not use the same solver for a best-effort problem where I can program strong heuristics, vs a problem that must be optimized and no natural heuristics exist. Sometimes problems have a natural type of formulation, and it is best to use a solver for that type of problem (linear programming, mixed integer programming, SAT, pseudo boolean, constraint programming, ...).
Choosing a solver also very much depends on the deployment situation; how should a model be used and what operational concerns are there.
Doing all sorts of things will introduce you to all sorts of problems; from there, it’s just a matter of habitually looking around to try to see who/what has already worked on whatever problem is currently in front of you.
For example, I first found minion when I was learning about scheduling problems, initially to help a friend who was a Chief Resident on a pediatrics ward and later, to automate the administrative hassle of scheduling incident responders. (I wrote about that a little bit many years ago here if you’re curious: https://mstone.info/posts/scheduling/)
Concurrently with that, while I was working with and supervising security researchers, I spent quite a bit of time implementing various program analyses to answer questions about what certain programs of interest might do, and to find inputs that would make them do interesting things. For this purpose, though, minion is much less immediately relevant than Z3 since the interfaces and research interests of the relevant communities are so different.
Finally, this year, I am focusing on problems that have a more conventionally geometric flavor as opposed to the more discrete search spaces mentioned above. Here, convex optimization is a starting point that cvxpy makes incredibly accessible, especially in combination with Stephen Boyd’s Stanford SEE EE364A lectures (or similar): https://see.stanford.edu/Course/EE364A
If it helps, one connection I see is that recommender systems often involve causal questions like: “how will user behavior change if we change the order these results appear in, or if we change which results appear in the first page of results, etc.”. Additionally, since we can only show one ranked set of answers for each query, counterfactual questions also rapidly arise about what would have happened if we had answered past queries differently.
What I think may emerge though is a better shared understanding of “what’s needed” as well as a great (ideally libre) library for implementing those ingredients consistently and interoperably — e.g., a DOM/CSSOM for these graphs as well as maybe an adaptagrams- or cairo-like library for the common layout and drawing capabilities.
(For example, have you noticed how many general-purpose drawing tools now have “align and distribute” capabilities that all work similarly? AIUI, many of those tools are either directly using or are inspired by Kim Marriot, PJ Stuckey, Tim Dwyer, & colleagues’ work in this area?)