I actually just started doing this in my architecture notes.

You can inline source code in org-mode that renders a graph png.

  #+BEGIN_SRC dot
  digraph {
    a -> b
    b -> a
  }
  #+END_SRC

You can look into graphviz and dot language for intricacies. Once you learn it, it's super useful to jot down ideas and change structures on the fly.