Adapton is definitely an inspiration, and I have cited it in previous iterations (and even had it in an earlier draft - if you check the Markdown source, the link def is still there).
Here's my current thinking on the topic. It all depends on whether you want to model your problem as a tree or a graph (see also [1] for some great discussion of that tension). If you model your problem as a graph, then a general purpose incremental computation engine like Adapton (or Incremental or Salsa) is great. However, if your problem is a tree, then constructing the explicit dependency graph requires a nontrivial amount of ceremony. What Xilem does is represent the most common tree-structured flows of information in a very lightweight manner (mostly just plain code that builds views), while allowing you to insert arbitrary graph edges if you like with more work.
I think it's a discussion worth continuing. One thing you could do is integrate Adapton and Xilem together, where the implementation of most view nodes in the latter becomes queries into the Adapton engine. How well would that work? Really only one way to find out.