As someone who helped lead the Polymer team in the transition from HTML-first Polymer to JavaScript-first lit-html/LitElement, I have some experience building both approaches.
I think that JavaScript-first is far better for templating the more general case (or the lower level foundation) because JavaScript is where your data lives. It's generally much easier to bring markup into JavaScript than it is data and data manipulation into HTML.
In HTML you need re-invent expressions, scopes, control-flow, references, and imports. You're going to spend more time and code implementing a less expressive, slower, and more proprietary system.
In JavaScript you just need a way to describe fragments of the resulting DOM (whether you prefer JSX, function calls, or tagged template literals), and the rest is just JavaScript.
Now, I do see benefit from the HTML-first approach for a lot of people and some use cases. One reason I also push on web components so hard is that with interop comes flexibility in allowing a mix-and-match of approaches. As a side-project I'm working on an HTML-first declarative component system layered on top of LitElement: https://github.com/justinfagnani/stampino-element
This can interop with any other web component, so you can write your app components in HTML, but drop to JS for the most complex components and use JS-first third-party components and design systems.
There are - as with most things in life - pros and cons to each approach. With plain JS, it's absolutely true that you can let the language do its things when it comes to things like scope, control flow, etc as opposed to reinventing them. But at the same time, there are a million ways to loop in JS, and a more restricted DSL can offer very good optimization opportunities that cannot be reasonably implemented on top of a JS-based templating engine. You can also create custom syntax for cases where the language doesn't gel so well (e.g. each/else or await in svelte)
I've been slowly seeing more and more of a third approach where there's very little in the way of control flow in the template itself, where data is wired up using different constructs. Solid.js is an interesting example where it uses reactive constructs on top of "plain JS(X)", while encapsulating control flow inside components, which it is then able to optimize aggressively.
> optimization opportunities that cannot be reasonably implemented on top of a JS-based templating engine
You very much can do those optimizations in pure JS. lit-html's repeat() directive implements the list diffing approach used in ivi, Vue. It's invoked just like a higher-order function over an array and a mapper function: https://lit-html.polymer-project.org/guide/writing-templates...
I also don't think custom syntax is necessary or that useful. JavaScript has enough abstraction power in functions and objects that you can build very nice embedded DSLs without a compiler or custom template syntax.
but my main point is that both approaches are still valid, and rather than choose a framework based on its particular DX, we should move away from frameworks and be able to write individual components with whatever DX the developer wants, and have them all work together.
Yes, you can do some optimizations. Keyed list diffing is obviously doable (any respectable vdom does it). Marko-style optimizations (i.e. compiling to no JS hydration logic whatsoever) is far more difficult. IIRC Dominic from the React team looked into it at some point a few years ago but it truly is a very very difficult class of optimizations to implement generically while also supporting unrestricted JS syntax.
I think that JavaScript-first is far better for templating the more general case (or the lower level foundation) because JavaScript is where your data lives. It's generally much easier to bring markup into JavaScript than it is data and data manipulation into HTML.
In HTML you need re-invent expressions, scopes, control-flow, references, and imports. You're going to spend more time and code implementing a less expressive, slower, and more proprietary system.
In JavaScript you just need a way to describe fragments of the resulting DOM (whether you prefer JSX, function calls, or tagged template literals), and the rest is just JavaScript.
Now, I do see benefit from the HTML-first approach for a lot of people and some use cases. One reason I also push on web components so hard is that with interop comes flexibility in allowing a mix-and-match of approaches. As a side-project I'm working on an HTML-first declarative component system layered on top of LitElement: https://github.com/justinfagnani/stampino-element
This can interop with any other web component, so you can write your app components in HTML, but drop to JS for the most complex components and use JS-first third-party components and design systems.