My understanding of strict textualists' arguments (especially Gorsuch), is that they view it as a separation of powers issue.
By only looking at the text of the statute, the Court—which consists of unelected officials with lifetime terms—is trying to limit its power. If the text is unclear, they feel it should fall on elected officials to clarify it.
Certainly, but there's a problem with that. Because an outcome will still happen; an outcome determined by the law, -and its interpretation-. Even an interpretation of "we should (overturn lower court's decision/uphold lower court's decision) because the law is unclear" is an interpretation of the law.
After all, it is as equally fair to say "The wording of this law leads us to a clear understanding of the intent to mean X, and if the legislative body feels that to be in error they should pass new legislation to clarify it" as it is to say "The wording of this law leads us to an unclear understanding, and irrespective of intent we will decide !X until the legislative body passes new legislation to clarify it".
While the non-textualists may be explicit in trying to understand the intent of a law in accordance with their own biases, the textualists are still interpreting the law according to their own biases. There have even been some...really interesting cases of logic to try and do so (i.e., earlier generations' understanding of 'sex' matters when interpreting anti-discrimination statutes, but earlier generations' understanding of 'gun' does not matter when interpreting the 2nd amendment). It's incredibly rare (pretty much unheard of in the case of a few of them) for them to decide against their own predictable biases because a law is ambiguous, or a situation is novel.
By only looking at the text of the statute, the Court—which consists of unelected officials with lifetime terms—is trying to limit its power. If the text is unclear, they feel it should fall on elected officials to clarify it.