> So, what would you have them do? Do you even have an example of how you think general purpose formatters should be "safe" under your model of the world?
Yes. In my very first comment I posted a link to Java's upcoming feature, which uses the type system to ensure proper validation in a general-purpose string templating mechanism. Here it is again: https://openjdk.java.net/jeps/8273943. As I also said in my first comment, our security experts were so concerned about this problem, which was empirically found to be one of the most dangerous programming operations in recent years (famously so among those interested in security), that they wouldn't let us add it to Java without a solution to the security problem
> This is not a defect in the SQL formatter, I am using the wrong tools.
But 1. people do use the wrong tools, which is why it's such a common and dangerous vulnerability, and 2. a typed language certainly can prevent that, as in my link.
> I can't do anything about the fact that people will make grave logical errors when programming
That's a strange position from a Rust user, especially as in this case there certainly is something the language can do.
> I suspect your 0.1% figure is pulled out of your backside, but, sure, somebody will get it wrong.
Pretty much all top security vulnerabilities lists list this problem near the top, and some languages are so popular (many millions of professional devs) that even 0.1% of their users are sufficient to make this problem a common one, deserving of its spot. So I figure that 0.1% is about the right number to make this a very common problem.
> Again, if you fear strings you can use special-purpose languages which don't have any strings so that you can't possibly make this mistake.
Or use Java's upcoming string templates. Or, if you prefer less popular languages, Scala, which uses a similar technique.
But that JEP doesn't actually provide the safe general purpose formatter you're insisting Rust should have built here.
The exact same programmer who you insist will write
format!("\"{myString}\""); // in Rust
will also write:
CONCAT."\"\{myString}\""; // in Java with this JEP
The JEP argues that it'll be all OK when Steiner attacks^W^W^W so long as you only use the potentially tainted "strings" via an API which doesn't allow general string objects. But, that's also the exact situation you've dismissed as unable to prevent abuse in Rust.
Java can't stop you writing your supposedly "JSON" data made with CONCAT to a file or over a network socket, and Rust can't stop you writing some made with format! either, it's just data, who knows why you wanted to write this or that gibberish?
According to you we should expect 0.1% of the "safe" Java using these templates to be vulnerable.
The key is understanding why programmers make security mistakes. As you can imagine, this is an active area of research that, for obvious reasons, is of much interest for language and API designers, but one of the causes that is currently believed to be a significant one is that programmers reach for the wrong tool for the job -- security wise -- not because they're idiots, but because many programmers don't understand the security implications of something that they believe is completely innocent, and so they reach for the more convenient option, which might be less safe.
So while preventing someone from constructing JSON (which can be output as a string) is not always as fool-proof as preventing someone doing the same with SQL (as the driver API will simply not offer an option that takes a string) the reason such a feature is added in the first place is because it is easier, and that is what makes it more attractive. A more convenient mechanism attracts people to use it.
Some things, like SQL and JSON, are often convenient to create using templates. Using something like CONCAT."""{"x": "\{x}", "y": "\{y}"}""" is certainly no more convenient, and so no more attractive, than writing JSON."{x: \{x}, y: \{y}}", but it is more convenient in some situations than using an API that requires defining or generating a type for the object in the source language. So while safe JSON libraries in Java and Rust exist today without a built-in template mechanism, being able to offer them with that mechanism will make unsafe options less tempting by comparison.
That is why experienced security experts recommend that languages do not add a string templating mechanism that is more convenient, and so potentially more attractive, than safe options for security-sensitive uses. Their general rule is, "when possible, don't make programmers jump through more hoops to do something secure than something insecure." You're free to find this advice misguided, but I wonder if the people who added this feature to Rust consulted with security experts before adding it. I don't think I would have been aware how sensitive this feature is if it weren't for the advice of security experts.
Maybe you just aren't familiar with serde_json again:
json!({"x": x, "y": y})
... is valid and even idiomatic Rust today to express a JSON object with two entries named x and y containing whatever is in the variables x and y, or, if that doesn't make any sense (e.g. variable x is an operating system Mutex) it's a compile time type mismatch.
That's much easier than the complicated dance envision in the JEP and which you insist will be "convenient" and dissuade Java programmers from choosing the easy option, yet of course you always get the intended JSON out, even if say x is a malicious input intended to trip up naive JSON encoding.
That json! macro does the right thing. What the Java feature does is give that exact capability to any library that can benefit from templates.
The entire point of my comment was that, surprising as it might seem to some, templating is now known to be a particularly dangerous area -- quite possibly the most sensitive aspect of language and API design after buffer overflows -- and that's why templating features require a security review.
If Rust's designers' answer is that their security analysis has led them to the conclusion that the right stance against code injection is for template APIs to role their own templating macros from scratch rather than use a higher-level templating mechanism -- then they're doing what I suggested, and macros are their mechanism that corresponds to Java's pluggable templating design. If they did not consult with security experts on their string formatting feature, I suggest they do so. Perhaps all that's needed for Rust is to include components in the standard library that would help library authors write correct and secure templating macros.
Yes. In my very first comment I posted a link to Java's upcoming feature, which uses the type system to ensure proper validation in a general-purpose string templating mechanism. Here it is again: https://openjdk.java.net/jeps/8273943. As I also said in my first comment, our security experts were so concerned about this problem, which was empirically found to be one of the most dangerous programming operations in recent years (famously so among those interested in security), that they wouldn't let us add it to Java without a solution to the security problem
> This is not a defect in the SQL formatter, I am using the wrong tools.
But 1. people do use the wrong tools, which is why it's such a common and dangerous vulnerability, and 2. a typed language certainly can prevent that, as in my link.
> I can't do anything about the fact that people will make grave logical errors when programming
That's a strange position from a Rust user, especially as in this case there certainly is something the language can do.
> I suspect your 0.1% figure is pulled out of your backside, but, sure, somebody will get it wrong.
Pretty much all top security vulnerabilities lists list this problem near the top, and some languages are so popular (many millions of professional devs) that even 0.1% of their users are sufficient to make this problem a common one, deserving of its spot. So I figure that 0.1% is about the right number to make this a very common problem.
> Again, if you fear strings you can use special-purpose languages which don't have any strings so that you can't possibly make this mistake.
Or use Java's upcoming string templates. Or, if you prefer less popular languages, Scala, which uses a similar technique.