I don't know if I can make the claim that if a program behaves to spec, the types must be right (that seems like an academic statement out of my own wheelhouse). But I've always scratched my head at the canard of "static types saves time from obviating the need to write a whole class of type validation tests", that just doesn't map to my own experience.
I challenged my intuitions on this recently and checked out some of my existing unit tests suites to see what my type error coverage looked like. It was exceedingly rare that I was able to fine a behavior assertion which after introducing an intentional change to cause a type error, didn't blow up the test.
The cases where I did find type errors that snuck through with false negatives, the test itself was actually written poorly and the same behavior assertion would fail in other ways unrelated to types.
I'm sure there are situations (in my own stack and in other folks') where my above findings are contradicted but, given that static typing for something like JS is not in any way a zero-cost abstraction, I'm not sure it's really a net win.
I may be misunderstanding what you write, but I've had many (many, many) programs that worked for the wrong reasons, which led them to break in very (very, very) ugly ways due to unrelated changes.
The only ways I know of to get rid of (some of) these bugs are to:
- use as much static typing as possible;
- wherever static typing is not sufficient, sprinkle this with a paranoid dose of assertions.
If I read what you write correctly, that's way beyond "testing for the correct behavior", no?
> which led them to break in very (very, very) ugly ways due to unrelated changes.
Sounds more like a problem with tight coupling than a problem with types per se. A function should not change behavior due to an "unrelated change", regardless of static or dynamic typing.
Let me give you a trivial example (with TypeScript-style syntax, but assuming no TypeScript-style type-checking):
```
function foo(config: {hasYakShaver: boolean}) {
if (config.hasYakShaver) {
// ...
} else {
// ...
}
}
foo({hasChuckYeager: false});
```
This will accidentally do what I want, but for bad reasons. And one day, quite possibly, a change that appears unrelated will break this.
And that's me being nice. If I start using reflection to do complicated stuff, or if I'm using C++ and the reason for which my code works is memory corruption... well, when I need to figure out what suddenly broke the code, I'm going to be up for serious a moment of loneliness.
Sure, static typing will catch such issues. But there is still no guarantee that the foo function (or the function calling foo) works correctly just because it passes the type checker.
If you have unit tests (or other types of automated testing) you have a much bigger chance of catching logical errors and it will also catch trivial error like misspelling an identifier.
> Testing for the correct behavior should be enough.
Also, in my experience, automated tests tend to catch many bugs, static typing also catches many bugs and defensive programming catches many bugs, too. There is an intersection between all three, but it's by no mean 100%. So I use all three of these.
And of course, let's not discount monitoring, fuzzing, etc.
You're not going to cover all of the correct behavior in any nontrivial software with unit tests unless you plan to spend 99% of your time writing unit tests. Static typing gives a good baseline of correctness.
> Static typing gives a good baseline of correctness.
Not really. Lets say you have an `add(a, b)` function. Static typing can guarantee that the function returns a number, but not that it returns the correct number. So you need unit tests anyway.
A unit test `assert_equal(4, add(2, 2))` actually tells you something about the correctness about the function, and it implicitly also verifies that it returns a number. So static typing does not save you anything.
Static typing does have advantages, for example for documentation and for automated refactoring. But it doesn't replace any unit tests.
Sure, in this trivial example there may not be clear benefit, but with more complicated code, operations on complicated structures, with potential state changes, etc. there are many benefits. Even in your trivial example you don't cover the correctness or correct usage of your function - since it's dynamically typed it could be called with non-integer arguments - what's the expected behaviour in those cases?
Static typing certainly have many benefits, just as it has a number of drawbacks. But I'm not making a general argument about static versus dynamic typing, I'm just rejecting the specific argument that static typing saves you from writing any unit tests.
Nobody made the argument that static typing removes the need for unit tests. It does check for many conditions that would have to be unit tested otherwise, such as calling functions with the wrong types.
I'm saying static type checking does not replace any unit tests, because you wouldn't write tests which only checks for "the wrong types" in the first place. You should check for valid behavior - i.e. is the output correct, not just of the expected type. Types are just an implementation detail. But if the tests verify that the output is correct, then it implicitly follows that the types are correct, so you get type checking for free.
No, as I've pointed out - even in your trivial example you're not testing for all correct behaviour. What's the correct behaviour when the function is called with invalid argument types?
> But if the tests verify that the output is correct, then it implicitly follows that the types are correct, so you get type checking for free.
Your unit test checks the correct behaviour for one out of infinite possible inputs. You don't 'get type checking for free' , what you get is no type checking at all.
This is true - unit tests generally does not test all possible inputs. They test some representative examples and edge cases. In most cases there are infinitely many possible inputs, and you get diminishing returns the more cases you test.
If I understand you correctly, you a suggesting unit tests in a dynamic language should test all possible invalid inputs. Since statically typed languages eliminate a subset of invalid inputs (because they won't compile), statically typed languages saves you from writing some of these tests.
My argument is that you wouldn't write such tests in the first place since they provide little to no value.
If function `foo()` calls function `bar()` you want to ensure that foo passes the correct arguments. You don't do that by checking that bar() rejects all possible invalid types. You do it by checking that foo() actually works and provide the correct result. The types involved are an implementation detail.
> You do it by checking that foo() actually works and provide the correct result.
But you're testing a tiny subset of possible scenarios of behaviour of the function. If you can anticipate all possible input types and value ranges, maybe unit tests are enough, but that's not realistic in a dynamic language/program where complex values are non-deterministically generated at runtime - based on user input, db, file, etc.
In any statically typed language I know, it is possible to create invalid values which looks correct to the type checker. E.g. if a function takes an interface as parameter type, you can create a dummy implementation which throw exceptions for any operation. So basically you have the same problem as you describe for dynamic languages.
If you are talking about untrusted external input then yes, you need to carefully validate and reject invalid input. If you receive JSON or a CSV file or whatever, there are an infinite number of valid inputs and an infinite number of invalid inputs. But this is the same for statically typed languages, and the type-checker will not help you, since a valid JSON string and an invalid JSON string look exactly same to the type checker.
> E.g. if a function takes an interface as parameter type, you can create a dummy implementation which throw exceptions for any operation. So basically you have the same problem as you describe for dynamic languages
That's not at all the same problem, because here you're intentionally writing code that throws exceptions for some reason? Also, many languages, such as Java will not allow you to throw a checked exception that's not declared on the interface method - such issues will be statically checked and caught - thanks for proving my point ;)
> of valid inputs and an infinite number of invalid inputs. But this is the same for statically typed languages, and the type-checker will not help you, since a valid JSON string and an invalid JSON string look exactly same to the type checker.
Not at all, in a statically typed language the JSON would typically be deserialized into a structured type with all the benefits of type checking, validation and usage of the resulting values. Of course it's possible to just use a JSON string directly, but that's not idiomatic and generally not the way it's done in quality codebases.
This kind of thing is really what contract-based programming is meant for; it shouldn't properly be a test, except that we have to resort to that for the lack of language support for DbC.
I really disagree with this. I use static types as a quick correctness check.
I've added types to large python projects to reduce the complexity of reasoning about the code base, and the typing does not make it run faster (actually, the opposite).
I've got lots of experience in developing with both static and dynamically typed languages, I know the differences and it has very little to do with anything you are talking about.
Dynamically typed languages tend to be simpler and easier to use. This means on the whole they have less bugs than their static typed counter parts.
The real issue with dynamically typed languages is that their performance sucks.
(That and according to this thread people not having a clue on how they are properly used. You can't just use the same development techniques as you use in a statically typed language. It's different.)
> I've got lots of experience in developing with both static and dynamically typed languages
And yet your comments demonstrate the opposite.
> Dynamically typed languages tend to be simpler and easier to use.
Very debatable.
> This means on the whole they have less bugs than their static typed counter parts.
Not at all.
> The real issue with dynamically typed languages is that their performance sucks
Every time you say something like this it just makes it obvious you have no idea what you're talking about. You can write almost completely typeless, highly performant, c and assembly code, while high level languages with advanced type systems are generally not the most performant.
You seen to be confused about interpreted/jit/compiled languages and static/dynamic typing - which are not the same thing.
> The article covers a study of the same name. In it, researchers looked at 400 fixed bugs in JavaScript projects hosted on GitHub. For each bug, the researchers tried to see if adding type annotations (using TypeScript and Flow) would detect the bug. The results? A substantial 15% of bugs could be detected using type annotations. With this reduction in bugs, it's hard to deny the value of static typing.
> Yes, I'm sure the LOAD instructions take pictures of cats as operands via their typeless instruction sets... What in earth are you on??
Have you written any assembly? Assembly generally has no or very rudimentary type checking, you're generally dealing with words/bytes and addresses, you can arithmetically add parts of strings, divide pointers, etc. Errors due to these operation will surface at runtime, not be typechecked.
> well once someone does that let me know.
You can use void pointers as return types and arguments for all functions in c code. The effect is significantly less type checking while having equivalent performance.
"A substantial 15% of bugs could be detected using type annotations. With this reduction in bugs, it's hard to deny the value of static typing."
Actually, that makes it really easy to deny the value of static typing. If the total number of bugs in dynamic and static code is the same. But some bugs in dynamic code would be caught by static typing checking.
We most conclude that adding static typing results in a large number of non-typing related bugs being added to the code base. It's simple maths.
"Have you written any assembly?" Yes, I'm an emulator author, thank you very much.
The registers and opcodes are typed with things such as u8, u16, u32, u64, i32, i64 and only work with data of the right type.
"arithmetically add parts of strings, divide pointers" You mean standard C stuff, you know the statically typed language.
"You can use void pointers as return types and arguments for all functions in c code"
Dynamic types is not the same thing as type eraser. That void pointer doesn't carry the information that it points to a picture of a cat for example.
The fact you don't understand the difference between a void pointer and dynamic typing doesn't exactly surprize me. It's more like a giant vtable.
> If the total number of bugs in dynamic and static code is the same.
Completely baseless assumption.
> Yes, I'm an emulator author, thank you very much.
Ahahaha let's have a link then.
> The registers and opcodes are typed with things such as u8, u16, u32, u64, i32, i64 and only work with data of the right type.
Those are sizes, not types, and the same opcode generally applies to signed and unsigned integers. You consider that to be a static type system and you think the purpose is performance and not correctness? Lol
> You mean standard C stuff, you know the statically typed language.
You're trying to debate against the need for static typing by pointing out unsafe parts of c? Ahaha
> That void pointer doesn't carry the information that it points to a picture of a cat for example.
First of all you can totally carry around runtime type info and value with a single void pointer. Not to mention many dynamically typed languages have type erasure and many statically typed languages have runtime type info. Also, you've claimed multiple times that there's no need for type checking whatsoever. You need runtime type information at runtime now?
Not OP, but as I understand it, it is about adding tests to check what the behavior will be if the input parameters have values outside of the expected type. E.g. what will be the behavior of "add(a, b)" if a or b are a string, null, array, object, undefined, etc. instead of numbers.
But why would you do that? Sounds like a waste of time.
If some other function uses `add()` but pass an invalid argument, then this bug would be discovered by testing the function which pass the invalid argument. Presumably that function would work incorrectly.
Say you have a python function that expects a dictionary, but instead a string gets passed in. At some point in the function the python function may or may not explode at runtime depending on how it's written. If your functional tests doesn't cover this scenario and you haven't guarded for invalid data types in your python code, you may never see this issue until it hits production or a refactoring of the code base is done.
Ok, now that entire scenario goes away with a language that has a compilation phase and has type enforcement. It's boiler plate inside the function for type validation your don't have to do inside the python function (or blame the caller) and a testing scenario(s) you don't have to write. That's the work I'm talking about.
The bug is in the function which passes a string instead of a dictionary. Presumably this function will work incorrectly, right? So the unit tests of that function will find the error.
Unit tests which verifies a function has the correct behavior gives you "type checking" for free.
You just have to shift the mindset to testing behavior instead of testing types.
The problem is that the function may work incorrectly by e.g. quietly causing an invalid state change. For example, that string that was passed instead of an integer gets stored somewhere. Nothing visibly breaks; it's just that later - possibly much later, and in code much removed from where the bug actually is - you reach for what you expect to be an int, but get a string instead. Good luck debugging where it went wrong.
You can also have bugs in statically typed languages. Perhaps not the trivial bug of passing a string instead of a number, but you still can have the (more likely) issue of passing the wrong number, i.e. a logical error.
Lets say `add(a, b)` contains a bug. Perhaps it doesn't handle negative values or overflow correctly or something else. But it still returns a number, just the wrong number. This kind of bug is much more likely. If this wrong number is stored somewhere you also might get obscure errors down the line and a static type system will not discover it. And this bug will be much more insidious and harder to track down.
So you need unit tests anyway to protect against logical errors. If a unit tests verifies the behavior of a function it will also implicitly verify the types without any extra work.
The case you describe is handled by some form of checked types. In Ada, I'd just declare the range and let the compiler handle it. In a more conventional OO language, I can have a wrapper type for that number which enforces all constraints in the constructor. Either way, the point is that things blow up as soon as possible.
Broadly speaking, this is the contract-based approach: at every data flow boundary, explicitly describe your contract, so that any violation can be detected immediately instead of propagating and poisoning derived state. Eager type checking is just a subset of that, true. That means that static typing is not enough, not that it's unnecessary.
OTOH unit tests cannot fully express contracts, because even with parametrization
you can only test so many of the possible inputs.
How do you define a contract which guarantee that the result of `add(a, b)` is correct? Not just that the result is a number, but that it is the correct number?
Of course `add()`is a silly example since this is already a built-in in any language, but consider a function `calculateSalesTax(product)` - how do you define the contract which guarantees the result is the correct number?
There is no way, in any language with any type system or any amount of tests (except maybe in a pure language with tests that cover the entire domain of the function, in which case you're just reproducing the entire function in the tests), to guarantee that calculateSalesTax produces the correct number in all cases. So the problem you describe exists equally for both dynamic and static type systems with tests. Dynamic type systems just rule out fewer possible errors (and fewer possible correct programs as well, to be fair) before the program runs.
Think of static typing as an autogenerated set of unit tests that ensure that for any given function, all arguments of the wrong type will be rejected.
For example, if a function accepts a string as an input, test it by passing in all other possible primitives and classes and ensure that the behavior is strongly defined and results in the expected exception class.
Then realize that if you haven't written all of those tests by hand in a dynamic codebase, you don't actually have code coverage. Anyone who works on your codebase could pass you an argument of the wrong type, and it might be in a code path that is rarely invoked and you will find out it's broken in prod.
> Think of static typing as an autogenerated set of unit tests that ensure that for any given function, all arguments of the wrong type will be rejected.
My point is that you wouldn't write such unit tests in the first place. Unit tests should test behavior, not types.
No, it's actually different, because in a statically typed language it would not even be possible to write code that would pass the wrong types of inputs, because it would not compile!
Also, if there are input values of the right type that cannot be handled properly, that should also be tested! That is the kind of thing tests are for!
Example: you write a function that divides two numbers. It's not possible to divide by zero. You should absolutely have a test that passes in a divisor of zero and asserts that some kind of custom DivideByZeroException is thrown. If you know that certain inputs are out of range, you should test them explicitly -- otherwise you're being sloppy. If there is a large number of inputs out of range, you should test all of them, and you can use a `for` loop if you have to.
The difference is, if you have a valid input out of range, that is all you need to test -- the language itself takes care of all the "wrong type" tests for you. Otherwise you have to write them yourselves, and if you don't, you don't actually have good coverage. I understand that it is customary in dynamic languages to not write those tests, but that is merely another argument for the superiority of static typing.
What you are saying all breaks down with larger codebases.
A codebase with 1000 people working on it benefits dramatically from static typing. Everyone who has tried to do the same thing with dynamic typing has learned that. And quite a few companies that started off with Python had to rewrite everything in Java.
If you honestly think that nobody ever passes the wrong type, you've never worked on a complex dynamic codebase with a lot of people. It happens all the time.
The more modern solution is to break the large codebases into self contained microservices. In which case dynamic typing scales indefinitely.
I've worked on dynamically typed codebases for a long time. Passing the wrong type is very rare. I mean how would the code pass the unit tests if it was passing the wrong type?
> Passing the wrong type is considerably rarer than the wrong value so need not be considered.
Passing the wrong value doesn't just happen randomly. It happens because of errors in logic many of which can be caught with a strong static type system.
> You shouldn't write test to validate types in the fist place. Testing for the correct behavior should be enough.
Yeah but with dynamic typing there's a whole bunch of behaviors you need to think about. You pass an int into a method that expects a string, what's the expected behavior? Or maybe something more nefarious, like a method that expects a Python dictionary but is passed in a pandas Dataframe, which has similar syntax with totally different meaning; what's the expected behavior?
Much easier to just specify what types you're expecting to work with and let a compiler or type checker deal with it.
> You pass an int into a method that expects a string, what's the expected behavior?
Probably an exception, depending on the language. E.g. if the method expects the argument to support the method "substring()" it would be expected to throw an exception if the passed object does not have such a method.
If you pass an object which support the expected interface but with a completely different semantics, then you probably get a weird unexpected behavior. But that would also be the case in a statically typed language.
The flip side of all that is that a lot of times those other data types that have the same interface will just work without needing to go through generic hoops. Of course, if your software is the kind that cannot tolerate errors, that's not good enough. Most software isn't like that, and the flexibility is often more useful than extreme safety.
The two examples are human errors easily avoidable by reading the documentation. And some mechanism that uses types like method overload should be transparent from a behavior perspective.
> The two examples are human errors easily avoidable by reading the documentation.
Okay, but that doesn't answer the question. We're in dynamic typing world; my method accepts any data type by design. What's the expected behavior when it doesn't receive the expected type? Do I add assertions and fail with an assertion error? Let the runtime fail for me? Silently convert the type? In any case, I need to add a test for that.
If your answer is "humans shouldn't make a mistake when calling the method," that's arguing for static typing, not against, because that's exactly what static typing prevents: a human making a mistake. Turns out static typing makes for a really good baseline level of documentation, as well.
> And some mechanism that uses types like method overload should be transparent from a behavior perspective.
Your method might work today for both a dict and a dataframe because it's only using simple accessors. But what if someone wants to go back and change it and they use something that's only on dictionaries, like the * operator? Your code would fine, your unit test cases probably all use dictionaries so they're fine, and it's only when you deploy to production and find out that someone's code elsewhere was incorrectly passing in a Dataframe that just happened to work, no longer does.
Static typing has a solution for behavior-based types too, they're called interfaces. Even better in languages like Go and Python (with Protocols) that use structural type checking for interfaces, rather than explicit interface implementations.
> In any case, I need to add a test for that.
You don’t test types. If you have a ‘trim’ method that accepts a string, you don’t unit test it with an array, unless you plan to add a test for every possible types. Static typing can help prevent a human calling it with the wrong type, but won’t prevent calling it with the wrong runtime values, like after casting an Any to string. Static typing do syntax check, not runtime check.
When I’m talking about testing behavior, it’s about testing that the code does what it’s supposed to. Not what the arguments is. So you test that ‘trim’ works on all strings, including empty. What static typing helps in checking assumptions when programming, but every bet is off when executing. Especially when interacting with the outside world.
If someone changes the code, it’s on them to notify everyone about breaking changes. If the documentation says that the method accept dict, and you call it with dataframes, it’s on you to check that it will continue to work with dataframes, not on me to ensure that someone is calling it with dataframes if I use specific code related to dict.
> If someone changes the code, it’s on them to notify everyone about breaking changes. If the documentation says that the method accept dict, and you call it with dataframes, it’s on you to check that it will continue to work with dataframes, not on me to ensure that someone is calling it with dataframes if I use specific code related to dict.
This is equivalent to re-implementing static types and a compiler with human processes. This isn't scalable.
Maybe, but using static types is not a panacea either. It only make it easier to assume that someone have correctly constructed the value and that all manipulation have been correct so far. The type checker helps by checking that if the assumption is true, everything down the line is also true. But only in the context of the code, not in the context of the process itself, where the assumption may be incorrect, or the chain is broken somewhere due to bad logic (that’s when off-by-one errors arise). Nothing is a silver bullet. Static types can make life easier by preventing some meaningless code, but they don’t ensure correct programs.
Program behaving to the spec is something you'd write a functional test for; but OP is most likely talking about unit tests. If you're testing e.g. individual functions in isolation, you don't know how they're actually used on the outside, so you have to test for invalid inputs that the language allows - just like in a functional test, you have to test for invalid inputs coming from the environment.
(There is a separate argument over what proportion of functional vs unit testing is optimal.)
I like static typing, but his this is just a silly argument. If the program behaves correctly according to spec, how can the types be wrong?