There's definitely xor type nonlinearities going on in factor models of personality types. The interaction between two factors is often very different at the extrema of each factor - often flipping sign.
Regression models can't capture that. Now it's possible that the behavioral nonlinearities are somehow emergent, and that despite those, the ECG can be captured by a linear model. But I wouldn't want to bet on that.
Nonlinearities in ML existed long before DL and ReLU. What's at the end of a deep CNN? Probably global pooling followed by a dense layer? That final dense layer is taking a weighted combination of the last stack of feature maps. So deep learning is fundamentally the same as the old approach: Construct features with a nonlinear transformation of the input. Then calculate a score by taking a linear combination of those features. The only difference with deep learning is that the manner of constructing those nonlinear features is left unspecified.
Interesting comment. I'm not an expert in CNNs but what you're saying makes a lot of sense.
Question: are you saying the final layer is "in effect" a linear combination? At least in transformer architectures, the dense end block is iirc three layers deep and uses relu. Even if the CNN's dense final part is one layer deep, wouldn't it also be using relu activations?
Even a one layer dense ANN can capture nonlinearities if it has a nonlinear activation fn. But maybe in practice the activations don't do a lot of work? Or am I simply mistaken about final layer, does it simply have linear activations?
Also, can you share your intuition about the nonlinear features? Spectral/wavelet analysis? Or something more complex?
> Question: are you saying the final layer is "in effect" a linear combination?
Not just in effect a linear combination; it is a linear combination. There are some exotic nonlinear NN layers that are used in particular niches. But in general, NN layers are syntax sugar over matrix multiplications (i.e. linear functions). The nonlinearities are the activation functions between the layers.
> But maybe in practice the activations don't do a lot of work?
No, the activations do do a lot of work. There are real nonlinearities in the network.
> Or am I simply mistaken about final layer, does it simply have linear activations?
Sometimes you actually do use a linear activation on the final layer for certain regression tasks, but that's not the main thing I'm getting at.
Let's say that your final layer has is a ReLU activation. What is this conceptually? You are taking a linear combination of the features from the previous layer and them clamping the result to >= 0. Sure, that's a nonlinearity, but it isn't going to have much in the way of emergent modeling capabilities. You need to stack many, many nonlinearities before you get that.
So my point is that a deep neural net of N layers boils down to a complex nonlinear function of N - 1 layers, followed by a "dumb" linear combination in the final layer. You can do this with traditional ML methods as well, but you have to handcraft your nonlinearities.
> Also, can you share your intuition about the nonlinear features? Spectral/wavelet analysis? Or something more complex?
There's innumerable possibilities here. It could be starting with a method that's inherently nonlinear, like nonlinear PCA, polynomial regression. Or it can involve transforming the output of a linear function (like the Fourier transform) in a nonlinear way.
Admittedly this very tough. And for really tough problems, like video synthesis, effectively impossible. But NNs get thrown at much simpler problems all the time.
It seems every language starts with free functions and classes (even mojo). They then realize that isn't so great: third parties are at a syntactic disadvantage (no dot fn's). And then, the language designers "fix" the problem with extension methods. Now you have three different function kinds, and we haven't even broached async.
Why not have only structs, free functions, and UFCS?
the historical expectation is that class methods will dispatch dynamically but free functions will not. so if you only have structs, functions, and UFCS you either:
1. don't dispatch on the first argument,
2. make the first argument privileged and dispatch on it, or
3. dispatch on all the arguments
the first solution is clean, but people really like dispatch.
the second makes calling functions in the function call syntax weird, because the first argument is privileged semantically but not syntactically.
the third makes calling functions in the method call syntax weird because the first argument is privileged syntactically but not semantically.
the closest things to this i can think of off the top of my head in remotely popular programming languages are: nim, lisp dialects, and julia.
nim navigates the dispatch conundrum by providing different ways to define free functions for different dispatch-ness. the tutorial gives a good overview: https://nim-lang.org/docs/tut2.html
so to sum up the answer to the original question: because it's only obvious how to make it nice and tidy like you're wanting if you sacrifice function dispatch, which is ubiquitous for good reason!
OK, so here's the deal with SSA and the running time of your compiler.
- You can convert code to SSA in something like O(N log N) or even close to O(N) if you get fancy enough. The conversion step is not very expensive to run, though it is annoying to implement. Converting to SSA is fine, that's not the problem.
- Optimizations implemented on top of SSA are cheap to run. That's sort of the point of SSA. So, that's not the problem.
- SSA optimizations achieve goodness when you have multiple of those optimizations running in something like a fixpoint. It's pointless to say "I have SSA" and then just implement one of those optimizations. I don't think MIR does that; like most SSA optimizers, it has multiple optimizations. The problem sort of starts here: SSA sort of implies that you're building an optimizer with multiple optimizations. Even if each opt is cheap, having SSA usually implies that you're going to pile on a bunch of them. So, you'll have cost either from death-by-a-thousand-cuts, or you'll eventually add an optimization that's superlinear.
- After your run SSA optimizations, you'll have nontrivial global data flow - as in, there will be data flows between basic blocks that weren't there when you started, and if you do a lot of different SSA optimizations, then these data flows will have a very complex shape. That's caused by the fact that SSA is really good at enabling compilers to "move" operations to the place in control flow where they make the most sense. Sometimes operations get moved very far, leading to values that are live for a long time (they are live across many instructions). This is where the problem gets significantly worse.
- Even without SSA optimizations, SSA form adds complexity to the data flow graph because of Phi nodes. You almost always want the Phi and its inputs to use the same register in the end, but that's not a given. This adds to the problem even more.
- Probably the reason why you went for SSA was perf. But to get perf from SSA-optimized code, you need to have some way of detangling that data flow graph. You'll want to coalesce those Phis, for example - otherwise SSA will actually pessimise your code by introducing lots of redundant move instructions. Then you'll also want to split ranges (make it so a variable can have one register in one part of a function and a different register in another part), since SSA optimizations are notorious for creating variables that live for a long time and interfere with everything. And then you'll definitely need a decent register allocator. No matter how you implement them, the combo of coalescing, splitting, and register allocation will be superlinear. It's going to be a major bottleneck in your compiler, to the point that you'll wish you had a baseline JIT.
Those things I mentioned - coalsce, split, regalloc - are expensive enough that you won't want them in a JIT that is the bottom tier. But they're great to have in any compiler that isn't the bottom tier. So, it's a good idea to have some non-SSA baseline (or template, or copy-and-patch, or whatever you want to call it) JIT, or a decent interpreter, as your bottom tier.
Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns.
I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little crestfallen.
I know old habits die hard - I myself used to use such language. So I don't want to be too harsh, and equally, maybe you know more details such that the 'man' is justified. But your comment is ultimately about manners, so I think it's appropriate for me to point this out.
I would drop it where an interlocutor is specifically referenced. So 'oh man' is ok. But c'mon man usually expresses disagreement with a specific person. So while I would certainly say c'mon man to a trans or cis masc (which, if I had any shadow of a doubt, I would determine by querying their pronouns, after sharing mine so as not to inadvertently put them on the spot), I would definitely not say it to a trans or cis fem, and also not to an enby.
Someone's life obviously takes priority over gender sensitivity :)
Again, I realize it's a bit of (cognitive) work, and takes some getting used to. But this is hacker news - a certain intellectual pedigree is assumed.
Oh man is something I say to all sexes. Your comment just seems to be policing language with no point. This kinda stuff hurts your cause, which I wholly support.
Might be a regional thing but "my friend" sounds very condescending / patronising to me! I'd personally feel quite upset if somebody began a critique of my behaviour with that.
I think at this point it's just best to assume that "dude", "man", and "bro/bruh" in slang are just gender neutral.
There seems to be something about the English language where people think gendered language relates to gender. But in other languages, a table or a bridge may have a gender. A cat may be referred to as a she, despite its physical sex or gender identity.
> There seems to be something about the English language where people think gendered language relates to gender.
This.
Sometimes when people talk about using gender-neutral terms, I wonder how it would work in other languages.
As you said, for some languages there is a gender for inanimate objects.
In Arabic for example, apparently everything is either feminine or masculine. All countries are feminine, table is masculine, and chair is feminine [1], book is masculine, and car is feminine [2].
Other languages such as French, Russian, and Spanish also seems to assign genders to objects.
The Sanskrit language on the other hand seems designed with feminine, masculine, and neuter, but some objects have a non-neuter gender anyway.
Quoting from [3] (on Sanskrit):
> "Fruit" is neuter, but "tree" is masculine! "City" is neuter, but "village" is masculine! "Army" and "knowledge" are feminine, but "action" is neuter! In truth, most nouns take a certain gender by default, and we cannot guess a noun's gender just by looking at a noun's meaning.
> Other languages such as French, Russian, and Spanish also seems to assign genders to objects.
Those languages assign gender to words, I think, not objects. A car in German can be "der Wagen", "die Karre" or "das Auto".
There may be languages that assign genders to objects. Perhaps Dyirbal, which Lakoff refers to in "Women, fire and dangerous things"? I'm not familiar with any such language so I can't say.
Btw, of all the languages that I mentioned above, Sanskrit is the only one that I know atleast a little of. And that was because I took a Sanskrit 101 class in college many years ago.
Indeed there's no question that Nim is basically following C++'s lead on this. Nim iirc always had constructors and destructors. Final piece of the puzzle is move semantics, and I recall a blog post where Araq came up with something very similar.
yes, Nim has move semantics, but takes care of you more than c++ does.
for example, if you use an object that was previously moved, you dont get garbage, the compiler turns the first move into a copy (and tells you)
It very much is, and the point is, it _used_ to be more like java. Araq basically pulled off a very daring switchover from reference based language system to a value based one.
So now the language can credibly claim the same as c++ - no room left closer to the metal. But it's packaged in a much nicer syntax (imho), and has features like macros which we can expect I'm C++ in maybe 10 years, if we're lucky.
You can just slap a resistor in series, which will induce a drop in potential over it. That, in turn, will mean the potential difference across the capacitor is lower, leading to a more shallow exponential decay of the current over the circuit - ie a slower discharge.
And that's just a passive circuit, with an active circuit you have a lot more options.
Note: I studied maths and computing so I could very well be wrong. I would appreciate being set straight by elec eng majors!
Sorry but this is simply par for the course for any of the above.
You can certainly wrap a lot of that stuff, but you need to make assumptions, and the person that uses likely is writing demanding app, and they want full control over literally everything - but they also would like to cut time to port to Linux by half (say).
Regression models can't capture that. Now it's possible that the behavioral nonlinearities are somehow emergent, and that despite those, the ECG can be captured by a linear model. But I wouldn't want to bet on that.