This reminds me of Seeing Like a State[0] (I know, it's not new... but I only got around to reading it earlier this year.)
Automating a process makes it more standardized and legible, but takes away a lot of the nuance and of the resilience that human institutions tend to bake in. Do it too little, you get chaos; do it too much, you're destroying the very thing you were trying to nurture.
It's certainly deformation professionnelle, but the parallel with software is eerily relevant.
One thing on my list for CRUD software is an „I know what I’m doing” button that bypasses validation. Usually it’s behind manager or admin approval.
It’s pretty tricky to get right and is very case by case in what it specifically means. But it’s been critical to handling edge cases.
For example, in 99% of cases, you can’t progress to stage X without doing Y. But in this case „I know what I’m doing”.
Though this puts a lot of pressure on downstream processes to handle missing data. Fortunately that hasn’t been an issue too much because validation grows and changes over time and must naturally deal with records that were created under prior validation when their state was originally valid.
One time I was involved in automating a business process. Nothing super complicated - scrapping data from a portal, filling forms, some data wrangling. It had to handle a few edge cases and involve a human in the loop. I tried to include as much data validation as possible, but this being a business in real world there were of course some exceptions. I implemented a Superuser Override which was basically a button that launched a popup with another form where the app users requested an approval from their manager to accept the unvalidated data. It was red and scary and it had Bootstrap .danger all over because it was a cool thing to do back then. Don't judge, ok?
Things worked as they should have. The automation saved tons of time, even if some fields were missing. Occasionally, there was a Superuser Override request which sent an email to the manager overseeing the process. Actually, things worked too well - the volume of processed data increased and so did the number of the Override requests. The manager - busy as they usually are - started approving all the requests. Manually at first and then they ran a macro that did it for them.
Eventually the app users got wind of it and they started using the Superuser Override liberally. Before they shown more restraint.
I wish there was some clever punch line to this story but I don't know what happened after.
If I had to do it again, I'd be more aggressive with validations, make the approval process more like a code review, involve the most experienced app users as the Superuser, and have audit logs. Oh and Tailwind instead of Bootstrap.
Yeah. My first choice for approval is admins because some managers are better than others on this for sure. But that really is only possible because I build for a fairly small user base.
Audit logging is definitely another must for me. I literally won't make an app without it. The app I'm working on now had a case where a PM promised up and down that edit history wasn't needed for a particular section, and lo-and-behold 2 years later it turns out that actually we do need the history.
This is so important. You don’t want to stop flows from happening because of a bug or bad config. Just keep track of the override who did it when and if possible a quick blurb. Then ensure those exceptions are caught and fed to dev.
I like this idea, but seems like a bug factory. Going to break lots of downstream assumptions and create duplicate workflows for these (hopefully) rare events.
When you write a whole essay like that it's more than a bit ridiculous, but the same idea on a smaller scale can work very well rhetorically. For example, take this section from Churchill's notorious speech:
We shall fight on the beaches, we shall fight on the landing grounds, we shall fight in the fields and in the streets, we shall fight in the hills; we shall never surrender.
every word except "surrender" has an Anglo-Saxon root.
Earlier in the speech Churchill used the same rhetorical device of in a positive sentence full of words with french/latin roots:
A miracle of deliverance, achieved by valor, by perseverance, by perfect discipline, by faultless service, by resource, by skill, by unconquerable fidelity, is manifest to us all.
(That form of repetition seems to be called anaphora, I just learned.)
It’s interesting that people often comment on the Anglo-Saxon/Germanic roots of that famous phrase – but nobody finds it weird to use Germanic words in a (defensive) war against Germany and eschewing French words in support of France in this war. The explanation seems to be the specific English context: there the connotation of Anglo-Saxon as the language of the people seems to override any germanic associations.
Have you read Farnsworth's Classical English Style? He talks about Saxon vs. Latinate word choice (and invokes Churchill frequently!), which makes me think you've either read it already or would enjoy it if you haven't yet :)
AOPS is a great community but it's for competitive mathematics (AIME, AMC, USAMO, etc.)
For OP: I think the problem with your question is that "math" is too broad, e.g. Math Overflow is for professional mathematicians, things like AOPS are very specific and most online communities are about what you would call "recreational" math (think 3blue1brown or standupmaths), likely far below what you want.
ok, I guess from what you say Math Overflow might be my best bet then, I'd like to have the competence of a professional mathematician, even if I work as a software engineer because it pays more :)
The kind of maths they do on Mathoverflow is PhD level and above... they probably get paid more if they leave academia, but then they are hired to do the kind of maths on math.stackexchange.
Not really, as a former theoretical physicist who also got offers from many math PhD programs and have many math PhD friends, by far the most recruiting emails we get are from quant trading firms, and you won’t find that kind of math on Math.SE.
But yeah, roughly speaking you need to at least be a math PhD student who have passed your quals to be at the entry level of MathOverflow.
At least it's not the Alapin(2. c3). As a sicilian player from the black side this is like half my games now and I'm considering switching to some other defence because it's boring me to death. Not sure what I'd go for though, feels like the sicilian is the only non-dubious defence to e4 to offer real fighting positions. Caro-kann is passive, French is pretty passive unless they play Nc3 and you get the Winaver, Scandi is a snoozefest, and the Petrov is frankly disgusting in its lack of ambition. Two Knight's defence is very fun if they play Ng5 or d4 but now everyone's playing d3 to make the game as boring as possible.
I guess I'm stuck with the Sicilian. If anyone knows a way to make the Alapin wild and tactical as black, I'm listening.
Archangel does indeed look like a fun one. Lots of good options in the Ruy Lopez really. The Open Spanish and the Marshall gambit can be a lot of fun too.
Funnily enough I actually play the Italian as white, but I don't play the boring c3 d3 lines. After 3. Bc4 Bc5 I play 0-0 and on Nf6 play d4. exd4 e5 transposes to the Max Lange and Bxd4 Nxd4 Nxd4 Bg5 gets into the Italian gambit which is very sharp. My main issue is that on 4. 0-0 d6, d4 doesn't really offer much. I've been playing a "deferred Evans" with b4 and getting some fun positions that way though. I'm also working on learning the minutiae of 2. Bc4 so I never have play a Petrov ever again...
> A machine is coming along that genuinely might be 80% of your skill set in a few years
Is this supposed to be scary? This scenario is absolute stonks if you're a developer worth your salt. A machine makes me 5 times more productive, and I don't even have to commoditize my complement because they're taking care of that themselves?
Worry not, we are already doing that! GDPR + AI act is almost the definition of unstable regulatory regime, compounded by the fact that the specific implementation of those is up to the member states.
I don't have weird AI fears and I'll be shocked if LLMs end up being more than "it helps writing code sometimes, kind of", but there are potentially great use cases to investigate and I don't see what all of this is getting us.
I could be wrong, but my impression of the post was that it is not meant to be a novel solution for a hard problem, but rather a demonstration of an interesting technique on a simple enough problem that the problem doesn’t distract from the technique itself.
In an ideal world, the nurse to patient ratio would be high enough that patients could be seen on regular rotation frequently. I've never been in a hospital where this was the case. So a system that can correctly prioritize resources for critical cases even if it's pulling resources away from non-critical cases will probably result in a net improved outcome.
I don't know...If every 3rd time I was alerted it was some relatively serious issue, vs how often there is a serious issue when just doing rounds that you stumble upon, I'd think that would be a pretty good alert rate compared to the norm. But then again, I'm not in healthcare.
Essentially, it depends on workplace integration, i.e. how much effort it takes to discover the alert trigger. From personal experience, I'd say the upper limit of inconvenience is 'click the alert' on mobile, and 'move mouse on alert label to see tooltip' on desktop. Anything more will be quickly discarded, especially if it involves a popup or opens a new window.
Most of those alarms will warn about trivial everyday results, that may (with rather low probability) cause death down the line. My bet is they'd get mostly ignored very quickly.
Wow you're right. I mean it's all maths articles on Wikipedia, not just statistics.
I think there are two causes of Wikipedia maths articles' general awfulness:
1. They're probably written by people that just learnt about them and want to show off their superior knowledge rather than explain the concept.
2. The people writing them think it's supposed to be a precise mathematical definition of the concept, rather than an easy to understand introduction. It's like they're writing a formal model instead of a tutorial.
Often the Mathworld articles are a lot better than Wikipedia, when they exist at least.
Fun bit of trivia (though depressing) from the wiki
The term "MARS" is trademarked and licensed to Salford Systems. In order to avoid trademark infringements, many open-source implementations of MARS are called "Earth".
R is cursed beyond reason, but traditional software engineers are sleeping on it, IMO. It's very easy for quantitative people that are not software developers to get something done quick. The downside is exactly what you described, most projects are not just the model, they eventually tend to incorporate generic data wrangling, UI/web code, etc, and a general purpose language tends to work better overall.
I have a similar anecdote: I was brought in on a project where a group of terrorists implemented a solution for a TSP-like problem directly in R. We eventually replaced that thing with OR-Tools.
+1. I am a software engineer but I double majored in statistics and wrote a lot of R in undergrad. The library ecosystem is incredible. Essentially any technique in statistics has a well-documented R package that is one library() call away.
I keep wondering if I should learn the Python data science ecosystem at some point but it just seems like a waste of time. One of my personal projects is written in Python but calls into R for statistics/plotting.
The same thing can be said about Python. Python itself is not such a great language, especially in terms of performance. However they managed to have every simple package in the world of analytics and ML added to the Python ecosystem, so it is impossible to stop using it.
I think of MATLAB, Mathematica, R, and Python together as "practitioner's languages". These are languages that are designed from the core to be highly productive to a specific kind of technical worker (in the sense of developer velocity).
MATLAB for engineering. Mathematica for mathematics. R for statistics. Python for software engineering.
You could also say "Python for ML", of course, and that would be true, but Python is also used for general purpose programming much more than the other three. I think that "Python for software engineering" is more correct.
I think that each of the languages is shaped to the way that its users think about the problems that they want to solve with it.
MATLAB is shaped around linear algebra. Mathematica is a term-rewriting system. R has lots of magic around data and scopes to make the surface syntax for stats nice. Python is shaped like a traditional OOP language but with a pseudocode-like syntax and hooks so that libraries can act magically.
This is kinda half-baked, I'm trying to express this for the first time. But essentially I think that Python is what you get when you have real programmers (^TM) try to create the programming equivalent of something like MATLAB, Mathematica, and R.
And so of course ML, which is a field dominated by real programmers, adopted Python in order to create their ecosystem.
Generally agree, though Mathematica is for physics, I rarely saw it used by mathematicians, especially compared with MATLAB, which was closer to the programming language of choice. That or Julia were both more common.
> I think that Python is what you get when you have real programmers (^TM) try to create the programming equivalent of something like MATLAB, Mathematica, and R.
I think this is a better description of Julia tbh, at least relative to MATLAB.
I believe the advantage of Python for the domain of ML is just how easy it is to take tested C and Fortran code and add it as a bona-fide software engineering package. In other words, it is a language that allows standard scientific software to be consumed according to the latest software development trends.
I guess I should also mention Inform7 for interactive fiction as another example. That is an extremely weird language. It's shaped to the minds of fiction authors, who are very, very different from software engineers.
One concrete example: R has 5 distinct, actively maintained class systems, at least 3 of which are somewhat commonly used for new projects. I.e., there are 3+ reasonable ways to declare a class, and the class will have different semantics for object access, method calling and dispatch, etc. depending on which one you choose.
Another: R can’t losslessly represent JSON because 1 and [1] are identical. That’s a float (well, float vector) literal by the way, the corresponding int literal is 1L, though ints are very prone to being silently converted to float anyway.
The R development community is (was?) consciously focused on single operator managing their stuff on their workstation.
Considerations like repeatable procedures, reliable package heirarchies, etc. were clearly and more or less politely Not Interesting. I spent several years with one of my tasks being an attempt to get the R package universe into Gentoo, and later to RPM packages.
I wouldn't say the R devel community was rude about it, but the systems-administration view of how to maintain the language was just not on their radar.
At the time I was trying to provide a reliable taxonomy of packages to a set of research machines at a good sized university. Eventually, I gave up on any solution that involved system package managers, or repeatability. :)
So if you're a researcher driving your own train, R is freakin' FANTASTIC. If you're the SA attempting to let that researchers' department neighbors do the same thing on their workstations, anticipate fun.
I've wrapped R to python before. That was okay, a bit stilted but still could take to production if absolutely necessary.
You're 100% right that R is great for data scientists (my background) for frontier level academic implementations as well as toy/simple models. It's generally a poor runtime for computation and suffers from much of the same issues as Python for data quality and typing. Python is better for battle-hardened type stuff, has better debugging tools for certain.
R _can_ be done well, but the juice isn't worth the squeeze typically.
as a maniac who has written a full stack application in R: I agree. It is easy to get something out of the door quickly for the average R user, but maintenance will show you quickly how brittle everything is.
in addition to neighbor, there are multiple standards for everything, even core language features like objects, which makes reasoning about code difficult. dplyr and friends, which make the aforementioned data analysis easier, require interacting with one of the least consistent metaprogramming systems I've ever dealt with.
It runs pretty much as described - you enter `=py` and it becomes a place to dump python code directly without configuring anything. The "catch" is it runs in the cloud and you need to buy the appropriate license level.
from what I understand from seeing youtube videos of it in action, you can use python functions libraries for the calculation part, but one of the strenghts of VBA was that you could interact with the Excel software itself, and that bit hasn't been replicated in python.
people liked they could set up scripts in VBA for automating routine tasks, but hated the VBA language syntax, from what I've understood, and wished they could program those functions in some more popular language like Python etc. But that isn't what this is.
Automating a process makes it more standardized and legible, but takes away a lot of the nuance and of the resilience that human institutions tend to bake in. Do it too little, you get chaos; do it too much, you're destroying the very thing you were trying to nurture.
It's certainly deformation professionnelle, but the parallel with software is eerily relevant.
--- [0] https://en.wikipedia.org/wiki/Seeing_Like_a_State