Hey fellow reinsurance actuary! I totally agree that Excel has its place in modeling, especially one-offs, and your criticisms make sense. That said, we have been moving a lot of our calculations to Python. We have had way too many rickety tools to move files or send emails (“first you open this spreadsheet and click this button, then you open this spreadsheet and click this button, then...”), and way too many version control issues over the years. Python solves those nicely.
I’m curious about docker + pip, why do you like that better than poetry or pipenv?
One reason why Python is so successful is that it places very nicely with C code. Many of Python's libraries are thin wrappers around native DLLs.
For example, numpy is a wrapper around a BLAS DLL (e.g. Intel MKL). Pipenv manages the python side of things, but don't exert control over the system DLLs (like Docker does). Anaconda gets very close to what Docker does (by managing DLLs). Have not used poetry, so can't comment.
Ultimately, like most dependency management issues, lacking a stable DLL environment won't be a problem until it is :)
numpy is much more than a wrapper around a BLAS dll. BLAS implements three sets of operations:
Level 1: unary and binary vector vector operations, one transform.
Level 2: Matrix vector operations.
Level 3: Matrix matrix operations (most famously the dgemm routine).
Perhaps some blas implementations offer more features, but that would defeat the purpose of a standard interface.
Okay, I can totally see it now. We are still at the stage where people seem to think that I’m neurotic for worrying about the python side of things, so DLLs have not been on the radar. ;)
In Nodejs native libraries are a PITA. The compatibility API breaks on a schedule every 6 month, dependencies get updated by OS distros sometimes breaking stuff, then you rely on the OS being able to compile the library. I hope Python has better native interop.
It would be considered incredibly bad practice for such a dev's machine to be used to perform almost any calculation of significant business importance. It's why mediocre Tech Executives are able to appear like they got some work done by focusing on "no access from dev to production."
With Excel there is no such separation, and when there is it would make a great punchline to an XKCD or Dilbert cartoon.
... that’s a laugh of bitter jealousy. I’ve dealt with XML, but I’ve never with XML that came with a schema, or which would have reliably followed one. Not saying schema-less formats are great, but at least I can eyeball them to see what is going on.
XML turned everyone into a language designer during an era where we already knew that language design was a rare skill.
If I saw a schema, which I often didn’t, it usually didn’t say what the author thought it said. To a first order approximation, all the good ones I saw came from one tool (XMLSpy possibly?)
Namespaces ended up in a sort of uncanny valley that I can’t quite do justice to.
how about docbook?
It's been a while, but I think even eclipse's project.xml used to have a schema, and would validate it if you tried modifying it yourself.
While I understand why authors may not like the IA’s emergency program, I find the language to describe it confusing. A number of comments here use the term “uncontrolled digital lending”, and it took me a bit of reading to discover that the files still had DRM and would “expire“ after a set time. The phrase paints a picture of the IA sending unprotected PDFs to anyone and everyone. I think it is confusing at best.
Authors, I don’t think the confusion does you any favors. I was more sympathetic to your concerns before I read your comments.
They distributed an unlimited number of copies - not backed by anything physical. That being the case, there is effectively no time limit on how long you can borrow something (you can just re-check it out).
That is just straight up piracy. They lent copies they did not own.
They tried to justify it by saying libraries were closed but they ignored the fact that more than 90% of libraries already have digital lending programs that went through the due processes and are legal.
I have thought about using DVC at work, but I have held off in part because it isn’t clear to me how the development is supported financially. I recall seeing a job posting (almost put in a resume!), but I don’t see any pricing info, so I’m a bit confused. Am I missing something obvious?
Hi! Great question. It started as a pet project, but now is being supported and belongs to a VC funded entity - Iterative. + it has independent contributors, it's an Apache 2.0 project after all. In terms of business long term goals, I would you can compare us to Hashicorp. There are no plans to monetize or even build premium features in DVC or CML.dev project, rather we'll be building enterprise layer on top of them. I really hope that clears out your concerns. If not - I would be happy to answer any question.
In my experience with auditors, harsh/rigorous does not necessarily mean better because the auditor’s mindset makes an enormous difference. I’ve had process-focused people spend days of their time and mine digging into things that do not have a material impact on results. Unfortunately, even if people have the best intentions, I think that process-focused-but-loose and results-focused-but-rigorous can be easy to mix up, especially for people not in the weeds.
This really resonates with me. I have seen the “buy before you build” attitude turn into “buy because we can’t build”. I know correlation is not causation and there are always plenty of other things going on... but it sure seems like the pride in our work dropped off and we were on the wrong side of every tipping point.
I've seen this at a few places now. Learned helplessness is definitely a thing and a lot of it comes down to an inability or unwillingness to hire broadly. When you're hiring Bolt Turner #3, they might know a little bit about turning bolts (but might not, either) but expect everything that isn't a turnable bolt to be handed to them. But the screw turners and the pipe welders you've also hired don't think a working ecosystem of tooling is their responsibility either and they're not really equipped to build it either, so the only option becomes to buy. And at that point you lack the breadth and the depth of knowledge to effectively choose the right thing to buy, and the outcome is as if by dart board.
There's a reason to hire specialists in this industry, but we conflate one-track ponies with specialists and it's to our detriment.
Thanks for writing/sharing this. It puts words around some dynamics I’ve seen, and not just online. It got me thinking about people and my mental models of them, which I think is helpful.
There are a lot of places where Know Your Customer rules apply. Further, there are places where identity really does matter, such as insurance. The approach you outlined sounds like a headache when it comes to those things. Not sure who the headache is for... :)
Mostly because of the concept of “insurable interest”. Insurance pricing doesn’t hold up unless the buyer has a reason to avoid the outcome being insured against. The simplest example is life insurance.
Why do you question whether identity would matter?
I’m curious about docker + pip, why do you like that better than poetry or pipenv?