I'd argue that it's greater than that these days, many programmers really just can't do what a lot of good programmers can do. Even if they're given all the time in the world they'll just never get a result. So whats that make them infinityX programmers?
Exactly. At my last job, a colleague of mine was tasked with creating a tool to generate a monthly summary report based on 100s of millions of records. It took him weeks of trying to get something working in Python (naively read in all the data into Python objects and then trying to iterate over them), coming up with convoluted algorithms trying to speed things up. His final solution took multiple days to run (too slow for the requirements). An equivalent solution using PostgreSQL (using the built-in COPY to parse the CSV records) and couple hundred lines of SQL (a day's work) could do the job in less than 10 minutes.
At the beginning of my career I've done comparable things. I once spent multiple weeks implementing a solution to a problem that would have been trivial if I had known about unification [1]. Thousands of lines of complicated C++ code, countless bugs,... If you don't even recognise the problem, how can you expect to find a reasonable solution in a time that is only 10x as long as it would take someone who recognises the problem and knows how to solve it?
I don't understand this. I'm not saying I don't believe you, but it seems ridiculous to me. I actually feel vicariously annoyed just reading that story.
In my experience, most competent programmers encounter problems they can't solve off the cuff frequently. Then they think about it, research a bit, formulate the problem, etc. They come back to solving it after stepping back because they realize they're in the weeds and just treading water.
At a certain point, your colleague must have realized this task was simply beyond his "off the cuff" abilities and gone looking for a more efficient way. Surely he had the presence of mind to realize he was beginning to be unproductive? How did he spend weeks on a problem that can be summarized as "retrieve a lot of records from a database"?
Furthermore, what was your organization like that someone that grossly incompetent wasn't checked in on about progress? There was no "mercy rule"? No referee to call it when it was clear it was taking weeks instead of hours or days?
I guess I'm just confused because I don't consider myself a particularly spectacular programmer, but I'm self-aware enough to search for a solution and RTFM for the tools at hand. I just don't understand how weeks were spent here. I've never encountered a team where it seemed like one person was that far off from the median (to be fair the companies I work with likely wouldn't use them as a point of contact though).
> I don't understand this. I'm not saying I don't believe you, but it seems ridiculous to me. I actually feel vicariously annoyed just reading that story.
It was ridiculous, and I was annoyed at the time too. Especially because I had basically laid out how to solve it just before I left that company. Somehow that didn't get communicated to him. When I went back for one day 5 months later, I learned that he was basically still nowhere (he had been doing other things too). I spent an hour with him laying out the solution in detail. From what I last heard, he listened, and that is the solution they are currently using.
> At a certain point, your colleague must have realized this task was simply beyond his "off the cuff" abilities and gone looking for a more efficient way. Surely he had the presence of mind to realize he was beginning to be unproductive? How did he spend weeks on a problem that can be summarized as "retrieve a lot of records from a database"?
If you only have Python experience and no intuition about how much time/memory certain problems should take, you don't necessarily know when you're out of your depth. It was a Django shop. The only exposure most of the people in that company had to the RDBM was through the Django ORM. Multiple times I have seen them do things through the ORM which would be trivial and multiple orders of magnitude faster with a couple of lines of SQL.
That data also didn't come in a database, but in a series of CSV like files. If you don't have RDBM experience, you don't necessarily see how you would use a database for this problem. And Python conveniently has a csv module that will happily parse your files into objects... If you buy into the marketing that Python is 'fast enough', and 'you can always use Pypy if it needs to be faster', you might not realise when you get stuck.
> Furthermore, what was your organization like that someone that grossly incompetent wasn't checked in on about progress? There was no "mercy rule"? No referee to call it when it was clear it was taking weeks instead of hours or days?
Management didn't know how long that task was supposed to take. The technical competence in that company was very limited, and they were used to deadlines slipping all the time.
> I'm self-aware enough to search for a solution and RTFM for the tools at hand.
That puts you ahead of the pack, in my experience. I had almost no experience with RDBMs before I joined that company, but reading the first 100 pages of the PostgreSQL manual the first week put me ahead of people who had been using PostgreSQL as their principal tool for more than a year (a different team, the metrics team which used PostgreSQL directly, not through the Django ORM). They had never heard of simple things like VIEWs and WITH queries.
> I've never encountered a team where it seemed like one person was that far off from the median
I don't think he was that far off from the median in that company, maybe just a bit below. To be fair, it was technically the worst company I've ever worked at. I was only with them for 6 months (and I never would have started there except for personal circumstances).
Consider yourself lucky never having worked at such a company. They are more common than you think.
yes exactly, funny when I was writing that I had the same thought about what I was like when I first started - I had a flashback to spending days writing out some C code for a linked list. I thought it was cool at the time, comparing that me to this me, then it would easily be a 10x difference. qed.
