How was Keller able to pull off so much impactful work over so many organizations? i.e. what proportion of his impact was attributable to novel ideas, solo engineering implementation ability, engineering leadership, or just darn good timing?
I was at Intel when Jim Keller came. He was one of the most inspiring technical leaders I have ever encountered. He obviously had very strong engineering knowledge as well. When he came in, he took some time to really understanding our process and challenges. Then he created 5 areas that Intel needed to improve. He said “we are going to fix these, or burn down the place trying”. And these weren’t vague goals like “execute more effectively” or “be customer obsessed”, but really concrete and measurable things like “reduce empty silicon area by 50%”. He had a task force of principle engineers on each task, and every meeting he would spend almost the entire time going into detail of the progress and challenges on them. Anytime someone would complain and say the goal couldn’t be done, his response was “AMD did it, why can’t you?”. He was always trying to simplify things and cut through complexity. His vision was that if we hit those 5 targets, we would win the market. As an engineer, his clear vision and strong technical understanding made him the best leader I have ever been under.
it’s a pity he left Intel so quickly. After he was gone, his 5 goals basically fell by the wayside. I don’t think people like being told they’re doing it wrong. I still try to listen to any podcast or interview with him, because I think he has many deep insights.
Moar. Maybe you have some more stories or insights from him. I'm a shameless fanboy. I also listened to I think every podcast (not many of them unfortunately, Lex ones seem the best as expected). This guy oozes wisdom.
I will share one more story. Some of the leaders were saying they couldn't hit the validation milestone because of all the things that had to completed before they could sign-off. Jim Keller's response was:
"This is how were going to validate this chip:
1. go to the beach
2. open your laptop
3. run the chip in emulation through boot
4. tape-in! (send to the fab)
5. take the rest of the year off at the beach
"
His point was that we need a simple focus on proving the chip works and then shipping it. If any validation milestone or objective is not directly connected to that, it's not worth doing. We don't need all this extraneous validation and multiple layers of rechecking. Just emulate the thing and then your done.
In his interview with Jordan Petersen he talked about how some people spend all their time "admiring the problem"; Trying to understand and categorize it. Keller didn't want to spend any more time talking about the problem then needed. He was focused on finding good solutions (and it had to be a good solution), and then executing them.
You should see his interviews with Dr. Ian Cutress. The man basically knows everything in the industry in and out and knows how to create great teams. I want to interview for Tenstorrent at some point and would love to work for him. I feel being part of great teams is what he has been fortunate with. People very easily forget the rest of the team in chip design and make it about one person. I feel he's a true leader in chip design teams, but I also think he has worked with many great people to achieve that.
People like this are extremely rare and talented, AND they are able to build amazing teams.
Other super talented people want to work with a Jim Keller. So a company hires a Jim Keller not just because you want his brain, but because you can then go to the next 20 amazing people in the industry and say "come work on Jim Keller's team" – now you have 20 min Jim Kellers on your team and that is enough to move mountains in an extremely specialized field such as chip design.
The cult of personality is always an enticing idea, it's pretty much certain that Jim Keller had a stellar team to work with in all of those places.
That being said, even a strong team is still usually only as good as its leadership. You can have all the great ideas in the world, but if the organisation does not allow you to pursue them (or on the other extreme, allows everyone to work on what they think is best) you will still fail to execute.
At the end of the day you need someone to make the correct decisions every step of the way.
One thing I gathered from interviews is that Jim Keller is good at identifying good people already on the teams and inspiring and working with them to set the priorities and goals to make a better product. The team that built Zen isn't that much different than the team that built Bulldozer, but Zen is a much better product. He's also pretty good at spreading the credit around.
Having a good grasp of everything in the whole stack is obviously helpful for setting priorities and goals, and there's a lot of leadership skills involved in getting people to work together and follow the organizational goals. Having a track record of success is obviously helpful, too.
To address the article, this will be very interesting because in the market Tensortorrent is targetting the real challenge is software support for PyTorch, TensorFlow, distributed training, etc... and less about the hardware itself. It's not like AMD GPUs can't do matrix multiplications at competitive TFLOPS but instead that if I wanted to use them to run a model I'd have to work through several workarounds to get similar performance to an Nvidia GPU that is mostly plug-and-play.
We will see if Jim Keller's many talents include managing software.
And they're blurring the line by adding more AI-focused acceleration instructions to their GPUs (maybe focused on their CDNA line rather than the consumer RDNA, but they're expanding their capabilities on both).
There seems to be a bit of a philosophy difference between NVidia and AMD acceleration - NVidia add new specialised "cores", while AMD adds those similar capabilities as accelerated instructions to their shader units.
Though this might just be marketing - I don't think we've got an open reference for the NVidia shader ISA like we do with AMD so difficult to compare, but the capabilities and integration in shaders of many of these acceleration paths seems to imply they might also be implemented as shader instructions, which stretches the definition of a separate "core" to me, and so might just be a marketing difference rather than anything technical. Same with things like the BVH acceleration used in ray tracing - AMD added shader instructions, while NVidia talks loudly about "RT Cores".
Yes that's my point. AMD has the hardware, but using that hardware with the existing DL ecosystem is much more painful and they haven't been able to bridge that gap.
> Erm, Itanium never had any kind of market leadership. It was a failure.
That's true. But part of the reason Itanium as a failure is because AMD came out with x86-64, which was so successful that Intel ended up being forced to adopt it (it didn't help that the first generation of Itanium chips was... underwhelming).
I think the general consensus is that AMD could only swoop in as it did because Itanium was such a disaster of epic proportions, both technically and on a management level. There was no way Itanium was ever going to be successful. x86-64 was just the final nail in the coffin.
I found that Lex interview enlightening for two reasons. Firstly, Keller is a profoundly talented and interesting guy. And secondly, Lex is a truly painful and incompetent interviewer; virtually every time Lex spoke I begged my screen for him to stay quiet and let Keller speak.
His guests are often so interesting that it's worth persevering through Lex's un-profound musings. (Every time he uses the word 'beautiful' or 'love' it reminds me of a pretentious teenager who lacks life experience)
Haha, I kind of agree but I’m starting to think that dumb interviewers are a feature, not a bug. They ask obvious mid questions that are difficult to answer without giving away any tricks if you had them.
Asking obvious mid questions is one thing, but Lex often interjects with his own mid opinions. Or he takes what could have been an interesting line of questioning and diverts it to banal cod-philosophy often using words like 'beautiful' or 'love' in an attempt at profundity
I had the same feeling with a lot of interviews from the past, but somehow, about a year ago, something changed. Not sure if it was a conscious decision or just the result of so much practice, but for quite something I haven’t screamed at my screen when listening to Lex.
I've noticed that as well. He frequently also gets distracted thinking about taking ideas to such an abstract level they don't really make sense any longer, he looks off at the ceiling somewhere, and then the guest continues off as if they didn't see that. It makes me crack up every time, hah.
This is in contrast to many other podcasters who stay engrossed in surface level details and never graduate to thinking about overarching concepts. Lex just zooms there too fast sometimes.
I think Lex is a very talented interviewer. His questions often seem low quality (to me), but he pushes the discussion in the correct direction. He seem to be e.g. build sentences where after the first part there is a moment for guest to throw in something or for Lex to judge guest reaction. Or maybe I'm reading too much into it, but regardless, he seem to be able to get something out of these guys which others interviews couldn't. Shining example is Stephen Wolfram who seem to be extremely hard to interview.
Lex's done some of my favorite geek interviews. Just from memory: Stephen Kotkin, Roger Penrose, Lisa Feldman Barrett, Everyday Astronaut, Jim Keller, his own father. Many more I'm sure, if I consulted Lex's archive.
I skip over the episodes with culture warriors. I've already had my fill, thank you very much.
That's exactly what I thought. Jim Keller had a lot of deep insights that he was boiling down to simple terms. It was obvious he had thought about these ideas for a long time and that what he was saying came from a lot of experience. He would talk about things like switching lanes in driverless cars being a matter of ballistics.
Lex Friedman just interrupted him in a whiny voice to say "it's not thaaat simple". He basically just got in Jim Keller's way. It would be like someone interrupting a john carmack presentation every few minutes to say "I don't know about all that". It was a shame.
Fridman's interviews are an unusual combination of being well researched and incompetently conducted. When the interviewees are so interesting and well-chosen, and otherwise not the sort of person who would get a lot of media attention (Jim Keller is one example), it is indeed painful when he interrupts the guest or... sigh... interacts with them verbally in any way, really.
Unrelated, but Fridman's proclivity for injecting Elon Musk as a discussion topic into the vast majority of his interviews is genuinely weird.
He had guests more famous than Rogan before ever going on JRE.
Steven Pinker and Eric Schmidt were in the first ten guests and Elon Musk wasn’t long after. I’m sure Rogan got him more high profile cage fighters and that type of guest, though.
> Does anyone have any insight into how hes managed to get such high profile guests?
Once you have some, they might have introduced him to others like a chain or linked list (if that sounds better ref)
Lex shared this on his podcast as well: It's not easy to find or get hold of Donald Knuth. But, he kept trying and finally got him. Persistence pays, I guess.
Keller was great. He was actually great in all public appearances I saw him in. And I find it hard to find something I disagree with him. Usually with people I look up to, there are _some_ things I'm not on board with, but I don't seem to have that with Keller.
I understand that until recently he kind of stayed out of the public spotlight, and I wonder why, because he has such a great presence.
Some design decisions are also really not obvious. I've seen engineering decisions of the kind... We'll put another microcontroller because we would need less wiring between these 2 boards and the wiring would be more expensive.
A clever one I saw was where they milled a PCB thinner in a section to make it bendy and did not need 2 boards because of this.
Can confirm. Most people probably wouldn't believe the effort that is spent on improving the physical engine models in the ECUs in order to save a few bucks on sensors. Precisely, as you wrote, because those few bucks multiply while a software effort is one-time.
Yes, they were planning ahead. They knew they would sell more vehicles in the future, and thus the effort was worth it.
Not just for V1, but the following version as well. Once you have an internal team, you can vertically integrate and costume the solution to your needs exactly rather then buying a generic part from a supplier.
Things like OTA updates are just easier if you control the software and hardware stack yourself.
And once you have a team with expertise you can use them for other projects as well, like Dojo.
I remember in the mid '70s looking under the hood of my family's enormous station wagon and being greeted by a solid surface of hoses and valves, all dedicated to meeting the recently-boosted emissions standards. Open an internal-combustion hood now, and it's much, much simpler.
At this kind of scale, finding a "small" saving may pay for tens of engineer years and justify hiring people just for that. Same thing to optimize cloud costs.
https://wccftech.com/jim-keller-tenstorrent-wants-to-compete...