> Then there was AWS re:Inforce – the annual security conference – which was themed “Security in the era of generative AI”.
This tagline is representative of every part of the hype around GenAI. It makes it sound like security has fundamentally changed and we all need to re-learn what we know. Everything to do with GenAI is treated like this: we need new security plans, we need AI Engineers as a new job title, we need to completely reevaluate our corporate strategies.
Security in the world of generative AI is not substantially different than infosec has been for a while now: User prompts are untrusted input. Model outputs are untrusted input. Treat untrusted input appropriately, and you'll be fine.
The same goes for "AI engineers", who are in the business of wiring up APIs to each other like any other backend engineer. We take data from one black box and transfer it to another black box. Sometimes a black box takes a very long time to respond. It's what we've always done with many different kinds of black boxes, and the engineering challenges are mostly solved problems. The only thing that's really new is that the API of these new black boxes is a prompt instead of a deterministic interface.
Don't get me wrong, there will be things that will be different in the post-LLM world. But my goodness do the current crop of companies overestimate how large that difference will be.
As a person replying to your comment in the era of generative AI, I'm inclined to agree the hype is a bit much, even considering how impressive the technology can (sometimes) be.
Another big area of hype is "prompt engineering." That one seems to have calmed down slightly, but for a while, there were large swaths of the Internet who were amazed that the set intersection of "talk like a decent human being" and "be precise in your communication" could generally lead to good results.
In many ways, "AI" right now is magic marketing sprinkles that you can put on anything to make it more delicious. (Or, if you're inside a big company, it's magic prioritization sprinkles.)
Maybe the prompt engineering should have caught on more. I'm convinced that the large swaths of people commenting here and elsewhere "I don't get AI, it's just a parrot and it's always wrong and hallucinates, it's not useful" just don't understand that the prompt matters and the idea isn't to one shot everything. It writes good code for me every day, so I can only assume they're asking "Write me an OS from scratch" and then throwing their hands up when it obviously fails.
I think that calling it "prompt engineering" is what made it fail to catch on. We didn't call it "Google engineering" back in the day when you could actually craft a Google search to turn up useful results, we called it "Google-fu" [0].
"Google-fu" sounds like a fun skill to learn and acquire, where "prompt engineering" sounds either like something well out of reach or like pretentious nonsense depending on the audience.
>"Google-fu" sounds like a fun skill to learn and acquire, where "prompt engineering" sounds either like something well out of reach or like pretentious nonsense depending on the audience.
More likely, "prompt engineering" is a marketing term made up by AI marketroids (cf. androids) hoping to make developers feel better about their reduced roles in this "grand new AI age".
> I think that calling it "prompt engineering" is what made it fail to catch on.
I don't think so.
I mean, clearly calling it "engineering" threw some people off, in the same blend as some gatekeepers cringe at calling train drivers "railroad engineers". But that's puerile gatekeeping that misses the whole reason why there is a vast need to know how to "engineer prompts".
The truth of the matter is that the focus of "prompt engineering" is being able to put together inputs that solve business problems in professional settings. You need to have full control over the generative process to integrate it's output in a business setting. That requires specialized knowledge way beyond naive requests expressed in natural language.
Complaining about "prompt engineering" because that only focuses on specifying queries and operating a specific service makes as much sense as complaining about SQL/database/postgres engineering because that only focuses on specifying queries and operating a specific service.
Before trying to dismiss "prompt engineering" through gatekeeping logic, first you need to justify why there is no need to know what you're doing to get outputs by feeding the right inputs. Even in subreddits dedicated to using generative AI to create images and videos,they started to outright ban posts where the contents are posted without the prompts used to create it.
To me it’s more like, if I have to carefully craft English language prompts in a conversational back-and-forth to get things done, then I am not really interested in doing that job, which sounds like being a manager or a teacher, and in practice just makes me feel totally dead, sad, and quite frankly bored.
That’s just not an interesting or rewarding way to interact with a computer, and the last thing I want to do is add long wait times and nickel-and-dime cost to the process. Layer on using different LLMs for different tasks or trying them out against each other and cross-checking output and it’s a mind-numbingly indirect way to get anything accomplished that in the end teaches me nothing and develops no useful skill that I enjoy practicing.
If it works for you, great, but even the most honest and genuine fans make it sound like a nightmare to me.
Could not agree more. If the utility of this thing is based entirely on the right sequence of magic words why aren't we calling it "prompt wizardry" or something that better encapsulates the nature of it.
If the utility of all computers is based on the right sequence of magic words why do we call them software engineers instead of something better like "code wizards" that encapsulates the nature of it?
I guess the difference is code is (almost always) totally deterministic. Or at the very least, they're designed so that is a mostly safe assumption.
It doesn't seem likely an LLM will ever do that. Maybe at a certain point of sophistication? But if the model is regularly changing - which they almost all will be, if they're expected to be up-to-date - there is a strong change they'll be different every time they're used.
(I've been getting different behaviour in even relatively narrow ML-based systems for years. Google Assistant is my prime example - I regularly use the phrase "add to my calendar on the 20th of September at 5pm, go to the park". Almost all the time, it works perfectly. But a couple times a year at least, it won't process this as an action - it just does a Google web search for this string.)
Code can be deterministic if you're only doing trivial things or have very simple systems. Beyond that it only has sufficient determinism to cross a threshold into being considered useful. Dead letter queues, uncaught errors, kernel panics, race conditions, deadlocks, cosmic bit flips, dumb avoidable bad choices from unexpressive languages, malicious actors, resource constraints, and so on; the real world of software we live in is a duct taped together mess of half measures that only mostly do what we want at its best. So much of the work in product programming is handling all the things that can go wrong.
So yeah, prompt "engineering" is indeed a silly term, but software "engineering" kicked off the dilution of that word ages ago. And GPT models can be inspected and measured for input and output, prompts can be analyzed for their effects and usefulness, temperature settings even directly control some degree of determinism. It's not like models change on a whim unless you're just using end user products. Anthropic, Huggingface, AWS, OpenAI, they let you pick a release model version in your API calls and stick with it for a long time. If you're self hosting a fine tuned Llama 70b, nobody will ever force you to update it if you get it doing a task to your expectations. The quality of deterministic behavior in AI is currently lower than that of Excel or C code, but it's also serving a wholly different purpose, people want it to be creative and create novel nondeterministic outputs, comparing them is a bit silly.
To be fair to what OP is saying, it's not so much that you have to carefully craft the prompts every single time, it's that there is a linguistic register you have to adopt in order to get results out of an LLM. The initial learning process for that register can be hard, but once you've learned it it comes naturally.
I think of it as similar to Googling in the early days. What started as a skill I had to pick up became second nature and I could find things faster than my family without even really thinking about what I was doing. It just became natural.
to be fair, expecting most software engineers, who typically have a bachelors degree, to be able to communicate well in english is not asking for a lot. Via a textual medium no less! But apparently it is…
Most of my colleagues communicate with chatgpt in broken english, or they ask a question while leaving out crucial details about their problem. They’re always surprised when i am able to get a useful response from chatgpt when they couldn’t. it’s comical sometimes.
I 100% hear you on the “not a fun way to interact” though. To each their own. I personally enjoy it, it’s like a rubber duck that can actually talk back. :) not for everyone though.
It's not so much that communicating in English is the problem, at least for me. I'm a native English speaker and have a reasonably strong command of the English language. I'm able to craft my words to convey specific tone or meaning.
The problem is that GenAI is a complete black box with nondeterministic outputs. I can write code and I know with a very high degree of confidence what I expect it to do. Asking an LLM or a generative image program for something, I have no idea what it'll give me. It gives no feedback other than results, which may or may not be what I want. If not, I have to reverse engineer what I think it might want me to say in order to get desired results. And the same query placed another time might give a completely different answer. I don't deny that it can do some impressive things given the correct inputs, but I am not inclined to spend my time searching for the magic words.
> To me it’s more like, if I have to carefully craft English language prompts in a conversational back-and-forth to get things done, then I am not really interested in doing that job, which sounds like being a manager or a teacher, and in practice just makes me feel totally dead, sad, and quite frankly bored.
You're showing a fundamental misunderstanding (or ignorance) of the whole problem domain.
For starters, you place an awful lot of emphasis on what you think is "carefully craft English language prompts". That makes as much sense as characterizing the job of a database engineer as "carefully crafting quasi-English language prompts". The language used is completely irrelevant, and being able to use in some circumstances something resembling natural language to build up context does not take away from it.
Any remotely honest and objective analysis of the topic would start from similar activities, and to start off the areas of work where Llama are being used. For image/video generation you need to look at graphics design, video editing, video production, illustrators, etc. These activities, by their own nature, are iterative and exploratory. Then for text you have the work of copywriters and editors, and even writers and essayisgs. The work is fundamentally iterative and exploratory. Then you have work like exploratory data analysis/statistics/data mining. Every aspect of that work is iterative, even the reporting part.
to me it sounds like a job that would be similar to a search engine optimization engineer - studying the output of a third-party program when providing that program with different sets of keywords.
If it only solves the problems I already find trivial then it is a parrot. Nowadays we all have a calculator with us but if you emphasize that fact and choose to not practice and excel at basic arithmetic then you will be unable to perform higher mathematics that require it at every step. Of course if your problem is always already a solved problem then sure a parrot can be convinced to spit it out.
So yes, the actual question for software engineering would be how to get AI to produce and iterate on an OS. The hallucinations aren't the only problem then, the lack of predictability in the answers is the biggest issue.
Been quietly wondering something similar to you for a year: I've ended up 95% confident that phenomena is due to people evaluating it in terms of "does it replace me?"
Cosign prompt engineering. My startup is tl;dr "what if i made a on-every-platform app that can sync and let you choose whatever ai provider, and you pay at cost. and then give you a simple UI for piecing together steps like dialogue / ai chat / search / retrieve / use files"
Seems to me the bigs are completely off the mark, lets cede the idea there's an omniscient AI available. Literally right now.
Cool.
It still has no idea how you work.
you could see 42, in hitchhiker's guide the galaxy, as a deep parody of this category error
I appreciate this perspective on prompt engineering. I’d love to think that one of the great outcomes of LLMs are people returning to more decent and precise forms of communicating. Imagine the progress if we could get that to transfer to human-human communication as well.
> Another big area of hype is "prompt engineering." That one seems to have calmed down slightly, but for a while, there were large swaths of the Internet who were amazed that the set intersection of "talk like a decent human being" and "be precise in your communication" could generally lead to good results.
I think your comment conveys your obliviousness of the problem domain.
The main driving need for prompt engineering is not an inability to "talk like a decent human being". That's just your personal need to insult and demean people who are interested in a problem domain you know nothing about.
The main driving need for prompt engineering is aspects like not being able to control how context is formed and persisted in a particular model, and how to form the necessary and sufficient context to get a model to output anything interesting. Some applications require computationally expensive and time-consuming runs, and knowing what inputs to provide to a system which by it's very mature is open-ended is a critical skill to adequately use the system in professional settings.
Let's put it like this: GitHub copilot is a LLM service which is extremely narrow in what are their applications and use cases. Yet, you can't even get it to add unit tests to a function following a specific style without putting the effort to build up the context it needs to output what you expect.
I think the problem is they're trying to introduce nuance and a narrow path to allow this. They want an acceptable level of risk to using untrusted model output for the efficiency/productivity gains it will bring, notwithstanding hallucinations.
Generative AI would not have flown in the security theater of Yesteryear, but CTOs see productivity multipliers.
Right, but that's not a new problem either. We want to allow people to send emails with some acceptably-low level of risk that spam will get through. We want an acceptably-low risk that our image upload feature won't be hosting CSAM. And we want it while still getting the benefits of allowing our real customers to pay us for the services we offer. Businesses have been figuring out the balance of risk:reward for as long as infosec has been a concept.
While that was indeed billed as the theme of reInforce, there were plenty of sessions and workshops that did not involve GenAI at all. There was a great chalk talk about the underpinnings of how the AWS IAM service works across services and regions, for example.
IAM is eventually consistent. And they do a lot of derivations of hashing off an original signature and distributing individual, bespoke versions to services in different regions to limit the blast radius of a compromised credential.
If you go to an AWS event in the future, the name of the chalk talk was "The Life of an IAM Policy"
My favorite was when everyone was looking for prompt engineers.
I was trying to understand what prompt engineering was, because I thought there is no way this is a discipline for how to ask ChatGPT questions... And then I realized it was...
Sure, I get that there is much to learn regarding formulating effective prompts, but a new career path?
hype is the deeply engrained norm in our industry, bro. just sit it out. as that famous saying says, "this too shall pass.".
until the next one, of course. ;)
for example, in rough order, some past hype trends: 3GLs, structured programming, initial AI (then AI winter), expert systems, CASE tools, 4GLs, OOP/OOAD, UML and round trip engineering, design patterns, dot com boom (and bust), agile, functional programming, Web 2.0, SaaS, crypto, Web 3.0, big data, data science, ML/AI.
most of them had or have some actual benefits, but nothing like the hype parrotted, by those with and without vested interests.
been there, seen them, from the third or fourth one onwards.
Security for a long time has been log parsing and auditing for compliance (pdf reports) and some tooled posture report !! It took a while security team to do this tedious task now an ai with 1/10 price can do this !! And can make big companies rich !! Do it once and give it to all !!
AWS is way too bandwagonny these days. Back when it was all engineers they built things on the basis of "this is cool technology". These days marketing runs large parts of AWS and plans are decided more on the basis of "this will look cool on a PowerPoint slide".
I keep going back to the basics: Serverless is servers. Machine learning is servers. GenAI is servers. And, from what I've heard, most of AWS revenue is servers and storage.
(For the record: I am also an AWS Hero, and an AWS customer since 2006.)
> AWS is way too bandwagonny these days. Back when it was all engineers they built things on the basis of "this is cool technology".
I don't agree. I think AWS has always been extremely customer-focused, and they scramble to offer whatever service might have any traction at all from customers. It's just that they are already providing the low-level baseline services, and now they are progressing to offer increasingly higher-level ones.
I'm talking about machine learning-driven firewalls, backend for mobile applications, video streaming, edge computing, even Blockchain and now LLM services.
As much as it might surprise you, there is plenty of real-world demand for these services. You might accuse it of being "bandwagonny", but if you take an objective look at it you'll find that they are playing the role of supply store owners during the gold rush. It comes at no surprise that AWS is the one part of Amazon whose revenue is growing massively year-on-year, with the last report pointing to a 17% growth year over year.
The way that cloud businesses work, you sell the servers for about as cheap as you possibly can do. Instance prices are all a race to the bottom among the providers, because servers are largely commodity hardware that's easy to get from any number of providers, and it's one of the first prices customers see, and often plays a big role in their choice of provider.
So that's not where you make your profits. So you're right, lots of revenue, but crucially, there is no real profit. There never will be. That makes it a boring product, not worth focusing a lot on from a marketing perspective etc.
Same tends to go for all of what you might think of as the basic building blocks of the cloud. e.g. object storage prices are often really close to what it actually costs to provide the service.
You make your profits on what you sell that runs on the cloud. All those additional things like databases, streaming services, kubernetes bits, functions etc. Those are where you make your actual profits. GenAI is a big potential profit driver for AWS, so that's where they're pushing. A couple of years ago it was "$foo, but on Kubernetes". Before that it was "$foo, but Serverless".
They're just pushing where the profit and interest is, and pretty much always have done.
sort of side-note: Gartner's evaluation of cloud providers got really absurd around kubernetes stuff. Because one cloud would do it, you'd miss out on points if you didn't also add it, even if being on kubernetes literally added zero benefit, or arguably was worse. Same for "Serverless". It didn't matter if customers were actually using it, or wanted it, if AWS/Azure/GCP launched it, you'd better have it too.
> A couple of years ago it was "$foo, but on Kubernetes". Before that it was "$foo, but Serverless". They're just pushing where the profit and interest is, and pretty much always have done.
I don't think that's it. Those who migrated their EC2 apps to ECS/EKS/Fargate/App Runner have already migrated, so there is diminished returns in pushing those technologies.
The same goes for serverless. The whole world already adopted this to it's full extent. Those who want/can use these services, are already running these services, and AWS is already getting better utilization rates from their idle computational resources from this.
These are not fads. They are already infrastructure.
What we are seeing is additional high-level services being released to meet customer demand. There's now a massive need for training and running your custom private LLMs. There is absolutely no justification to skip the revenue you can generate by serving these markets.
You're commenting on services provided by a cloud provider. I think it's obvious this is about which services are being used by clients of cloud providers, and not how your grandma stopped using a kettle to instead make tea by invoking a lambda through API Gateway.
Having worked for storage services for the cloud, it's not a massive premium. Things are kept pretty close to costs. We have to, it's a similar race to the bottom case. Each time one of the clouds drop storage costs, the others rapidly follow even to the point of it losing them money while they figure out how they might return to profitability.
The biggest cost for cloud storage services is not disks. Never has been. They're pretty cheap all things told. It's the per rack operational costs that dwarf them. The cost of electricity, cooling etc. You're also paying for the durability and accessibility that is built in to the software. It's why you see, e.g. backblaze obsessing about their server specifications and how many disks they can cram in to the server. Everyone is trying to maximise the rack data density, and trying to tow a really fine line on having just enough compute power in the individual server.
Durability and accessibility is solved problem using erasure coding with automatic recovery. Nothing interesting in this space going on for like 10 years unless you’re pushing planetscale storage.
> It's the per rack operational costs that dwarf them. The cost of electricity, cooling etc
Ok lets do some simple math - lets say you have a few racks with 50PB in there, colocated. Power + floor space + remote hands will not cost you more to than 250k/y and that’s being very generous. Dividing by 50PB thats .5c per gig PER YEAR. Can you explain why GCS/S3 charges 50x that not including egress which is also ridiculous?
Have you seen their private rate cards for this level of buying? I bet if you had your calculus would change here.
Also, it’s easy to beat most costs if you get consumer grade hardware and don’t refresh it more than once a decade. I see that often when folks compare costs, but that’s dishonest and just kind of shoots their credibility in the foot.
Note that we haven’t even started discussing hw costs - tp correctly noted that amortized capex is less than the opex. It doesn’t change the math that much. I had pulled the relevant quotes to make a google sheets model and even with top line hardware i couldn’t stretch the break-even point beyond 2 years.
Private rates are cool except they come with strings attached and even discounted 90% its still not even close to break even on decent sized commits. Purely on costs the cloud cant win whichever way you slice it.
How is Lamda, servers? Obviously it is underneath but because there's all sorts of weird tuning shit to get them to work right, it's is own separate thing to deal with and so deserves some sort of way to differentiate it. Are we just unable to handle new definitions for new, different things?
Having worked on both (servers and serverless), I personally find the serverless programming model to be fundamentally different. I don’t have time to write, operate or troubleshoot servers. Serverless is a huge productivity boost. I can set and forget some glue piece of functionality and it will operate without the issue. Then I can focus coding up the interesting and differentiated bits.
That’s where I (respectfully) disagree. It’s true that there are new things to thing about, and there are use cases where it’s not appropriate. That being said, overall it’s a net productivity boost and YMMV but for me I’d say it’s 4x or 5x.
I have a python script that fetches a website every few minutes and makes an API call when some conditions are met. I never really figured out all those other AWS or cloud tools and usually just resort to renting a vortual server and running the script via ssh.
That seems the way to go unless you have insane amounts of data. We had some could guys write a pipeline in AWS for us with several steps connected by lambdas that get triggered at each step. The damn thing is just not reliable. It skips some changes or sometimes doesn't work at all. I am sure it can be made reliable but I am beginning to be very skeptical of all this complexity. I think we could write the whole pipeline in 300 lines of Python with the same results but somehow this is not "cool" because it doesn't scale. Never mind that the complex stuff doesn't scale either
That sounds like it works great for your use case. The serverless stuff is best for when you want to do things like customer order processing workflows or distributed packet mirroring orchestration, etc etc
I didn't get it. All of the new features they're adding are for AI, but the old features still exist, right?
If nobody wants you to use them, is that because everyone already has as much conventional architecture as they need? Perhaps the new opportunities are all in AI because we've pushed conventional stuff as far as it could go, and we were just rearranging deck chairs.
I'll be honest that, if we've run out of ideas, I dunno if AI really solves any problems I want solved. But even if not I don't see how appealing to AWS fixes anything.
The problem is that these type of very much needed but boring new features don‘t grant you a promotion. But if you work on a GenAI product right now the odds are in your favour.
> The same goes for feature releases. If the vast bulk of all new feature releases are geared towards GenAI, it implicitly means AWS is rerouting investments from classic infrastructure to shiny GenAI. It means that the products I love get smaller budgets. It means that the products I use will not get the next feature I want, or only at a slower pace.
I think the article does hyperbolize a bit, but this seems like a hard truth. Unless AWS has hired an entirely new swathe of AI-focused engineering talent, or if their public face at events is significantly disconnected from where they're spending their real money.
AWS doesn't even have proper IPv6 support yet. If they can't find anything to improve in their conventional stack, it's only because they aren't looking.
I think the main issue here is brand and growth. AWS needs to convince CIOs/CEOs to use them over Azure/GCP, not engineers. And even if AWS cared about convincing engineers, we already prefer AWS so introducing new services for container orchestration wouldn’t move the needle. What does move the needle is being perceived by enterprise leadership teams as just as cutting edge as the competitors. “Generative AI” is the only signal those teams understand these days.
It's strange because as a senior engineer I'm telling everyone that genAI is not ready, has glaring quality, safety, and security issues, is underpriced by VCs planning to crank prices later, and even if the magic was real offers less promise than tried and true conventional solutions we haven't tried yet.
AWS should be spending significant time explaining how their giant portfolio of conventional tools are improving. If they have stopped, they've lost focus. But hopefully it's just the marketing team focused on magic beans.
> I'm telling everyone that genAI is not ready, has glaring quality, safety, and security issues
The problem being that nobody listens of course. You still have to build heaps of genAI crud because leadership is so excited about it.
Then you end up building a free text box to automatically determine one of 4 different problems the customer can have, and scratch your head wondering why we don’t just provide 4 options instead of letting the customer write a whole story...
>AWS should be spending significant time explaining how their giant portfolio of conventional tools are improving.
See but that won't work because leaders are dumb. The central tension of civilization is that the smart ones are needed down in the boiler/operating room to solve complex low-level problems, and whoever's left must steer the ship.
It’d be nice to think of it like that, but I think the leaders just have completely different incentives.
It doesn’t cost them anything to re-route resources from the keep the lights on work, and has a potentially massive career impact if they’re the first ones to do something sensible with genAI.
Even if most of the work fails it’s still worth it. It’s to their benefit to hype AI beyond all reason because it justifies the resources they spend on it.
Good reflections. It kind of remind me of the big data era where everything needed to revolve around big data.
However, what happened is that it became apparent that not everything needs to be big data. Business needs will shine through as they always have and dictate what is truly important.
I'm not afraid of the wave of gen AI. Think of it as the new power tool that just came out that everyone's currently talking about. You'll add it to your toolbox because you don't want to be obsolete. It'll blend into everything else once the hype wave is over.
I don't know, if you get existential worries about your job because one supplier jumps on a hype bandwagon at a few conferences, then aren't you a bit too married to that supplier? I don't mean from a company perspective, I mean personally. I don't understand how AWS conference topics prevent anyone from being a cloud engineer.
when a company focus and strategy shifts to a single domain/product, everything else gets impacted.
Enhancements expected by the community will be delayed, engineers need to come up with temporary solutions which needs to be rewritten next year, new ways of solving problems will not be shared, new content will be created only for a couple of products.
This is just in general true of generative ai. In many ways it commoditizes skilled labor. They do not care about people in the posters situation. It is meant to lower the bar and make the labor cheaper.
AWS is desperate to climb up the value stack. Compute and networking is a commodity (with fat margins at retail prices to be sure), and the second and third place providers are willing to make deep discounts to land big deals. That's not going to justify those future lofty valuations.
The problem is, for all it's talk over the last few years, AWS remains a complete non-player in the GenAI space, much less so than Azure. In my opinion the problem is exactly the same as for every other high-level service they've tried to launch. QuickSight, Lex, Polly, Cognito, CodeGuru, SageMaker, etc: they're not good. Nobody ever said "I really like QuickSight, I sure wish it had GenAI capabilities". So when the hastily-expanded QuickSight team(s) then goes on to release 42 different Q enabled SKUs, nobody cares. For various reasons, AWS is organizationally incapable of launching a non-infrastructure product that is simply great, as doing so would take attention to detail and deeply caring about things like UX which are anathema to Amazon.
On the positive side, GenAI model access will be commoditized and part of the basic undifferentiated cloud infra, and AWS will do fine there.
I finally went to an AWS event this year: the AWS Summit in DC.
It was an awesome (and awesomely overwhelming) experience, but I completely agree with the author. GenAI EVERYWHERE.
The other topics that the author brought up from re:Invent 2022 were still present, but not without heavy mentions of how AI contributes to them.
That said, I have some predictions that might make OP happier.
DevOps and Platform Engineering is still a hot topic, especially in a world where companies are repatriating back to the data center (or are at least going hybrid). All of the 2010s bare metal tech (Foreman, Ansible, etc) are going to come back in big ways, and Kubernetes consumption will only increase. eBPF and systems engineering is still hot and will really help here for high-performance observability.
Companies that won't repatriate or want to use the cloud for prototyping will want to focus on cost optimization. This requires serious cloud engineering skills (using spot instances and S3 lifecycle policies is table stakes; much more can be done, especially on reporting and automation).
GenAI will help here (super helpful for analyzing time series data and surfacing patterns), but having the fundamentals will always be useful here.
Hear hear. I'm a founder of a GenAI startup, left Google to do it. And yet, cannot believe how much debasement has occurred, and trust has been lost, by every. single. product. and. company. thinking it has to have an AI story, and it has to be the story.
c.f. Google IO keynote this year. I couldn't tell you a single thing Google is launching this year, beyond limited, rushed features where Gemini chat is in a side pane.
And that's not on me: it's because Google literally didn't talk about a single other thing.
And as usual, Google is out of touch and doesn't get the joke, c.f. at the end, Sundar presenting their own count of how many times they said AI.
I sorely miss tech industry of the 00s, I simply cannot imagine ex. 2000 Apple/Steve Jobs falling for this. There's this weird vapid MBA brain drain in charge everywhere. But hey, stonk goes up.
> And that's not on me: it's because Google literally didn't talk about a single other thing.
Most tech companies don't have a whole lot to show right now, so AI sucks up all the oxygen in the room. This becomes a feedback loop with the stock market, too.
> There's this weird vapid MBA brain drain in charge everywhere.
Yep. And Apple's playing along as well. Their latest WWDC presentation has the most weird tone I've ever seen in their presentations ever: "we added some AI features and they're pretty cool I guess...also it's super private! Here's all the ways it's kept private."
So much hedging going on. So little excitement. Because they're just playing to someone else's tune, and they're not good at doing that.
2004 Apple with AI: we need to get 100 people locked down and working on Finder 2.0 with this, and keep execs focused on reviewing it wholeheartedly. this year, let's lol at vista security via confirmation dialogs, vs our best in class OS.
2024 Apple with AI: this is our best year ever. Look at our CTOs hair. We fixed Siri being eons behind, now if it thinks ChatGPT can help you can tap a confirmation dialogue.
> I simply cannot imagine ex. 2000 Apple/Steve Jobs falling for this.
I wouldnt be so quick to assume that. Let’s not forget that Jobs bought Siri and then integrated it into every platform they sold.
AI has been a buzzword for literally decades. It’s just exploding in popularity right now because the capabilities of GenAI have recently exploded.
It’s a little like how VR and AR has been around for decades. It’s just taken this long for the technology to make mixed reality a possibility for the masses.
> And as usual, Google is out of touch and doesn't get the joke, c.f. at the end, Sundar presenting their own count of how many times they said AI.
Haha be kind to your ex-employer. This was supposed to be a joke. I agree they don't get it but they at least tried this time. I ranked this high on the Sundar-joke ladder (which is a low bar I know).
> I simply cannot imagine ex. 2000 Apple/Steve Jobs falling for this.
No he would not entertain any of this nonsense. IIRC he had a hard time with the Siri demo too. It appears Apple is happy to take itself and shareholders on a side quest with all their cash…good time to be earning interest on it too.
it's not only cloud services. The AI hype has hit self-hosted services and "normal" / "offline-first" applications as well.
For example, the team / leadership / foundation behind Home Assistant has been pushing AI features hard in the past 18 months or so. This coincides with my feeling that there hasn't been any relevant improvement in Home Assistant's core features and usability — it's in stagnation for over a year now.
This is of course my own opinion, but it makes sense: if a significant share of resources is spent on AI stuff, that share is not available anymore for other needs.
This is why we need small company and community owned and run software, preferably open source. Zulip's post about their policies regarding AI training rubbed some the wrong way due to its perceived anti-Slack positioning, but it was at least sensible and measured. Venture backed businesses are all vulnerable to investor desire to chase fads in the name of speculative returns.
> Your first leadership principle is customer obsession: “Leaders start with the customer and work backwards”.
> I’m your customer, and I’m begging you: please let me be a cloud engineer again.
However, as with many enterprise products, the author is not the customer; it’s the user.
The customers are the companies that buy AWS because it’s an essential technology for their strategy. When the whole tech world is talking about generative AI, they want to be there, and Azure seems to be ahead because of the MS deal with OpenAI. (even if they are not ahead, customers' perception matters most).
So basically, what Amazon is trying to do by making all of these conferences and announcements about GenAI is to send a message to their customers: we are ahead on the wave of GenAI and you can still trust that our products are going to help you be on the hype.
AWS mainly earns from new modern managed services because ppl do not want to manage anything and AWS does it for them. All those services are too expensive and AWS markets those and keep talking about those as conventional managed services have less margin.
> Your first leadership principle is customer obsession: “Leaders start with the customer and work backwards”.
> I’m your customer, and I’m begging you: please let me be a cloud engineer again.
Only AWS knows how many H100 GPUs they have, how busy they are. How many people are paying for them, how many people want them and can't get them, and how many people just don't care at all.
It's possible that the focus on GenAI for Re:Invent 2023 wasn't based on any hard data like that, and is really just up to the whims of Adam Selipsky since Jassy moved over, but maybe someone who better knows their planning process can comment.
I’d wager AWS makes more money from GenAI than any other domain. So it makes financial sense for them to sell that part of the business hard at the moment.
This opinion is based on admittedly anecdotal experience, but I’ve worked in a large range of domains on AWS over the years and by far the biggest AWS bills were for startups specialising in GenAI.
Interesting. I figured all their AI efforts were motivated by FOMO rather than actual returns. Why is AI stuff making so much money? Wouldn’t a new area like AI be a loss leader as they try to get market share?
The only way cloud providers are making money on "AI" is via obscene mark-ups on access to GPUs used by companies who are deluded into thinking that (re-)training their own LLM is what their own shareholders want to see.
The problem is that legal world are still undecided about the safety of public models.
Plus often businesses need GenAI to be trained on their own IP (ie stuff sensitive to their own business that they don’t want in the public domain).
—-
Point 1 will be decided over the next few years as creators take companies to court (or “ethical AI” starts to displace the current models trained on unlicensed content)
Point 2 cannot be resolved without training your own models.
——
Let’s also not forget that LLMs are just one part of the GenAI movement. There’s audio and image generation too (Plus video, but that’s more an extension of image). In fact it was the latter that I worked on.
And then you have other areas of AI outside of the generative space too. From hundreds of different applications of image recognition to sound processing to searching for other kinds of bespoke patterns. These are all areas I’ve worked in too.
Often a GenAI product will require multiple different “AIs” to function, as part of a larger pipeline that appears like a single opaque box to the customer. And most of those models in the pipeline likely aren’t generative, let alone LLMs.
You make it out as if all the executives at AWS have some master plan surrounding the probably absurd number of GPUs they bought, but the likely answer is its just a bunch of fallible people bandwagoning on the latest trend.
Recursive doom loop, we need to mirror everyone else's models at the same price and add Enterprise™ verbiage*, lest Wall Street say we don't have a story.
Ironically, trying to compete like that, and then focusing on that, causes the problem they're trying to avoid IMHO. They're always going to be third as long as MS gets to serve OpenAI and Google has in-house AI talent building models that are top-tier competitive. And if you set this one farking thing aside, they're #1.
* ex. first time I've seen "FM" as an acronym is on the page you link. they mean "foundation model", which itself is a term I'm likely to see in the Economist, but never on HN. Colloquially, it means "big AI like ChatGPT"*
Not sure about Bedrock, but I used Q to write some Python code, and I have to say - it was pretty good. Even ended up covering an edge case I didn't think of. In the end it was all pretty basic (walking a directory tree and compiling some analytics on content) - but it saved me time and I never felt frustrated with it. My anecdotal $0.02
It is one of those typical moments for all big corp when fight for customer is less important than fight internally for employee survival. In this case, AI is the tag to keep oneself "relevant" and in the job.
AWS really feels they were left behind so they are overcorrecting a bit.
In private they are truly thirsty for AI applications they can write uses cases on that they even offer upwards of 100K credits for Gen AI purposes only.
TBH AWS re:Invent is frequently like that. In 2017 or so there was an absurd amount of “Alexa integration” events that were completely useless. I get the sentiment but, you know, it’s really just a giant marketing and pre-sales exercise.
I feel I am going to be downvoted for this, but...
I think the technical specialty that will be most at threat from automation by AI would be the exact job that he authored has -- solutions engineers that build commodity cloud infra on AWS, Azure, G cloud, etc.
Look at progressions and range of abstraction between standard sys admin IT work to serverless deployments, especially with IaaC tools.
You can describe your architecture to chatGPT and it can spit out a CloudFormation YAML. It will be rudimentary and poor, but I could see a Gen AI tool offered by cloud providers where al you do is describe your app and then deployed infra on your behalf, and optimize form there.
Not trying to talk down on folks who do this type of work, but sharing my opinion on where I think the author is ultimately coming from.
> the technical specialty that will be most at threat from automation by AI
Can you point to any actual AI product in this space that functions? Everything I've seen is like, if you squint then it kinda looks like it's doing something, but it's actually producing something embarrassingly wrong, unsafe, or otherwise unusable. And no, having a SME repeatedly prompt until it does the right thing doesn't really make sense.
If we're just talking about hypothetical tools that someone could make, but haven't, we're talking about magic.
GenAI can definitely churn that code out, but most of the code that we write is extremely situational. We rely heavily on specific customer needs and very specific use cases within the products we're trying to sell.
Current SOTA genAI tech is more than capable of generating a comment like that - no insight, just a “+1” comment dressed up as an emperor-has-no-clothes hot take for an audience that already agrees with it.
This tagline is representative of every part of the hype around GenAI. It makes it sound like security has fundamentally changed and we all need to re-learn what we know. Everything to do with GenAI is treated like this: we need new security plans, we need AI Engineers as a new job title, we need to completely reevaluate our corporate strategies.
Security in the world of generative AI is not substantially different than infosec has been for a while now: User prompts are untrusted input. Model outputs are untrusted input. Treat untrusted input appropriately, and you'll be fine.
The same goes for "AI engineers", who are in the business of wiring up APIs to each other like any other backend engineer. We take data from one black box and transfer it to another black box. Sometimes a black box takes a very long time to respond. It's what we've always done with many different kinds of black boxes, and the engineering challenges are mostly solved problems. The only thing that's really new is that the API of these new black boxes is a prompt instead of a deterministic interface.
Don't get me wrong, there will be things that will be different in the post-LLM world. But my goodness do the current crop of companies overestimate how large that difference will be.