for narrative/dialogue, yes, text generation is currently useless. censorship, slop, extreme positivity bias. even jailbroken Opus is shit.

but audio generation we already have is pretty much good enough, and this is big. it's not AAA tier yet, sure, but still lightyears better half-assing it with mediocre voice actors. it is now an option to only use real voice actors for a few key characters, and even that won't be necessary within the next decade. even indie video games will be fully-voiced soon.

Similar for textures - it would be really neat if textures were auto-generated to add detail when you get close to something. As it is, the sprites just look bad.

Same for dubbing.

Voicing a game in English is doable, voicing a game in n languages is only doable if you're the size of EA / Activision / Ubisoft.

Those don't have to be mutually exclusive though. To take AI out of it, think of those murder mystery parties where actors interact with attendees. Actors have roles to play and things they must do to move the story forward, but they improvise their dialog when talking with the players and sometimes each other. Or if you've ever played D&D, you have experienced talking with NPCs that are controlled by your DM. I think video game AI could be a lot like that, where NPCs use natural language instead of rigid dialog trees, but otherwise, they behave a lot like they do today.

Yes, that's the intuition for where the technology might go. Someday.

But actors and DM's are much more disciplined than LLM's, partly because they have careers and friendships on the line for misbehavior. For what amazing things they can do in good weather, LLM's are not really reliable when you want them to consistently deliver something very specific, very secure, or very artfully crafted. They may get there, but their design makes it a very hard problem that we're still a long way from seeing commercialized.

Agreed that "live" in-game LLMs for NPCs is probably a little while away. Need better tools than now that allows to strike a balance between constrained/directed output and variation. I suspect that LLMs are already used by (some) game developers to aided dialogue creation tools for game developers, where they curate and select more or less a fixed vocabulary. I do think that there are many viable steps to "generic". For example, if the game developer could specify possible dialog trees (as in the possible branches and outcomes), then an LLM could ideally fill in the concrete text when reaching the different paths. Or an LLM could add in "small talk" - meaning things that reflect the world state, recent events, but still have the quest type dialog be practically hardcoded (so there is no risk of unreachable states etc).

Ah, like this: https://www.youtube.com/watch?v=Kw51fkRiKZU

I do think there is a big opportunity for widely supported hardware-accelerated matrix algebra in games. Currently most of that is geared towards graphics (naturally) but being able to easily encode arbitrary models and have them run on-device would open up a lot of opportunities for games (like deep simulation) that weren't possible before. Its currently possible of course but requires custom tooling and (relatively) niche hardware like a high-end graphics card.

I see the development energy around LLMs as a way to open up support for that.

Perhaps Llama are good enough now to add flavour NPCs to the world, i.e. NPCs not on your critical path, but maybe on the town square where you can hear them talk about mundane life stuff.