DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn.
That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war’ in a strategy game.
This is a big open problem in AI research right now. I saw a post around the time of AlphaGo challenging DeepMind to tackle StarCraft next, so it is very cool that they have gone in this direction.
When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.
> When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.
I don't actually agree with this. Unlike Go, StarCraft is not only a game of strategy; "micro" (micro-managed tactics, basically) also plays a big role. An AI is going to be able to issue a LOT more commands per second than even the most skilled humans, giving them a natural tactical advantage.
Strategy is more difficult for an AI, but not the only deciding factor in StarCraft.
Agreed that micro is a big advantage that AIs have held in previous AI vs. Human StarCraft contests, but the OP stated that the SC2 research environment will limit the AI to (elite) human APM levels, so that edge will be somewhat blunted. It's probably fair to question whether all actions are counted equally; I could believe that an AI could allocate their action budget more widely than a human can, even while hitting the same total APM.
More importantly, in my opinion, I'm not aware of any other AIs that have been competitive at the StarCraft problem using a Neural-Net-based implementation; from talking to PhDs in the field (but not from direct technical experience, so take this with a pinch of salt), my impression is that state-of-the-art RNNs just don't have the ability to encode the history of the game-world in a way that would enable them to be competitive. So it will require significant advances to the theoretical models to be able to compete using the sort of implementation that DeepMind will build.
When I was playing SC2 frequently a few years ago, someone made a program that used genetic algorithms to construct build orders [1]. It would produce a build that technically worked, but in a lot of cases they weren't very viable.
I tried to make a blink stalker rush that'd be the rough equivalent of the build shown in the post liked above, just with the Protoss race in the game. The program would tell me to do things like make two extractors but put fewer than necessary probes on them, pull probes to extractors after X trips or at X time (can't remember which), things like that.
I remember that program!
Interesting though that my experience with it was different; I recall that letting it run for enough time it was able to come up with viable and often quite efficient build orders.
By the way, I do seem to remember that putting 2 probes on an extractor was slightly more efficient (per probe) than 3 :)
Part of my problem was definitely what I was asking it to do. Going after blink so fast left me open to faster all ins like 2rax/2gate, 6 pools, etc. Being slightly slower than a 4gate and with more tech investment puts you even further behind if you can't do damage with the push. A better way to phrase it might be that the build worked, just not in the context of the metagame at the time.
The program would tell me to do things like make two extractors but put fewer than necessary probes on them, pull probes to extractors after X trips or at X time (can't remember which),
One thing that helps (fwiw) is to recognize a diagnostic.....for example, if at minute 4 you are unable to build a gateway, it means you messed up the probe count at minute 2. That gives you almost immediate feedback on where the problem happened, without having to watch the replay.
I agree. I think you'd have to limit that machine's APM to somewhere around the human limit, and make it choose which actions it wants to spend its time on.
Otherwise the AI is eventually just going to absolutely destroy humans with micro, which is not remotely interesting. We've known transistors were faster than fingers for ages.
'Computers are capable of extremely fast control, but that doesn’t necessarily demonstrate intelligence, so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”.'
Yes, this is definitely necessary for it to be interesting. Even so the AI will have some significant advantages over a human, in terms of being able to perfectly time events like injects, chrono boosts, etc., and watching every visible area of the minimap at once, so never being unnecessarily surprised by an attack. There are plenty of other examples. Nothing that's technically impossible for a human, but the ai will be perfect at those aspects as a given. For the human player to win, even limiting the machine's APM, they will need considerably superior strategy to make up for these differences.
Also, I'm not sure how they plan to limit APM, but if it's done in the naive way - the AI simply isn't allowed to surpass an APM of X, there will be plenty of optimizing it can do in terms of spending those actions more optimally than a human could (in some cases by performing fewer but more difficult actions to accomplish a task), as well as by remaining at peak APM continuously throughout an entire match. (Something that is a hallmark of great players, but like map awareness and macro timing, no human is able to keep up their peak speed indefinitely without any lapses.)
Why not limit it to a certain number of keys typed per second, and a certain rate of moving the cursor? Maybe randomly wait 0-150 ms before sending a keypress to simulate human imprecision?
What "somerandomness" what pointing out is that nobody in this thread seems to have actually clicked on the link. DeepMind's article clearly states that they would limit the number of actions per minute (traditionally referred to as APM in esports) taken by their AI, yet people are still saying that this wouldn't be impressive because the machine has an unlimited number of actions.
Clicking the same button twice in a row in rapid succession is much easier for a human to do than clicking two small buttons on different parts of the screen. A naive counter of "actions per minute" would count both things as exactly the same number of actions.
Whenever the harder action is better than the simpler one, the AI would gain an intrinsic advantage even though it's limited to the same number of APM.
My post (the grandparent to yours) fully acknowledges that. However, there are many different things that count as an action in starcraft, and even with a cap on APM, there will be many ways for a computer to pack in more useful actions than a human. Also a computer will have other innate advantages, like perfect timing and map awareness. An equally skilled human will therefore need stronger strategy to make up for these disadvantages. Your parent post appears to suggest mitigating these other advantages in various ways (although I don't personally agree with that approach, since the computer isn't doing something that's out of reach for humans, just things that aren't practical for humans to achieve consistently).
If you did NOT limit APM, it wouldn't even be a contest. The AI could simply mass first tier units and crush any human opponent with micro.
I do not understand the point of an AI if you want it to be equal in all ways to a human. Why not just fight a human?
The goal is to build an AI that is at least as good as pro players (which would involve building models and technology capable of taking complex decisions etc.). The goal was never to implement something that is strictly equal to a human.
You cannot blame the AI for having a better memory and perfect timing, or next thing you know you'll ask them to limit the RAM to something comparable to the memory in a human brain and use older processors to restrict the unfair advantage current computation clusters have.
Regarding your last point: even without limiting APM, there is no AI that can currently compete with decent players . If it was as easy as you say, people would've done that with Broodwar which has open APIs. The fact is that micro alone cannot win a StarCraft game. Perfectly micro-ed zerglings cannot do much against a terran turtling behind a closed wall and shooting with marines, or a protoss with immortals and adepts.
> I do not understand the point of an AI if you want it to be equal in all ways to a human. Why not just fight a human?
Well, if you want just an AI to win with humans, you can stick to playing chess - nowadays the AI can drunk-walk its way to victory through the game tree faster than humans can make leaps in strategic thinking. The point of such experiment here is to impose limits on what an AI can do in order to cancel out "trivial" advantages it has, so that the challenge for AI developers will be to improve the core of strategic thinking & planning. Victory by perfect micro is not interesting[0]. Neither is victory by perfect macro. You need to curtail the avenues which would let the AI win by executing utterly dumb strategies with machine-like precision if you want the AI to develop smart strategies.
TL;DR: because humans are not programmable, so they cannot be rapidly improved ;).
The examples you previously gave are different from what you're saying now, but like I said the article clearly said that they will limit the number of actions per minute which means: no gimmicky/weird micro tricks.
Any other advantage you mentioned (computers having a larger memory and faster reaction time than humans) shouldn't be removed, as that doesn't make any sense: machines will be machines. You're building an AI on such a complex game because you want to be able to model hard problems with incomplete information, and to do so you do need to use a lot of memory and the computer's fast processing... even if that means being faster than a human. That is "fair".
> Your parent post appears to suggest mitigating these other advantages in various ways (although I don't personally agree with that approach, since the computer isn't doing something that's out of reach for humans, just things that aren't practical for humans to achieve consistently).
As far as your last paragraph, I suspect the issue is that people building AIs generally try to follow the spirit of the game. I believe it would be possible write an AI that wins by perfectly macroing out tier-1 units, rushing, and perfectly microing them (and obviously it would be much easier than to build a more general Starcraft AI). If it hasn't been done, I expect that's because it would be little more than a gimmick. It certainly wouldn't be any fun to play against, nor would it advance AI in general.
I don't think it's as easy as that. Even with perfect macro + micro, your opponent could simply scout you, see what build you're going for, and counter your build.
Pro players do have (quasi-)perfect macro: They build units and buildings exactly when they should be built (especially at the early stages of the game). They also have an excellent micro. What makes one player win against the other is often the choices they make (going for a surprising build, or simply executing an attack in a way their opponent wouldn't be ready for).
So I'm pretty sure an AI that would just mass zerglings and micro them "perfectly" still would be crushed by decent players that know how to counter that rush. (build a wall, and have siege tanks or a similar unit)
This is one thing that I think would be fascinating when (if) we get to a competitive AI in this arena.
Do we eventually see the hypothetical human (or AI!) opponent trying to metagame against the way an AI system interacts with the game? E.g. human-pointless feints in an attempt to pollute AI logic?
Under the premise that a competitive AI will reason and act differently than a competitive human (due to aforementioned difference in speed/memory vs abstract thinking resources).
The problem with that is, most counters to first tier units rely on splash damage. As zerg for instance, if your opponent is rushing with lings or marines, you need banelings. But it would be trivial for an ai to neutralize the splash damage, so that wouldn't work.
Generally speaking, the counters are balanced for human players. Biased toward high level players, but still human.
We've seen exactly this happen in competitive play. There was a player who would always play "Mech" Terran units, even though it wasn't the best strategy, because he had a lower APM than the Korean pros did.
How does the AI "know" which Zergling is going to be fired upon? Are the tank shots not instant-hit? Or is this just a demonstration of what actual optimal control would look like.
The tanks aren't given overriding manual commands, so it likely could just predict how their automatic targeting works. If a player would give commands, it could get at least some information from the animation frames before the target is aquired and the shot fired.
(The implementation in the video likely just cheats and gets the information from the engine, but at least the first case should be deterministic enough)
There's almost certainly a bunch of timing-based micro with animations/game ticks/etc that a machine would find. For example, if a machine does a sequence of actions with sub-millisecond precision they can pop a unit into range, fire, and then pop back out again without being fired upon over and over.
A lot of units would be rendered useless because it's balanced for humans. That means the units they counter would be a lot stronger. I'm afraid it would probably just be a race to get the fastest rush.
What you're describing is called "kiting", and is already being done by regular players (not to mention the pros). It is only possible for some combinations of units, for example Marines with stimpack vs regular zerglings ("slowlings"), or Viking vs Void Ray because one unit needs to be fast enough to be able to shoot + move out of range before being shot.
They do say that they will limit the number of actions taken by the AI to focus on building a smart and strategic AI instead of learning gimmicky micro moves, but if one thing is sure, it's that even the best micro cannot win a StarCraft game if the macro is not on point, and bad choices are made (good luck micro-ing siege tanks against a swarm host!)
Rushing is a bit of a rock paper scissors situation. You can beat a fast rush by doing a slightly slower rush that focuses more on economy, and winning via defenders advantage.
I haven't played SC in a few years, but I think I can give a basic answer.
There's would not be one optimal strategy, but many optimal strategies depending on the matchup between units. By taking advantages in differences in movement speed, attack range, attack projectile speed, and attack cooldown, a weaker group of units can potentially beat a stronger group.
This is an older video that demonstrates some examples of winning with good micro: https://youtu.be/CdSKD3LRHV8?t=11. An AI knowledgeable of and capable of executing such tactics would be pretty strong.
Just on the surface, these sorts of games are notoriously difficult to balance properly. Someone discovers a strategy that tips the balance, and units/abilities/whatever have to get nerfed. The odds that there's a stable balance at a level of play not actively tested (e.g., infinite APM) are extremely low.
One example is a sibling post's video of 100 zerglings against 20 siege tanks. For humans, the zerglings are capable of destroying only a handful of tanks before being killed of. For an AI with infinite APM, they can move all but an individual targeted zergling out of the way of the tank's splash damage. This allows them to take out all of the siege tanks with only moderate losses. Many such matchups will be tossed on their heads, and it's likely that one of them is totally game-breaking.
There is a lua widget in spring rts that allows for units with at least one farsighted unit to automatically stay out of los. Sneaky army will wriggle through all defense lines.
Void rays are countered by AOE anti air. With infinite APM there is no need for two rays to be hit by the same aoe. Further AOE tends to do less direct dammage, so VR with infante APM may hard counter a lot more unit types than intended. IMO, the risk is more from something like blink stalkers becoming insanely resource efficient.
Simple example would be in combat, like being always out of range ect ...
Imagine one army where every single unit is controlled by one human. Each individual unit would be very very strong.
Infinite micro? I'd just send all drones/probes/SCVs directly to the human's base and kill their drones. The extra drones that the human has wouldn't be able to make up the difference.
This is already an infamous strategy (worker rush) and it os easily countered as long as you have all your workers united and attack at once (not much micro can be done at that point, and the numerical advantage trumps that... unless you're playing protoss and constantly trying to replenish shields, and still!).
Also, I'm pretty sure this is not something that DeepMind would do to "solve intelligence".
still a tough question but they could do some insane things. for instance if they had 200 marines and controlled each individually i can't imagine what they'd do. for humans it's requires insane mechanics to even break them up into 8 seperatel groups and control them for more than a few seconds
From my naive perspective; in GO you do not need to be concerned with resource consumption/mgmt... in Starcraft and other RTS' you do.
How much more complexity is added by resource mgmt vs just "legal play possibilities" as now you have "this move isnt legal because you dont have the requisite resource, thus you need to depend on completing those moves first"
Are there any efforts applying deepmind to, say, commodities markets? Or something along those lines? Supply chains for complex hardware manufacturing logistics?
Robot assembly of complex parts in an assembly line?
Sure, AI's playing games is cool...
What I would like to see is an AI playing "Farm Simulator" to manage a vast swatch of farm equipment making and picking food over a vast expanse - with little human interaction.
> Supply chains for complex hardware manufacturing logistics?
That would go well with the concept of Seed Factory, which is a factory that could reproduce itself, from raw materials. If we can build a self replicating device, we will reach another kind of singularity. At the moment, the whole human race is such a self replicating system, but if we can take humans out of the loop we could ship it to an undeveloped area (another planet?) and have it create a city and industry for us.
Another application would be to ensure self reliance for humans in the post automation world. A self replicating factory, capable of solving the whole logistics puzzle of building repairing itself, as well as supporting its owners, would ensure that its owners don't depend on UBI or jobs.
Such a factory would initially need some industrial machines and robots, probably at minimum a 3D printer, some raw materials and high end pieces such as CPUs. Then it would gradually print and assemble more equipment.
It would eventually bootstrap like a compiler, being able to "compile" an identical factory from raw materials.
Keeping count of each object, its source parts, construction waste, energy spent in building it, processes and time they could "compile" an ecosystem of devices that carefully work together, recycling everything that is wasted in the process. As the capabilities of the factory increase, the process could be "recompiled" to take advantage of the new capacity.
The place of AI would be to optimize the process. Instead of a hand-made industrial compiler, it could search for a way to combine its resources in more masterful ways. It can keep track of more things than human planners (project managers?) can. It could potentially assess the needs and organize the industrial production for a whole country in a more efficient way than capitalism or communism could.
The beauty of a seed factory is in its initial price - under 5% of the price of a factory built from buying equipment, because it only needs the seed of the factory and raw materials. The factory would grow like an organism, expanding itself. Most of the factory would be self made. Just like a cell, or an animal.
This is what I'm looking forward to with AI. With humans we tend to If This Than That and be methodical. AI on the other hand given enough processing power does everything at once and is omnipresent. Movies like iRobot kind of point to this, still in that movie there was a linear progression on the part of the AI side.
There is a massive advantage to seeing everything all the time and being able to arrange and organise everything at the same time.
Is it just me, or is anyone else a little disturbed that some of the greatest minds in AI are optimizing it for winning at a highly strategical/tactical game of warfare..
As a StarCraft 2 amateur, it is my opinion that it has a lot more to do with being able to manage a lot of actions at the same time -- production, defense, expansion, scouting and attacking your opponent. SC2 is definitely not only about war. The pro players are periodically uploading videos explaining SC2 economy and the huge importance of proper timing and type of production in order to achieve maximum growth -- and consequently, aggression towards the opponent.
Don't get me wrong; I'd prefer DeepMind's efforts be focused on a huge and realistic economics simulator but there are simply no such games (not ones that are good and/or realistic anyway).
IMO StarCraft 2 is the best economy and war simulator these days so that's why DeepMind chose it.
It will not get to strategy in these games at all. I can already see the AI dominating any human player just by splitting army. Imagine a terran AI dropping at 4+ places at the same time, just 4-5 marines each. This can be done within the first 6-7 minutes and no human player will be able to hold. I know they limit the APM for AI but you can easily do such drop with very few APM if you simply react fast enough on damage events.
There is a lot scope for mimicking a human player in terms of how long it takes to react and how the APM is actually used. I think the more effort they put into the AI restrictions, the more interesting AI you will end up with.
Having an AI that wins because it is faster isn't very interesting.
They are bad mostly because they try to not be obvious so that Blizzard would ban them. High APM, precision clicks without camera move or unnatural multitasking gets you reported and banned fast these days. Could be because of other things but there seems to be a lot less cheating recently.
Sure, but even then it's difficult. Micro'ing three fronts at once would be difficult for a human but could easily fit within the AI's budget.
You could solve this by coming up with some sort of context-switch effort function but it starts to feel forced.
Or at least, once you start calibrating the parameters of the problem around human limitations, it sort of feels weird to talk about the computer being better or worse than the human.
I think you just need to require the computer to do the same thing a human does and only look at a specific part of the map at a time. Then switching between fronts will require clicking on the minimap to move the view to a different front, so frequent context switching is naturally penalized by the APM limit.
>You could solve this by coming up with some sort of context-switch effort function but it starts to feel forced.
Doesn't seem too forced to me IMO. Humans have to expend more effort to context switch, so if the goal is to put humans and computers on a level playing field that's something you have to take into account.
no one is even mentioning the role of vision in the game. for a human it's pretty tough to keep track of the minimap and play at the same time. you end up missing a lot of things. imagine something that could monitor both simulataneously.
Maybe at high levels of play it would, but limiting the APM to even very low levels (e.g. 60 APM, i.e. 1 action per second) would still allow the computer to macro very efficiently, so I don't think it would be a balanced difficulty adjustment. It would be simulating highly skilled player with limited mobility rather than a player that is overall less skilled.
I'm guessing that a limit of X actions per minute would be implemented by imposing a delay of 1/X minutes after each action during which the computer cannot execute another action.
>Computers are capable of extremely fast control, but that doesn’t necessarily demonstrate intelligence, so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”.
They address that in the article. The context they're talking about is a bot beating a human with human-like dexterity limits in place.
> I don't actually agree with this. Unlike Go, StarCraft is not only a game of strategy; "micro" (micro-managed tactics, basically) also plays a big role. An AI is going to be able to issue a LOT more commands per second than even the most skilled humans, giving them a natural tactical advantage.
Seems like they should probably tackle a game that involves strategy without micro as a stepping stone to StarCraft. Are there any games with good AIs like that already?
> strategy without micro as a stepping stone to StarCraft
Never played StarCraft(assuming micro/tactics refers to simply continuous time based ability to give orders), but based on my experience AOE AIs, may be a game with turn-wise playing instead of continuous orders in time playing?? (for ex: battle of wesnoth)
I don't agree with this. This was more true with the original StarCraft but with SCII a lot of the fine graining micro is automated. And although the computer will be able to reach 1000s of APM, a solid strategy with perfect timing will defeat it.
It doesn't matter how well you split your zerglings or move your marines, when there are tanks busting through your front door.
> When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.
More like a time to be very excited!
But I totally agree, long-term dependencies are an difficult problem.
The DNC[0] by DeepMind is in my opinion a very important step in the correct direction to tackle this.
Honestly you can solve fog of war using regular computing (TM) to remember the last known state of a particular area, then feed that to the network along with information that A. it's an area as it was seen some time ago B. how much time elapsed since it was last uncovered.
I can't imagine that this is where DNCs come in unless you want to needlessly complicate the matter (and create a network that is much more expensive to train).
Yes, it's interesting what help will be given to the network.
From the article it seems the answer is: very little. It is allowed to use only the pixels on the screen. (During play, not training apparently.)
It will even have to spend actions on moving the screen/window to inspect things.
Very cool stuff :)
I think you're thinking about the difference between full information games vs partial information games, rather than stateless games (the state of a starcraft game is stored on a central server, the computer just has no access to it).
There's already been excellent progress in poker AI based on traditional game theory, see http://poker.cs.ualberta.ca/.
It's not just the limited information space, however. There's also the solution space or "action space". A poker game-space is quite narrow compared to that of StarCraft because of pixel-wise differences. This is the main reason that makes StarCraft "much more realistic". In real life you have infinite moves and options with (almost) analog differences, whereas a game like Poker is much more discrete. Go is somewhere in between.
For instance, as any protoss player knows this all too well, a Zealot placed a mere pixel away from where it should could allow a stream of zerglings in, completely throwing any game plan you had up to that point out the window and enter crisis management mode. Usually this is a "mistake", which a computer may never make, but it needs to learn the relevance of such pixel-sized mistakes, which is unlikely to have as much of an impact in a poker game or Go game as it would in StarCraft.
> but it needs to learn the relevance of such pixel-sized mistakes, which is unlikely to have as much of an impact in a poker game or Go game as it would in StarCraft.
I agree with your point, but you need to look no further than the game that AlphaGo lost against Lee Sedol.
It committed a very bizarre mistake that made no sense gamewise.
You're right, that's the technical term for it, thanks. I was referring to the agent's evaluation function being stateless (i.e. it does not _need_ to persist state from previous turns in order to play optimally).
I think that there are two distinct aspects of the incomplete information that are significant; first, ignoring temporality, you have the fog of war, so you can't see the whole board. This is probably easier to address in a RNN, since you can play quite well as an amnesiac that just reacts to things that are currently visible. But you need to scout less if you have a memory of what's out there, so the amnesiac won't be able to play optimally.
Then there's the temporal aspect. The set of previous states of the game is not stored in the game and made available to the player, and so to play optimally you have to have a memory. This is where new techniques will be necessary.
These are separate problems, I think, so it will be interesting to see if DeepMind can make progress without reaching human performance on the second part.
your last sentence, and this comment in general sounds exactly like what people were saying a year ago about Go. (Chess was easy, because it's perfectly suited for AI, but when a computer can beat us at Go, that's when we should be afraid!)
Even if you don't imagine robots becoming self aware and deciding to kill humans on their own, you can imagine a fleet of drones and self-driving tanks controlled by a deep learning network. They don't need to understand what they're doing to make effective tactical decisions.
If the US doesn't do this, someone else probably will.
We're going to see an AI-based blitzkrieg attack in our lifetimes.
Once you can launch an invasion without losing any soldiers, war seems like a much less dangerous proposition.
They are quite terrible at all of the other aspects: macro, broad strategy, tempo & timing, planning tactical surprises, etc.
This results in a very low overall competency, when it comes to winning games against highly skilled opponents.
It will be a challenge for researchers to improve performance in these areas. The problem space is very complex -- it makes solving Go look like a cake walk in comparison.
As you note, that's only micro, and it's the easy part since you can do most of that with rather simple rules and just high apm (which is where the benefit compared to human players comes from). The interesting part for AI is how to get there, the longer-term planning, recognizing the enemy's strategy based on what it observes, reacting appropriately, etc. So far those are the things AIs have trouble with.
I suspect this will eventually lead to AI as a service for games. Rather than build a terrible AI that delays a game by months, approaching a company that can build a decent AI initially which gets better overtime would probably be ideal and create better experiences.
Is it harder to make an AI that is mediocre at playing the game vs one that is good? That is the main issue with making good AIs for games now. We want AIs (if you can call them that) that play kinda like humans but worse then the player as the player wants to win.
Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
I think it's really important to realize is that a game AI isn't really there to try to win, or maybe even to "convincingly" beat you the same way a human is. Perhaps you could train an AI to make the game more "fun", but that's pretty hard to measure and people are so different. It's a really hard optimization process.
One commonality, it seems, is that players want to get better at the game. Perhaps, an AI that TEACHES the user to improve their play, and reduce errors would be an easier problem. So far we mostly do this using heuristics of gradually increasing difficulty, but it should really depend on what the player is weakest at... and what the user can improve at fastest! That is something nicely measurable, which you might use to train an AI.
Then, if/when people play against other human players they are as strong as they can be... often an FPS goal.
> Perhaps you could train an AI to make the game more "fun"
This is actually a way more interesting problem to solve, for the gaming industry.
Beating games is cool for industries where gaming is not the final problem (autonomous cars, various decision making, etc). But if AIs are to be exploited by gaming industry, it needs to learn things like : "is the user bored?", "what usually triggers new interest?", "how should difficulty be adjusted given current way the user is playing?", etc.
>an AI that TEACHES the user to improve their play
Wouldn't you be able to do mutual training that way? By mutating the AI's "strategy" only after losses, the player would continuously need to figure out a weakness in the AI's game-plan. And the AI tries to evolve its strategy every time it loses.
You could use some kind of library of example "mistakes" made by very-high-level players to identify when lower-rank players are making similar errors; then point out to the user how everyone else who made that mistake 'updated' their behavior.
Well, starcraft AIs aren't particularly good. Any player who has completed the single player campaign could beat all the AIs that came with the game that didn't cheat.
And a player in the top 20 percent could beat any AI that has ever been created.
It's like saying that a Go AI would have a huge advantage because it can move instantly, giving the opponent no time to think. Perhaps it contributes, but strategy is far more important.
This essay reminds me very much of the "Playing to Win" series by David Sirlin: http://www.sirlin.net/ptw/
It's also worth noting that AlphaGo optimizes its perceived chance of winning the game, rather than its score. It "gets complacent" and mostly plays very conservatively the moment it's ahead on points, because it doesn't care about margins, only about win/loss.
It does, but the AI has to do useful things first before doing them quickly is a benefit.
Additionally, humans can use economic reasoning (player built 3 barracks to they can't be building x, y and z). This can lead to AI being excessively safe (economically inefficient).
Definitely, but my guess is that Starcraft AI's are really good at Macro play (building the right units and tech trees), but at the upper levels of SC players win or lose in Micro play (specific positioning of units), which is much harder for an AI to accomplish well, since there are a lot of little tricks. Basically, Macro can be programmed easily, but Micro requires the player to respond creatively.
It's actually the other way around, I'd say. Individual unit control follows very naturally when you have unlimited APM. And most of the time you can define simple and good criteria on how to position the units (e.g. concave, or exactly the right distance apart to avoid splash damage, or one unit a tiny bit more towards the enemy to be the first one targeted by incoming fire, ...). Build orders can be programmed, yes, but transitioning between strategies properly based on observations and guessing or judging the enemy's strategy is quite hard.
The ones I saw in the competitions lost against humans on such micro rules since the humans quickly intuited their blind spots. The ability to bluff and screw with AI's, a human strong point, is what made me predict Starcraft was going to be way harder than Go. I'm still betting on the human in this one.
And the last line you said is also true to make things even more difficult. :)
First person shooters can have super accurate AIs that are really frustrating to play against. Try beating the last "boss" in quake 3 on nightmare difficulty. A game like Starcraft relies much more on decision making that raw 3d math calculations and that makes making a decent AI pretty difficult without giving it advantages like extra resources or vision. That said, even though a game like Quake is easier to write an AI for, top players can still be top level AIs with superior reasoning about using the map and timing power up spawns and such.
> First person shooters can have super accurate AIs that are really frustrating to play against.
The Worms series is also notorious for this. Their AI targeting seems to flip between "clueless" and "godlike", with very little middle ground. E.g. https://youtu.be/gqPITW04vRQ?t=1m20s
Probably not, to be honest. For a game like chess, the challenge is that there are so few things you can actually do. Inefficiencies aren't really a thing, every single move has a semi-discrete value.
Starcraft however has all kinds of minor inefficiencies you can build in. You could inconsistently build workers, you could occasionally supply cap, your army comps can vary in sophistication level, etc.
I bet you could run an analysis of "move quality" for the human player, find out where the moves he is making would rank in a list of possible moves sorted by value of the AI's quality function, and select the same-percentile moves for the computer AI to make (from its available moves).
> Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
It is actually about making a good AI. Civ 6 or SC2, in both cases they simply "cheat" by providing multipliers to the AI in order to solve its limitations. The basic function is largely the same. Providing the AI with cheat codes is not "good" AI.
A good AI design would remove the need for such arbitrary advantages that are out of context for human players and leads to weird issues.
It depends. AIs can do some things much better than nearly all humans. Twiddly little micromanagement for example. Total Annihilation (and it's successors like Supreme Commander) do this. The AI will send out a constant stream of micromanaged harassers constantly order to annoy the human opponent to death.
Where the AIs tend to be very bad is in long term thinking and overall strategy.
Blizzard's AI in Starcraft generally plays like a slightly more aggressive but novice human. It's more fun than the TA AI, but not terribly strong once you understand the game mechanics.
Some AIs for the Spring RTS Engine had this too. The funny thing is though- harrassment doesent scale. In these Exponential Economy type of games, the harassment only works as long as the macro-strategy is unable to take off. After a certain treshold, you can micro all the funny dances in the world- but the wave of enemys is going to carry this surfer to the shores end.
I think it's about options for players, some are strong others fresh and both want to choose an AI to their likings. Sometimes they are in the mood to get beat maybe to learn new tactics and occasionally just a victim to fight frustration.
What wrong with a strength option? I'd include an unbeatable entry in any case.
I'm not sure that accuracy/HP are the right things to tweak. I've always wanted to be "surprised," in an unscripted way, by AI enemies in FPS and RTS games.
One game experimented with smart enemy units that could sneak up on a player and attack them from behind, or outflank them. The players hated it and assumed the computer was cheating and just spawning enemies behind them.
The same is sort of true in RTS games. The new rerelease of age of empires 2 has vastly improved AI that was developed by modders. Lots of players thought it was just cheating.
But in general it has been very well received. I think players can get used to good AI. They've just been trained to expect stupid AI that cheats.
For the vast majority of players it's probably irrelevant whether the AI cheats or not. The goal is not to play fair and beat the human usually, but to put up a good fight and make things challenging and, most importantly, fun for the human. Of course, competitive and professional players have different expectations or requirements in that regard.
Still, AI in games is mostly a trade-off. It usually can't take too much resources (because it usually has to run on the same machine the human is playing on), it has to be believable and fun to play against. This usually rules out too fancy algorithms and approaches with dubious returns.
I think the majority of players would prefer AIs that are challenging through better strategy and tactics, rather than just being bullet sponges or having better aim. People also get frustrated when the computer wins through cheating, rather than actually being better.
That would be funny. The difficulty slider changes to "How many hours of playtime" instead of easy, medium, hard. Actually it could even just scale it automatically based on a metric like steam hours played. that would be awesome.
I don't think so. Many of the fun things AIs do in games is attack at different stages of the game. They send a small force at the beginning, and then a little larger force a few minutes later, and then a big attack at the end. No AI trying to win would end up at this strategy. You would have to define the fun goals and have the AI train on that for it's success criteria.
I've been to the Game Developer's Conference (once) before. Believe me, there have been startups that have provided A.I. services for years (it was kind of like the equivalent of the Havok engine for physics, you had to implement it into your game). I don't know how well used they are, but I remember them being mentioned at one of the lectures about A.I. that I attended. They had tables and swag, too. I don't remember their names.
There are a lot of problems, but mainly I think it boils down to game people, like a lot of programmers, wanting to write something themselves. You'll always understand it better, and it'll be perfectly tailored to your problem. Unless the problem is REALLY hard and takes a long time coding tedious things, they'd rather just do it themselves, since coding something is always more fun than figuring out someone else's code.
Also, out of my experience with many different fields of programming, game programmers seem to want to write more of the stuff themselves than most. This would be another huge subject to go into, but I think the biggest reason is that historically you couldn't ever really punt on performance in games, and you had to tweak everything to your problem. If your game ran slowly or looked bad, people just wouldn't buy it.
In previous years I've seen a lot of AI tech for action-games -- not neccessiarly FPS but shooter in a flat world with obstacles. Eg. marking objects for cover, marking positions to defend from etc.
I thought a neat way to make NPCs less dumb in games would be to let other players (internet connection required) to be the NPCs. They would have limited coverage area maybe set tasks and restrictions and let them fight a real person to move onto the next part of the game.
Basically set up a queue of player and what NPC character to play and when a user gets to that part of the game where NPC needs to load, you jump in and play. When the NPC dies you the game (scene) and get put back in the queue of playable NPCs.
You're storming the beaches of Normandy. You duck behind cover. Suddenly a shot rings out. You're dead. The kill cam reveals you were killed by a human-controlled enemy, standing on a seagull in the skies above. [1]
Unfortunately, humans will find advantages that shouldn't be used in any sort of game that tries to build a believable world.
That isn't to say it's a bad idea. You'd just have to design the game with human enemies in mind. I was very interested in The Crossing [2], which incorporated that of mechanic. Unfortunately, it was canceled. :(
AI is terrible in games because it's literally such a hard problem. Thinking like a human, working in a group, overcoming obstacles and reacting to dynamic behaviour from the human player. This is far harder that Go which has 100x simpler rules. Additionally they had a mega ton of training data to feed into the Go model to begin with, which a game won't have as the data structures will all be different under the hood.
The reason they picked this challenge is to work on problems where the whole state of the game is not immediately accessible (games of partial information, as opposed to chess and Go). Compared to AlphaGo, it is much more similar to the situations humans encounter in daily life.
Most games don't need this. Smoke and mirrors work a lot better than honest AI to create an illusion of intelligence, and it's much easier to change game mechanics and balance with a simpler AI.
Allowing researchers to build AIs that operate on either game state or visual data is a great step, IMO. Being able to limit actions-per-minute is also very wise. The excellent BWAPI library for Starcraft Broodwar that is referenced (https://github.com/bwapi/bwapi) provides game state - and was presumably used by Facebook to do their research earlier this year (http://arxiv.org/abs/1609.02993). For mine, the significant new challenges here not present in Go are the partial observability of the game and the limited time window in which decisions need to be made. Even at 24 frames per second, the AI would only have 40 milliseconds to decide what to do in response to that frame. This is more relevant to online learning architectures.
The open questions here are how freely this will be available - and in what form. Will I need to buy a special version of the game? Clearly there will be some protection or AI detection - to ensure that competitive human play is not ruined either by human-level bots, if they can truly be developed, or by sub-par bots. Starcraft 2 (presumably the latest expansion, Legacy of the Void, will be used here) does not run on Linux, whereas most GPU-based deep learning toolkits are, so having a bridge between the game and AI server may be necessary for some researchers.
Besides being great for AI researchers, this is probably good for Blizzard too, since it will bring more interest to the Starcraft series.
BWAPI asked in the past, I think, and got pointed to the game's EULA, which forbids it. It may be a different thing if Google asks. Especially with a somewhat more remarkable result (AlphaGo) than "just" a bunch of student AI experiments.
If DeepMind is planning on building an AI that can beat the best human SCII player, they have their work cut out for them.
I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style.
> I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style.
I wouldn't be surprised if an AI comes along that shows this human perception of what it takes to win isn't required at all. I'm sure similar things were said about Go and Chess.
Both Go and Chess are perfect information games with a (mind bogglingly huge but) vanishingly small number of states/permutations compared to Starcraft. It's an entirely different beast, altogether.
Similar things weren't really said about Go or Chess. Both are perfect information games so, by definition, they can't involve deceit There's no need to try to detect subtle patterns in your opponents behavior because you have the entire game in front of you. That's not to say it's impossible but they do very much have their work cut out for them. It's an entirely different type of game that is much more difficult than Go and Chess.
> Similar things weren't really said about Go or Chess. Both are perfect information games so, by definition, they can't involve deceit
I took deceit to mean tricking your opponent into thinking you were doing one thing when you were actually going to do another. You can't do that in Go or Chess?
What I generally meant was maybe there's some really boring numerical number crunching or some simple mechanical algorithm that solves games like StarCraft but when humans talk about it, they talk about mind games, deceit, intimidation etc. when maybe these things are irrelevant.
>I took deceit to mean tricking your opponent into thinking you were doing one thing when you were actually going to do another. You can't do that in Go or Chess?
Well you can technically but it's horrible. No above amateur player tries to play 'trick moves'. Skilled players try to find the objectively best moves in the given position because tricks are just way to easily spotted.
The "betcha didn't see that coming!" thing is only something they do in dramatised films. Stracraft and other RTS games on the other hand abuse lack of information quite heavily. 'Cheese strategies' or unorthodox openings are very frequently used at the highest level.
Yes, people play silly moves once in two decades, but again, in 99.9% of games nobody swindles or plays a6. In starcraft a good third of pro games or something involves cheesy openings.
Deceit boils down to assuming your opponent will misjudge the situation and make a mistake. In Chess or Go, computer players choose the best move assuming that the opponent won't make any mistakes (will never be fooled). There's no attempt to model a weaker opponent.
This is why computer players in Go tend to fall apart when they start losing. They don't see a path to victory against a perfect opponent, so all moves look the same.
A Go AI that can win from behind (given a handicap) might be an interesting challenge.
> In Chess or Go, computer players choose the best move assuming that the opponent won't make any mistakes (will never be fooled). There's no attempt to model a weaker opponent.
What I meant was there's no reason an AI couldn't try to fool a human in a perfect information context. For example, you could bait a (probably not world class though) opponent into capturing a piece which puts them in a bad position, where certain baits are more likely to work on humans than others. There's plenty of situations in real life where two people have the same information but one comes out on top because the other overlooked a detail.
> This is why computer players in Go tend to fall apart when they start losing. They don't see a path to victory against a perfect opponent, so all moves look the same.
Is this not a limitation of the AI though instead of something inherent to AI?
> A Go AI that can win from behind (given a handicap) might be an interesting challenge.
It's more of a testing limitation. These programs are only occasionally tested against humans. You need expert players to test, they're not available all the time, and it takes a long time to play a single game (unlike poker). You can use databases of online games but it won't tell you how the player would have responded to a different move 37.
If you're writing a program to beat anyone, there isn't much point in optimizing the algorithm for a specific opponent's weaknesses when there are improvements to make that would work against anyone and the improvement in strength can be evaluated using computer testing alone.
Work on exploiting specific opponents' weaknesses is typically done using much simpler games (check out what's been done for rock paper scissors).
Top players go to great lengths to just deny scouting information (for example killing inoffensive hallucinated phoenixes flying their way) just to keep their opponent in the dark.
They have their work cut out for them, but not in the way you think.
Remember that it's easy for an SC2 AI to beat a human if there are no constraints such as APM. So if and when the AI wins, the humans are going to complain that the constraints weren't set tightly enough.
So the contest is going to devolve into a war over the constraints to use, just like the endless bickering between Protoss Terran and Zerg about balancing the units.
The best thing DeepMind will be able to say is that they were able to beat Flash (or whoeever) with only 150 APM. But Flash will complain that it wasnt fair because DeepMind was able to be so efficient with those 150 APM so that really it should be limited to 100.
I suppose once the APM cap is well below what humans do, then the humans will have to admit a sort of defeat.
Hopefully there will be a ladder running where humans can play against the AIs under development and get a feeling for how they play. Will be really fun & interesting!
> it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style.
Isn't it also possible that the strategies employed by top human players (e.g. deceit, having a gut feeling of what the opponent will do) could be less effective or even useless against an AI that doesn't think or act like a human player? Given how human-influenced the meta seems to be (at least, to my understanding), it seems plausible to me that this could be the case
It's quite possible that AIs will reveal completely new and different types of strategies. It's very unlikely that a concept as broad as deceit is going to be disproven though. I mean just thinking about what that would look like that would mean an AI discovered that it's actually better to show your opponent what you're doing rather than trying to mislead them. I have a hard time believing that could possible be the case.
> It's very unlikely that a concept as broad as deceit is going to be disproven though. I mean just thinking about what that would look like that would mean an AI discovered that it's actually better to show your opponent what you're doing rather than trying to mislead them.
I meant it in the other direction; what if top players rely extensively on deceiving each other, but an AI is better able to predict its opponent's strategy than a human an isn't fooled? Again, I'm not saying this is necessarily the case, but I don't think it's implausible
What about scissors paper stone? An AI can be trained to have >50% odds against a human player, given a copious amount of games. Humans have predictable patterns and they can definitely be exploited.
No the same thing very much could not be said about Go.
Go is a perfect information game so by definition it can't involve deception. It's a completely different problem for AIs.
I'd love to see a e-sport league where the teams are AI human hybrids (Centaur teams). We know that AI human hybrid teams are great at chess [1],
and I'd love to see rts games played by 'Centaur' teams. In the same way the innovations made in F1 often trickle down to consumer cars, can you imagine the advances that could be made in human-machine interactions in the crucible of a real-time Centaur gaming league?
I'd rather that the coding was something that was left to the player or to the third party market. Right now programming games are kind of boring, more like a way of prettying up homework than anything else. A really fun game where programming could give you a competitive edge would be something to see.
The closest thing I can think of is add-ons for MMOs, but there the game developers specifically try to prevent add-ons that are too performance enhancing instead of making the coding an integral part of the game.
This would be so much more fun with a turn-based game where speed isn't a variable, like Civilization. I'd love to play against several AIs that were better than me because of code, not because they get a bunch of extra in-game bonuses. With a nice AI API, you could have online competitions where AIs battled every month.
This paper is a few years old, but it gives a good overview of the problems faced when building an AI for starcraft and the methods used.
Ontanón, Santiago, et al. "A survey of real-time strategy game ai research and competition in starcraft." IEEE Transactions on Computational Intelligence and AI in games 5.4 (2013): 293-311.
Looks like they are going to limit the APM of the AI. I wonder how they are going to decide the limit? I've never played StarCraft, but from what I understand very high APM is needed to play the game at the highest levels.
There was a Starcraft brood war AI competition that did not have a limit on APM.
The top contestants were employing strategies that required APM an order of magnitude higher than what players could do. For example, in brood war, SCVs can repair any Terran building or vehicle but it is not worth the effort to repair goliaths or tanks for a player.
However, the AI could easily manage repairing it's tanks which makes the Terran army much more cost effective.
In SCII there is auto repair and other improved UX that probably limit the ability of novel strategies like repairing vehicles.
Because the tanks are using the default unit AI, which AFAIK amounts to "when your weapon is ready to fire, shoot the closest enemy in range". The zergling AI knows the attack range and rate of each tank so can ensure that the closest zergling is always on its own when a tank is ready to fire.
What would be really interesting is putting a similar smart AI on the tanks, which could make decisions like shooting at the highest concentration of zerglings instead of the closest one, or holding its fire if one lone zergling is in range, but a clump of them is about to enter range.
Human players already do both optimizations with widow mines, which are a unit that takes several seconds to prepare its shot so it's feasible to retarget. It's not feasible for humans to do this with tanks though as they fire immediately and automatically.
You could even see counterevolution as two AI algorithms become better at predicting what the other will do.
Game theory dictates that you shouldn't anticipate what your opponent is likely to do, but rather assume that they will chose the best choice available. Given that you need to move the zerglings before the tank fires the tank should be able to retarget and always hit the highest concentration. That being said organizing your lings to run in a formation to never expose any to splash shouldn't be too hard.
If you slow it down, it would be apparent to you too. High-level human players do this kind of thing to, just not this fast. But effectively, it's watching where the attacking unit is facing and it knows when the next attack is coming, so it knows what the viable targets are and can react when it becomes obvious.
(it's also possible the system is receiving information directly from the game engine, but it's definitely possible to figure this out from nothing but the information on screen)
I've followed the professional StarCraft 2 scene for a long time now. There were a few players who were famous for having high APMs, here is an over-the-sholder view of one of them [1]. I don't have a reference but believe he used to play in the ~500APM range. That would probably be the upper limit of a human's ability.
I wouldn't be surprised if a 500APM player is making closer to 100 meaningful actions per minute. It looks like most of the actions are spam that may register on the APM meter but not affect the outcome of the game.
An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.
Back when I played SC2, you really didn't need a high APM to succeed, unlike SC1. SC2 offered some helpful mechanics that SC1 did not, such as the ability to select infinite units at once, or give commands to multiple structures at one. I think Sheth, a high level Zerg player, was famous for having under 100 APM.
Sheth did not have under 100 apm. He was infamous for not having a high apm in the early game (people like to spam keys in early game for many reasons). He still had over 200 apm once the game got going. This is on the low end for pros in sc2.
That may be an ok definition for you, but it most certainly isn't for SC2. I wasn't anywhere near the pro level and I would peak at 300 apm and would spend most of the game around 200. 12 APM is borderline not playing.
Consider it a statement about what I'm looking for in a game. I'm not really interested in competing over who can work their mouse button harder, or in driving myself to excel in giving myself repetitive motion injuries. Those are things that take away from starcraft; they don't enhance it.
I would strongly encourage you to go against your preconceived idea of what StarCraft is (a game where people compete on who's the fastest to bash their keys) and see a professional game and all the strategy it entails, such as in the semi-finals and grand final that will be on today.
Pro players do have valid reasons for having such a high APM (even if a portion of it, maybe 30%, is just "spamming" to keep theirselves active and ready), but you can be an excellent player with rather low APM (~120 maybe?). Why would you need to do 2 action per second? A lot of it is simply building units periodically (e.g pressing 1 then "S" every 10 seconds) but then APM really gets high when there is a battle and you need to give actions to multiple sets of units to get a good engagement.
High apm is required, yes. But most people push it by pushing it through spamming 1-2-3 (groups of your army). There was a pro player who didnt do it and he was around +- 70 APM.
It would be interesting to see what percentage of a pro player's APM is actually useful. Like you said, most of the time they are spamming buttons to keep their momentum up.
There is a replay analyzer that attempts to figure this out, generally the effective APM will go up during a fight, but the players will keep the APM fairly high even when the eAPM is not high just because it is easier to go from many useless actions to many useful actions than few useful actions to many useful actions.
There's probably no reason to restrict the computer the below human pro players (200-300 APM). My guess is the engine just wasn't designed with superhuman APM in mind and could be exploited. If that's the case then the AI would in some sense not really be playing the same game as the humans.
Superhuman APM makes it a really different game. Current Starcraft AIs can do thousands of commands per second. That means that essentially each unit has its own AI and can act independently of all the other units. There's almost no delay between different unit commands.
Unlimited APM makes AI much less interesting from the perspective of game theory.
Think of it like a robotic boxer, except instead of an android, they build two 30 foot long walls of spring-loaded boxing gloves that close in on the human boxer. Yes, the robot punches the guy a lot, so the problem seems "solved" but it isn't really.
Buts it's intentionally dumbed down to keep the game fun, loosing all the time would get frustrating real quick. Theres no ROI for AI devs to kick the humans ass (except for maybe now with our AI/Cloud/VR/ craze).
It would be cool to see a project like this for an open source game, such as OpenTransportTycoonDeluxe (http://www.openttd.org/). The AI developed by interacting with the OpenTTD economy might even prove useful for urban planning of real geographic regions.
Training data? After all, AlphaGo trained on a database of over 30M expert human moves. I suspect one championship round from Team EnVy is worth billions of iterations of random exploration ;)
Kudos to both Blizzard and DeepMind. Anticipating a lot of fun with this. StarCraft 2 could indeed become the AI pedagogy standard.
That's so cool! I wish they could start doing AI for team based competitive games like League of Legends where meta-play and team decision making is important.
Is that too complicated to tackle yet?
Are you imagining a single AI playing all 5 heroes on a team, or 5 independent AIs each playing a single hero with limited communication between them, or a single AI controlling a single hero on a team with 4 humans? I would say the first scenario (1 AI controlling entire team) is roughly equivalent to the the RTS case. It's just fewer units with more abilities, and character progression instead of base building. The other two cases are probably a lot more complicated.
I can't wait to see the first team of 5 separate AI learn to insult each other. It'll be a watershed moment in history when the first vote to kick another AI occurs.
Honestly all these scenarios are interesting research avenues.
Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice.
Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.
Either one could work. If the single AI that controls 5 heroes is easier to program, let's imagine that.
I have the feeling that it's more complicated than a 1vs1 RTS game, but I don't know why...
Controlling five units is a lot easier than playing an RTS. In an RTS you don't just control five units, you control potentially hundreds of units, and you have base-building and an economy to manage.
AI controls one champion, maybe a bit weaker than player controlled ones.... and it controls the minions. every x wave, they can go into jungle or do other things rather than push lane
Wow that's a very interesting idea. Several models working together. Plus you limit their method of communication by being able to use the chat only. Inventing and learning new languages made by other models!
restricting controls while typing is an arbitrary restriction considering they can communicate with audio asynchronously.
this brings up another interesting topic all together- which is how inefficient human language is at communicating complex ideas, quickly, with people you don't know.
that is- just like how AI has an obvious APM advantage, it also has an obvious communication advantage (raw serialization and deserialization. there is no room for ambiguity or misinterpretation).
basically, all information known by one agent will be known by all other agents in near-realtime. something humans can't do yet.
> restricting controls while typing is an arbitrary restriction considering they can communicate with audio asynchronously.
On the other hand, if four people are simultaneously speaking to you, it's hard to understand all of them at once (especially if you're also talking at the same time); of course, an AI would presumably be able to communicate information much more quickly to other AIs than humans can to other humans, so limiting one to "speaking" at a time would probably not be that detrimental to them
Best would be if they had a language model, but no memory of communicating together in the past, so each game the AI would have the additional hurdle of building up communication.
depends on the goal, right? considering they want to limit APM in the SCII model- we can assume that goal is something like "beat a human under human conditions". in that case yes, communication needs some limitation. but this seems uninspiring. "assuming" a no-limitations AI can beat a human in RTS, the question seems more like "how many inefficiencies must we introduce before the AI is 'fair' game".
but, there is a different use case where you have an AI team try to beat another AI team under optimal conditions, which i think is more interesting, as it has the chance to completely change the metagame (like what Deep Blue and DeepMind can do for Chess/Go respectively).
By the ways, if anyone's interested, there was a deluge of deep learning papers today and one of them basically used deep learning to make deep learning models and it did better than humans.
Seriously... Source? I have looked at new papers indexed by Google scholar and I don't see anything that boasts this. It would be the feature of the abstract and would probably be a Nature/Science paper. So... Source??
That's great.
AIs on SC1 relied on many hacks. Initially I thought that DeepMind was going to create a bot for the original SC.
I hope some of the advances in SC2 AI can be integrated into the in-game AI. e.g: a trained neural network that plays better than the "hard" AI, but can run on a consumer box and not on a massive cluster.
Typically they train the AI against itself. That gives it a continuous skill gradient. Otherwise it would lose all of the games without learning anything. Since there are no successes to learn from. Or it would win all the games after it figures out how to exploit the built-in bot. With strategies that probably won't apply to humans.
This will be good for games moving forward due to the meta changing for players as the AI adapts to their tactics and vice versa. Lessons learned from this can then be applied to other areas. And as an added bonus it creates more interest in AI research.
I applaud the idea, but I'm worried about how open the results of the research will be in the end.
I think the worst possible outcome for society would be if we ended up with capable AI but with the algorithms only accessible for a handful "AI-as-a-service" companies.
The second concern is understandability of the algorithms: from what I've read, it's already hard to deduce "why" an ANN in a traditional problem behaved like it did. An AI that is capable of coping with dynamic, unpredictable situations like an SC game (with only pixels as input) is impressive but it seems less useful to me if it's not understood how that is done.
DeepMind always published papers about their research (the AlphaGo paper goes into a lot of details on how everything works). Not sure why you're doubting the openness of this initiative when they are pushing an open API for everybody to do research on, not just DeepMind.
Well unfortunately im fairly certain that outcome is the source of funding for these research projects, and i dont think socialized funding could hold a match to it nor potentially even provide as much value
> We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers next year.
This is awesome. I've only ever reached the Platinum league in Starcraft II (1v1), but I'd almost feel more driven to create bots to (hopefully) surpass that skill level, than actually playing the game.
Can a simulation of a complete Starcraft game be done quickly and assuming it can't doesn't that present problems to training an AI? For example, I'm guessing complete games of Go are order of magnitudes faster to simulate which makes it more practical to do things like getting AlphaGo to play against itself to train.
No, the fastest they can do right now is 8x. And yes, that will become a substantial bottleneck for AI training. I imagine they're going to have to have a large sim-farm for just running the games at 8x for training.
Comparing AI and humans in games is not useful unless all limitations are controlled for both parties. The artificial intelligence only gets the video output of the game and output to simple controls with a human reflex - like lag on how long it takes for the controls to take effect. It just comes down to the scientific method.
This is a wasted opportunity for other strategy games to become the most played game on the planet.
I sometimes play strategy games and I always find the AI disappointing. Any game with a great AI would be my favorite for years. Heck, I would even pay a few dozen cents/hour to be able to compete against a proper AI.
For anyone looking for more information about Starcraft 2, the world championship is on this weekend and the broadcast for today has just started (16:30EST)
This is so exciting. I've always wanted to program bots to play online games -- mainly for learning purposes. (Can I make a bot that plays better than me?)
But I've never done it because of the risk of bans. I'm glad that Blizzard has opened it up for people to experiment with this. I wonder how it will interact with any sort of anti-cheat systems in place, etc.
This has been around for awhile [1]. They give you an API to use and then you upload your JAR. They have a Twitch channel [2] that is constantly streaming AIs playing against each other in their tournament.
You should give Adventure Land [1] a try. Its a browser MMO where you can code a bot to play for you. Its still in alpha but it has been pretty stable for the past few days that I have been playing it.
Starcraft isn't troops on the battlefield, in the same way that Risk, the boardgame isn't troops on the battlefield.
Just because the 1s and 0s or game pieces represent "troops" to humans, doesn't mean that the underlying game mechanics have anything at all to do with a real war.
For all we know, the problems solved by a monopoly AI, are more applicable to a real war than those solved by a Risk or starcraft AI.
You must admit that there is an underlying aggressive theme in games like StarCraft. The visuals show clear resemblance of people being killed by projectiles and by other war technology. The AI will be rewarded for skillfully decimating the enemy, for protecting the units it has control over and for coming up with strategies like ambush, raids, patrolling etc. I am more convinced by the idea that an AI will be able generalize to the real world from playing StarCraft than from playing Monopoly. However, I think it will be quite a leap to the real world either way.
> You must admit that there is an underlying aggressive theme in these games. The visuals show clear resemblance of people being killed by projectiles and by other war technology.
An AI doesn't know or care about any of this.
Literally its applying statistics to optimize a cost function or assign labels to input states.
These numbers could represent number of kills or number of dollars; the computer doesn't care.
It's just absentmindedly optimizing a cold value function! No more humanity! More more external input! Just cold machine logic!
We've doomed ourselves!!!!
But seriously. No, there is nothing inherent about an AI maximizing wins in StarCraft than one maximizing profits on the stock exchange. Although I feel like there's a bunch of Sci-Fi where this obviously goes awry.
I do research in "Deep RL" for a living. The same algorithms (and advances related to them) that can be used in such a scenario can be used to create machine learning solutions for social good.
Similarly, algorithms used to detect cancer in medical imaging can be used to do automatic targeting on killer drones. Should we stop doing research altogether?
It's going to happen anyway. I think we are better off doing such things in the open and having some visibility into it rather than be surprised when something happens.
How is 'saying no' going to stop the military of this or any other country from doing it? How do you measure the disadvantage we'd be at by not allowing our military to do it, but letting other militaries (over which we have no control) do it?
So, basically another arms race to the bottom, similar to nuclear warheads ? 1940s thinking with 2016 technology. I hear what you're saying but not all progress is good progress.
The limits of our ability to think a global scale are an issue, we still have a tribe mentality, this discussion demonstrates that. "The Chinese, The British etc"
We still think in terms of tribes, but we have technology change the world. Until we evolve some more we're probably not fit to wield more and more powerful weapons.
It would be good if we could stop worrying about weaponry all together and focus on real issues like deforestation, poverty, pollution of the sea, climate change etc.
I get it, and I wholeheartedly support your position on the issue. The reality, unfortunately, is that we are horrid little creatures always scheming and fighting for power. Stopping nation X from doing, will only encourage nations Y and Z to develop it faster as an advantage.
The only way I can see an agreement to limit the escalation of AI warfare is on the same context we saw the START treaty: once the weapons are developed by at least two key players as "deterrent" there's an incentive to reach an agreement to stop further development.
Call me cynical, but if this election has shown me anything is how tiny-minded people still command an inordinate amount of power in this country.
What are you willing to sacrifice to say no? How about your immediate comfort and free time. Thats the real cost. Its not "when it happens we'll stabd togethor", Its "youtube videos or protecting the world from a potential future threat that everyone will think Im crazy for worrying about"
I'd be curious to hear more of your argument. The world is a complex place. How do you protect the one's you love when others exist who wouldn't say no?
Legends trace the origin of the game to the mythical Chinese emperor Yao (2337–2258 BC), who was said to have had his counselor Shun design it for his unruly son, Danzhu, to favorably influence him.[64] Other theories suggest that the game was derived from Chinese tribal warlords and generals, who used pieces of stone to map out attacking positions.[65][66]
If you are going to do war you should do it well. An AI commander could be programmed to maneuver troops in a way to reduce civilian casualties, reduce damage on infrastructure, and even maximize the chances of the enemy surrendering rather than having them killed.
Of course, I doubt that an AI commander would replace a human making the actual decisions in the foreseeable future. It would just be another intelligence tool for military command. From this point of view you are essentially arguing for our military to be ignorant, I don't think that is a good bet.
This worry and the implication (control use of AI) is similar to the development of cryptography. Try to stop people from doing evil with it is a completely fruitless task.
There are better war simulators if that were the purpose. There also wouldn't be a need to publicize it. Finally, big data/A.I is already used by the military. It probably isn't "decision making" levels, but it doesn't need to be. "Decision helping" is a good enough advantage.
the press is going to make a huge clickbait deal about this, even if the complexity of starcraft 2, even if huge, is extremely small compared to the complexity of the world
This is really good news!
Lets hope DeepMind can improve even further on their Differentiable neural computers (DNC) which seems like an requirement for this kind of AI to work (exploiting long-term dependencies).
I also hope that other research/industry teams will join on the competition to create competing AIs. Very exciting!
I understand the challenge, and the importance of the proof-of-principle. But again? Having done Atari games, then Go, at what point exactly does google deepmind start attacking some real world problems of consequence?
Or maybe the answer is never, other companies are supposed to do the hard work? We only play games?
Machine learning is currently being used all over the place, in almost every single of Google's products.
Projects like these may have a "fun" goal, but while reaching towards that objective, a lot of new techniques and theory is developed, which then gets used in real products.
Systems like this are much easier to use as a playground and test in than the real world. But as they mention, the messiness and stateful nature of games like SC2 will bring a lot of innovation which will be very relevant to the real world.
What real world problems do you have in mind? Games are convenient because they can be easily simulated, and you can compare the performance to expert humans.
The reinforcement learning they work on is extremely general and can be applied to many other types of problems. Particularly robot control, which is sort of the same sort of task as a video game.
The Berkeley Overmind was designed for a 2010 Brood War competition. The website (http://overmind.cs.berkeley.edu/), although no longer updated, still contains many videos and links to articles that describe the internals of the AI and the design process. The Ars Technica article in particular goes quite in depth.
I can't wait to see how far DeepMind can go in this area. I was initially skeptical that AlphaGo could defeat top human players, but then it happened. Who knows, perhaps one day AI can compete against progamers in GSL!
Competitive AI in broodwar already can destroy the best humans. AI can use strategies and tactics that human can't effectively do. Most notable is perfect splitting which changes the game quite a bit.
>Competitive AI in broodwar already can destroy the best humans
Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.
I'm pretty excited about this. I think some kids out there will really enjoy an environment like this to mess with, and maybe learn a thing or two about machine learning along the way.
Starcraft is a really fun game, and I think it's enough to engage kids a little more than something like Minecraft where there's plenty of room for some cool ML hacking, but not enough stimulation from it. Instead of just seeing blocks here or there or whatever, starcraft has hard goals that will force them to use critical thinking skills, their knowledge of the game, their own personal strategic insights, and the ML skills they accrue.
So exciting! Love the feature layer idea also, well done!
I agree. SC2 is a very strategic game and has tons of potential for scouting, fake attacks, distractions, diversions and all sorts of deceitful play. And that's coming after having a good micro-management of units, using perfect timing for expanding your economy, upgrading your units, proxy offense, and a ton of other mechanics.
DeepMind really chose well. SC2 has to be the most demanding game nowadays in terms of strategy and execution.
DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn.
That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war’ in a strategy game.
This is a big open problem in AI research right now. I saw a post around the time of AlphaGo challenging DeepMind to tackle StarCraft next, so it is very cool that they have gone in this direction.
When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.