[Tom]

Some data

A poster on TeamLiquid.net recently pointed out the following distribution in the Korean Grandmaster league (which I posit represents the most competitive ladder):

81 players in Grandmaster. 40.5% of the total
64 players in Grandmaster. 32% of the total
45 Players in Grandmaster. 27% of the total
1 Player in Grandmaser. 0.5% of the total

Top Ten: 9 s, one and no .

Top Twenty: 15 , two and three .

Overall race distribution in 1v1 Korean ladder

It makes little sense to analyse the proportion of races in the Korean GM league in isolation from the overall race distribution. To illustrate this point, if 40.5% of all Korean players used terran, the fact that 40.5%of Korean GMs played simply means terran is"batting average".

I did a search of www.sc2ranks.com this morning using the following parameters: Region=Korean, Team=1v1, Activity=Last 30 days. I did this search to eliminate the "noise" in the data from inactive and team game players (in other words, I am only interested in the data for active 1v1 players).

The overall distribution for active 1v1 players on the Korean server is:

10.1%
34.8%
34.8%
20.4%

Analysis of the Korean data

The most striking feature of the data is the substantial under-representation of players in Korean GM league (0.5% of Korean GMs play random, compared to 10.1% of active Korean 1v1 players). Korean players are an extraordinary 20 times less likely to be in GM league compared to the general Korean 1v1 population.

While has the lowest proportion of players overall (excluding random), statistically they are the most over-represented in the Korean GM league. 27% of Korean GMs play , compared to 20.4% of active Korean 1v1 players overall. The data suggests Korean zerg players are 1.3 times more likely to be a GM compared to active Korean 1v1 players generally.

are also proportionally over-represented in the Korean GM League, with 40.5% of Korean GMs playing , compared to 34.8% of the Korean 1v1 population. This data suggests Korean terran players are 1.15 more likely to be a GM compared to active Korean 1v1 players generally.

are slightly under represented in the Korean GM League, with 32% of Korean GMs playing compared to 34.8% of the general Korean 1v1 population. This data suggests Korean protoss players are 0.92 times as likely to be a GM compared to active Korean 1v1 players generally.

The top 10 has a very high representation of (9 out of 10 at the time of writing this), but I think it is difficult to say anything meaningful based on such a small population.

Conclusion

There are many data sets one can look at in examining balance issues. It seems unlikely to me any of these will provide the final "answer". However, to the extent that it has become fashionable and indeed tacitly acceptable for zerg and terran players to complain about protoss being "overpowered", there is little in the above data to support these complaints.

Whilst the above data suggests Protoss is statistically the weakest race amongst Korean GMs, the result should be treated with caution for several reasons. Most obviously, the same analysis for GM Leagues in the other regions may paint an entirely different picture. As stated at the start, I have looked at Korean because my hypothesis is Korea is the most competitive ladder. I would really like to see similar analysis for the other ladders (I have run out of time to do it in this post!)

The second reason for caution is the Korean results suggest Protoss is the weakest race at the Korean GM level. It is possible the picture is completely different at the Master and Diamond level, for example. The above data suggests that with very good mechanics (as possessed by a Korean GM), Zerg are the strongest race. However, with the mechanics of a Diamond player, it may be that Protoss are stronger. In the absence of data, this is just conjecture.

To the extent that players attribute their perceptions about game balance to their own (anecdotal) experience, I would caution against such an approach. This type of analysis is most likely to be affected by personal biases (and people tend to remember and attach great importance to their losses, to the extent where conclusions not based on solid data risk being distorted).

Finally, in relation to recent Blizzard comments in relation to Protoss warp gate times, it is interesting that Blizzard's primary aim appears to be to alter PvP, and they are concerned that the changes should not make Protoss "weak" in the early game against the other races. We can infer that Blizzard are not particularly concerned about Protoss balance (presumably based on their own analysis of mass ladder data).

Thanks to: http://www.teamliquid.net/forum/view...opic_id=216212



*** Update - data for SEA as at 27 April 2011 ***

On 27 April 2011 I performed the exact same analysis for SEA as set out above. Of the 196 players in GM League:

60 players (30.6%)
67 players (34.2%)
62 players (31.6%)
7 players (3.6%)

The overall distribution for 1v1 players (activity = last 30 days) on SEA is:

24.9%
33.9%
32%
9.2%

Discussion of SEA results

As was the case for Korea, in SEA zerg are the most significantly overrepresented race in the top 200, relative to the proportion of zerg players overall (30.6% of GMs are zerg, compared to 24.9% of active 1v1 players overall).

Random is again significantly underrepresented (3.6% of GMs are random, compared to 9.2% of active 1v1 players overall).

Protoss and terran are both very slightly overrepresented (the difference is 0.4% for terran and 0.3% for protoss).

Comparison of SEA and Korea

Overall, the results for SEA are similar in relation to the significant overrepresentation of zerg and the significant underreprentation of random in GMs league.

However, the results differ significantly in relation to terran and protoss. The proportions terran and protoss players in SEA GMs league is very close to the overall proportion of active 1v1 players, in contrast to Korea where terran was significantly overrepresented and protoss slightly underrepresented.

The consistent trend is random and zerg. Zerg players are significantly more likely to be GMs on both servers, whilst random players are significantly less likely to be GMs on both servers. However, the difference from the average is somewhat smaller for SEA than for Korea. This is consistent with nirvana's hypothesis below that zerg is the strongest race with very good mechanics - ie, if we assume the general skill level on SEA is lower, it follows from nirvana's hypothesis that zerg is less likely to pull ahead of the other races. It probably also follows that random get a better "trot" on a less skilled server.

The big difference between SEA and Korea is terran. Terran were found to be significantly overrepresented amongst Korean GMs, but in SEA terran appears to be "batting average". This could possibly be explained by the theory that terran performs better amongst more mechanically "skilled" players, but I am not really sure. Conversely, the fact that protoss is performing better on SEA than on Korea may be consistent with the theory that protoss performs better amongst mechanically "poorer" players.

+ [Ignore This] +

Posting this image just so the article has a thumbnail, haha!

Quick Comments

nirvAnA:

excellent post, moved to articles

[cure]

Interesting statistics, although this thread would be closed if it becomes another flame/QQ thread, i hope this can remain clean with only constructive comments.

[pikk0n]

I've been waiting for someone to make a post like this for a while now. It's a really interesting and good read. I'm a little worried about protoss being vulnerable to zerg cheese if the gateway and zealot build time is increased any further. You wouldn't be able to spawn a zealot out in time to block your ramp in the event of a 10 pool. 7pool requires a forge regardless so it wouldn't matter and sacrificing a pylon to hold a 7 pool off is worth it seeing that the zerg is at a severe econ disadvantage. 10 pools are annoying though.

I'd say that terran's definitely the hardest race to play because it's so micro intensive. Zerg ranks second. I find protoss to be less micro intensive, hence the reason why some player may feel that protoss is OP but it has pretty much been this way since SC1. That's the reason why I played protoss back in the old days. The koreans and their crazy micro definitely makes terran seem so much more OP. Your protoss ball with HTs would not be EMPed but instead be sniped by those APM crazy guys. LOL! You'd most probably find 6 drops happening at once with those Korean terran GMs.

[inFeZa]

more zerg qq? god damnit

[Eldrid]

Well the random stat is an obvious answer... 3x less games as 1 race. 3x less practice. they're "good" at all the races, but not "super incredible awesome" at all 3. thus 20x less are in gm.

just sayin'

[pXAero]

I would like to see data from all the other servers alongside the korean data as SC2 isn't a korean only zone like BW is. Not really surprised by the fact that there's not many random players in the GM league.

With a new patch coming up soon I'm wondering just how much Blizzard is willing to change things, like whether they will keep making tiny continual adjustments to intentionally keep the game evolving.

[AlphaWhale]

More people play protoss and terran than zerg and random. I'm waiting for the part where I learn something. These numbers don't tell you much other that what people prefer to play. I'm surprised the numbers are that even at 40T/32P/27Z.

[mR.bONG789]

alot of pros dont ladder anymore since gsl and most other tourney's map pool are differant to the ladder one so cacluating the balance based the ladder's stats is bais

[Meatex]

Unfortunately the data is inconclusive without additional info about length of matches that were won and lost and not to mention the data would need to be adjusted to account for stronger players playing a certain race (eg MC is really strong player)

[nirvAnA]

The second reason for caution is the Korean results suggest Protoss is the weakest race at the Korean GM level. It is possible the picture is completely different at the Master and Diamond level, for example. The above data suggests that with very good mechanics (as possessed by a Korean GM), Zerg are the strongest race. However, with the mechanics of a Diamond player, it may be that Protoss are stronger. In the absence of data, this is just conjecture.

i like how this supports my belief that protoss is the easiest race to learn but also the race with the shortest skill ceilling. once u improve ur forcefields and all the other normall stuff, there isnt much room for improvement whereas zergs/terran can always do things faster and cleaner.

also supports my thought that zergs is the hardest race to master

[jason]

Hmm,

I wonder why top players like MVP, MKP, Nestea, MC thinks that protoss is the strongest race of all.

And hmm, good players don't always ladder. They train with AMAZING players in custom games.

And based on the current TSL, 3 toss 1 terran 0 zerg.

Hmm ???

[nGenLight]

Thank you for OP for at least trying to use a reasonable and significant sample size to discuss balance in a more appropriate way, I would like to see continuity in this rather than seeing small, insignificant, and obviously samples pulled from a ridiculous huge resource pool to justify whining.

In theory, I am kinda with Nirvana on this one. However I feel that Protoss has and needs that potential to play above this "Protoss Ceiling" in order to match their Terran and Zerg counterparts that are capable of playing at a higher level.

[ROOT`iaguz]

I disagree. Mechanically the ceiling to play Protoss is probably really short, but I feel that in terms of strategy and optimizing protoss is as complicated as the other two. I only did play protoss for about 40 games but that was definitely the impression I got after studying them pretty hard too, there's an awful amount of tiny inefficiencies in regular Masters Protoss play that the Pr0-tosses have managed to iron out, and could still probably iron out a bit more!

In general I'm a touch leery about just looking at ladder results to imply imbalance, especially in Korea where most people probably play tons of customs in their team houses and probably have tons of smurfs too. Blizzard would probably do well to send David Kim to Korea where he can mingle within all the team houses and really see how and what to do as far as balance goes (I believe he does speak the language, correct me if I'm wrong).

I'm not surprised to only see 1 random player though. Anyone in the Korean GM is either the Korean version of Combat Ex (grr...) or is trying to get into a team house/already is in a team house and looking to be a gosu. No Gosu plays random, it's just too difficult. TLO is just a weird example (And I think he plays straight Terran now).

[ZergGirl]

Originally Posted by jason

Hmm,

I wonder why top players like MVP, MKP, Nestea, MC thinks that protoss is the strongest race of all.

And hmm, good players don't always ladder. They train with AMAZING players in custom games.

MKP keeps his main and 2 smurfs in GM on ladder lol ^^;
that's 2 terran in the top 10, and another in the top 50.

My julycherg will teach me how to defeat pigtoss, I will wait!

[Tom]

Originally Posted by Tom

Posting this image just so the article has a thumbnail, haha

nirvana has spilled perhaps the worst kept secret on SEA, but just to confirm - yes I really am Tom Cruise.

[TASanchez]

Comparing the tiny amount at the top to the masses does not really mean anything because we cant judge the other factors affecting that distribution.

For example I would say the majority of koreans in grand master played SC1 before. Most of them just picked the same race they played in SC1. Maybe this or other factors meant a more even amount of top rts players played each race.

The bronze-diamond population you are comparing them to generally came from playing the campaigns and other rts and found it easier to start as terran or protoss.

Looking at the results like you are would make sense if everyone had just started at an even field with those race distributions and improved from there. However, since grand masters is made up of mostly players that were already top rts players before sc2 the races they chose strongly influences the race distribution at the top.

It seems much more reasonable to look at results from tournaments around the world to see which races are actually performing best at the top level in comparison to each other. Overall though to me the races seem fairly well matched at the moment.

[Tom]

Relying on small sample size of top players

Originally Posted by TASanchez

Comparing the tiny amount at the top to the masses does not really mean anything because we cant judge the other factors affecting that distribution..

I agree that any results need to be treated with caution given the size of the dataset. But I disagree that results based on a sample size of 200 players is "meaningless" just because one can't judge other factors affecting distribution. If I was really serious about this, I suppose I should perform a regression analysis to determine the correlation between race and performance, as well as whether the results are statistically significant rather than just "eye in" the data. If the results are statistically significant, it does not matter that they do not allow us to judge other factors affecting distribution.

One reason it is interesting to look at GMs League is precisely because it is at the pointy end of the bell curve. The problem with looking at the middle of the ladder leagues is that Blizzard is aiming for a 50/50 win/loss ratio, and one would expect this to mask differences in the relative "strength" of races. Namely, to maintain 50/50 win/loss, players playing Race X may be getting matched against weaker players who play Race Y.

The other reason why it is interesting to look at GMs League is that it allows us to test the hypothesis (to some extent) that certain races perform better as a player's game mechanics improve. So to use zerg as an example, it at least appears that zerg perform very well at the GM level (both in Korea and SEA), and proportionally much better than terran and protoss. This may suggest that zerg is the "strongest" race, provided you have excellent game mechanics. (Please do not take me to expressing this conclusion - I am just saying it seems to me the best way to test the hypothesis is to look at a dataset of the "best of the best".)

Incidentally, both these reasons apply a fortiori to examining data from top level tournaments. But (for the reasons below) I actually feel looking at the GM League dataset is more meaningful than looking at tournament results.

Which sample to use - GMs League or Tournament Results?

Originally Posted by TASanchez

It seems much more reasonable to look at results from tournaments around the world to see which races are actually performing best at the top level in comparison to each other. Overall though to me the races seem fairly well matched at the moment.

As you rightly point out, looking at top level tournament results achieves the same thing. But I think it suffers from the following problems:

First, the tendency of people to look at very selective/small samples (eg if I play protoss, I say "look as GSL - two terran final!", if I play terran I say "look at Dreamhack, two protoss final!", if I play zerg I say both (ignoring eg SEASL which was stacked with zerg). I want to be clear - I think each of these approaches is wrong. They are based on a sample size so small and selective as to be meaningless for the purpose of generalising the results to a wider population. I know you are not saying this Sanchez, but there are a lot of people who trot gems out from time to time, with no regard random variation amongst other things.

Second, and related to the above, I actually think the sample size for top level tournament results may be too small as to be meaningful. At least Korean GMs League gives us a sample size of 200 players (many of whom appear to be playing hundreds of games per week) and probably tens of thousands of games. In contrast, looking at recent tournament results give us perhaps a sample size of perhaps 24 to 50 players and a few hundred games at most.

Third (and most important of all I feel), a dataset based on top level tournament results is most likely to be affected by extreme outliers in the distribution. In layman's terms, the results are most likely to be skewed by players who are just plain "freaks" of nature when it comes to playing RTS games (or who happened to have a good day!). Hence, the results are likely to have least to do with balance (although it obviously counts to some degree), and most to do with absolute player skill. The data from Korea GM League probably also suffers from this problem, but I would say to a more limited extent.

[CZE]

T is doing better on KR compared to the rest of the servers because of the way the race lends itself to players with near-flawless mechanics. P does better on every other server because of the lower mechanic requirement. It's not until that super-top level that you see a large difference (compare players like White-Ra/nani to cheesy NA P's like incontrol)

[TAavanar]

Originally Posted by CZE

T is doing better on KR compared to the rest of the servers because of the way the race lends itself to players with near-flawless mechanics. P does better on every other server because of the lower mechanic requirement. It's not until that super-top level that you see a large difference (compare players like White-Ra/nani to cheesy NA P's like incontrol)

i find it funny u think that incontrol is cheesy, can i ask you why u think that?

[Tom]

For those that haven't seen the thread, I recommend you check out http://www.teamliquid.net/forum/view...opic_id=218558

The author has plotted the data from the TLPD database - ie TeamLiquid statistics for tournament results (excluding MLG Dallas) at the International level and at the Korean level.

It appears that the international level, tournament results are approaching close to a 50% winrate for each match up. However, in Korea Protoss has been getting absolutely murdered in recent tournaments, with win rates of 33% of ZvT and 30% of ZvP. The Korean data set is a much smaller sample size, so you would expect some volatility, but even so, the results are quite extreme. I guess the real question will be whether this is an outlier or the beginning of a trend - perhaps we will see some reversion to the mean in the data for next month.

If there is a real difference between Korea and the rest of the world, I am really curious to know what is driving this. Currently, Protoss appears to be fairly even in international tournaments, but almost unplayable at a professional level in Korea.

[Noradrenaline]

Originally Posted by avanar

i find it funny u think that incontrol is cheesy, can i ask you why u think that?

I bet he's still hungup on what incontrol did to TLO (was it MLG dallas?) with hidden 4gate and another cheese... he was 'an apologetic winner'

But, if you watch his stream and more general games I'd say hes a pretty reliably 'standard' protoss, cheese is more effective when unexpected.

As to the racial disparities, well, Korea has had a very long lineage of S-Class terrans and a general over-representation since early broodwar days, Boxer Nada and Flash have been inspirational to many, and it seems to be a very 'successful' race with a significant contingent of the old guard carrying their race into the new game. I remember earlier in the GSL seasons Boxer said nearly all his matches were tvt and that he had to ask for practice partners in order to prepare for a tvz match since he never encountered many.

I'm inclined to agree with most of the above analysis concerning protoss, and it seems fairly evident to say Zerg has the steepest learning curve and the most onerous requirements for success at a given level with limited early-mid game information, a wide variety of cheese and rush builds to survive and a constant challenge to sustain optimal drone timing and macro efficiency in spite of the above. When those challenges are met, the true power of the swarm is then vivid. Random was simply never a serious competitive choice, learning 9 matchups over 3 requires genius level talent and that kind of capability will always be more successful chanelled into one race alone; broodwar and wc3 were no different.