The Dangers of Online Polling


Warning: This is an editorial article. You are allowed to disagree with my opinions.


Since I’m finally finished with my Wario epic (Go read it!), I decided to start doing active research into who Japanese users are requesting online. This will be a sequel to the highly popular, often sourced Perception of Smash DLC in Japan. Before I begin, I wanted to clear my thoughts on the Smash Ballot, the effectiveness of polling and the size of our community.

First, as SmashChu said in his article Smash Ballot: More Suggestion than Popularity, the Smash ballot is NOT a popularity contest. It’s a suggestion box. The notion that someone is going to “win” is completely unfounded.


Meet the true ballot "winners".
Meet the true ballot “winners”.

Furthermore, it’s highly unlikely that Nintendo or Sora LTD will release the results. The only Smash Ballot data that we’ve received was the If There Were a Smash 2 PolI way back in 1999. Melee’s and Brawl’s Smash Ballot were never released. In that case, some of the top characters were not chosen because the choices were uninspired, or the series was over-represented. Look at the “Retro character” selection. The Ice Climbers were on the bottom of the polling, yet they were chosen over Sukapon, who had to most votes. but was originally delegated to a mere item. Votes matter but not to the extent that many believe. Simply put this isn’t like voting for prom queen or king.

In addition, more votes does not equal better data. If you interview 10 or 40 people holding “Frosted Flakes” cereal in the grocery store the majority of people are going to say they enjoy eating “Frosted Flakes”. Accurate polling is actually very difficult. You need to try to actively search out a variety of demographics and opinions. With polling there’s always a margin of error anyway, so we really shouldn’t take our attempts at polling as the ‘word of god’. By posting polls in Smash communities, we are only getting the hardcore Smash community’s opinion. That’s not a variety of opinions, that’s an echo-chamber. In the end, the only poll that matters if the official Smash Ballot and no matter how hard we try we can never recreate the votes that it received.  

Our polling is not 100% effective. If you look at a variety of polls, they all give slightly different data. That’s because within each community there is bias that is clearly evident through polling. If I had to really simplify the wishes of each of the communities: Wonder Red is popular on NeoGaf, Daisy on Tumblr, Shovel Knight on r/smashbros, King K. Rool on SmashBoards, Chrom on Miiverse. None of these communities by themselves represent the wishes of the Smash community as a whole, and these characters represent which characters get the most “feverish” support.

Your ballot choices are g-g-g-g-great!
Your ballot choices are GRRREAT!

To make an analogy, it’s like polling the hardcore Democrats (Or Republicans) about who they want to see President. Sure, they have strong views on what makes a good candidate but (theoretically) they need to pick someone who will appeal to the moderates. Otherwise, their candidate will lose the popular vote as they aren’t aiming for the middle.

Right now, the Smash community is not doing that. We are picking characters that we want. This is an issue, because the community isn’t as large as we think it is. We are grossly exaggerating our importance. Chris Pranger recently discussed this on the Part-Time Gamer podcast. Go listen to it, as it clears a lot of misconceptions about Nintendo’s relationship and the hardcore community.

“People are like, “why don’t you support the tournament scene? support the hardcore players?” Because– you guys are loud and like ever present but you don’t make up a great enough number to justify cost on this game.”

Even though he is talking about the tournament scene, his sentiments can be easily applied to the speculation scene too. I would highly suggest reading SmashChu’s article, Econ 102: Opportunity Cost and Character Selection which has been highly influential in forming my thoughts. Whoever is picked from the Smash Ballot needs to sell. DLC needs to be popular among all gamers, not just the people who talk about Smash all day.

Is it good?
Apparently…this is a movie? Never heard of it.

Ryu is a perfect example of this. In my Mega Smash Poll, Ryu placed 39th out of 50th. Viewtiful Joe and Crash Bandicoot placed higher than Ryu. Even if we looked at “likelihood” (Wants/ potential), Ryu was thought to be extremely unlikely. I know there was that misconception of the “No Fighting Game” quote, but I’m asking the Smash community to exercise practicality in our expectations. From one super fan to another, please remember that we are super fans, and our opinions do not represent the “moderates” or the general public.

PushDustIn continually pisses off the fanbase. If you would like to give moral support, or just argue with him you can follow him on Twitter.

Share this!


  1. Something about the No Fighting Character rumor.

    I once read a Japanese thread about Smash DLC shenanigans and one said something like:
    “Ryu was chosen despite Sakurai said no characters from fighting game”

    I’m not even sure why this happened, but like many of those (including the one about ballot as I told you in the translation conversation) they are flat out ignored by the rest of the thread.

    Not trying to break anything here, but just why did that happen? Are there misconceptions that went to the Japanese even though they have the original source in their language?

    1. It was a misconception/misunderstanding people had on what Sakurai said last year or so. He mentioned that adding in fighting game characters aren’t that interesting as they already are a fighter character. He prefers the more interesting additions to the roster.

      However, fans misinterpreted as “No fighting game characters allowed” rather than Sakurai is just not as interested in them.

      He managed to get Ryu working by adding those fighter inputs and the double final smash. If not for that, Ryu would probably be very uninteresting.

      1. Like a lot of things Sakurai says about the game, it’s something that should be taken more as a general idea than a strict rule. He’d probably have been far less interested in Ryu without his unique qualities – though he’s still a gigantic icon of the entire medium known all over the world, and that puts him ahead over all the other Capcom characters.

        His comment about “series without a future” is also like that; clearly there are series like that in Smash (or they were at one time, like Metroid), but he thinks it’s worthwhile to emphasize characters who are more likely to continue and be more well known to a general audience. “Golden Sun” on the Virtual Console doesn’t “lock” Isaac’s chances and the reception to “Dark Dawn” doesn’t kill them, but the series becoming less prominent generally makes him less ideal of a choice.

        I think we as a community tend to latch onto specific things – the various unofficial polls, for instance, or current releases – without considering that we’re only getting a small part of a very complex picture.

      1. Okay, thanks for clarifying. That explains it then.

        So you mean, The quote is translated from Japanese to English, released to public, and translated back to Japanese for them? That’s like those “bad translator” shenanigans lol.

        1. GamesTM interviewed Sakurai through a translator. They used the translation of what he said.

          Then they broke up the quote; half of it appears in the article half of it appears under the header ‘what about characters from fighting games’ (something like that).

          The two pieces aren’t pieced together, and the second half is translated back into Japanese by forum users not realizing that half of the context is missing as it was reported like that.

          It’s a lot of shenanigans!

    1. There’s been a lot of additional content besides the “suggestion box not ballot” propaganda. We released a 27 page article on why Wario is the way he is in Smash, Sakurai’s thoughts on Smash 64 and Melee, translations of 2 pages of the Smash 64 project plan and a translation of Sakurai’s latest Famitsu column…all in the last week. The “suggestion box not ballot” type posts accounts for less than 10% of all content we posted in terms of time and effort for the last week.

      As I stated in the article this is a precursor to my research into who is heavily requested on Japanelse message boards. It’s more about how the community can’t fully expect our polls to be accurate because Nintendo is looking at very different data. Some of characters we want (like Isaac) are lacking general popularity which might make them unlikely choices for Dlc.

      Soon we will be posting who we all expect to get in through the Smash Ballot. All of the writers have different opinions, so it should be interesting to read the rational behind everyone’s choices. We have a lot more content planned, so stay tuned.

  2. Very important text. One text that I would love to read from Source Gaming is an actual research on how big the competitive Smash competitive is.

    I just never see a number bigger than 200k. That’s as high as tourney streams went in viewers, that’s about what Smashboarsd and /r/smashbros have in users and the likes of VGBootCamp is also at around 100k subscriped.

    I mean, everyone knows the community isn’t that big when you look at the whole picture, but 200k against more than 10 million copies sold between the two Smash 4 games is a tough comparison. The voice that wants to be heard the most apparently doesn’t even make 3% of the userbase!

    So I would like some clarification on that, Source Gaming style. I’m sur the community would be pissed of to no end but it would surely help them understand that Smash is aimed at a much, much bigger audience that has different opinions.

  3. Thanks for the article, PushDustIn. The hardcore Smash community needs to hear the truth, even if it’s what they don’t want to hear. Despite not being quite so, this is why I was telling countless people on Miiverse “Your DLC suggestion isn’t likely to happen and here’s why…”

    Though I have my doubts on the influence on the hardcore community, it may be a bit more than one may think in relation to the ballot. Even if X% of the community is hardcore, how many remaining Y% is even aware of the ballot, and those that are aware, how many care enough to vote? And for the ones that do vote, how many of those votes can be taken seriously? (IE non-Spongebob votes.)

    It’s a tough subject to tackle, especially without data. So that said, all we can do is speculate like the speculators we’re speaking of.

    Looking forward to those upcoming articles too.

Leave a comment below!