On Facebook and Twitter in China, and Getting the Numbers Right

Rick Martin
4:30 pm on Sep 28, 2012

image: FastCompany.com

A number of publications have been citing Global Web Index research that claims Facebook and Twitter have 63 million and 35 million users respectively. We thought those figures smelled funny, so we stayed clear. But those numbers are being widely reported by big name publications, and I think that’s regrettable [1].

I’ve exchanged a few emails with the folks at Global Web Index regarding the Twitter user count [2], just to question their methodology, and to their credit they were very receptive and made a few good points [3]. They tell me their results are survey-based from a sample of 8,000 total respondents [4], with surveys conducted in Chinese. The following is parts of my email correspondences, adjusted and supplemented to be posted.

How many actually climb the wall?

Like many, my main concern was that Twitter is blocked in China and that circumvention tools were not used widely enough to support such high user counts. The Berkman Center published research back in 2010 about the use of circumvention tools in China, and a the time they gave a generous estimate of no more than three percent, qualifying that the “actual number is likely far less.”

So perhaps considering that this is now two years later, we could be generous again that say that “far less than three percent" has now reached three percent [5]. And if we are again generous and assume that all of those users are using those circumvention tools to use nothing but Twitter, that would mean about 16 million Twitter users. Again, still far less – and that’s being generous.

Notice how many times we had to be generous there, and we still didn’t reach half of GWI’s 35 million figure.

Comparing to Sina Weibo

Sina Weibo had about 300 million registered users as of last May. Those figures came from Sina itself, with no mention of what portion were active. But…

  • If we are generous and say that 100 percent of them are active, it would mean that there is about one active Twitter user for every 10 active Sina Weibo users in China (35 million to 350+ million).
  • If we are conservative and say that 50 percent of those 300+ million are active, it would mean almost one active Twitter users for every four active Sina Weibo users (35 million to 150+ million).

Neither of those two estimates seems realistic to me. Twitter doesn’t have the same visibility in China that it has in the West. It is not a part of mainstream vocabulary (Weibo is), it is not plastered all over the Chinese internet in the form of share buttons (Weibo is), and it is blocked. If it could reach 10 percent of Sina Weibo’s user total despite all that, it would be a miraculous feat indeed.

Can we measure language use?

We could also consider examining the languages used on Twitter, which has been done to some extent by Eric Fischer in this visualization from late 2011. In the graphic, you’ll notice that China is not very active, especially when compared to neighboring Japan. But Italy for example (which GWI estimated to have 3.1 million active Twitter users) is far more ‘visible’/active. There are numerous other examples too. This is hardly conclusive, but again, another indication that China’s Twitter count is very low.

chinese-twitter

A final note

It’s entirely possible that the use of circumvention tools is much higher than we think, and that more and more people in China are accessing Facebook and Twitter. But I think we have to call bullshit that the numbers are anywhere near the level that GWI claims [6]. While none of the facts listed above disprove their data, collectively they certainly support the case that the figures from this extrapolated survey are off-base.

But perhaps the biggest argument to be made against GWI’s figures, and what should throw huge red flags to anyone even remotely familiar with China’s internet, is that the average Chinese user does not really give a fuck about internet censorship, nor about getting on Twitter or Facebook.

Admittedly, we’ve cited survey-based research in the past too, and while I don’t think any of it was off base like this GWI research, I expect we might be even more careful moving forward. We still have a lot to learn about which information we can trust. But I’m glad to say that our team makes a good effort to be consistently accurate over just being quick to publish.


  1. The Next Web has done a 180 on GWI, and rightfully so I think. The update to the original post leaves much to be desired, however, if it’s intended to stop misinformation, as there is not much of a disclaimer/correction up front besides a note to say see below for ‘additional statistics for context and comparison.’  ↩

  2. The Twitter user count was making headlines two days ago, which was when I started questioning GWI. I believe first from the folks at The Next Web, via eMarketer. The Facebook user count is in the news today, most notably Bloomberg and CNet (with some questioning).  ↩

  3. Perhaps the best point was that if Chinese users were logging in to social networks via a VPN or proxy server, they would not show up in measurements recorded by organizations like Semiotics, as their IP would appear to be a foreign one.  ↩

  4. GWI’s full explantation:

    Our methodology is survey based. We surveyed a representative sample of more than 8k (total) respondents in China over 7 Waves of research from 2009. Our panels are provided by LightSpeed, the market standard for Online research, and recognised as the more credible panel provider.

    I also questioned about whether surveys were representative of China’s entire population, as opposed to just urban respondents. I’m told that of the total figure, 1.85 million are from rural areas, 3.71 are from suburban areas, and 29.91 from urban areas. It makes me a little nervous that they extrapolated to within two decimal points. I would think there is some law of significant digits being broken here!  ↩

  5. A GWI representative casually cited the Harvard study, noting the three percent estimation, and saying that this is “likely to be 10 percent” now that two years have passed. But that remark was made casually, and I’m not certain that I want to hold them to that 10 percent claim.  ↩

  6. GWI also offered to discuss further, which was nice of them. I’m still awaiting a reply on some of the points I made above, although unless Twitter and Facebook actually release some stats about China users (which is unlikely), I’m not sure what we can actually resolve. GWI appears comfortable with extrapolating the survey results, while myself – and apparently many others – are not.  ↩

(And yes, we're serious about ethics and transparency. More information here.)

Replies
  • Marcello

    This is GlobalWebIndex point of view in regards to their research in China:

    http://globalwebindex.net/thinking/china-the-home-to-facebook-and-twitter/

  • http://thenextweb.com Jon Russell

    Hey Rick,

    There she blows…I was waiting for you to (rightly) publish something on this issue, given our shared appreciation for scrutinizing statistics.

    Agree with your points, as you may have already read. It is definitely a concern to see an 8,000 person survey used to make big claims about an Internet population that it is in excess of 500 million.

    That said, I don’t have a problem with people citing this information per se, but I am uncomfortable when it is used without suitable comparison or challenge. That was the point of our update (which btw has now been updated to include our ‘full debunk’ – not enough hours in the day!) and we positioned the first article as ‘this is what a report is saying’, rather than ‘hey guys, did you see how many Twitter users China has’. Hence why we added the extra info and then went to the length of a fuller post when GWI published its blog post with extra details overnight.

    It’s kind of a shame that you position our second link in the annotations below, but I do appreciate the recognition – we were very keen to get the data debunked since (bar our disclaimer) no-one else was asking questions online (at least on blogs) and that’s obviously important when news is breaking out via Bloomberg and other high profile channels.

    Cheers
    Jon

  • http://www.1rick.com Rick Martin

    Thanks Jon,

    After my email correspondence with GWI, I had hoped to not even chime in on this at all. The Bloomberg piece was the catalyst though — so for better or worse, there’s my two fen.

    Yeah, I tried to avoid naming names in the body text (hence the relegation to footnotes), as I thought it would distract from the issue at hand. Though as I said, I think the original update on TNW was very weak in terms of addressing the issue and delivering the new information to readers up front. The second piece was more encouraging, for sure.

    Thanks for swinging by!

    Rick

  • http://www.leonacraig.com Craig Mattoli

    First, as a physicist, I hate to see extrapolation done with small samples. That said, I think that you would be surprised how manyn people in China do use VPN’s. There are free ones, and I was and am still surprised by the demographics of the people who use them. So, while the large number may not be correct, it may turn out that your small numbers are an underestimate. I teach at a large University in China, and I have students telling me that they’ve checked me out on FB and Google, and that’s not surprising, but the many people who I’ve encountered who have never gone to college and are using the free VPN’s blows my mind.

  • http://lighthouseinsights.in Prasant

    Surveys and their numbers are always going to be looked upon with lot of suspicion. Glad to see Rick that you coming up with better thoughts on the whole issue.

  • http://blog.hootsuite.com DaveO from HootSuite

    Still confused why number of Twitter users is China is the question they are trying to answer … how about studying how Chinese audience use social and why they use the tools they do. Now that i’m using Sina Weibo a bit at HootSuite, i see a compelling product and an audience using social in whole new ways compared to western tactics.

Read More