On Facebook and Twitter in China, and Getting the Numbers Right


image: FastCompany.com

A number of publications have been citing Global Web Index research that claims Facebook and Twitter have 63 million and 35 million users respectively. We thought those figures smelled funny, so we stayed clear. But those numbers are being widely reported by big name publications, and I think that’s regrettable [1].

I’ve exchanged a few emails with the folks at Global Web Index regarding the Twitter user count [2], just to question their methodology, and to their credit they were very receptive and made a few good points [3]. They tell me their results are survey-based from a sample of 8,000 total respondents [4], with surveys conducted in Chinese. The following is parts of my email correspondences, adjusted and supplemented to be posted.

How many actually climb the wall?

Like many, my main concern was that Twitter is blocked in China and that circumvention tools were not used widely enough to support such high user counts. The Berkman Center published research back in 2010 about the use of circumvention tools in China, and a the time they gave a generous estimate of no more than three percent, qualifying that the “actual number is likely far less.”

So perhaps considering that this is now two years later, we could be generous again that say that “far less than three percent" has now reached three percent [5]. And if we are again generous and assume that all of those users are using those circumvention tools to use nothing but Twitter, that would mean about 16 million Twitter users. Again, still far less – and that’s being generous.

Notice how many times we had to be generous there, and we still didn’t reach half of GWI’s 35 million figure.

Comparing to Sina Weibo

Sina Weibo had about 300 million registered users as of last May. Those figures came from Sina itself, with no mention of what portion were active. But…

  • If we are generous and say that 100 percent of them are active, it would mean that there is about one active Twitter user for every 10 active Sina Weibo users in China (35 million to 350+ million).
  • If we are conservative and say that 50 percent of those 300+ million are active, it would mean almost one active Twitter users for every four active Sina Weibo users (35 million to 150+ million).

Neither of those two estimates seems realistic to me. Twitter doesn’t have the same visibility in China that it has in the West. It is not a part of mainstream vocabulary (Weibo is), it is not plastered all over the Chinese internet in the form of share buttons (Weibo is), and it is blocked. If it could reach 10 percent of Sina Weibo’s user total despite all that, it would be a miraculous feat indeed.

Can we measure language use?

We could also consider examining the languages used on Twitter, which has been done to some extent by Eric Fischer in this visualization from late 2011. In the graphic, you’ll notice that China is not very active, especially when compared to neighboring Japan. But Italy for example (which GWI estimated to have 3.1 million active Twitter users) is far more ‘visible’/active. There are numerous other examples too. This is hardly conclusive, but again, another indication that China’s Twitter count is very low.


A final note

It’s entirely possible that the use of circumvention tools is much higher than we think, and that more and more people in China are accessing Facebook and Twitter. But I think we have to call bullshit that the numbers are anywhere near the level that GWI claims [6]. While none of the facts listed above disprove their data, collectively they certainly support the case that the figures from this extrapolated survey are off-base.

But perhaps the biggest argument to be made against GWI’s figures, and what should throw huge red flags to anyone even remotely familiar with China’s internet, is that the average Chinese user does not really give a fuck about internet censorship, nor about getting on Twitter or Facebook.

Admittedly, we’ve cited survey-based research in the past too, and while I don’t think any of it was off base like this GWI research, I expect we might be even more careful moving forward. We still have a lot to learn about which information we can trust. But I’m glad to say that our team makes a good effort to be consistently accurate over just being quick to publish.

  1. The Next Web has done a 180 on GWI, and rightfully so I think. The update to the original post leaves much to be desired, however, if it’s intended to stop misinformation, as there is not much of a disclaimer/correction up front besides a note to say see below for ‘additional statistics for context and comparison.’  ↩

  2. The Twitter user count was making headlines two days ago, which was when I started questioning GWI. I believe first from the folks at The Next Web, via eMarketer. The Facebook user count is in the news today, most notably Bloomberg and CNet (with some questioning).  ↩

  3. Perhaps the best point was that if Chinese users were logging in to social networks via a VPN or proxy server, they would not show up in measurements recorded by organizations like Semiotics, as their IP would appear to be a foreign one.  ↩

  4. GWI’s full explantation:

    Our methodology is survey based. We surveyed a representative sample of more than 8k (total) respondents in China over 7 Waves of research from 2009. Our panels are provided by LightSpeed, the market standard for Online research, and recognised as the more credible panel provider.

    I also questioned about whether surveys were representative of China’s entire population, as opposed to just urban respondents. I’m told that of the total figure, 1.85 million are from rural areas, 3.71 are from suburban areas, and 29.91 from urban areas. It makes me a little nervous that they extrapolated to within two decimal points. I would think there is some law of significant digits being broken here!  ↩

  5. A GWI representative casually cited the Harvard study, noting the three percent estimation, and saying that this is “likely to be 10 percent” now that two years have passed. But that remark was made casually, and I’m not certain that I want to hold them to that 10 percent claim.  ↩

  6. GWI also offered to discuss further, which was nice of them. I’m still awaiting a reply on some of the points I made above, although unless Twitter and Facebook actually release some stats about China users (which is unlikely), I’m not sure what we can actually resolve. GWI appears comfortable with extrapolating the survey results, while myself – and apparently many others – are not.  ↩

(And yes, we're serious about ethics and transparency. More information here.)

Read More