Can You Trust BuiltWith, W3Techs and Software Usage Statistics?

Can-you-trust-BuiltWith-W3Techs-and-software-statistics

The Drupal Association put out a press release today, quoting some statistics about Drupal.

The press release said:

  • According to Drupal.org stats, Drupal now powers more than 1 million websites.
  • According to Builtwith Research, Drupal powers more than 12% of the world’s top 100,000 websites.

I regularly see stats like this fly around the web, particularly on Twitter.

So, I decided to use these Drupal stats as an example and answer the question: “Can you trust software usage statistics?”

Sources of usage statistics

There are three websites that collect very large amounts of information available about Drupal:

  • Drupal.org
  • W3Techs.com
  • BuiltWith.com

There are some smaller sites such as Wappalyzer and TopDrops, but I excluded those because either have much smaller data samples than the big three or didn’t present their data in ways that could be easily compared.

Out of these three sites, not all the information is equally valuable. It’s worth giving priority to sources who are closest to the actual software installs. Because Drupal.org connects directly to actual Drupal sites, its data is probably the most reliable.

I would give the least credence to BuiltWith. Both Drupal.org and W3Techs publish their methodology, but BuiltWith has only the following, empty statement:

“Every website gives off “signals” that they are using a particular technology. We can track these signals and determine if a site is using a particular web technology from it.”

Question: How many sites use Drupal?

media_1393955131863.png
  • Drupal is used by 1 million websites according to Drupal.org.
media_1393955171282.png
  • Drupal is used by 796,045 websites according to BuiltWith.

Answer: These two numbers agree fairly closely, so we can estimate that Drupal is used by close to a million websites.

Is Drupal usage growing or shrinking?

media_1393955043492.png
  • Drupal use is trending up slightly (from 800,000 to 934,000 sites during 2013) according to Drupal.org.
media_1393954938260.png
  • Drupal use is trending up very slightly according to BuiltWith.
media_1393955011244.png

Answer: Taking an average of the three sites, Drupal usage is trending slightly up.

What percentage of the web is using Drupal?

media_1393955358275.png
  • Drupal is used by 0.3% of all websites, rising to 3.9% of the largest 10,000 websites according to BuiltWith.
media_1393955425495.png
  • “Drupal is used by 5.4% of all the websites whose content management system we know. This is 1.9% of all websites.” W3Techs

Answer: This is all over the place, but the midpoint of these averages is just over 2% with a noticeable lean towards large sites.

Conclusions on the Drupal press release statistics

So what’s the verdict on the Drupal press release?

  • Yes, Drupal probably does power around a million websites.
  • The 12% figure is odd. I’m not sure where it came from (BuiltWith says 3.9%). Even if accurate, it doesn’t reflect the other statistics on the percentage of the web using Drupal.

Conclusions on usage statistics in general

I started this exercise as a sceptic but was pleasantly surprised by the fact that could extract some general agreement from these different sources.

There were at least three points where we could get some consensus:

  1. The total number of sites
  2. The general trend in usage
  3. The general market position

The one point where the results were radically different was the % of the web using Drupal. I wouldn’t feel comfortable quoting any of that data. I suspect that BuiltWith and W3Techs have radically different ideas about the size of the internet.

However, it’s worth remembering that even the closest answers differed by at least 25%. The data gets even more wild when we start to include other sites such as Wappalyzer.

If you’re doing this research, always get at least two sources of data, and if possible three or more.

Author

  • Steve Burge

    Steve is the founder of OSTraining. Originally from the UK, he now lives in Sarasota in the USA. Steve's work straddles the line between teaching and web development.

0 0 votes
Article Rating
Subscribe
Notify of
4 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Robert Vining
Robert Vining
9 years ago

Hey Steve, if you notice the dip on W3Tech’s usage between May and June of 2013, there is a press release on their blog that states they started using a larger sample for their numbers.
Basically, W3Tech started out sampling 1 Million sites in the beginning, to gather their stats. Then, mid 2013, they moved to a 10 Million site sample for their data. Both WordPress and Joomla received a boost to their overall usage numbers when this occurred, however, Drupal’s usage number dropped a bit. Further enforcing the premise that Drupal tends to power the upper end of the market share.
I have 16 Month’s worth of monthly stats breakdown on Joomla using W3Tech and Builtwith as my basis. Granted, the numbers may not be exact, but used as the basis for statistical data over a time period, show growth or decline over that same period.
Thanks for the write up on this!

steve
steve
9 years ago
Reply to  Robert Vining

Thanks Robert. That is useful to know.
So w3techs changed their sample size 10-fold and caused a noticeable change in all their results … but don’t note that on any of the graphs.
That kind of fact should increase our skepticism further.

Jon Hearty
Jon Hearty
9 years ago

Very interesting post, Steve. Datanyze shows Drupal with 2,868 websites in the top 100k websites based on Alexa rank: [url=https://www.datanyze.com/market-share/CMS/?selection=1]https://www.datanyze.com/ma…[/url]

steve
steve
9 years ago
Reply to  Jon Hearty

Thanks Jon. Your site proves the articles’s point to some extent.
Your own site is showing 2.8% for the top 100k sites. Compare that to 3.5% for BuiltWith.
Those numbers agree about as much as we could hope for.

4
0
Would love your thoughts, please comment.x
()
x