WICS: A Usability Score for Web Software

wics

Over the last decade, I’ve spent my days teaching people how to use web software.

Some of that software has been easy to teach – students seem to understand the interface quickly.

Other software has been painful to teach – I’ve had to teach students to master long, complicated processes.

But, those were just feelings based on my limited experience. I couldn’t prove that one system was easier to use than another. I couldn’t prove that there were easy ways for this software to become more usable.

So, after a lot of thought, I’ve develped WICS: a usability score for web software. Allow me to present the first draft of WICS …

What is the WICS system?

The WICS system is based on tasks inside the software. Some examples are:

  • Add an image to your content
  • Add a new user to your site
  • Categorize your content

I’ve scored each task based on 4 criteria:

  • Words: How many new or unusual words must the user learn?
  • Ideas: How many new or unusual ideas must the user learn?
  • Clicks: How many clicks must the user make?
  • Screens: How many screens must the user visit?

Each task produces its own score. The scores can be added up to produce a total for the software platform.

There are other usability scores out there such as the System Usability Scale, but those are based on user emotions (the Likert scale). I wanted something more practical, more heuristic.

Word, Ideas, Clicks, Screens = WICS. Let’s jump in and I’ll show you how WICS works.

About these examples

  • I’ll use 5 examples from WordPress, Joomla and Drupal, because that’s the software I know best.
  • Don’t worry too much about the exact score. The difference between 11 and 12 or between 40 and 41 is not important. I believe this system is most useful for identifying pain points, rather than giving scientifically accurate results.
  • I’m using the default installs of each software. Yes, each platform can be greatly simplified by an expert, but that’s a different issue.

Example #1. Creating Content

Let’s start off with an easy example: creating content. I’ll start from the viewpoint of someone logging into the system for the first time.

With all of the platforms, you can create content very easily:

WordPress scores 6:

  • Words: 2 (Posts and Pages)
  • Ideas: 1 (The difference between Posts and Pages)
  • Clicks: 2 (Add New > Publish)
  • Screens: 1

Joomla scores 5:

  • Words: 1 (Articles)
  • Ideas: 1 (Articles are how Joomla stores content)
  • Clicks: 2 (Add New Article > Save & Close)
  • Screens: 1

Drupal scores 8:

  • Words: 2 (Article, Basic Page)
  • Ideas: 1 (The difference between Article and Basic Page)
  • Clicks: 3 (Add content > Article > Save)
  • Screens: 2

Example #2. Adding Images

Let’s take another example: adding images to content. I’m going to start from the viewpoint of someone who is in the process of creating an article.

WordPress scores 6:

  • Words: 1 (Media)
  • Ideas: 0
  • Clicks: 3 (Add Media > Drag the image > Insert into post)
  • Screens: 2

Joomla scores 8:

  • Words: 0
  • Ideas: 0
  • Clicks: 6 (Image > Choose Files > Open > Start Upload > Select the image > Insert)
  • Screens: 2

Drupal scores 5, assuming you use Article. If we used Basic Page the score would be over 20:

  • Words: 0
  • Ideas: 0
  • Clicks: 3 (Choose file > Open > Upload)
  • Screens: 2

Example #3. Adding Users

Let’s take a third example: adding users. This actually turns out to be easier than creating content:

WordPress scores 3:

  • Words: 0
  • Ideas: 0
  • Clicks: 2 (Add New > Add New User)
  • Screens: 1

Joomla scores 3:

  • Words: 0
  • Ideas: 0
  • Clicks: 2 (Add New User > Save & Close)
  • Screens: 1

Drupal scores 5:

  • Words: 1 (People)
  • Ideas: 0
  • Clicks: 3 (People > Add User > Create new account)
  • Screens: 1

Example #4. Adding Extra Features

Now we really get into the differentiators. How hard is it to add extra features to the site?

WordPress scores 6:

  • Words: 1 (Plugins)
  • Ideas: 0
  • Clicks: 2 (Add New > Search Plugins > Install Now > OK > Activate Plugin)
  • Screens: 3

Joomla scores 6:

  • Words: 1 (Extensions)
  • Ideas: 0
  • Clicks: 2 (Extension Manager > Click the Extension > Install > Install)
  • Screens: 3

Drupal scores 17:

  • Words: 1 (Modules)
  • Ideas: 1 (Modules are outside the site on Drupal.org)
  • Clicks: 9 (Drupal.org > Copy URL > Modules > Install new module > Paste URL > Install > Enable > Check the box > Save configuration)
  • Screens: 6

Example #5. Organizing Content

Let’s look at organizing content. Again, I’m going to take the perspective of a user who is writing content.

WordPress scores 5. You can organize your content at the same time as you are writing.

  • Words: 1 (Categories)
  • Ideas: 1 (Categories are used to organize content)
  • Clicks: 2 (Add New Category > Add New Category)
  • Screens: 1
media_1395167727505.png

Joomla scores 12. You need to leave your content writing to create categories:

  • Words: 1 (Categories)
  • Ideas: 1 (Categories are used to organize content)
  • Clicks: 5 (Save & Close > Add New Category > Save & Close > Article Manager > Edit)
  • Screens: 5
media_1395168798441.png

Drupal has a score of about 41.

  • Words: 4 (Taxonomy, Vocabulary, Terms, Term Reference)
  • Ideas: 2 (Vocabularies contain terms, Terms are added via fields)
  • Clicks: 18 (explanation below)
  • Screens: 17 (explanation below)

Structure > Taxonomy > Add vocabulary > Save > Add terms > Save > Structure > Content types > Manage fields > Add new field > Select a field > type > Save > Choose Vocabulary > Save field settings > Save settings > Find Content > Edit.

media_1395168331398.png

This was by far the hardest Joomla and Drupal task.

Takeaways

  • WordPress was the easiest system for these tasks. WordPress had a total of 26, Joomla had 34 and Drupal had 76.
  • WordPress is making a smart choice in version 3.9 by allowing users to upload images by dragging them onto the editor. That would reduce their score for images to 1.
  • Joomla and Drupal could make life much easier for users by allowing them to create categories while writing.
  • Drupal would benefit from an appstore to install plugins automatically.

How this system could be improved

This system, as I’ve outlined it, is far from perfect. Here’s at least three ways it could be improved:

  • Weighting the results: Ideas are much harder to learn than clicks. The book Developing Technical Training has been invaluable in helping me think about this. Ideally, ideas would score as much as several clicks or screens.
  • Ideas and words are subjective: Click and screens can be counted exactly. What is a difficult idea? What is a new word? Anyone using these criteria does need to account for the technical experience of their users. Or, perhaps user could self-identify which words and ideas are confusing.
  • Include a way to account for busy screens. I tried to account for possible confusion by including Screens as a metric. However, there’s a huge difference between a cluttered screen and a clean screen.

Who is WICS for?

I think WICS can help trainers like me, but it could be even more useful for developers trying to make easy-to-use software.

It would be very useful to create a list of the 10, 20 or 30 common tasks and see what the score is for your software. Unlike scores based on the Likert scale, developers can run these scores with or without external testers.

Over to you

I’d love to hear your thoughts on this system. Where does it fail currently? How could it be made better?

Instructor

  • Steve is the founder of OSTraining. Originally from the UK, he now lives in Sarasota in the USA. Steve's work straddles the line between teaching and web development.

0 0 votes
Blog Rating
Subscribe
Notify of
9 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
brianteeman

Thats a really good concept and builds on the standard user testing approach of asking users to complete small tasks and recording the process.

steve

Thanks for the positive feedback, Brian
I do think this could integrate well with standard user testing.
In this blog post, I always took the fastest and most accurate route … most users wouldn’t.
In a real testing environment, the tester could record how many clicks they take and screens the user visits.

Randy Carey

Your criteria (words, ideas, clicks, and screens) corresponds to my mental model. But I do think these need to be weighted. Even more importantly, we must factor in the user. The use of non-intuitive terms or steps is a larger burden for the casual, infrequent user versus the frequent power-user.
But in my mind, the biggest difference is between the site integrator and the client’s content managers. If we tailor the UX for the client, we no longer compare CMSs out-of-the-box, but by [a] how easy each is to tailor to the client, and [b]

the UX we can deliver to the client. If we accept that site integrators deliver a customized UX to the client, then site integrators have a need for some metric or principles that guide them in customizing the client’s UX.
For what it’s worth, last week I delivered a presentation on streamlining the client’s workflows, and it relates to your metrics and the perspective I just expressed: [url=http://www.slideshare.net/careytech/streamlining-the-clients-workflows-in-joomla]http://www.slideshare.net/c…[/url]

steve

What a great link, thanks Randy. The ideas in your presentation are very close indeed to the ideas in the post.
Yes, you’re right on the users too. It could well be that tasks need scoring differently for different types of users.

Yaeger Design

Steve, example #1 is not accurate for Joomla! (or Drupal?). Articles

cannot be viewed without having an associated menu item attached to

them. Thus, you must create a menu item in a separate section of the

admin that links to the article after the article is created. I believe

the same is true for Drupal. Thus, WordPress is head and shoulders above

the others as the articles are accessible via permalink immediately upon publishing them.

steve

Hi Yaeger
That depends, although yes -we could be harsher with this grading.
By default, Drupal is actually the easiest out-of-the-box for making menu links.
WordPress just dumps it on the frontpage and Joomla does the same if you click “Featured”.

Yaeger Design

Point being, creating content is useless unless it’s accessible. With WordPress, the content can be accessed immediately (if you know the ID or permalink) without creating a menu item. Not so much for the other two. Try using your Joomla! article alias in the URL to accesss the content you created…it doesn’t work. Just pointing out there are additional WICS you left out.

steve

Yes, I see your point there. That involves a slightly longer process than we covered here, but it’s a valid point of confusion.

Yaeger Design

The example would be valid only if you changed the title to “Drafting Content” and the WordPress button was changed from “Publish” to “Save as Draft.” Then you would be comparing apples to apples. Otherwise, you need to add in the additional 20 WICS to create a menu item for the article.

9
0
Would love your thoughts, please comment.x
()
x