Usability Test of Steam & CDKeys

Usability is determined by a user experience, not the designers

For this project my team and I were tasked with conducting a usability test with 2 different game sites: Steam and CDKeys.

Using a template of questions provided by our professor, we reworked them to suit our needs and created a survey for friends and family to take. The questions ranged from demographic information, to rating the UI of each site from 0 (poorly designed) to 5 (excellent design).

• Conducting the survey:

  • First, we had them use each of the websites and then complete the survey describing the overall experience they had with the sites and how they ranked amongst each other.

    • Each participant filled out a pre and post task questionnaire. The tasks included visiting the sites and completing 3 scenarios.

• Scenarios:

  • We gave our participants two scenarios to perform to better test the usability of the websites

    • Browse through the PC games by category/genre you enjoy. Adjust the filter to display games below $40

      • The purpose of this scenario was to have users look for products they were interested in and how simple it was to filter their searches based on price.

    • Visit the on-sale games, and compare the prices and/or games offered

      • The purpose of this scenario was to have users explore the website’s navigation.

    • Search for two new open-world RPGs to buy and compare their prices. 

      • The purpose of scenario 3 was to see how easy it was for users to look for titles based on release date and genre.

  • We gathered our data and calculated the average age, gender, how often they use the internet, and how they access the internet (i.e. a personal computer)

    • In addition to finding the averages, we also calculated percentages and the standard deviation for certain questions.

• Analyzing the results:

  • We were able to draw a variety of conclusions from the data collected. For example all of our participants were frequent users of the internet, meaning they’ve likely encountered both good and bad user interfaces and user experiences.

    • 100% of participants owned a computer and used the internet daily

      • 50% has made 10 or more purchases of a similar product (video games) on the internet within the last year.

    • 83% of participants were men in their early to mid 20s

      • This demographic has a greater likelihood of intending to purchase a product from either website as seen by the 4.75-4.00 mean of each site, respectively. Approx. 55% of gamers in the U.S are young adult males.

• And the winner is:

  • After crunching the numbers we compared the data in order to determine which site was better.

    • CDKeys ranked better than Steam in most categories, specifically the perceived ease of learning, product offerings, and impression of the usability of the website.

    • Steam ranked better than CDKeys in the intention to use and utility categories.

    • Together there was a great intention to use and a moderate rating of their usabilities as a whole.

  • From this we can conclude that CDKeys provided users with a better experience compared to Steam.

    • Perhaps it’s CDKeys competitive prices or their streamlined UI that makes it more user friendly than Steam. Although participants were less inclined to use CDKeys, likely because Steam is more familiar to people compared to CDKeys.

A users experience is always in flux, usability testing can inform researchers and designers how to optimize to better suit a users needs. This became clear to me after completing this project, despite being two well established websites both Steam and CDKeys fell short in at least one category. This isn’t to say their websites are bad, but that they can always be better for the user.