ProCard Card Sorts

Two card sorts show the value of iterative improvement

The ProCard card sort reports below show the value of taking an iterative approach to UX testing. The information learned in the first card was applied to the second card sort, dramatically improving the results.

ProCard Card Sort #1

Invitation to participate to selected ProCard users was sent December 10th and resent December 16. Tevin Cleveland invited an additional 15 participants on December 16. As of January 3rd, 19 of 26 (73%) participants completed the study and 7 abandoned. It took participants a median time of 02:37 to complete the study.

Reminder: What is a card sort?

As defined by usability.gov, a “Card sorting is a method used to help design or evaluate the information architecture of a site. In a card sorting session, participants organize topics into categories that make sense to them and they may also help you label these groups.”

Results Analysis

We would like to see at least 75% of a navigational item sorted to a single category, or possibly an even split between two categories. With ProCard’s 19 participants, that would be an agreement of 14 of 19 items. In this trial only 6 of the 15 navigational items reached this level of agreement.

In the image below, dark blue indicates higher agreement.

All but one of the proposed navigational categories had two or less items with majority agreement (10 or greater).

• ProCard Project - None
• FAQs – One
• ProCard Contacts – One
• ProCard Database – One
• Amazon Business – Two
• Citibank – Two

The Policy and Guidance category had three items with majority agreement but two were low majority agreement (11).

Five items had no majority agreement on any category - Misuse Report, ProCard Reference Kit, Application Form, Cancellation Request Form, Exception Request Form, Reports and Recommendations.

The only strong matches occurred on matching Category/Item names.

• Amazon Business – Amazon Roles (19), How Can I Join Amazon Business (17)
• FAQs – FAQ (18)
• CitiBank – Citi Manager (16)
• ProCard Database – Database Forms Overview (16)

Discussion and Suggestions

Based on the card sort, the ProCard navigation as proposed would largely not be successful for your users. Only very obvious navigational items would be found easily. The data indicates that either the names of the items or the names of the categories do not match your user’s expectations, or they are simply not understood. Additionally, the proposed navigation may have too many categories.

Most items were sorted into Policy and Guidance, which became a default placement category if there was not an obvious choice.

Additionally, there are many single-item categories (Procard Database, FAQ, ProCard Contact)

Moving forward, we can:

1. Refine/rename the categories and items and retest.
2. Perform an open or hybrid sort, where your users name the categories themselves. This will provide us some insight into how users categorize your content.

Category Changes

Policy and Guidance

• The only high match here was “ProCard Guidance for Cardholders”.
• There appear to be no policy items.
• This category became a generic category for items that did not obviously fit elsewhere.
• Unless there is additional Policies or Guidance, this category name only applies to one item. You should consider a more generic or more specific name such as “User Guidance”,  “Manager Guidance”, “Guidance and Reports”, etc. This latter category would encompass “Cardholder Guidance”, “Misuse Report”, and “Reports and Recommendations”.
• Create a new category “ProCard Forms” that will capture Application Form, Cancellation Request Form, Exception Request Form, and Database Forms Overview.

Amazon Business

• Has obvious placement of “Amazon Roles” (19/19) and “How can I Join Amazon Business” (17/19)
• I assume this has high need/high visibility.
• Could keep unless combined under an obvious new category such as “ProCard Programs”

ProCard Project

• No items were sorted into this category over 26%.
• I believe the name is too generic.
• Replace with “About Us”. About Us is a standard and well understood website category.
• Could contain Governance, Teams, Contact Us, and historical project information.

FAQs

• This category contains only one item of the same name, “FAQs”
• Could keep.
• Without FAQ as a category, would likely be sorted into the replacement for the Policy and Guidance category.
• Have to decide if it’s worth its own navigational category.

ProCard Contacts

• This category contains only one item, “Governance and Teams”
• Eliminate this category and the underlying Governance and Teams would move to new About Us category

ProCard Database

• This category contains only one item, “Database Forms Overview.”
• Eliminate this category and the underlying Database Forms Overview would move to new ProCard Forms category.

Citibank

• This category contains two item, “Citi Manager” and Global Card Management System.
• Keep unless you create a category that would encompass Citibank, Amazon, and other programs.

Item Changes

• Page names can remain longer, but we recommend you shorten navigation names:
• Shorten “ProCard Guidance for Cardholders” to “Cardholder Guidance”
• Shorten “Database Forms Overview” to “Database Forms”
• Shorten “How can I Join Amazon Business?” to “Join Amazon Business”

Procard Card Sort #2

As a quick reminder, the navigation as proposed and tested in the first ProCard card sort would largely not be successful. The data indicated that either the names of the items or the names of the categories did not match ProCard user’s expectations, or they were not understood. Additionally, the proposed navigation may have had too many categories. 

Changes made for the second card sort included: 

• Added "About Us" category 
• Added "ProCard Forms" category 
• Removed "ProCard Project" category 
• Removed "FAQs" category 
• Removed "ProCard Database" category 
• Changed "Policy and Guidance" category to "Cardholder Guidance" category

ProCard Card Sort #2

An invitation to participate was sent to selected ProCard users January 31th and resent February 11. As of February 24th, 18 of 22 (82%) participants completed the study. It took participants a median time of 01m49s to complete the study.

Reminder: What is a card sort? 

As defined by usability.gov, a "Card sorting is a method used to help design or evaluate the information architecture of a site. In a card sorting session, participants organize topics into categories that make sense to them and they may also help you label these groups."

Results Analysis

We would like to see at least 75% of a navigational item sorted into a single category, or possibly evenly split between two categories. With ProCard’s 18 participants, that would be an agreement of 14 of 18 items. In this trial 16 of the 18 navigational items reached this level of agreement, a huge improvement over the first proposed navigation. Eleven achieved greater than 90% agreement.

Only two items had lower than 75% category placement agreement:

• Coronavirus Resources: 61% Cardholder Guidance; 39% About Us
• Global Card Management System: 67% Citibank; 28% Cardholder Guidance; 6% About Us

Recommendation

The high level of agreement indicates the proposed navigation will be very successful for website visitors. We can place the Coronavirus Resources into two categories, both Cardholder Guidance and About Us using the menu builder feature of Drupal. I think Global Card Management System can stay in the Citibank category.