Learning, evaluating and testing with usability heuristics

On the last project I worked on, I had to test the multiple websites of a financial institution. The context was that they redesigned their websites to attract more clients. Besides functionality and compatibility testing, the stakeholders – the marketing department – asked me to provide information about their websites usability.

The usability of an application, usually, is designed into the product by a user experience (UX) designer.The process of testing involves gathering potential users and observing them while they are using the specific interface and completing a set of given tasks. The usability of a product can also be evaluated by a person with experience in the field. This method is called expert review or heuristic evaluation. The notion of Heuristic means: strategies or approaches from experience with similar situations, that are used for problem-solving, learning or discovery.

What the stakeholders wanted from me was to gather information about how usable the product is. Being a tester who loves to question everything, I questioned every element, action and piece of information found on the website. My mission was to review the usability of the product and the strategy was to use heuristics I rely on and my work and life experience.

During the project, I wanted to improve my skills on evaluating the usability of a product. So what I did was to sign up for a usable security course on Coursera because the first three lessons were describing usability and there were some examples on how to review different interfaces/functionalities. I took a basic web & app design class where we focused on the deeper understanding of how UX designers think, on how usability can be improved and about the impact of good and bad usability on users. And, of course, I read many articles on the topic.

Based on the stakeholder’s requirement to test the usability, I went through the websites and saw if I could find issues on this matter: functionalities that are difficult to use, if the navigation through the website pages is complicated, if all information can be easy to find, to read and to understand it and so on.

So this article will be about sharing my experience in how I managed to test the usability of this particular financial institution’s websites.

What is usability and why is it so important?

First, I would like to share a few notes that helped me through this project.

Jakob Nielsen, a web usability consultant, defines usability in his article as being “a quality attribute that assesses how easy user interfaces are to use”. He lists 10 heuristics in one of his articles, heuristics anybody could use while testing an application. These heuristics also helped me.

  • Visibility of system status
  • Match between system and the real world
  • User control and freedom
  • Consistency and standards
  • Error prevention
  • Recognition rather than recall
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
  • Help users recognize, diagnose, and recover from errors
  • Help and documentation

From the usability course on Coursera I learned that when I’m evaluating the usability of different functionalities, it is useful to raise questions about how the users will know to complete the action and how they will understand the feedback the actions provide. Answering these questions helps the reviewer discover issues that are making some features difficult to use.

Many other articles made it easier for me to understand how design helps the users to interact with products. Thinking takes a large amount of energy. Around 15-20 percent of our energy is used by the neocortex. So, if an interface is heavily loaded with information and complicated actions, cognitive load increases. When this happens, users get tired and they need to rest … away from the website.

Applying what I learned on my project

When I got a website to test I was careful to observe how I felt about the product while I was getting used to it. For every observation I had, either positive or negative, I took a note of it and later carefully analyzed the observations. For example, if I encounter functionality that I’m not quite sure what it does, then it’s important to be noted. Maybe other users will also have trouble understanding it. After I get used to that functionality I can tell why I had issues in getting to know how it works and I can help with a more constructive feedback – instead of just saying that the functionality is hard to use.

Then I started my tests by looking at the design and the layout of the website. Mostly I was paying attention to how HTML elements were arranged on different screen sizes and if the found issues would diminish user experience.

For example: On one of the websites, in the navigation menu, there were two buttons placed one under the other: the My Account and the Log out button. The issue was that both buttons had the same color and had no space between them. It looked like, instead of two buttons there was only one with two contradictory actions, so the users would not know where to click for those functionalities. One of Nielsen’s heuristic is to prevent the user from making errors with a careful design. In this case: with the delimitation of the two buttons.

Another important thing I was looking at was the visibility of information on different web pages. I thought it’s very important that the font and color, both on desktop and device, should be visible and easy to read. Typical of issues in this category are paragraphs with extra breaklines in between making the text was hard to follow. Sometimes breaklines appeared in the middle of sentences.

Text overlapping images had to be checked both on devices and desktop. There were situations when the dark grey text couldn’t be seen because it ended on a dark part of the image.

Next on my list was to scan all the links the website contained and then request removal of all the links that no longer worked. As the stakeholder had links with different partners, there were situations where partners changed their link or removed their websites. The user’s experience has to be a positive one and broken links do not help with this. Also, as Nielsen lists in his article shared above, the interface has to be minimalistic. Only relevant text/dialogues should be displayed, and HTTP 404 returning links are not improving user experience.

For almost every website, these were the steps I chose to begin with. After going through these tests, I switched to specific functionalities.

As mentioned above, I learned four questions which can help when exploring a feature. I used these as a heuristic when testing how usable the functionalities are. This heuristic helped me see more easily and more clearly some of the existent usability issues. Let’s see the questions and then I will give you an example how this heuristic guided my testing:

  • will users know to perform the action?
  • will users see the control?
  • will users know the control does what they want?
  • will users understand the feedback?

At one point I had to test a “Send invitation” functionality for one of the websites. The website was created for parents who are saving money for their children. The functionality was sending invitations for baptism/birthday/or a custom party.

The form had four fields:

  • a dropdown where the user selected if the invitation was for a baptism party, birthday party or a custom invitation,
  • an input field where the user introduced the recipient,
  • an input field for his child’s bank account,
  • and an input field for the text that should appear on the invitation.
  • The result was an email with a cute drawing: sun, clouds, animals all around, and in the middle the text that the user wrote.Let’s see the heuristic in action!

Will the users know to perform the action?

Yes. The functionality seemed pretty simple to use. The users fill out the form and click on the send button. Every field has a helpful text preceding it, telling the users what they should enter in that field.

While testing the form, it felt easier first to fill out the field where the invitation text goes and then select the type of invitation and deal with the other fields. But when I got to selecting custom invitation from the dropdown, my text was erased.

My conclusion: the action is performed easily only if the fields are filled out in a specific order. And this is a bug.

Will the users see the control?

In the ”My account” section, where the users deposit money for their children, they encounter a “Send invitation” button, so it’s easy to both access and understand what the functionality is for.

Will the users know the control does what they want?

The answer seemed pretty obvious: it sends invitations to children parties.

While testing the functionality, I sent all the test invitations to my email address. So I knew the input and also the output of the functionality. But then I realized that if I were a regular user I would send invitations to my family: how would I then know what the output looks like? And then I started to ask myself, why would I use this functionality? Wouldn’t it be easier to open my email client and send those emails myself? That way I could edit the email how I wanted, I could add attachments and so on. So, why would I use this functionality?

Without a preview, I don’t really know what my invitation looks like. I don’t know, for instance, that the output has a drawing around the invitation text. The users should know why they are using that functionality and not other similar applications.

So I suggested adding a preview functionality.

Will users understand the feedback?

To answer this question, I tried different scenarios to see if I’m prompted with different messages. For example:

  • What happens if the users leave a field unfilled – do they get a useful error message;
  • Does the text input field have a visible character limit specification – and is it enforced
  • Are users receiving tips on format when manually entering e-mail addresses? Clicking on the send button – users are prompted with a confirmation message for successful send, or an error message when there is a failure.

Every step of the way should be a clear action for the users so the cognitive load does not increase.


While testing, I tried to be aware of the errors I made using the application. If I made the same error more than once, I tried to understand why I was doing that. What was the cause of my tendency to use the interface in a way that was not designed to work?Often users use intuition and/or mental models in dealing with something for the first time. So this was important for me to understand, why I made the same error more than once. Was my intuition dictating something else? Did my mental models interfere? Could this be a problem for other users as well?

For example, on one of the websites, the search bar only worked if the users entered an item to search for and then clicked on the magnifying glass icon next to the input field. But if the users entered an item to search for and then pressed the ENTER key from the keyboard, nothing happened.

When I noticed this behavior I took a note to log it later (I was trying to reproduce another issue). But after using the search bar over and over again, and every time wanting to finalize my search by pressing the ENTER key, it got really frustrating making the same error. My mental model referred to all the other search bars on various other websites: Google, Facebook, Wikipedia and so on. Remember Nielsen’s heuristic about recognizing versus recalling?

Using usability arguments while logging bugs is very helpful in describing why we think something is a bug and why it is important to be dealt with. For the previously mentioned ”Send Invitation” functionality, I logged a design bug. If the invitation was too long – over 10 lines of text, the text that appears in the received email is displayed as being horizontally truncated, so it wasn’t pleasant to the eye.

The developer’s fix was to set a maximum number of characters – 150 – for writing the invitation. This wasn’t the fix I expected. Actually, it wasn’t a fix at all. 150 characters for an invitation could be far from enough for some users. Invitations can be written as poems with breaklines between the lines, so users can reach 10 lines even with 150 characters. The truth is, that when I logged the bug, I wasn’t specific about why the character limit would be an issue for users. I only said that it was not a lovely output when you saw your invitation text being truncated.

To be more persuasive after that, I searched for examples of invitations, invitations in the form of poems and invitations longer than 150 characters. When I reopened the bug I explained both situations from the users’ point of view, in what way users might use the functionality, and provided examples.

The character limit was increased to 500 characters and the design changed so the text isn’t truncated .

The websites were pushed to live

In the end, I was very excited about how the project went. Almost every bug I found was fixed. I felt that by using usability as a point of reference I managed to build a stronger case for my bugs and developers understood their impact. And a great feeling: I’m happy with the final product. And the stakeholders are happy too.

Don't be shellfish...Tweet about this on TwitterShare on FacebookShare on Google+Pin on PinterestShare on LinkedInEmail this to someoneShare on RedditShare on StumbleUpon
Go back


2 responses to “Learning, evaluating and testing with usability heuristics”

  1. Cem Kaner says:

    Excellent! Clear and straightforward. I’ll suggest to students this as recommended reading in my testing class at Florida Tech.