I wanted to share my experiences of user testing with investment bank sales staff and traders on the trading floor, since it is a world away from the relative piece and quiet of a dedicated usability lab.
In order to test anything with users on the trading floors, we have to go to them, since there is little or no chance of dragging them away from their desk, even for short periods of time. This presents the first challenge. The trading floors are noisy, hectic and intense environments where you can expect frequent interruptions. Sales staff and traders can be very short on time and prone to distractions. Their ‘day job’ has to come first. Setting up an appointment in their calendars can occasionally work, but you are just as likely to walk into the middle of a small financial crisis using that approach, as you are by just ‘cold calling’.
Most banks will not allow the installation of prototypes or recording software on participants’ computers, so your test materials will have to be portable. This is where I’ve often encountered my next challenge, the lack of desk space. What with notepads, files, books, iPods, multiple keyboards and the telephone ‘dealerboard’, there is often little space left to set up your test materials.
Some of these challenges can be mitigated very easily. In the first instance, turn your laptop into a portable usability lab. There are plenty of software choices for both PC and Mac, but I prefer a combination of Silverback and Axure prototypes on an Apple Powerbook, or better yet on a Macbook Air.
Silverback is very lightweight, but is does a wonderful job of recording the screen as well as the participant’s face and voice. Clicks are highlighted in the recording by small ‘sonar pings’ and you can even add markers into the recording using an Apple Remote. Since everything is being recorded, the think aloud protocol works very well, and you will not need to crowd your participants with another person for note taking. If you are planning on recording video, take along a piece of electrical tape or a lump of blu-tack, just in case the participant does not want to be on recorded. Stick it over the camera so you can still record the conversation, if your participant agrees.
If you are using a laptop for testing, remember to bring a mouse and ensure it is similar to one the participants will be used to.
Try to keep the test sessions as short as possible, with the bare minimum of preamble. Focus on the key tasks and flows and keep them as short as you can. Remember that even a 15 minute session with most of the sales people and traders is likely to be pushing it, but you should be able to squeeze five to six short tasks into that time. Set up the next recording session before you get to the participant so the only thing you have to do is your short introduction.
So there really is no excuse for not testing, even in a challenging environment.
I don’t think I have come across two web projects that have been the same, each one presented unique challenges and required a slightly different approach. In addition, two developers may each use a slightly different interpretation of a specification, two art directors will invariably come up with very different visual treatments, so even if the process is the same, the results almost certainly won’t be. At the end of the process however, the final product has to be usable.
So how can we ensure that the product is working, that it is in fact usable? The simple answer is to just ask some users. This may seem like a slightly flippant remark, but I believe it really can be that simple, especially during the early stages of the design process. There are many techniques for testing websites with users, ranging from very simple corridor or guerrilla testing, to full on usability studies. Each one is appropriate in certain situations, but not in others.
I believe some level of user input is essential during all stages of the design process. After all, most of us follow same variation of a user centered process, so rather than second guessing the users, let’s get them involved. It doesn’t have to be particularly formal or expensive. Just some simple qualitative feedback can be enough to make a real difference. It could be as simple as asking a colleague for their opinion on a wireframe or an early concept sketch. More feedback could be obtained by building a simple prototype and asking people to click through it. Again, this does not have to be a full blown usability study, a small group of people will be enough to get valuable opinions.
A more formal technique for user input is a card sorting exercise, which is usually designed to create a taxonomy for content on a website and by extension a navigation schema. Again, there are a variety of approaches, depending on whether a taxonomy exists already or not. If there is a current taxonomy, a reverse card sort or navigation test will deliver valuable insights into the validity of the taxonomy and whether it is still suitable. This could be performed on a prototype of the navigation or on a simple paper prototype, which could take the form of index cards labelled with the taxonomy items, which the test candidate is then asked to ‘navigate’ through. If no taxonomy exists, a closed card sort can quickly reveal how users would group content items into predetermined groups. An open card sort allows the user to determine the group headings, but this may be problematic during the early stages of the design process.
The benefits of testing early on should be clear, the more we involve users, the more likely we are to address their needs and concerns early on in the process, thus laying stable foundations for the remainder of the project. That said, there is no substitute for more testing later on. As the visual design starts to mature and the first components are developed, it should be straightforward to build accurate visual prototypes. These can be a simple as static images with clickable hotspots produced in, dare I say it, Powerpoint or they can be more involved Flash or HTML prototypes. As with all testing, simple qualitative feedback is more valuable than not testing at all, so again, simple corridor tests are better than second guessing.
Ideally, a formal usability study of the finished product should be undertaken before launch, but it tends to be one of the first things to get cut from the timelines as they approach crunch time. That is another matter for another post though…
