This qualitative research method refers to the verbalized account of a subject’s mental process; in particular, the sequence of cognitive events, or thought processes, between the introduction of a task or problem to the final product. Also known as concurrent verbal protocol and thought listing, it is predominantly used as a research tool in the field of cognitive psychology. It can be thought of as “self report” or “thinking aloud.” This allows a researcher to find out how a person approaches a problem or task and describes the problem solving techniques or interpretations he or she uses. Provides rich, qualitative information about how a person reasons his or her actions in specific situations.
Weaknesses of method, the observer may influence or affect how the person doing the “think-aloud” behaves. The need to give a verbal report of their mental process may change how a subject interacts with the task at hand and affect their decision process. A subject can only report what they are aware of and not the underlying unconscious processes that take place in addition to the verbal decision
“Talk-aloud protocol is based on the simple idea that to get at someone’s thought processes while they solve a problem, you get that person to talk aloud what they are thinking, everything that comes into their head, as they solve a problem. This assumes that as someone solves a problem, they retain a small amount of information (7 plus or minus items) in a volatile memory store, their Short Term Memory (STM). The validity of protocol analysis rests on the assumption that if you can get at the contents of someone’s STM as they solve a problem, you learn a significant amount about their problem solving process”.
Observation Observation is very unique way of gathering information as you can see for yourself the problems that different people encounter, while they are carrying out the same tasks. You can see at first hand the problems being encountered and maybe understand why the problem has occurred as you can see why participant made the mistake. There are essentially two observation methods you can use during usability testing. The first method is ‘unobtrusive’ observation, this where you focus on observing what the test user does and avoid doing as much as possible from influencing the participant by explaining the design or asking questions. The second method is ‘obtrusive’ observation, which means you are allowed to describe the design choice, ask questions, or communicate with the test user in a conversation.
We as a group have decided to use the ‘Unobtrusive’ observation as to ‘Obtrusive’ observation as we do not want to influence the research results whatsoever as we want accurate information from our research methods. The ‘Obtrusive’ observation requires the observer to interact with the user and may influence the results that are collected. Therefore we as a group have decided to choose ‘Unobtrusive’ observation to collect the information as we believe it is better to capture information without interacting with the participants as we do not want to influence the results in anyway.
c. Unobtrusive Observation With unobtrusive observation you learn whether people can use the design in an easy and efficient way, and why this may not be the case. You learn how people behave and how people explain their behaviour, which amazingly are two different things. If you want to learn about behaviour, you have to study behaviour even thought that might not be what you want. If you want to learn whether users can use the design you have presented them with, you have to observe how they use it, not ask them what they think of it and this is where behaviour is monitored. Opinions are bad predictors of behaviour, as it has been known in the past that participants only score 1 out of 5 on basic tests but if asked about the system their opinion may not be what you expected if you were going on the findings of the results.
d. Obtrusive Observation Unobtrusive methods will not tell you what the test users think of the design. This will involve interaction between the observer and participants where the participants are asked questions continually regarding the design of the system and the participant must explain their answers. If you want to learn more about the usefulness and the acceptance of the design, you will have to ask the test users. This method involves opinionated questioning as you will not ask “Do you understand what this means?” as you will be expected to ask “What do you think when you see this?” as you want the users opinion.
Time Measures Time Measures is an effective way to see how a users interacts with a computer and how long it takes the users to carry out or and finish the task. It is clear the time taken to complete a task is some reflection of the amount of effort the user is making in order to complete or carries out the required task.
The group have chosen to use the questionnaire to collect the required information that is combined with open-ended and closed-ended questions. The think aloud aspect requires the users to say what they are thinking and this way the information gathered is more accurate as to information on a paper just answering questions. The observation method gives the observer the chance to see how the participant behaves during each task and why they are likely to make mistakes that they make. Finally we combine the time measure to our information collecting methods as this allows us to see how long it takes the users to carryout each task and compare novice, intermediate, experts at each task. To identify the main shortcomings of the website Pogo.com and this allows us to use this information to design a prototype that can iron out the mistakes making it more users satisfactory, enjoyable, efficient and effective.
Conclusion/ Recommendations As a group we have evaluated an existing system using the ten heuristics to appoint the error that consist within the report. To test if these are affecting the user we used a method like questionnaire, talk aloud observation and time measure to test the system to capture the user requirements. From the user requirement we were able to create a low fidelity prototype which fulfilled the usability of the system.
Improvements We as a group think that the prototype could be improved by changing the colours and making the errors much clears and giving a popup error message which would stop the user from enter unless the message is triggered. If we had more time we as a group would have created a high fidelity prototype improving it from the low fidelity prototype. Overall the group has contributed and carried out task well and on time.