Qualitative and Quantitative Research

i. The type of research provided by the evaluation technique is also an important consideration.

ii. There are two main types: quantitative research and qualitative research. The former is usually numeric and can be easily analyzed using statistical techniques.

iii. The latter is non-numeric and is therefore more difficult to analyze, but can provide important detail that cannot be determined from numbers.

iv. The type of measure is related to the subjectivity or objectivity of the technique, with subjective techniques tending to provide qualitative measures and objective techniques, quantitative measures.

v. This is not a hard and fast rule, however. It is sometimes possible to quantify what is, in fact, qualitative information by mapping it onto a scale or similar measure.

vi. A common example of this is in questionnaires where qualitative information is being sought (for example, user preferences) but a quantitative scale is used.

vii. This is also common in experimental design where factors such as the quality of the user's performance are used as dependent variables, and measured on a quantitative scale.

Interview Technics

i. Questioning domain experts is often a direct and quick way to get information about a task. Remember that the expert is not necessarily the manager or supervisor who knows about the job, but the worker who actually does it although there may be advantages to interviewing both sorts of expert, as the views of the manager or professional instructor are based on years of experience even if they are likely to be `idealized' versions of the task.

ii. It may be particularly appropriate to interview the 'professional' experts after doing some formal or informal direct observation. It is then possible to ask them to reflect on the various expected and unexpected behaviors; this can become a form of third-party walkthrough.

iii. More normally, one would begin with a general set of questions, possibly asking the expert to describe a typical day, or task. This can then be followed with more leading questions such as 'Why do you do that?' or 'What if this develops a fault?' The aim is both to uncover detail and to increase the range of behavior discussed. Where appropriate, the expert can be asked to produce lists of objects/actions associated with the task, although it may be unwise to demand too structured infor-mation during a first interview, as this may limit the range of material discussed.

iv. The exception to this would be HTA where one often starts with a top-down decom-position. In this case one can begin by asking the expert 'What do you do to make a cup of tea?' and then successively expand the explanation.