Archive

Archive for the ‘Human Factors Methodology’ Category

Interface design: The gulf of execution explained in a funny way

Paper prototyping – 2D and 3D

February 21, 2016 Leave a comment

What is it

Paper prototyping is a classic method for usability testing, specifically in the early stages of the product development process. Jakob Nielsen described the method in his blog as one of the fastest and cheapest rapid prototyping methods in the design process. All it needs is an idea for the conceptual interface design, paper, scissors, and glue. The conceptual design of an interface is sketched on paper. The paper sketches are shown to a user who is then asked to fulfil a task on the interface. The user then e.g., presses a button and in consequence the designer (playing the “computer) changes the picture in front of the user. Users can interact with the paper interface as they would do with a real product. It is an easy method to compare different conceptual designs without worrying about the implementation. Paper prototypes should be simple and should not include a finalised colour concept or high quality graphics as that is not what a paper prototype helps to evaluate. At best it is simply a sketch. The following characteristics of an interface can be evaluated with a paper prototype:

  • General Concept
  • Understandability
  • Navigation
  • Information Architecture
  • Functional Requirements (test if complete to fulfil the task)

How does it work in general

The video shows how a paper prototype and interaction with it look like. Different menus, tool tips and pop-ups, all that can be designed. Remember to think about which tasks you want to evaluate and how they can be achieved in the interface. What interactions does the user need to make? For each interaction an according change in the paper needs to be prepared. Users might take different interactions then expected. So prepare paper prototype “reactions” for unexpected interactions as well. That could be a site with “lorem ipsum” or just a blank page with “under construction”. Those preparations help to let the user explore the interface and helps you to see where users have difficulties in getting along with the interface. Are the actions might want to take to fulfil the task clearly presented in the interface? Does the user get apropriate feedback for each interaction he/she does on the interface?

The advantage of a paper prototype is that it can be easy redesigned. In the next iteration you could, e.g., try a different concept or rename a menu and see if that supports the user to better fulfil the task. You do not need many users for a usability test. According to Jakob Nielsen about 85% of usability faults can be found with about 5 users (“Why you need to test with 5 users”).

If you want to read more about paper prototyping, Carolyn Snyder wrote a book about it.

Paper prototypes in 3D

I recently came over an article were paper prototyping is applied to a 3D design. The article from Säde et al. (1998) is quite old, but because of the 3D paper prototype I thought of it worth explaining herein. They used a paper prototype to test a design for a drink can refund machine. The prototype was build out of foamcore cardboard, glued together with a glue pistol. The interface was represented by coloured print-outs. Lights in the interface were represented as coloured paper attached to the panel. To design 3D paper prototypes it is way to look at industrial designers. Here is another example for a 3D paper prototype of a toaster (yes, a toaster):

Last but not least you find some tools for paper prototypes on the bottom of this website. More information of how to to design 3D paper prototypes with cardboard can be found on this website.

Other Sources:

Säde, S., Nieminen, M., and Riihiaho, S. (1998). “Testing usability with 3D paper prototypes – Case Halton System”

Jakob Nielsen (2003). “Paper Prototyping: Getting User Data Before You Code”. (online) https://www.nngroup.com/articles/paper-prototyping/

How to Conduct a Qualitative Interview (Part 1)

November 1, 2015 Leave a comment

This is a guide which may helps you to understand what a qualitative interview is and when you can apply it as a method for your research. Please be aware that it is different to everyday communication and different to a counselling interview. In this part you can learn:

  • what a qualitative research interview is
  • forms of qualitative interviews
  • when to apply a qualitative interview as methodology

Check here for the second part (it contains information about how to prepare questions, about the dynamic about expectations and roles of the interviewer and interviewee).

Also related is my post on the thematic map analysis, here.

1. Introduction

This guide introduces research interviews, when to apply them, how to plan and conduct them, required skills, and the relationship between interviewer and interviewee. The concepts and knowledge explained herein are common in all types of interviews as part of the communication and dialogue. The guideline provides a set of tools rather than prescribing rules how the interviewer should behave. It is due to the dynamic communication process in an interview between the two persons talking with each other. The interviewer needs to know the tool set and decide about the most appropriate way to communicate dependent on the cues perceived from the interviewee, the interview’s goals, and selected method and structure of the interview.

An interview is a communicational dialogue and conducting it requires special communication skills. Prerequisites for an interviewer to successfully conduct a research interview are: openness, resistance to interpret the perceived information straightaway, and a sense of patience and listening. This is contrasting everyday communication where everyone wait to say the next bit. The communication process of the interview is strategically motivated towards the research question thereby the interviewer aims to avoid own interpretations. In contrast, avoidance of interpretations is not a typical part of everyday communication. In difference to counselling interviews, research interviews are directed towards the research question rather than to the needs of the interviewee. If the interview aims to explore problems, it typically focuses on a deeper understanding of the sense making context of the problem compared to a counselling interview.

This guide provides basic skills, but their flexible and most suitable application in a dynamic interview situation needs to be learned. Skills should be improved in forms of a technical failure analysis after an interview, for the next one. Besides the communication skills of the interviewer, of course, a successful interview requires a solid research question. A research interview, independent on how good it is conducted, will not help the research if the research question itself is not properly set. The methodological background of how to formulate a research question and knowledge about the topic of the interview are out of scope of this guideline.

2. When to Apply Interviews

Before a suitable method can be selected the research question needs to be clarified. The table below provides an overview of potential search areas (left most column) and whether they are suitable to be explored with qualitative interviews.

When to apply qualitative interviews.

When to apply qualitative interviews.

3. The Planning Process of Interviews

The following four steps are considered the main basic decisions to plan an interview. Other decisions concern only smaller aspects and are dependent on the decisions made in the steps (1) – (4).

Steps to plan a qualitative research interview.

Steps to plan a qualitative interview.

The following table provides an overview of interview types. There exist more types, the table shows just an extract. The interview types support different aspects of your research, e.g., interviewers get a wider range of experience within group interviews, as trade-off it is not possible to delve deeply into social and personal matters compared to individual interviews. If group interviews are conducted, do not forget to observe and note the group dynamics and interaction dynamics, as they are important for the interpretation of the data. To find a suitable interview form, think about your research question that you want to answer with the interview and consider the following criteria (one or more):

Interview forms

Forms of qualitative interviews (interview types).

After the 4 main steps of planning an interview are done, the planning can process to more detailed steps:

  • Write up a guide for action/reaction for the interviewer;
    1. What kind of questions are allowed? How are questions of the interviewee handled? How are inconsistencies in the narration handled? How much empathy is allowed?
    2. Handling of games of power? What other difficult situations could occur during the interview and need to be considered?
    3. How to handle disruptions through third parties?

(this could be solved by selection of an adequate location, see below)

  • Develop a set of questions
    1. Start with questions about facts (e.g., demographic questions) to warm-up the participants before starting with personal questions (e.g., opinions or experiences)
    2. Write up of a first request for narration, consider the interview type – e.g. expert interviews should not be started with an open request for narration
    3. Questions concerning ratings or attitude should be prepared in different versions, to have another formulation in case it is not understood
    4. Include a question at the end for the interviewee to ask “Did we forget to talk about something, that you would like to add?”
    5. Interviews are better comparable if the same questions are applied in all in the same form, but this does not take the natural flow of conversation into account. It is suggested to keep the flow flexible. If the questions gather facts, it might be worth to conduct an expert interview (even if the potential interviewee is not considered as expert in a common sense)
    6. The SPSS principle can be applied to generate an interview guide:
      1. S – collection of potential questions,
      2. P – check of the collected questions in relation to openness and knowledge (some have potential for a question, others for a keyword, others for an introduction, and other that require just a short answer could be either in a separate questionnaire or stored as back-up in case the information is not automatically told),
      3. S – Sorting of the remaining question in a topical / time order
      4. S – for each sorted package of questions a request for narration needs to be formulated additionally keywords should be added in a column next to it as they could be used for more detailed questions and the questions should be sorted obligatory (in a certain formulation) and optional
  • Even in a guided-interview, cues for provision of more information (out of the guide) should be taken and asked for, to stay in a conversational flow and to keep the atmosphere of openness. To enable this a guide should not contain too many questions.
  • In a semi-structured interview, note a set of open questions beforehand, it eases the conversation in the interview situation
  • Decision about the duration
  • Decision about the level of similarity between interviewer and interviewee (alienness vs. familiarness)
  • Decision about depth and required extend of knowledge an interviewer must have in the interview topic
  • Decision about the location
    1. Create an open, calm, and friendly atmosphere. The location could be decided by the interviewee, but this involves the risk to deal with very different location during the interviews, and it may include locations where the interview is interrupted by third persons
    2. An open atmosphere is created when two chairs are positioned at a desk over a corner (positioning them in front of each other is offensive)
  • Recruiting of interviewees
  • Creation of the interview elements, e.g. introduction, interview guideline, guide for the interviewer, material to present the project (e.g. participant recruitment)
    • Write up of an introduction, the introduction should include the purpose of the interview, confidentiality (including anonymised data, e.g. by using a participant number), explain the format of the interview, explain how the data if and where the data will be published, indicate the duration, how the participants could get in touch with you after the interview and finally ask if they have questions.
    • Do not forget to inform the participant about when the interview is recorded and ask for their permission (prepare a sheet, so you have a written proof of their permission to record them during the interview session)
  • Sheet for data protection, and form of consent
  • Creation of a template for the interview protocol
    1. Notes of the participants remarks, or recording
    2. Remarks of the atmosphere during the interview, as it could indicate critical situations which need to be considered in the interpretation
    3. Be careful about note taking, it helps e.g., for follow-up questions but do not jump on notes as this influences answers
    4. Audio recording should be preferred as it is more accurate and you store the exact wording from the interviewee, except there is good reason not to use it (e.g. in a very noise environment)

For quantitative research methodological control means repetition with the same results. For qualitative research it is not expected to get the same story in an interview, as the narration depends on the context and “sense” might change. Control for comparison of a series of interviews means to write down rules for the interview and a method for interpretation of the data and to strictly apply those in the interviews. Further difficult situations and potential reactions should be clarified and written down beforehand. The structure and choice of question should be integrated into the interpretation of the interview – “interview mistakes” can give important information.

The social context of the interviewee needs to be considered for interpretation of the narrations as it is important for the definitions of truth and ambiguous meanings – e.g., in some cultures it is not a manner to talk about certain aspects, so they likely will not be talked about. The use of language indicates the social context and intersubjective relations.

Before you start to conduct the interview it is important to run a pilot. A pilot helps to determine the accurateness of the assumed interview duration, if the order of the questions is logically fitting to the narration of the participants, and if there are flaws in the research questions, e.g., ambiguous questions. Questions that are not effectively eliciting necessary information should be dropped and be replaced by new, improved ones. Further a pilot could help to explore when and how it is productive to depart from the planned structure to follow the interviewee’s interests and knowledge. Participants for a pilot should be similar to those that will participate in the implemented interview.

 

Sources and Additional Information

Helffrich, C. „Qualität qualitativer Daten – Manual für die Durchführung von qualitativer Interviews“, Version 3 (title translated into English: Quality of qualitative data – a manual for the conduction of qualitative interviews)

Free Management Library “General Guidelines for Conducting Research Interviews”, (online) http://managementhelp.org/businessresearch/interviews.htm

Turner, Daniel (III) (2010), “Qualitative Interview Design: A Practical Guide for Novice Investigators”, (online) http://www.nova.edu/ssss/QR/QR15-3/qid.pdf

Knapik, Mirjam (2006), “The Qualitative Research Interview – Participants’ Responsive Participation in Knowledge Making”, (online) http://www.socsci.uci.edu/~castellj/sshonors/webdocs/methodolog_interviews_focusgrps/Responsive%20Participation%20in%20Knowledge%20Making.pdf

DiCicco-Bloom, Barbara & Crabtree, Benjamin (2006) “The Qualitative Research Interview”, (online) http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2929.2006.02418.x/pdf

How to Conduct a Qualitative Interview (Part 2)

November 1, 2015 Leave a comment

This is a guide which may helps you to understand what a qualitative interview is and when you can apply it as a method for your research. Please be aware that it is different to everyday communication and different to a counselling interview. In this second part you can learn:

  • roles and relationship of the interviewer and interviewee
  • about expectations and how they influence the interview
  • how to prepare questions and type of questions
  • to understand and influence the dynamics of an interview

Check here for the first part (it contains information about what a qualitative research interview is, forms of qualitative interviews, and when to apply a qualitative interview as methodology).

1. Relationship between Interviewer and Interviewee

The interview is a dialogue created between interviewer and interviewee. Both share a room for communication which they can use for their purpose, or not. The following figure illustrates this room.

Participants have their expectations for the interview. Typically they participate because they have an interest in the research topic and feel they can contribute to the knowledge. During the interview the participant applies effort to identify how much it is appropriate to say and how they say it. They attend to what the other expresses with words and with gestures. This perception in consequence subsequently shapes the responses, considering reactions before and anticipated consequences of what they say. Some researchers recommend to avoid spontaneous reactions, but considering the spontaneity of a face-to-face dialogue the way a communication develops naturally this is not possible. No reaction will be interpreted, likely negative, by the interviewee. Reactions towards the communication and different expectations are a natural process of communication and nothing bad to happen. The researcher should be aware of the own reactions and of differences in the interpretations, and possibly discuss them.

 

2. Expectations towards the Interviewee and Strategies of Narration

Qualitative research aims to reconstruct sense or subjective points of view. Sense is otherwise presumed from a pragmatic point in quantitative research. Sense is not objective but subjectively formed through interaction with people. Sense is recognised through repeating patterns in the interaction. Further each articulation needs to be considered as it is context bound and needs the context in which it appeared to interpret its sense. Because the sense of the interviewed person is potentially different to the sense of the interviewer it is important to create an atmosphere of openness in the interview so that the interviewed person is able to narrate her sense.

It includes that the interviewer avoids interpretations of what the other half “means” and remains more reflective. It helps to avoid interpretations if expectations towards interviewees are thought of before the interview is conducted. Expectations can reach from a similar relevance system like the own, to application of basic rules for communication to characteristics of the interviewee being communicative and competent. It is important to be aware of those prejudices to create an open atmosphere in the interview and to be able to react to unexpected contradicting statements of the interviewee. Interviewees may challenge and clarify the researcher’s language use and suppositions. That is part of their role. Because the interview aims to understand what the interviewee wants to narrate, they should be encouraged in doing so by explicit or subtle ways. The interactional style should be flexible to respond to this. It might help in such situations if the researcher adopts the learner stance.

Sometimes there is an alienness between interviewer and interviewee based on their different culture, age or social context. But it is not necessarily a problem. To generate the necessary feeling of nearness in an interview, it is enough when interviewees have the feeling they can be understood. For the interviewer it is important to understand and be able to handle the feeling of alienness. A shared background of interviewer and interviewee influences breadth and depth of the interview. With a shared background fewer words are needed to explain information, it costs less effort for the interviewee and it makes it more likely that the interviewee gives a credit of trust. Typically an interview is broader and more detailed with a shared background. In the preparation for the interview the interviewer should be prepared for potential differences and their meaning, it eases to “feel into” the position of the interviewee. However, sometimes it can be better if interviewer and interviewee do not have a shared background, so the interviewee will give more detailed explanations which would have been otherwise potentially presumed and not narrated.

 

3. Understanding another Perspective, and Positioning Questions in an Interview Situation

Openness means that the interviewee gets the room to structure the conversation. It leaves a choice for the interviewee to decide about the relevance of the question in his system of meanings, interests, and importance. It is different to standardised methods which define the answers and questions and predefine a relevance system. Questions in an interview in general should be neutral, open-ended, concern one topic, and be clear. The following things should be avoided in interview questions, because they harm the flow of conversation:

  • No ambiguous questions or such that cannot be answered from the interviewee
  • No questions with alternatives, or such that are more than one question in one
  • No judging aggressive questions
  • Questions need to remain in the language used by the interviewee, avoidance of terms
  • No questions that cannot be expected from the interview context or such whose intention is not clear
  • Careful, if any, application of closed questions (if, they should be used as detailed questions)

The first sentences of an interviewee articulates need to be carefully analysed. They reveal how the interviewee interpreted the question, which type of self-expression he/she uses and how he/she uses the room for narration given in the interview. There is a difference if the interviewee starts with “I” or “My father” – it defines in which context he/she wants to talk in the interview, more on a personal level of experience or more including relationships to others. Not all interviewees require / want an open atmosphere to narrate. Some need the reassurance from the interviewer, if they are talking about the “right” thing and to reassure if they are understood correctly.

It is important to pay attention to the non-verbal signals of an interviewee. Some researchers suggest that the interviewer should avoid emotional reactions in the interview at all and give at best no signals. Considering the dynamics of a communication process it would lead to an artificial situations. Be aware that giving no signals at all will be interpreted as signal by the interviewee as well, and most likely as a negative one, e.g. the lack of interest. Take care to provide subtle positive non-verbal signals, they encourage the interviewees their narration:

  • Eye contact
  • Friendly tone
  • Gestures like a smile, a assuring “Yes”, “hhm”
  • To endure breaks
  • Calm but not stiff posture
  • To select a posture towards the interviewee

Specifically if the interview is of a topic diving deep into the social context and personal matters of the participants it is important to rapidly develop a positive relationship. The relationship starts with a broad and open-ended first question of the interview. The question should give the interviewee time to hear what is being asked and to think about how to respond. As an encouraging strategy to ask for further details without disrupting the interviewee, the interview could repeat words used by the interviewee asking for their clarification. If the first step in the interview is mastered successfully there is an atmosphere of comfort in which the participants are not afraid of offending another and find satisfaction in the interviewing process. This may open up the possibility to ask questions which were too sensitive to ask at the beginning. Throughout the interview the interviewer’s goal is to encourage the interviewee to share as much information as possible, this involves non-verbal signals as well.

The researcher’s / interviewers own assumptions experiences can influence the way and manner questions are asked – e.g., leading questions or “why” questions indicate that the interviewer, if been in the same situation, would acted differently. In case questions do tell more about the person asking the question rather than they provide additional information from the interviewee. In narrative / ethnographic interviews it is suggested to keep questions concerning understanding to a minimum and leave it to the interpretation to clarify the incomprehensibleness. In a dialogue interview the clear interpretation of the narration is part of the dialogue, also if there should be conflicts they should be clarified by additional questions. The following is an overview of types of questions:

  • Questions to stay in a flow of conversation
    1. They are typically free of presupposition
    2. Such that remain in the situation, e.g., ”Could you describe that in more detail?”, “Please, tell me more about this.”, “What does it mean?”
    3. Such that push the flow of the conversation forward, e.g., “And then?”, “What happened then?”
  • Questions to direct the conversation
    1. Direction refers to the speed of the interview and the topic of the interview
    2. Such as detailing of already named aspects, e.g., “Could you please name an example for …?”
    3. Such as introduction of new topics, e.g., “Was … important?”
    4. If specific information, potentially such that should be compared over the set of interviews, could be requested via rating scale. This should be applied only when the context of the rating has been properly discussed in the interview beforehand (see Section 1, when to apply an interview).
  • Paraphrasing
  • Questions to evaluate contradictions, self-expression
  • Leading questions
  • Questions about facts and knowledge

An example of a good open-ended question and a more restricted one. Be aware that the second question is open-ended as well but assumes a certain influence of the participant’s previous experience which we do not know. Researchers should avoid those assumption in their question.

  • Have your previous experiences with technology influenced you or not influenced you to buy the driver assistant systems in your current car?
  • How does your previous experience with technology influence you to buy the driver assistant systems in your current car?

Paraphrasing is typically not used as a strategy to direct interviews, but it is good to improve attention as especially long narrations can overload the short-term memory of interviewers. Also paraphrasing helps to reduce the own impulse to comment from the own experience.

It is difficult to avoid presupposition in an interview, it should be checked that it does not affect the research question negatively.

 

4. Interaction in an Interview Situation – Dynamic and Formation

Be aware that the style selected in the introduction phase of the interview remains typically the one for the interview, and is hard to change. Power in an interview can be used by both. The interviewer can use power in sense of directing and knowing the questions, and the interviewee can use power in sense of provision of information in a certain way and at a decided time. To utilise their power interviewees can apply different strategies – e.g., to break through the question-answer set by asking questions to the interviewer, by a narration which is clearly independent of the questions or by a clear statement to tell the topic by their interpretation. It is helpful to be aware of potentially occurring games of power beforehand and write down a code of how to react. Here are tips how to react to games of power:

  • Make it clear for yourself that it is just a game and not related personal to you
  • If there occurred personal question, advise to a time after the interview where they can be answered
  • Remain on the interview level, e.g. do not reprehend

To be practised before the interview:

  • Practice open questions and requests for narration
  • Think of strategies to represent security
  • Learn to endure breaks
  • Do not change the type of questions

Pauses are an element to reduce speed in an interview, presumed the interviewee gets to use the communication room and the break is not disrupted by the interviewer. The interviewer needs to carefully decide if a break was a measure of style or a request to switch the roles, e.g., dependent on where the interviewee is looking – if a closing statement came before the break and the look is focused on the interviewer it is likely that a request for a role change. Questions should be directed towards the research question. If the participant feels unease it can influence the interview with larger pauses and the interviewee narrowing down lengths of answers. The interviewer should be aware of this and avoid to suggestive questions, they should stick to open questions.

Statements indicating a pause in an interview.

Statements indicating a pause in an interview.

Be aware that there is not one truth. In varying contexts a person might tell a story different. Social reality in itself is already an interpreted and constructed image of reality. E.g., the experience of the same event can be told from very different points of view. To understand the truth of the narration it needs to be set in the logical context of the interviewee. Further there is a difference in how the event occurred in the past, how it was perceived and how it is narrated now.

 

Sources and Additional Information

Helffrich, C. „Qualität qualitativer Daten – Manual für die Durchführung von qualitativer Interviews“, Version 3 (title translated into English: Quality of qualitative data – a manual for the conduction of qualitative interviews)

Free Management Library “General Guidelines for Conducting Research Interviews”, (online) http://managementhelp.org/businessresearch/interviews.htm

Turner, Daniel (III) (2010), “Qualitative Interview Design: A Practical Guide for Novice Investigators”, (online) http://www.nova.edu/ssss/QR/QR15-3/qid.pdf

Knapik, Mirjam (2006), “The Qualitative Research Interview – Participants’ Responsive Participation in Knowledge Making”, (online) http://www.socsci.uci.edu/~castellj/sshonors/webdocs/methodolog_interviews_focusgrps/Responsive%20Participation%20in%20Knowledge%20Making.pdf

DiCicco-Bloom, Barbara & Crabtree, Benjamin (2006) “The Qualitative Research Interview”, (online) http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2929.2006.02418.x/pdf

Additional literature to understand how interview dynamics change with age, gender, and social class of the interviewer:

Manderson, Lenore, Bennett, Elizabeth, and Andajani-Sutjahjo, Sari (2006), “The Social Dynamics of the Interview: Age, Class, and Gender”, (online) http://qhr.sagepub.com/content/16/10/1317.full.pdf

Thematic analysis for interviews

December 17, 2014 1 comment

Recently I learned about the thematic map approach for analysis of interviews. If you are looking for information about qualitative research in general, interview types, and how to prepare and how to conduct interviews, have a look in my other articles: how to prepare and how to conduct a qualitative interview, part 1 and part 2.

The thematic map analysis approach is described for example in a paper from Braun and Clarke (2008). Perhaps you have already used the approach but did not name it as such. Roughly spoken it is a technique to generate “themes” from interviews based on keywords and keynotes that are generated in an iterative process, narrowing them down and ordering them hierarchically. A very useful tool for this process is a mind-map, but it is not required if you do not like mind-maps.

The following is a stepwise approach to generate themes from an interview. The process is proposed and described in the paper by Braun and Clarke (2008), but I left out step 3) as I found it overlapping with the other steps.

1) Familiarization with the interview – transcribe the interview data and read the interview

2) Start coding – start thinking of recurring themes and key events/concepts in the interview, read through the interview and gather all data about the themes, refine hierarchy

3) Review themes – go through the data again, narrow down themes, refine hierarchy

4) Check of the theme’s names – recheck if the theme’s names are clear and representative, for example if sub-themes and main themes fit together

5) Writing up the report – carefully select extracts from the interview that are most representative for your themes and use them in the report-text to support your themes

The authors emphasize the importance to clearly state the psychological approach the analysis is based on (such as the constructionist approach), because it is important for interpreting the results and you should use the approach to set your interview results in context. Further you should state what your themes are. They can be interpretations from interview statements or they can be on the level of extracts of the interview (without interpretation).

Speaking from my personal experience, I found it difficult to get into the process. My first mind-map covered a lot of things from the interview. Mostly because a lost of things from the interview seemed to be important while reading and making notes. It is easy to lose the sight of the research question and end up with a mind-map of summarized points of the interview. It is not as bad as it sounds to keep the first mind-map broader than the research question. I found it easier to sort out of a big mind-map than to add to a mini mind-map. However, starting from a broad mind-map will take extra cycles in narrowing the schemes down, finding a hierarchy and finding the correct names for the themes.

A mind-map is better suited to present a general overview. Beside the mind-map, it was useful to generate a table with each scheme in a column and in the rows below supporting extracts from the interview. It very much helps to structure the data, check if there is enough evidence in the interview to support your scheme and you see different aspects of the schemes in a glance.

The mind-maps can be created with any mind-mapping software, I used VUE (see below). Below you can also see the mind-map from the beginning and the final mind-map that I produced during my thematic analysis.

Mind-map beginning Mind-map final

Sources:

Braun, V. & Clarke, V., 2008. Using thematic analysis in psychology. Qualitative research in psychology, 3(2), pp. 77-101.

Lynass, R., Pykhtina, O. & Cooper, M., 2012. A thematic analysis of young people’s experience of counselling in five secondary schools in the UK. Counselling and Psychotherapy Research: Linking research with practice, 12(1), pp. 53-62.

TUFTS University, 2013. Visual Understanding Environment (VUE). [Online] Available at: http://vue.tufts.edu/

Functional Resonance Accident Model (FRAM)

December 27, 2013 Leave a comment

Functional Resonance Accident Model – for the analysis of accidents

The functional resonance accident model (FRAM) is used for accident investigation to find the combination of conditions, actions and events that contributed to the accident. This method specifically focuses on a combination of events, actions and conditions. Thereby interconnected variations of normal functions are seen as cause of an accident rather than a real function failure. The variability of multiple functions can combine in unexpected ways. It goes also with an understanding that normal activities can never be prescribed or regulated completely. Conditions of work are always underspecified to allow a flexible and efficient working. So a good coping with situations is related much to the adjustment and ability of anticipating risks by organizations and individuals.

Starting the analysis requires the identification of the essential system functions. Base could be existing task analysis, procedures and expert knowledge. For each function the following characteristics are described:

  • Input (I)— start of the function
  • Output (O) — result of the function
  • Precondition (P) — conditions that must be fulfilled before the function starts
  • Resource (R) — needed by the function to produce the output
  • Time (T) — temporal constraints
  • Control (C) — monitoring and control of the function

Dependent on the function it may is described in several aspects each with the characteristics. The functions can be characterized in a table. Visualized the analysis could look like this for one function:

FRAM 1

Source: Erik Hollnagel, Shawn Pruchnicki, Rogier Woltjer and Shawn Etcher, International Symposium of the Australian Aviation Psychology Association, 2008

In the second step the variability of the function in case of the accident is described. Also the interconnection of functions needs to be described, e.g. if the input for a function was not adequate then the function giving input must also be described.

Step three considers the couplings and dependencies between the functions. Graphically it is shown through linking the output of one function with the input of another function, see Figure below.

FRAM 2

Source: Erik Hollnagel, Shawn Pruchnicki, Rogier Woltjer and Shawn Etcher, International Symposium of the Australian Aviation Psychology Association, 2008

Step four is more outcome related for the analysis, it identifies barriers for performance variations, comparable to quality checks that help to avoid unwanted events from happening. A barrier thereby can be an organizational structure or a physical structure, e.g. for cabling of a computer on the organizational level a checklist that needs to be during the cabling to ensure the correct cabling or on a physical level different types of cables so that no wrong ones can be connected. In the outcome of the analysis it may suggests barriers dependent on found couplings.

As an example for accident analysis the above introduced FRAM method is applied on the accident analysis of the Comair flight 5191. Comair flight 5191 was the first flight to start at the specific morning in 2006 at Lexington Airport. The airplane crashed after an unsuccessful takeoff. Unintentionally the pilot took the wrong runway, did not recognize it that this runway was shorter and not suitable for the type of airplane.

Usual functions involved before the turn into runway and takeoff:

  1. review of weather and airport data
  2. taxi briefing
  3. takeoff briefing
  4. clearance(s) from air traffic control (ATC)
  5. perform a taxi checklist
  6. taxi to runway
  7. perform a before takeoff checklist
  8. turn onto the runway

Pilot error was soon assumed as error, but other factors contributed which in another combination of events maybe could have avoided the accident.

Step one the review of weather and airport data reportedly was done by the pilot. The variance analysis revealed a deviation in this step. Due to constructions the usual taxi way to the runway (where the airplane speeds up and takes off) was not available and the taxi way changed. This information was part of a so called NOTAM (Notice To AirMan). A NOTAM is a notification of potential safety critical events along a flight route or at a location. It is distributed via telecommunication means to airman. Government agencies and airport operators generate and distribute them. One reason for a NOTAM is e.g. a closed runway. NOTAMs important for the specific flight are included in the flight release procedure. However in this case they were missing. A second source for this information failed as well, the NOTAMs are usually broadcasted by the terminal service information available on the special radio frequency. But this specific NOTAM of the taxiway closure was not part of the broadcast.

Graphically the function “review of weather and airport data” gets input at least from three sources, it looks like this:

FRAM 3

Source: Erik Hollnagel, Shawn Pruchnicki, Rogier Woltjer and Shawn Etcher, International Symposium of the Australian Aviation Psychology Association, 2008

There was only one person at the tower and not two like an internal policy advises it. Usually when there are two operators in the tower one supervises the radar and the other one does the takeoff planning and communication with the airplanes, like checklists above. One person has more workload. He did not check during talk through of the checklists if the airplane was really at the transmitted position.

The pilot drove the airplane and the co-pilot (also called first officer) was busy meanwhile with the conversation with the tower as they went through the before takeoff check-list. At the stop line of the runway the first officer was not able to recheck the correct runway marking, he was not able to see from his position.

Another contributing factor was that the pilots used an outdated map of the airport. The changed taxiway was not correctly presented, even that would make the identification of the correct position at the airport difficult. Additionally resulting from the construction some of the taxiway and runway lighting systems were not functioning.

 Source: Erik Hollnagel, Shawn Pruchnicki, Rogier Woltjer and Shawn Etcher, International Symposium of the Australian Aviation Psychology Association, 2008

Overview of Task Analysis

Task analysis is a set of methods. This implies a selection before start of work. Different aims can be reach dependent on the selected method: workload analysis, post accident analysis to find the starting point of the accident, analysis if there is everything provided for the task, define requirements for the task and as analysis to find different ways to reach the task. The approach reaches therewith over different phases of the design process.

There are four major classes of task analysis presented comparable below. In two previous articles I introduced some hierarchical task analysis and some cognitive task analysis methods, see in category “Human Factors Methodology” to find them. GOMS can be counted as hierarchical task analysis but not necessarily as it provides a detailed level.

Name of the method class Aims / Difficulties Short description
HTA
(Hierarchical Task Analysis)
– hierarchical and functional approach (goal oriented not action oriented)
– goals can be reached by different actions – that makes HTA flexible and neutral with respect to its solution
– can be used to compare different actions used to accomplish the goals
– sub-goals can be analysed for task frequency, task difficulty and errors
– first identify top-goal
– redescribe it into sub-goals
– specify in which sequence sub-goals are reached
– specify conditions that activate sub-goals
Cognitive task analysis – framework for addressing cognitive elements in the task – first make a task diagram
– make a knowledge audit table with cues, might difficult aspects, strategies
– simulation interview table potential errors, critical incidents
Ecological task analysis – relationship between person and environment- relationship between person and environment
– analyses structuring of the environment
– goal related as well as method related questions are addressed
– compares perceptual surface (information available in the environment) and action surface (possible actions)
– it is the description of the mismatch between perceptual and action oriented surfaces
GOMS
(Goals / Operators / Methods / Selection rules)
– aims to create a complete description of experienced users procedural knowledge
– information about learning time, execution time and degree of transfer
– similar hierarchical ordering process like in the hierarchical task analysis but in a more detailed level
– description in form of goals, operators, methods and selection rules
– focus is the method how a goal is accomplished