Cognitive work analysis and task analysis methods have been used to guide the designs of health information technology (IT) through understanding the cognitive work of healthcare professionals. However, most patient care today, especially in primary care, is delivered by teams. Effective team functioning is critical to achieving high quality care, so technology design must also support teamwork, including both individual and shared cognitive requirements and the goals of all team members.
Existing methods lack:
- Focus on teamwork
- Ability to identify and consider the goals of healthcare professionals
- Ability to translate the results into improved IT design that supports individual and team work
Goal directed task analysis (GDTA)
- situation awareness (SA)-based cognitive task analysis method
- used to characterize the cognitive work of individuals and teams
- identifies design requirements for complex technologies
- identifies goals and how goals are achieved
High-level goals are generally consistent across workers even in dynamic and complex industries such as healthcare, therefore designing for goals is more comprehensive in capturing the diverse ways in which people complete work. SA-oriented design principles guide the translation of the GDTA into technology design.
Situation Awareness (SA)
Level 1 SA
“the perception of the elements within a volume of time and space”
Level 2 SA
“the comprehension of their [elements] meaning”
Level 3 SA
“the projection of their [elements] status in the near future”
Methodological Approach: Designing for Situation Awareness (SA)
SA = A critical aspect of cognitive work and decision-making in complex environments, such as healthcare.
People rely on SA to make sound decisions to achieve their goals. This model of information processing is a central basis for effective decision-making, especially within complex and dynamic systems. Team members’ mental models must also align in order to effectively collaborate and coordinate interdependent activities towards a larger team goal. Team SA is “the degree to which every team member possesses the SA required for his or her responsibilities.” Shared SA, or the extent that team members have the same SA requirements, is critical for a cohesive understanding of the current situation and overall team coordination.
Despite the importance of SA and shared SA to effective decision-making and teamwork, existing tools for health IT design rarely capture this aspect of cognition, with the exception of GDTA.
Our application of GDTA to understand primary care teamwork in context was based on the work of Mica Endsley, notably her book Designing for Situation Awareness: An Approach to User-Centered Design (2012). GDTA is comprised of five steps, with iteration between steps. We adapted the method to fit the primary care context and to support the rigor of scientific research in data collection and analysis. A description of each step and our adaptations is provided below.
Steps in Goal Directed Task Analysis Method
This is an accordion element with a series of buttons that open and close related content panels.
Step 1: Define Study Goals, Scope and Identify Team
We expanded beyond the typical one person or small team of GDTA experts by assembling an interdisciplinary team of researchers including physicians, human factors engineers, and research staff with expertise in IT design, public health, nursing and sociology.
- Our initial goal was to evaluate cognitive work requirements for teams during patient visits and performing the work to support those visits.
- We focused on primary care core teams, which we defined as a physician and/or advanced practice provider (APP), and their nursing staff, including both nurses and medical assistants (MAs) who work closely with that physician/APP to care for patients.
- We further expanded this team to include other patient care team members and staff, such as front desk staff, educators, pharmacists, social workers, and case managers.
- We realized it was critical to extend beyond the common paradigms of patient care and teamwork at the time we began our research to avoid constraining our GDTA and the EHR design.
- The definition of patient visit-related work evolved to include electronic and telephone visits, but it did not account for patient information available between visits and the monitoring of patient status to ensure appropriate health care and beneficial outcomes.
- In the end we expanded our scope to include the period between visits and all team task management including work scheduling.
Step 2: Domain Analysis
A domain analysis was performed to understand primary care in preparation for observations and interviews. Academic and grey literature were reviewed on the roles and responsibilities of primary care team members, primary care clinic team structures, and previous primary care task analyses. We then considered our research participants.
- A typical GDTA includes data collection and validation with at least six persons from each role.
- To produce generalizable results and theoretical saturation in data collected, we sampled two core teams and their support staff in 8 Midwestern primary care clinics recruited through a primary care practice-based research network (16 teams total).
- We considered clinic-level variability in size and location (urban, suburban and rural), the EHR vendor in use, patient populations (e.g., income level, non-native English speakers), and governance structure (physician vs corporate owner).
- We interviewed clinic managers to better understand clinic, role and team structures, job duties and current health IT in place.
- To better understand primary care team goals in context, we completed a 2-4 hour observation with each member of the core team and most other patient care staff, shadowing them as they performed their work. We used an observer guide to record detailed notes on work tasks, teamwork, communication, and use of technology.
- In each observation, clinicians and staff were shadowed for a minimum of five patient care visits and while performing other work between visits.
- We performed simultaneous paired observations of physicians or APPs and their nurse or MA who roomed patients. This allowed observation of teamwork of the the core teams.
Step 3: Interviews with Experts
Semi-structured GDTA interviews of individuals were conducted by trained two-person teams, audio recorded and transcribed.
- Core team members participated in up to 3 interviews
- Other team members participated in 1 interview
- We interviewed physicians, APPs, MAs and nurses for up to 1.5 hours each session. We used two interview teams and scheduled these interviews concurrently in order to be least disruptive to the clinic schedule.
Participants first described their high-level goals in caring for patients. Follow-up questions and probes were used to determine sub-goals required to meet the high-level goals and decisions needed to meet each subgoal. For each decision, the participant was asked about information and assessments needed for level 1, 2, and 3 SA to make that decision. Participant information about education, length of practice, role on the team and who they identify as their team members was also gathered. We found it difficult to elicit work-related goals from each team member instead of descriptions of their current work tasks or frustrations with current technology. Not all participants were equally able to articulate their goals or recall all work performed. We addressed these challenges by using information collected from observations and other members of a participant’s team. For example, physicians described the work of the nurses on their team; we would use these comments in interviews with nurses to prompt them on their decisions and information needs.
Step 4: GDTA Data Analysis and Mapping
A GDTA map is a visual representation of the goals, subgoals, decisions, and information needs, without focus on how that information is obtained by participants or how tasks are completed. Each map summarizes the cognitive work across all participants within a specific role, such as physicians and APPs, or nurses and MAs.
As originally devised, the process of GDTA mapping requires interviewers to meet as soon as possible after each interview to discuss new information and use it to build the map. With busy data collection schedules and multiple interviewers in each session, this process was challenging logistically.
- To ensure consistency and rigor in how information was used for mapping, we conducted immediate debriefs following each interview.
- During the interview and the debrief, interviewers made detailed notes about the new information they learned about goals, subgoals, decisions and information needs, including where in the current GDTA map the information could be added.
- We deductively coded interview transcripts to ensure all appropriate information was being captured.
- Codes were based on topics in the maps, e.g. preventive care and team communication/collaboration. A second coder reviewed and verified the initial coding.
- We reviewed all excerpts on each topic, and teams of 3-5 researchers met to build the information into the map structure. Review of the debriefing notes ensured that no important data was missed.
Step 5: Verification Interviews
We conducted verification interviews in Clinics 5-8, with 1-3 sessions of up to 1.5 hours per participant. We divided the map into 24 sections by topic and selected 2-4 sections that were appropriate for a participant based on scope of work and areas of expertise identified during observation, with the goal of verifying all sections at least once.
- Verification interviews began with a brief introduction of the map’s high-level goals, and then review of the subgoals that the interview would focus on.
- Interviewers asked about the accuracy of the map structure, including probing for additional decisions made to meet a specific subgoal.
- Interviewers would ask about the information needed to make each decision. If new goals, decisions, or information needs were mentioned, interviewers probed to collect all needed data, following the procedure used in Step 3.
Minimal new goal information was obtained, often relating to practice innovations at clinics. After the interview, the two interviewers would debrief about new data, share this with the research team, and determine where to incorporate the information into the maps.
Step 6: Iteration
GDTA steps 2-4 proceed in a linear yet iterative fashion. Based on our domain analysis understanding of the differences in training, licensing and traditional roles in clinics, initially we built separate GDTA maps based on a person’s title. Notably, our observations and interviews showed substantial overlap in cognitive work, therefore we combined physicians and APPs into a single map and nurses and MAs into a single map.
Interviewing and data analyses were also iterative in nature. As interview data were added to the GDTA maps, gaps were identified that were filled by questioning in subsequent interviews. For example, a physician mentioned the goal of ensuring that the patient and/or caregiver understand the care provided and the care plan.
For example, when building the map on patient education and how to provide it:
- The first physician described verbal cues used to assess the adequacy of education.
- Subsequent physician/APP interviews sought information useful in assessing patient/caregiver understanding, e.g., the ability to accurately verbalize next steps for care, non-verbal cues, and prior patient success in following the care plan.
Core Team Member Interview Guide
Pre-Visit Clinic Manager Interview Guide
Challenges Performing GDTA in Healthcare
Scheduling data collection due to clinic time constraints
Difficulty of clinical staff in articulating goals and information used to support decision-making
Variation across clinics in how work is performed and who is responsible
Disconnect between the current state of technology and its support of tasks
We successfully applied GDTA in the healthcare context, which allowed us to understand goal-driven care and shared situation awareness across teams. This understanding and its implications for design are not possible with other cognitive analysis or user-centered design approaches.
This is an accordion element with a series of buttons that open and close related content panels.
Arndt BG, Beasley JW, Watkinson MD, Tethered to the EHR: Primary care physician workload assessment using ehr event log data and time-motion observations. Ann Fam Med. 2017;15(5):419-426.
Beasley JW, Holden RJ, Otles E, Green LA, Steege LM, Wetterneck TB. It’s time to bring human factors to primary care policy and practice. Appl Ergon. 2020;85:103077.
Beasley JW, Wetterneck TB, Temte J, Information chaos in primary care: Implications for physician performance and patient safety. J Am Board Fam Med. 2011;24(6):745-751.
Beasley JW, Hankey TH, Erickson R, et al. How many problems do family physicians manage at each encounter? A WReN study. Annals of Family Medicine. Sep-Oct 2004;2(5):405-410.
Endsley MR, Jones DG. Designing for situation awareness: An approach to human-centered design. 2nd ed. London: Taylor & Francis; 2012.
Holman GT, Waldren SE, Beasley JW, Cohen DJ, Dardick LD, Fox CH, Marquard J, Mullins R, North CQ, Rafalski M, Rivera AJ, Wetterneck TB. Meaningful use’s benefits and burdens for US family physicians. J Am Med Inform Assoc. 2018 Jan 23. doi: 10.1093/jamia/ocx158. [Epub ahead of print]. PMID: 29370425.
Holman GT, Beasley JW, Karsh B-T, Stone JA, Smith PD, Wetterneck TB. The Myth of Standardized Workflow in Primary Care. J Am Med Inform Assoc. 2016 Jan;23:29-37. doi: 10.1093/jamia/ocv107. PMID: 26335987. PMCID: PMC5009941.
Karsh B. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. Rockville, Maryland: Agency for Healthcare Research and Quality; June 2009 2009. AHRQ Publication No. 09-0054-EF.
Karsh B-T, Weinger MB, Abbott PA, Wears RL. Health information technology: fallacies and sober realities. J Am Med Inform Assoc. 2010;17(6):617-23.
Wetterneck TB, Beasley J, Holden R, Otles E, de Silva D. Human Factors: Technical Series on Safer Primary Care. Geneva: World Health Organization; 2016. License: CC BY-NC-SA 3.0 IGO.
Wetterneck TB, Lapin JA, Krueger DJ, Holman GT, Beasley J Karsh BT. Development of a Primary Care Physician Task List to Evaluate Clinic Visit Workflow. BMJ Qual Saf. 2012 Jan;21(1):47-53. Epub 2011 Sept 6. PMID: 21896667. PMCID: PMC3568931.