GDTA Methods

We expanded beyond the typical one person or small team of GDTA experts by assembling an interdisciplinary team of researchers including physicians, human factors engineers, and research staff with expertise in IT design, public health, nursing and sociology. 

• Our initial goal was to evaluate cognitive work requirements for teams during patient visits and performing the work to support those visits.
• We focused on primary care core teams, which we defined as a physician and/or advanced practice provider (APP), and their nursing staff, including both nurses and medical assistants (MAs) who work closely with that physician/APP to care for patients.
• We further expanded this team to include other patient care team members and staff, such as front desk staff, educators, pharmacists, social workers, and case managers.
• We realized it was critical to extend beyond the common paradigms of patient care and teamwork at the time we began our research to avoid constraining our GDTA and the EHR design.
• The definition of patient visit-related work evolved to include electronic and telephone visits, but it did not account for patient information available between visits and the monitoring of patient status to ensure appropriate health care and beneficial outcomes.
• In the end we expanded our scope to include the period between visits and all team task management including work scheduling.

A domain analysis was performed to understand primary care in preparation for observations and interviews. Academic and grey literature were reviewed on the roles and responsibilities of primary care team members, primary care clinic team structures, and previous primary care task analyses. We then considered our research participants. 

• A typical GDTA includes data collection and validation with at least six persons from each role.
• To produce generalizable results and theoretical saturation in data collected, we sampled two core teams and their support staff in 8 Midwestern primary care clinics recruited through a primary care practice-based research network (16 teams total).
• We considered clinic-level variability in size and location (urban, suburban and rural), the EHR vendor in use, patient populations (e.g., income level, non-native English speakers), and governance structure (physician vs corporate owner).
• We interviewed clinic managers to better understand clinic, role and team structures, job duties and current health IT in place.
• To better understand primary care team goals in context, we completed a 2-4 hour observation with each member of the core team and most other patient care staff, shadowing them as they performed their work. We used an observer guide to record detailed notes on work tasks, teamwork, communication, and use of technology.
• In each observation, clinicians and staff were shadowed for a minimum of five patient care visits and while performing other work between visits.
• We performed simultaneous paired observations of physicians or APPs and their nurse or MA who roomed patients. This allowed observation of teamwork of the the core teams.

Semi-structured GDTA interviews of individuals were conducted by trained two-person teams, audio recorded and transcribed.  

• Core team members participated in up to 3 interviews
• Other team members participated in 1 interview
• We interviewed physicians, APPs, MAs and nurses for up to 1.5 hours each session. We used two interview teams and scheduled these interviews concurrently in order to be least disruptive to the clinic schedule.

Participants first described their high-level goals in caring for patients. Follow-up questions and probes were used to determine sub-goals required to meet the high-level goals and decisions needed to meet each subgoal. For each decision, the participant was asked about information and assessments needed for level 1, 2, and 3 SA to make that decision. Participant information about education, length of practice, role on the team and who they identify as their team members was also gathered. We found it difficult to elicit work-related goals from each team member instead of descriptions of their current work tasks or frustrations with current technology. Not all participants were equally able to articulate their goals or recall all work performed. We addressed these challenges by using information collected from observations and other members of a participant’s team. For example, physicians described the work of the nurses on their team; we would use these comments in interviews with nurses to prompt them on their decisions and information needs. 

A GDTA map is a visual representation of the goals, subgoals, decisions, and information needs, without focus on how that information is obtained by participants or how tasks are completed. Each map summarizes the cognitive work across all participants within a specific role, such as physicians and APPs, or nurses and MAs.

As originally devised, the process of GDTA mapping requires interviewers to meet as soon as possible after each interview to discuss new information and use it to build the map. With busy data collection schedules and multiple interviewers in each session, this process was challenging logistically.

• To ensure consistency and rigor in how information was used for mapping, we conducted immediate debriefs following each interview. 
• During the interview and the debrief, interviewers made detailed notes about the new information they learned about goals, subgoals, decisions and information needs, including where in the current GDTA map the information could be added.
• We deductively coded interview transcripts to ensure all appropriate information was being captured.
• Codes were based on topics in the maps, e.g. preventive care and team communication/collaboration. A second coder reviewed and verified the initial coding.
• We reviewed all excerpts on each topic, and teams of 3-5 researchers met to build the information into the map structure. Review of the debriefing notes ensured that no important data was missed.

We conducted verification interviews in Clinics 5-8, with 1-3 sessions of up to 1.5 hours per participant. We divided the map into 24 sections by topic and selected 2-4 sections that were appropriate for a participant based on scope of work and areas of expertise identified during observation, with the goal of verifying all sections at least once.

• Verification interviews began with a brief introduction of the map’s high-level goals, and then review of the subgoals that the interview would focus on.
• Interviewers asked about the accuracy of the map structure, including probing for additional decisions made to meet a specific subgoal.
• Interviewers would ask about the information needed to make each decision. If new goals, decisions, or information needs were mentioned, interviewers probed to collect all needed data, following the procedure used in Step 3.

Minimal new goal information was obtained, often relating to practice innovations at clinics. After the interview, the two interviewers would debrief about new data, share this with the research team, and determine where to incorporate the information into the maps.

GDTA steps 2-4 proceed in a linear yet iterative fashion. Based on our domain analysis understanding of the differences in training, licensing and traditional roles in clinics, initially we built separate GDTA maps based on a person’s title. Notably, our observations and interviews showed substantial overlap in cognitive work, therefore we combined physicians and APPs into a single map and nurses and MAs into a single map. 

Interviewing and data analyses were also iterative in nature. As interview data were added to the GDTA maps, gaps were identified that were filled by questioning in subsequent interviews. For example, a physician mentioned the goal of ensuring that the patient and/or caregiver understand the care provided and the care plan.

For example, when building the map on patient education and how to provide it:

• The first physician described verbal cues used to assess the adequacy of education. 
• Subsequent physician/APP interviews sought information useful in assessing patient/caregiver understanding, e.g., the ability to accurately verbalize next steps for care, non-verbal cues, and prior patient success in following the care plan.