Going with their flow 

Editor's note: Lesleigh Campanale is manager, IEEE corporate - strategic research, at IEEE, Piscataway, N.J. Carole Schmidt is vice president of Chicago-based Doyle Research Associates. She can be reached at cschmidt@doyleresearch.com.

The Institute of Electrical and Electronics Engineers (IEEE) is the world’s largest technical professional organization dedicated to advancing technology for the benefit of humanity. There are more than 420,000 IEEE members in over 160 countries around the world, encompassing engineers, scientists and allied professionals whose technical interests are rooted in electrical and computer sciences, engineering and related disciplines. IEEE does a great job at meeting the needs of engineers working in academia, as researchers, but to grow the organization’s value, IEEE recognized that it needed to do a better job of meeting the needs of practicing engineers, those who work in corporate settings.

So, we set out to learn about these engineers: How do they work? What are the processes they use to get things done? Where are their pain points?
IEEE initially explored this research question by conducting a survey and running a couple of focus groups. Sure, we gained some useful insights, but we found that the level of information we really sought was more subliminal, more about actual – rather than claimed – behaviors, something that was likely too granular for engineers to explain in a group setting. We really wanted to understand those innate behaviors that people engage in for getting their work done; the things folks “just do” without thinking when they’re working.

Imagine that you want to understand how people eat lunch. If you ask people to explain how they eat, you might get responses like, “I drive to the pizza shop and grab a slice of pizza” or “I make a sandwich and eat it at my desk.” But that’s really just the surface level, isn’t it? How do you get people to tell you what they do not necessarily think about? That they habitually toast the bread for their sandwich first because they like that crisp bite when they eat it four hours after they make it. Good! That they hold a napkin in their right hand as they bite into that pizza slice to prevent the juice from running down their wrist and staining their shirt cuff. Better! That they always chew on their left side first, before their right. Bingo! Now we’re in business!

This was the type of challenge that IEEE brought to Chicago-based Doyle Research. Doyle’s prompts helped IEEE drill down to identify and define the problem more specifically, revealing the core research objectives:

• Workflow content. Identify the types of information most valuable in solving engineering challenges. Additionally, what are the terms engineers use when defining the components of their workflows?
• Workflow process. Explore how information is sourced. What tools are employed to use information and move through the workflow process? What human or non-human influences are most impactful? How is information shared? When and how is information stored? How do engineers re-access information? What happens to information when projects are completed?
• Role of IEEE among practicing engineers. How do practicing engineers view and value IEEE? What types of experiences do they or have they had with IEEE? What opportunities are there to make IEEE more relevant in their everyday practices?

Grounded in real behaviors

IEEE and Doyle Research carefully determined and monitored screening criteria to not only find practicing engineers in corporate environments but those active in the design and research stages of project work. This mindful decision ensured that the behaviors observed, the pain points noted and the insights gained around practicing engineer workflow were grounded in real, not claimed or recalled, behaviors.

From the start, Doyle Research executed the project fully online, to be consistent in interactions while making it convenient for engineers to participate. The project’s data collection consisted of three components:

Pre-mobile journal Webcam interview to confirm participant qualification for the study, to introduce best practices for using the platform upon which the workflow mobile journals would run, to build personal intimacy and rapport with the participants (in spite of the online collection method) so engineers knew their efforts were valued and, finally, to convey our high standard to capture granular detail that will uncover those subtle influences and latent behaviors in the day-to-day workflow engineers may not be aware of. Respondents were also incentivized with cash, paid after their journals and the subsequent interview were complete.

Smartphone mobile journals to capture multiple workflow occasions throughout the day, “in the moment,” when and where they occurred, whether it was in planning, research, design exploration or collaboration, etc. Doyle programmed the journal to refresh after each journal entry and post an alert. This alert reminded engineers to record their workflow experiences as they happened and the refresh allowed engineers to report up to four workflow practices each day, over the course of four days. Engineers used text, photos, audio and video (where it did not interfere with privacy/IP protections) to record “what I’m doing right now and how I’m doing it” with prompts for details. This journaling frequency was selected to enable the observation of a variety of workflow experiences while not being overwhelming to participants. As is the case with all depth research, some engineers recorded more workflow practices than others and some in greater depth but all participated fully.

A one-hour post-journal Webcam interview was conducted with each practicing engineer, about a week after journals were completed and reviewed. During these one-on-one interviews, Doyle’s moderator and the engineer together explored Doyle-noted observations, clarified attitudes and behaviors captured in the journal to understand drivers behind them and explored the perceived relevance and usage of IEEE in the engineer’s day-to-day workflow.

Four emergent workstyles

Doyle identified four emergent engineer workstyles (within the original segment that IEEE wanted to explore) that led to the development of a target persona for corporate engineers, an exciting and particularly useful surprise for IEEE. With a more thorough understanding of practicing engineers’ workflows, Doyle discovered numerous inefficiencies and pain points, some of which have produced opportunities for new IEEE products and a series of wireframes against which to test product concepts. IEEE considers this study to be one of its most impactful; the insights are used every day to bring key learning to various departments and to assist in product development.

True collaboration doesn’t end when the report is issued. At the close of the project, IEEE and Doyle examined the work-flow of the project to identify the attributes that made this collaborative research effort so successful.

Flexibility. IEEE initially requested focus groups with engineers to explore the workflow topic, hoping to identify patterns in emergent discussion, but the team realized that the granularity of engineer practices being sought required a more personal approach. IEEE’s team was open to a newer methodology – mobile journals – to more successfully answer their re-search questions.

Trust. IEEE had to trust the professionals they worked with at Doyle when they were presented with a new methodology for collecting information. Doyle conducted demonstrations of the Webcam and mobile platforms and involved IEEE in the interface with each method to build team confidence in the methods used.

Continuous contact with authority. Commissioning Doyle Research allowed IEEE to step back from the logistical management of the project and enabled the IEEE team to be more involved in the content of every activity in the project, critical to obtaining actionable insights. IEEE staff participated in weekly standing meetings with Doyle to keep the entire team updated and moving forward. The IEEE provided quick feedback on recruiting tools, guides, interviews and reports, having the authority to make decisions without delay. The IEEE team members observed nearly every interview and debriefed with Doyle afterward to utilize the learning for each subsequent interview. Finally, Doyle made transcripts of the mobile journals available in real time so IEEE could log in and review practicing engineers’ raw text, photo, audio and video responses when most convenient to do so.

Project champion. Because she expertly spoke two “languages” – “research” and “engineer” – IEEE’s internal research manager (Lesleigh) was autonomously able to authorize many project decisions, which kept the project moving forward as rapidly as possible. This was especially effective because Doyle Research had not previously conducted mobile journals among this particular audience. Lesleigh served as liaison and ambassador to IEEE’s product development team and senior management, keeping IEEE engaged, passionate about the project and well-informed about its progress and emergent learnings.

Iterative spirit. A valuable part of this longitudinal research effort was in applying research learning gained to subsequent interviews and journal follow-up, helping us dig deeper to understand the “hows,” that is, the specific steps, tools and compensating behaviors practicing engineers use in getting the projects they work on done.

Collaboration. The IEEE team remained patient and collaborative at each phase of the research to allow Doyle to synthesize the more than 45 hours of practicing engineer workflow data (text, photos, audios and videos). Doyle Research distilled the data for insights, then translated those insights into actionable implications and recommendations for IEEE. The team re-viewed and refined the research findings together and jointly prepared for its exposure to and discussion among the larger product development team.

Used nearly daily

Every researcher’s worst nightmare is that the work they do will end up on a shelf somewhere, collecting dust. This is not the case for the research project conducted for IEEE with Doyle Research. The study, Unveiling The Secret Life Of Practicing Engineers, has been used nearly daily by the IEEE product development team. The learnings continue to be shared throughout the organization. Further, the principles of a successful partnership (flexibility, trust, continuous contact with authority, project champion, iterative spirit and collaboration) are engrained within the IEEE research and Doyle Research team cultures and will continue to be applied in future projects together.