Market research organizations must react to slower growth and lower margins brought by commoditization, clients’ reduced budgets and proliferation of self-service research tools. Delivering exceptional research is one way to stand apart. But to survive and still profit, research agencies still need greater efficiency and cost-savings. This article is the first installment of a three-part series in which author Drew West, director, product marketing at Deltek, a Herndon, Va., provider of enterprise software and information solutions, draws from his experience helping research organizations such as Millward Brown and Chadwick Martin Bailey to explain how a research firm’s own internal “big data” can help it (1) accurately plan projects, (2) efficiently deliver the work and (3) effectively evaluate results. —Joseph Rydholm, Quirk’s editor

A profitable research project hinges on every worked hour, billable or non-billable. With the pressures facing your research organization, wouldn’t you like all projects as profitable as possible? Since nobody plans for an hour to be unprofitable, maybe your plans need better information.

Big data – inside your firm. Think of all the operations information created during your research projects – from proposal to execution, on to delivery and finally, invoicing. Imagine how much internal data about past projects and current work is being created at each step. But if your data is typically spread across many different systems, it loses all relationships among scope, resources, time, expenses or revenue – and leaves planners with an incomplete picture of the firm’s activity. So proposals, resource schedules or project budgets are unguided and inaccurate – leading to unprofitable projects.

Big data, little leverage. Researchers consider data “big” if its volume creates severe challenges in aggregating and analyzing it. Your internal big data is no different – so much data, in so many places, with so much possible interpretation that any planner might hesitate to use it. Makes sense. But what happens instead?

Planners rely on experience, judgment (or gut) to scope work, staff jobs or develop budgets. But this guesswork leads to inefficient work and unprofitable time. We see research firms of all kinds:

  • Ignore the past. The business-development manager who can’t review the profitability of past work can’t determine if the same scope is still accurate for identical project tasks in the new proposal. Risk: When the actual work takes longer, does the firm have to absorb the cost of that time?
  • Neglect the present. The resource planner who can’t determine actual time-to-complete status on open jobs doesn’t have an accurate view of true resource capacity. Risk: Firms with capacity concerns may postpone or choose not pursue new revenue opportunities.
  • Avoid the future. The project manager who can’t anticipate the pipeline’s expected resource needs may assign resources to other tasks. Risk: If the new business is won, will the necessary resources be available?

Wouldn’t visibility to past work, current jobs and even planned work would lead to more accurate plans? Planners of all kinds need better ways to aggregate data, analyze it and decide on the best plan.

Aggregate – bring it together. The above examples illustrate the many cause-effect relationships within your internal big data. The problem is, they’re mostly unexposed because the underlying data is typically spread across so many different systems – such as a business-development system, a project-management tool, some kind of timekeeping spreadsheet and perhaps a separate financial package. (Big data – on lots of little islands!) These disconnected systems also have duplicate data (so you don’t know which to trust) or outdated information (so status isn’t accurate). No wonder planners can’t aggregate the planning information they need!

A path to better plans means bringing your firm together on a single management system. That way, the critical relationships among information are established across the entire firm.

Analyze – see the whole picture. From front-office to back-office, across all stages of the research project, planners need the full context – easily visible and always current.

  • New business. Not all planners need direct access to the pipeline, but necessary pipeline details should transfer to a resource-planning view, to a finance view, etc. 
  • People. Planners must see what people are working on – and what they’ve done in the past, their unique skills and even kinds of work they aspire to do.
  • Projects. Have a complete project view – the tasks and who’s working (or worked) on them, work completed and work remaining. Be able to identify actual margin against expected margin – so anticipated results of current work can guide decisions about upcoming work.
  • Time. Almost all plans must rely on worked hours in some way. Ensure time capture is immediate, accurate and complete. 
  • Clients. Planners need multiple lenses into your firm’s relationship with the client – work done, work planned, work-in-process – along with results of past work and complete history of the resources involved. 
  • Financial results. Of course, reasonably expose financial results like profitability (past and expected) to help planners guide decisions.

Decide – past, present, future. Constantly-growing information about past history, present status and future plans evolve your internal data into big data. Why not leverage all three contexts in your planning decisions?

  • Past. Leveraging history leads to more accurate proposals. See which resources were used, how long work took against expectations, how profitable it was. 
  • Present. Even as “right now” constantly changes, an accurate view of current work-in-process gives an equally accurate view of current resource capacity. Nimbly adjust resources across tasks, assign them to new work, eliminate unnecessary down-time and avoid expensive contract resources.
  • Future. Evaluating the likelihood of proposed future work determines if capacity is sufficient – and guides immediate staffing, medium-term resource planning or long-term financial budgeting decisions.

Moving ahead – look inside. Is your firm gaining a planning advantage from its own internal big data? Consider past research projects that didn’t have the profit you’d like. In hindsight, can you identify some ways more accurate planning might have helped avoid the issues that dragged down profit?

Now, turn to your internal data. See how it’s being used in the planning process. Find out what information planners have – and need – to establish views of the cause-effect relationships so critical to planning. If barriers among your internal big data prevent you from taking full advantage, look for ways to pull it all together.

In part two we will address ways to use internal big data to help your research projects deliver the profit margin you expect.


Drew West is director, product marketing in Deltek’s Woburn, Mass., office. He can be reached at 617-528-2331 or at andrewwest@deltek.com.  For more information visit www.deltek.com.