The communication audit has become a popular tool to measure audience satisfaction with the content and packaging of information. Typically, these audits are designed as surveys and/or focus groups that solicit reactions to important elements of the way that communication is managed, such as choice of media, relevance of topics, frequency and timing of publications and meetings, and the workplace climate. While this input is useful, it's what we might refer to as a "level one" assessment.
Today, most of our clients and sponsors are looking for more: They want to know whether the audience actually comprehended the material, whether they used it and whether its use had any impact on business outcomes. In the world of training and development, these are often referred to as "levels of evaluation," based on a popular model developed by Donald Kirkpatrick, as discussed in his 1999 book Evaluating Training Programs: The four levels. The levels of evaluation are:
Level One: Satisfaction
Level Two: Learning
Level Three: Transfer of Knowledge to new behaviors
Level Four: Impact on Business
Level Five: Return on Investment
For example, a Level One assessment might capture what your employees thought about the content, writing style and choice of intranet delivery of weekly sales tips. A Level Two assessment would capture whether they actually learned anything from these tips; this might be assessed by a quiz. A Level Three analysis would try to determine whether new techniques learned in these tips were actually used by your sales staff; this might be done by interviewing the sales representatives or their managers, or observing them in action. If you were able to determine that the sales representatives were indeed enacting these new behaviors, you'd try to determine whether they had any impact on business: Did these tips increase sales prospects, the number of sales, or the sales revenue? Finally, you'd try to determine ROI by capturing the cost of producing and disseminating the sales tips and comparing that to the direct business results. Needless to say, most communicators never get beyond Level One in their audits.
Unfortunately, even sophisticated measures of ROI often don't capture the true value of communication. That's because a lot of what we do as communicators is build systems—not just produce individual messages or discrete campaigns. For example, you might be able to measure the ROI of a six-month PR campaign on a new product or service—you could tie increased sales to the campaign by doing a before-and-after analysis. But let's say that you're in charge of the employee on-boarding and mentoring process—a program that supports an employee through his or her first two years at the company. How would you measure the outcomes of this? Certainly there may be many positive outcomes, including reduced turnover and increased engagement—but these may not show up for years. Communication programs like these are better considered as infrastructure assets, and their results need to be measured in other ways. These ongoing communication systems add to the long-term valuation of a company, as does a strong brand, a patent or a strategic location. I represent this concept as Layers of (E)valuation (Gayeski, 2005), ranging from evaluating the immediate satisfaction of clients or the audience (return on expectation), through the results of quick fixes and longer-term projects, to relatively stable systems that become part of the organization's valuation.
The layers of (E)valuation are as follows:
Immediate satisfaction
Quick fixes
Long-term returns
Infrastructure assets
However, whether we perform simple audits or more extensive business analytics, we need to keep in mind that we are only measuring what we have in place, or what our audiences might recommend. To truly examine your communication system, you should use systems design methodologies to model your current and ideal communication infrastructures. For example, a graphic model of an existing communication system might look something like this.
As the demands placed on communicators become more sophisticated, so too must the methods that we put into place. By re-framing our function from being message producers to being engineers of the organization's communication infrastructure, we can position ourselves as strategic partners and direct influencers of the valuation of our companies.
References
Gayeski, D. (2005). Managing learning and communication systems as business assets. Englewood Cliffs, NJ: Pearson / Prentice-Hall.
Kirkpatrick, D. (1999). Evaluating Training Programs: The four levels. San Francisco: Berrett-Koehler.
Diane Gayeski, Ph.D., teaches, consults and does research on organizational communication. She is associate dean and professor of strategic communication at Ithaca College's Roy H. Park School of Communications in New York, and she provides consultation through Gayeski Analytics. This material is derived from Dr. Gayeski's forthcoming book, Managing the Communication Function, second edition, to be released by IABC. Contact her at diane@dgayeski.com or visit www.dgayeski.com.
|
|
