publications

Just like the managers of other business functions, communicators need to measure how effective our work is. However, the metrics need to be ones that we can act on—either to keep doing what we’re doing or to know what and how to change.

Ask “actionable” survey questions
To make the best use of metrics from surveys, it helps to ask the right questions with an appropriate range of responses. All too often, I see questions like the following ones, where the response scales range from agree to disagree, with a neutral point. It’s nearly impossible to guess what actions to take if most people disagree with the following statements:

1. Communication from management is open and honest.
2. New webcasts should be available once each quarter.
3. I receive too much email from the company.

The first question is actually two questions in one. The communicator has learned something only if everyone agrees. For those who disagree, we cannot tell how many feel that communication is open but not honest, or honest but not open, or neither honest nor open. I’ve seen companies with all three types of communication cultures. Unfortunately, I also see this question appear in nearly every large consulting firm’s database of employee engagement survey questions.

The second question is actionable only if the majority of your readers agree with the current frequency. If most of them disagree, you won’t know if you should increase or decrease the frequency. A better way to pose this question is to list various options for frequency and ask respondents to pick the one they think is right.

The third question suffers from a preconceived notion of what people will say. Again, you learn something actionable only if the majority agrees. One company that asked this question found that about one-third of respondents agreed, one-third disagreed and the other one-third were neutral. Yet, because of the way they phrased the question, their conclusion was that email needed to be reduced. In fact, the two-thirds who did not agree were likely either receiving the right amount or too little email, meaning that reducing the amount of email would not meet the needs of the vast majority. A better way to phrase volume questions is to ask if people are getting too little, too much or the right amount of email.

Avoid short cuts when more detail is needed
Even when the scale is more appropriate for knowing which action to take, asking the wrong “short-cut” question makes it difficult to track true progress. For example, I often see surveys that ask people if they’re getting too much, too little or the right amount of information about a particular subject. At first glance, it seems that if 30 percent say they want more information, giving them more information should make that initial percentage get smaller over time. However, it might not. Providing more information on a topic might fill the need for people who were already interested in the subject, but it will also make other people more interested in the topic, which will suddenly increase the percentage of people who want more than they’re getting.

The more actionable way to get at this issue is to ask people two questions instead of just one: Ask how interested they are in a subject and how well informed they feel about it. By subtracting the percentage who are informed from those who are interested, you can still see the “gap” of how many want more information than they’re currently getting. In addition, you can track progress in getting more people interested in subjects you want them to be engaged in, and track improvements in how many have become better informed.

With all survey measurements, in order to see improvements in the numbers over time, it helps to dive deeply into the data by subgroups. For example, you may find that some subgroups have very high scores on an issue while others are very low. Focus your action plans on the locations, departments or levels of your audience with the lowest scores to make your overall average go up for the next survey. It might also help to find out from the subgroups with the highest scores how they have been communicating. These best practices might be highly adaptable for your low-scoring groups.

Put usage metrics into context
Usage metrics are great because they are generated automatically from websites. Just be sure to look at measures like page views instead of hits. Hits can go up or down based solely on how many text and graphic files open automatically when a user clicks on a page. It’s impossible to take any meaningful actions by studying hit levels.

In order to take appropriate actions you need to first track your key statistics over time to see what makes them go up or down based on your varying approaches to online communication. Since most usage-tracking software doesn’t show trend data, you would need to create your own PowerPoint or Excel charts for key pages you’re tracking and update them each month.

To interpret usage statistics meaningfully, it often helps to put them into context instead of tracking just the raw numbers. Here are two examples:

• If you’re tracking usage of internal views of a particular online page, you’re likely to see sharp drops in usage during the summer months or local holidays. Those drops have nothing to do with the page suddenly becoming less interesting or useful. It happens because so many employees have gone on vacation. For internal metrics, it helps to find out from HR the average headcount actively at work for the preceding month, and then divide your page views by the number of active employees. This way you’ll be tracking the percentage of employees who are at work opening a page. Now when you see drops or spikes, they will tell you something actionable about the content you published in those issues.
• For online newsletters, don’t track the number of links clicked through per issue because the potential maximum number of click-throughs will vary depending on the number of links available in that issue. A more meaningful metric would be to track the number of audience click-throughs divided by the number of hyperlinks available in that issue. This percentage will tell you more about the types of content that are likely to drive greater interest in the details of stories.

If you haven’t known what to do with the measurements you have been collecting so far, try these three steps:

1. See if the survey questions generating the numbers need to be fixed.
2. Put usage metrics into context instead of tracking the raw numbers.
3. Dive into the data by subgroup to find where your areas of greatest need are, and where other groups may hold the key to best practices for other ones.