A Checklist for PR Practitioners When Considering
Measurement or Evaluation Research
By Walter K. Lindenmann, Ph.D.
1. Don't reinvent the wheel. There's a good
chance that someone else has already done similar measurement
and evaluation research. Start out with a review of
what's already been done, by doing searches online or
checking the web sites and databases of organizations
such as IABC, PRSA and the Institute for Public Relations
Commission on PR Measurement and Evaluation.
2. Be flexible. Don't limit yourself to traditional
data collection techniques such as telephone polls and
focus groups. There are lots of other methodologies
you might consider that can provide richer and more
valid data, such as shopping center or mall intercept
studies; mail surveys; doing a content analysis of what
appears in the media or in other key publications; collecting,
coding and analyzing the many inquiries an organization
may receive from its key publics; conducting observation,
participation, or role-playing studies; doing face-to-face
or in-person interviewing; carrying out a fax poll;
fielding a survey online; doing a panel study; or piggybacking
some questions on a large market research company's
nationwide omnibus poll.
3. Define beforehand what you want the research to
do for you and how the results are to be used. This
will determine if a descriptive survey is good enough
or if explanations as to why people have the particular
opinions and attitudes (or behavior patterns) that they
do also are needed. It also will determine if measuring
opinions is sufficient, or if a better understanding
of attitudes in general is required. The eventual use
of the research has to determine how precise and sophisticated
the findings need to be.
4. Remember that for most PR practitioners, "why"
is far more important than "what" or "how."
It is relatively easy in research to find what people's
views toward given topics or issues are or how they
feel about something. But obtaining "what"
and "how" information, while useful, has its
limits. What is really needed is the "why"
- why do people feel the way they do and what did it
take to get them to change their minds? To get at the
"why" may require more of an emphasis on qualitative
rather than quantitative research. Or, it may require
the researcher to ask more open-ended, or free-response
questions in the quantitative research.
5. Give serious thought to the TOTAL population to
be studied, and after that has been decided, to the
appropriate selection of the sample that will represent
that population. Samples are usually of two types
- those selected randomly (a proportional sample) and
those selected judgmentally (a purposive sample). Random
samples are often stratified or clustered (selection
by chance, by specific audience segments or on an area-by-area
basis). If a sample is selected judgmentally (in other
words, not by random or chance methods), then questions
ought to be raised about what criteria are being used
to select certain respondents and not others. For example,
in measuring consumer attitudes, a random sample probably
makes more sense. But in seeking to determine the attitude
of the media toward a corporation and its industry,
a judgmental sample might be most effective. The basic
question to be asked when drawing a sample is how precise
and unbiased should the sample be in order for the study
to have any reliability and validity.
6. Consider the eligibility of potential respondents.
Keep in mind that for a given survey, you may wish to
interview only certain segments of the population (e.g.
members of a particular minority group, individuals
who fall into certain income categories, etc.) or individuals
who have a special knowledge or expertise about the
items being covered in the study. Therefore, it is crucial
to consider your screening criteria for potential respondents.
Ask yourself, are the individuals that you are planning
to interview the appropriate individuals to be included
in the sample frame for your survey?
7. Spend a good deal of time on the survey instrument
that will be used when collecting the data. All
too often, questionnaires are written in haste or in
a haphazard fashion. It takes time and pre-testing of
the survey instrument beforehand to come up with a reliable
and valid questionnaire.
8. Find out as much as you can about those individuals
who will be doing the interviewing or collecting the
data for you. Very important when conducting in-person,
mall intercept, telephone and focus group studies.
9. When focusing on the fielding of a study, consider
the 'no-shows,' the 'refusals,' and the 'no answers.'
Having several non-respondents or "don't knows"
may not mean that there are problems with the target
audience groups under study. It may mean that the researchers
have not fully considered the question-asking and fielding
process.
10. Protect the confidentiality and anonymity of
respondents. There are many codes of ethics that
research organizations rely on when they design and
carry out research projects. Most of these codes protect
the confidentiality and anonymity of those individuals
who are interviewed or who are participating in the
studies. Be sure to protect the confidentiality of your
own research project. If your organization sponsored
and paid for the research, the findings belong to you.
Make sure that the research supplier never refers to
or passes the research findings to others without your
permission.
Dr. Lindenmann is an independent consultant, specializing
in public relations research, measurement and evaluation
services. He can be reached at lindenmann@cstone.net.
Discuss this topic with other IABC members at: www.iabc.com/memberspeak.
|
