PR measurement and math-challenged communicators

It’s an amusing sight, a group of PR practitioners at dinner trying to figure out the check and the tip. Most communicators don’t do math. It’s a right-brain left-brain thing. Lack of math skills led most of us to writing as an avocation to begin with. I’m always glad when Angela Sinickas is at dinner with us. She’s a math whiz. But you’d expect her to be, since communication measurement is her area of expertise.

Angela’s not alone. Tudor and Ryan Williams also specialize in measuring the effectiveness of communication efforts. Both Angela and the Williams boys are excellent at what they do. I’ve often lamented that their businesses have not grown to the point where they need offices in multiple cities, but I haven’t been surprised. Measurement just isn’t important to most math-challenged communicators or budget-conscious companies. Communicators would rather churn out cool-looking and brilliantly written collateral while companies see measurement as an effort by consultants and agencies to squeeze a few extra dollars out of the project.

I’m equally unsrurprised by the lack of attention given to Andy Lark’s December 18 post about the reults of a Holmes Report “survey of more than 100 PR Agency principles—the results of which point to some of the problems that continue to make research and evaluation a major issue for the industry.” Measurement…math…yawn…

Only three comments were submitted to the post—one by Auburn academic Robert French. You expect the professors to have a better grasp of math than practitioners. But the simple fact is that PR efforts will never gain the kind of respect we crave until we can quantify the results in terms that resonate with the people who hire us. Lark quotes from the survey:

“In general, their responses suggested that an failure of commitment- rather than the absence of necessary tools and techniques—is behind the industry’s poor performance.”

Lark’s conclusion mirrors precisely my own beliefs about measurement:

I just don’t understand how any PR department or practitioner can operate without a measurement program in place. Measurement isn’t monitoring. Monitoring isn’t measurement. What I am talking about here is a deep understanding of how communications impacts business outcomes. An understanding based on research not just of what occurred in the media, but also in the minds of your customers. Without that you shouldn’t expect resources, budget, even a job.

Still, there’s only one PR measurement blog that I’m aware of—the one from Katie Paine—and it isn’t updated nearly enough. What will it take to make measurement an integral part of any communicator’s planning process? I admire the hell out of Steve Rubel’s effort to get agency honchos talking about practical implementation of new media in their efforts. But maybe they need to worry about the fundamentals before they scurry on to the Next Big Thing. Whether it’s a traditional media campaign or a blog/wiki combo, it’s not complete without the measurement component.

1. When you DO measure, or play with numbers in any other way, you should at least have access to a math-savvy advisor. Sooner or later someone will ask you to report that something is 37% faster or reduced by 82%, and when that happens you REALLY need to check the math, especially with percentages.

   (I may be the world’s only “B. Math, ABC”)

   Posted by Tim Hicks  on  12/21  at  10:16 AM

2. I work at a large R&D driven company and it wasn’t until we started measuring our comms performance (especially internally) that leadership started to take us seriously. Any number of humanistic theories meant little to the leadership, but show them a multivariate analysis and we’re talking!

   
   Since then our comms organization has nearly doubled in size, management are aware that they need to do much better in their personal presentations, we are more often on-message, we have better focus on our channel management and finally we can follow long-term trends in the organization.

   I would suggest in any large organization where R&D counts, measurement is your best way to comms success. It doesn’t hurt in other organizations either! Measurement and documentation is simply part of being professional. Now to convert HR too.

   Posted by Mark  on  12/21  at  02:12 PM

3. Thanks for the real-world validation, Mark!

   Tim, we don’t really need to DO any math (thank God), because plenty of PR measurement tools exist. We just need to start using them, and not as an add-on but as an integral part of the program.

   Posted by Shel Holtz  on  12/21  at  02:55 PM

4. I wish there were more discussions about this in PR blogs, Shel.  Yes, the respect for the discipline is required and why practitioners don’t (more often) employ depth in their pre/post evaluations, I still don’t understand.  It is one (or the) path to acceptance and respect.

   Your reference to math made me smile a bit.  I imagine you can envision the looks I receive in classes when we discuss “Points and Picas” (conversions) in Style & Design or crosstabs, t-squares and margin of error in Survey Research. 

   I also agree that measurement - alone - isn’t enough.  The efforts should run the gammit (pre/post) including everything from audits to focus groups and surveys to standardized tests.    And, those just scratch the surface.  All in reasonable application, of course.  Where it makes sense.  Yes, the tools exist.  Why aren’t they utilized more often?

   KDPaine’s blog is a good one.  One thought is that she has so few PR/Marcom specific examples to write about.  I refer to the (a) minimal amount being discussed, (b) the minimal amount probably being done, and (c) the claim (albeit legitimate) of it being proprietary information.

   Now, there is a transparency discussion I can get my mind around.  Why isn’t more public discussion of these practices taking place?  Hmm?  Another student blogging exercise?

   All the best.

   Posted by Robert French  on  12/21  at  06:44 PM

5. We’re working on something up here in Toronto.  What started as an ad hoc group of colleagues in the Toronto industry has led to the Canadian Public Relations Society making us a full-fledged national committee.  While we’ve made a little local noise with our peers things will really get rolling in the new year.

   Now, don’t expect too much on the outcomes side just yet.  We’re keeping with the nailing down some of the fundamentals to begin with.  We’ve come up with a tool to effectively measure media coverage effectively and consistently with a CPRS-endorsed methodology and a centralized data resource.

   The hope is that this gets the ball rolling and we come up with new tools that measure other outputs i.e. blog relations, street teams, internal comm and the like. 

   What’s exciting is that we’re pushing this out as an ope-source, user-driven tool.  We’ll have a blog that I hope will stay current with measurement info, white papers, examples, etc. and an old-fashioned discussion board where users can share best practices, ideas to make the tool better, rant, complain, whatever.

   I’ll keep you posted.

   Posted by David Jones  on  12/21  at  07:09 PM

6. >>Tim, we don’t really need to DO any math (thank God), because plenty of PR measurement tools exist.<<

   And yet, it wouldn’t hurt communicators to be forced to do so. My father is an electrical engineer and his response to my questions about why I should take math, algebra, calculas, etc. when I had no plans for a “hard sciences” career was simple.

   “Because it teaches you how to think,” he said. “It teaches you how to find solutions to complex problems which will be valuable to you regardless of what field you go into.”

   He was correct, of course. While I did go on and take business courses in college (which should be REQUIRED for anyone who wants to call themselves a “business communicator,” but that’s an argument for another day) it was the thought discipline learned by taking math in grade and high school that has been the most enduring benefit to me throughout my career.

   Posted by Craig Jolley  on  12/22  at  08:07 AM

7. Hey Shel. For the record, math was always my worst subject.

   You may have noticed a brief exchange in the IABC Café between myself and someone who was suggesting that, “There’s better things to spend your money on than research, if you’re any good and money is tight.” My response was that research should be a basic component of any communications initiative.

   Too often, as practitioners, we have not fought the good fight with our employers or our clients. As a consultant, I spend considerable time and energy convincing clients that it is simply bad practice to ‘skip the research part’. Fortunately, having spent most of my career in the public and not-for profit sectors, I have the speech down pat. Even if money is tight, research is critical. Otherwise you are simply shooting in the dark and more than likely throwing good money after bad.

   Posted by Warren Bickford  on  12/22  at  01:44 PM

8. Hey, Warren.

   So there are three elements to consider here: research (up-front), monitoring, and measurement (to assess whether outcomes were met). If those were split out in the kind of research Holmes did, I’d bet the results came out the same—people aren’t doing them.

   Posted by Shel Holtz  on  12/22  at  01:54 PM

9. You’re only partially right, Shel. Yes, I don’t update my blog nearly enough, but that’s partly because I’m too busy fielding calls from all the people out there who are starting to get “measurement religion” and want to know how our dashboard system works. Seriously though, the number of calls we’re getting has skyrocketed in the last six months. The good news is that at least as far as my narrow little world goes, there are twice as many people measuring now as there were a year ago. Even the state department has announced a “culture of meausurement” so there is hope out there. Where Andy is dead on is that PR Agencies, with the exception of a handful of really forward thinking folks like Kaplow and TechImage, are still in the 19th century, viewing measurement as something they have to do to keep the account, rather than something you need to do to make better decisions.

   Posted by KDPaine  on  12/23  at  03:31 AM

10. Bouquets to the group in Toronto for getting involved in some knowledge sharing around measurement. I would urge David to quickly expand the group’s focus in to the other two critically important aspects of measurement - the outcomes achieved, in terms of attitudes and behaviors chnaged, and the business impacts that result. Counting the outputs is like counting cars on the highway - the tallies don’t tell you where the drivers want to go, how they intend to get there and what happened when they arrived. And that is the flaw in many of the measures in PR and business communications. It’s a lot easier to count outputs but figuring out our objectives and goals as outcomes and impacts will resonate with our business leaders far more and give us a more credible case for the value we add.
    The web site of the Institute of Public Relations has spome great material (free) on measuring outputs, outcomes and impacts.

    Posted by Tudor Williams  on  12/23  at  11:37 AM

11. I think that’s great, Katie, that you’re busy. But you’re not nearly as busy as you should be. If companies (clients, etc.) understood the critical importance of measurement, you’d need to open offices in 50 cities and hire savvy staffs to handle the demand. Or, nobody would need specialists because every practitioner would already build solid measurement into every client engagement.

    Posted by Shel Holtz  on  12/23  at  12:25 PM

12. Great post - couldn’t agree more.

    Anyone who is interested in the quantitative side of PR should take a look at our website/blog.

    We measure influence on issues and brands.

    For examples of our work take a look at our blog (http://www.onalytica.com/blog) or website (http://www.onalytica.com).

    Examples of our work includes influence measurements in relation to Wal-Mart, Dell, Kleenex as well as lots of political and general commercial issues.

    I’m keen to get in touch with people who are interested in the quantitative side of PR. Where is the best hangout?

    Sincerely,
    Flemming

    Posted by Flemming Madsen  on  12/24  at  02:57 AM

13. Shel, Flemming, I started (on the NewPR Wiki) a page on PR measurement, just in case you need a discussion space:

    http://www.thenewpr.com/wiki/pmwiki.php?pagename=PRMeasurement.HomePage

    Posted by Constantin Basturea  on  12/24  at  09:02 AM

14. Thanks for doing that Constantin.  A space for ongoing sharing about the subject of measurement will be a great resource.

    Tudor, I couldn’t agree more with expanding our project in Toronto to include ways to measure outcomes.  As you know, we can’t even really agree on how to simply measure the outputs, so we’re starting there first.  I hope that we will stimulate thinking about measurement in general and really push each other to figure out ways measure outcomes effectively. 

    I hope to see all of you on the NewPR Wiki.

    Posted by David Jones  on  12/24  at  08:10 PM

15. Thanks for providing the link to the PR measurement page Constantin! This is an area I’d like to explore further with my clients as well.

    Posted by Mark Turok  on  12/26  at  04:34 PM

16. It’s not about math, it’s about a rational approach to making decisions.

    It’s true that people who are good at math also tend to think methodically about solving problems. But the big picture with PR measurement is not so much about math, but about using the right tool to plan and evaluate the effectiveness of one’s work.

    Take your example, Shel, of figuring out who owes what on the lunch bill. Yes, it is important to be able to do the algebra, but it is more important to realize you need to figure it out in the first place. (You can always hire someone to do the math for you.) It’s what you quoted Andy Lark calling a “failure of commitment.”

    Perhaps part of the difficulty here is that PR measurement has a confusing name. “PR measurement” is a name that implies that something is evaluated. But the real value of PR measurement is what you do with the evaluation after you make it: how that evaluation is used as a tool for decision-making. The evaluation is just the math, and you can always hire someone to do the math for you. The really difficult part is to know what you want to use the math for, to know what questions you want answered. And what to do with the answers when you get them.

    Katie Paine (with whom I work on The Measurement Standard, http://www.themeasurementstandard.com) is fond of dividing the world into word people and
    numbers people, implying (as you have, Shel) that word people might never be able to get PR measurement. You may be right. But I say that the numbers are just a tool to help make informed decisions. Can a “word person” learn to do PR measurement? I’d answer that by suggesting that there are plenty of CEOs out there who run profitable companies but who aren’t necessarily good at math, they just know the right questions to ask, and who to get the answers from.

    Thanks for starting up a great train of conversation, Shel.

    Bill

    Posted by Bill Paarlberg  on  12/29  at  09:04 AM