Proving Value through Strategic Impact Measurement – ROI & ROO (2/5)

In this post we want to show why measuring is so important and how you can achieve best results. In our last blog post, the first of the series around the topic of impact measurement and experience design, we elaborated on what actual meeting costs are.

In recent years, we have witnessed change in organizational accountability, especially toward investment in people, programs, and projects. In the communications & event industry stakeholders are clearly becoming more and more numbers-driven and structured. Why? Because measuring impact and ROI (return on investment) is not only a controlling and evaluation tool. It also serves as a continuous refinement and development tool that helps develop people and company cultures. Furthermore, the measurement process helps defining clear and measurable objectives at the very beginning of the design process.

So, how can we measure the monetary or non-monetary value of a communication campaign, a meeting or an event? Meaning, how can this principle of the monetary return on investment (ROI) and non-monetary return on objectives (ROO) be applied to our industry?

How VMP generally measures meetings & events

We start with analyzing the needs & expectations of the stakeholders as well as the meeting/event purpose and develop – together with our clients – clearly defined and measurable objectives. Based on that, it must be crystal clear what parameters will be measured. Finally, deep thought as well as profound analytics go into the survey design and the questions’ wording and phrasing.

At VMP we apply the industry-standard ROI Methodology framework by Phillips (2008)¹ and enrich it with the Theory of Planned Behavior (TPB) by Ajzen (1991)² that focuses mainly on the non-monetary ROO.

ROI Methodology¹: There are five levels of data to the ROI Methodology where the meeting objectives – often predefined by the meeting owner – can be classified in (cf. illustration below). The sequence of objectives (Level 0 to 5) is called “Chain of Impact”. The higher we climb the Chain of Impact, the more challenging the objectives are to evaluate and measure. Results are measured in the direction from Level 0 to Level 5. These levels are logically connected to each other.

 

Theory of Planned Behavior (TPB)²: his theory incorporates aspects of attitudes towards behavior³, subjective norm⁴ and perceived behavioral control⁵ into the measurement process. Together these interacting antecedents can either strengthen or weaken the individual’s intention to perform the behavior in question. The theory itself is “well supported by empirical evidence” (cf. Ajzen, 1991). TPB fits within the ROI Methodology (cf. illustration below) and has similarities when measuring ROO on Level 0 to 3. Furthermore, it adds a deeper understanding of the psychological aspects of behavior change in humans and offers measurement tricks.

In addition to the disciplines of psychology, sociology and human behavior, we use and apply our knowledge of behavioral economics not only while measuring, but also when designing the meeting.

A wide range of data collection methods can be applied. For example: Surveys by self-reporting (cf. explanation below), surveys by proxy⁶ , estimation methods (by focus group, participants or other stakeholders) and many more.

Example Meeting Measurement

In our example, we focus on the measurement of a stand-alone meeting with the general purpose of a cultural change – and assume that a) the objectives and b) the target audience are already fixed when measurement starts.
The VMP Model portrays the level of comparison that is based on the ‘before and after’ principle (cf. illustration below) where we implement surveys (cf. green boxes) over time to closely monitor the long-term behavioral change:

• Pre-meeting Survey 1: Before the meeting starts.
• Session Surveys: Right after every session at the meeting.
• Post-meeting Surveys 2 to 4: Survey 2 after the meeting, Survey 3 around three months after the meeting and Survey 4 around 6-9 months after the meeting.

When designing the survey, the focus is always on the objectives.

 

 

What are the benefits of measurement?

List of examples of benefits to our measurement approach (non-exhaustive):

• Presence of objectives: Survey itself can be an implicit reminder to implement the objectives. Additionally, Survey 1 can serve as a tool for participants’ expectation management.
• Tool of communication & support: Results of the surveys can be used to take further communicative action with the participants.
• Benefit to the meeting design: Results of Survey 1 can lead to a better meeting design as participants’ lack of knowledge are recognized at an early stage and the meeting design can still be adapted to the real needs.
• Measurement of relative and distinctive impact: Survey allows to point out which improvement is substantial and which only marginal (relative impact). As an example of a distinctive impact: Results could show that those who knew more before the meeting learned more at the meeting as well.

Conclusion

Research shows that within the meeting & event industry ROI is an important and challenging topic: A study of EventMB unveils that 54 % of the corporate event planners prioritized the topic of ROI in 2018 as one of their biggest concern.⁷

Generally, the higher we measure with regards to the levels in the pyramid of the ROI Methodology, the more challenging it gets. However, it is not always necessary to go through all levels. But it is always a must to have clear objectives and measurable parameters to start a successful measurement process.

 

Do you want to know how we can help you to measure the impact of your campaign/program or meeting/event? Contact us today !

___________________________________________________________________________________________________________________

More of our series on impact measurement (ROI and ROO) and efficient experience design:

• A Call for Transparency – Total Meeting Costs (1/5)
• Target Audience and the Importance of Personas (3/5)
• Perfect Meeting Environment to Optimize ROI & ROO (4/5)

___________________________________________________________________________________________________________________

List of references: 

¹ Phillips, J et al. (2008), “Return on Investment in Meetings & Events – Tools and Techniques to Measure the Success of all Types of Meetings & Events”, New York: Routledge.
² Ajzen, I. (1991), “The Theory of Planned Behavior”, Organizational Behaviour and Human Decision Processes, 50, 179-211.
³ Refers to the degree to which a participant has a favourable or unfavourable opinion or rating of the objectives of the meeting.
⁴ Refers to the perceived social pressure or accordingly support to execute the objectives of the meeting.
⁵ Refers to one’s personal beliefs about the ease or difficulty of performing the objectives of the meeting. It can be strengthened by education at the meeting.
⁶ For example: if a meeting focuses on cultural change and the target audience consists of top-level management, non-participating team members can be questioned about the roll-out within the company.
⁷ Survey conducted with 1000 respondents.

---