Modern trade promotion optimization (TPO) is far more demanding than consumer packaged goods (CPG) industry executives understand. Oversimplification of TPO is dangerous, costly, and, in the end, fails to achieve the business objectives. The most important aspect of optimizing trade spend is knowing and employing the right data. Surprisingly, few hit the mark.
Trade promotion management (TPM)/TPO is itself becoming more of a priority for operational process and technology transformation across the entire consumer products industry. As we pointed out in the previous blog post regarding revenue growth management (RGM), the data driving effective planning, execution assessment, and analysis of promotional performance is a mission critical component of this new RGM renaissance. Optimization is the process of leveraging that data to predict and prescribe future promotional success.
So, let’s take a closer look at the data that feeds a successful promotion optimization outcome and how even the most effective CPG companies often continue to lack precision and trust in their TPO results.
I often ask this first question in the planning workshops for my clients which are about to implement a TPO solution. The typical answers include:
OK, all of these are true for sure, but when the hard work of developing the complex algorithms needed to deliver the optimized figure, the answers are not always clear, or worse—unbelievable. Technology continues to grow in TPO functionality, but we have a long way to go.
RGM leadership will tell you that generating profitable revenue is the ultimate goal. So, delivering the specific objective against the forecast and, more importantly, controlling the outcomes will be the significant benchmark of good TPO.
Invariably, “ROI” is going to be mentioned as a primary, if not THE, reason for a TPO in the first place. But then I ask my second question: What IS ROI?
If this was a simple answer, the corporate integrated annual business planning process would take place in a single afternoon! However, there is a complex interrelationship between so many elements of the forecast, promotion plan and consumer engagement that has to be considered. The data required for the preparation and ultimate approval of a corporate annual forecast and operational plan are the “building blocks” of this mission critical effort.
These “building blocks” of data are intended to work in concert with each other to ensure the most effective outcomes. In my book, “The Invisible Economy of Consumer Engagement,” I spend a lot of time discussing the individual fiefdoms of a typical CPG company and how each leverage these building blocks of data to define their specific performance objectives.
The sales organization continues to concentrate on volume and profitability, of course, but historically, they have been less concerned with the actual performance of the promotion than they are the incentive they provide to their account buyer to agree to a specific volume of product purchased.
The supply chain organization sees the achievement of the forecast from the perspective of the physical storage, transportation, and safety of the products from the plant to the retailer’s loading docks. They look at time, cost of transportation and warehousing, and reliability of production.
Marketing tracks the consumer demand, determines product messaging, and competitive positioning.
Finance and IT all have their roles and requirements, primary among them being keeping costs down and developing sophisticated technologies to ensure efficient and effective operations.
So much focus and attention is now given to TPO technology as millions of dollars are invested in building an efficient optimization engine. Add to that the frustration in finding and securing the right data and you have a perfect storm that most CPG companies want to avoid.
This is all too often the case as I speak and work with CPG companies, helping to coordinate annual business planning processes. The number one reason for the length of time it takes for integrated corporate business planning is the time it takes to gather, harmonize, align, and analyze all the data. If you have ever been involved in this process, you know how difficult, frustrating, and often emotionally draining it is on all the participants.
In a typical TPO integration, most companies will include data from historical trade promotions, point-of-sale (POS) data directly from the retailer (typically accounts such as Walmart, Target, Kroger, Home Depot, etc.), as well as indirectly from syndicated data providers. Also, more sophisticated TPO solutions will include baseline, demand plans, and other S&OP (sales and operations planning) data sources.
However, while these integrations provide a set of data for basic “what-if” scenarios, data is often inaccurate or incomplete—rendering output that is flawed and dangerous to fully depend upon. An example of this is the lack of details within a typical three-to-five-year historical accounting of trade promotions.
The old saying, “Garbage in, garbage out” definitely applies here.
In the following graphic, the red dotted lines represent an ideal picture of the data necessary for true optimization—the eventual type of predicted outcome we will all see in the future. Some leading CPG companies already integrate data from marketing and logistics (supply chain, shipping/receiving, and warehousing), but the application of that data to a true projected outcome remains very rare.
If the TPO technology is sophisticated enough to provide a large enough array of scenarios based on the current scope of data, there should be enough data to populate those scenarios. At a minimum, the data represented in the graphic would be required.
Most CPG companies lack detailed historical data on prior promotions. Increased sophistication with TPM solutions has helped with this issue, but there are still too many great companies that continue to rely upon spreadsheets where, even if the data is there, it is virtually impossible to extract.
This is especially true for promotions that actually deviate from their originally planned content. Post promotion functionality such as claim and deduction management does not usually allow for a detailed update of the changed terms and content. If the TPO runs predictive scenarios based on only originally planned promotional data instead of the actual performance, your performance analytics and future optimization is bad.
Let’s take a closer look at what is considered a strong database for minimally effective promotion optimization:
It is necessary to maintain three (3) years of data in trade promotion history. This ensures you can create a full 12-quarter cycle of promotional performance for the account.
The more detailed the promotional data is, the more accurate the projected values from an optimization algorithm.
This is the single most important data in the process of effective promotion optimization. It is, after all, the final result of the promotion—what crosses the scanner at checkout. If this data is not exact, you cannot generate a true picture of the end result of the promotion. POS data has been provided to the manufacturer/supplier by two distinct routes: (1) direct from the retailer, or (2) through promotional analyses delivered by syndicated data providers.
The syndicated data providers will base their analysis of promotional performance on sampling methodologies of POS data which, over the years, has been marginally sufficient to understand the causes and effects of promotion results. Today, however, the more sophisticated AI-driven technologies are proving that sampled models are failing to adequately capture the total picture of how a promotion truly performs across all of the account’s locations and markets. The quality of the analysis of the consumers’ reactions to promotions, pricing, and positioning is high, to be clear. With today’s real-time consumer engagement, too much intelligence is lost in not being able to capture and include every sale at every store in every market.
Otherwise, how else does a brick-and-mortar retailer compete with e-commerce where every single piece of data connected to the sale, including looking but not buying, is available? The past rationale for not providing every single location in the POS data uploads has always been cost and size of the potential database. Today’s big data technology can handle the volumes of data, and AI enables rapid and accurate decision making which is critical to effective trade promotion optimization.
Smart CPG companies are beginning to go direct to the retailers, especially larger regional chains like HEB, Wegman’s, Meijer, and Publix for POS data. The value is that it will be complete—100% coverage of all locations and markets—and also timelier (yesterday’s data). Knowing what happens in the past 24 hours is today’s mandate for precision and trust.
Finally, the timing of POS data is more important now than ever before. Again, syndicated data providers are generally anywhere from a week to a month latent in analyses of promotional performance. Online shopping is forcing more CPG companies to look at promotional performance on a weekly basis—even considering daily analysis for the very near future.
We are seeing more frequent updating of baselines which enables more accurate and timely predictive planning of promotions. Optimization engines must be able to manage and maintain current baseline data with historical tracking and the ability to integrate the POS data on a daily basis to keep track of near real-time comparisons in volume, revenues, and profitability.
The need to keep track of available inventory at both the manufacturer’s warehouse and the retailer’s distribution centers is critical. We saw in the pandemic how the supply chain disruption is affected and how difficult it is to optimize promotions without full and complete understanding of where the product inventory is at each point in the chain.
We are seeing a renaissance in the technology and processes used to manage in-store execution of promotions. Many of the world’s leading CPG companies and brokerage firms are doing a good job of integrating this data with trade promotions to get a full picture of compliance.
Key data elements for retail execution include, but are certainly not limited, to the following:
We are seeing more collaboration and sharing of data between sales and marketing. Increased visual alignment between trade and corporate marketing event calendars is helpful, but we need to take the next step to integrate the data into the scenarios and algorithms that drive optimization. Some examples of this type of data include:
A variation on the old childhood game of “hide and seek” is called “Kick the Can.” It’s the same as hide and seek with the exception of a single empty can placed out in the open where the poor soul having to seek the hidden players has to protect the can from being kicked by someone that both frees the ones already captured and extends the anguish of the person having to hunt the ones hiding.
But it need not be the case.
More often than not, strong investigation into the multiple siloes of data within the company and a careful examination of where and how the data can be sourced leads to proper building of the right database for effective TPO. Combined with the cleansing and harmonization of historical trade promotion data, this process is a mandate to going forward with accurate and trustworthy optimization and predictive and prescriptive promotional outcomes.
Having a partner doing that for you that is not the one providing the TPO technology is the best way I have found to ensure success in that endeavor. Working in collaboration with the TPO solution platform vendor and an independent data collection and management provider is the most efficient way to make this happen.
Once the historical trade promotion data is reasonably accurate and comprehensive, start with the POS data collector and provider. My work and experience with Retail Velocity, for example, has taught me that making sure the data is clean, harmonized, and aligned with the CPG’s own master data ensures that the quality of the data trumps any level of sophistication in the TPO solution itself.
The key to successful promotion optimization is the quality and integrity of the data. There is no compromise when it comes to making sure your second largest line item in the corporate financials, trade spending, can deliver the highest ROI—no matter what the metric is.
Rob is a frequent podcaster, blogger, and speaker on the topics of revenue growth management and business process innovation. He is the author of the recently published and popular book “The Invisible Economy of Consumer Engagement,” which outlines how consumer products goods companies can achieve the highest value in both promotion ROI and consumer engagement.