Modern trade promotion optimization (TPO) is far more demanding than consumer packaged goods (CPG) industry executives understand. Oversimplification of TPO is dangerous, costly, and, in the end, fails to achieve the business objectives. The most important aspect of optimizing trade spend is knowing and employing the right data. Surprisingly, few hit the mark.
Trade promotion management (TPM)/TPO is itself becoming more of a priority for operational process and technology transformation across the entire consumer products industry. As we pointed out in the previous blog post regarding revenue growth management (RGM), the data driving effective planning, execution assessment, and analysis of promotional performance is a mission critical component of this new RGM renaissance. Optimization is the process of leveraging that data to predict and prescribe future promotional success. 
So, let’s take a closer look at the data that feeds a successful promotion optimization outcome and how even the most effective CPG companies often continue to lack precision and trust in their TPO results.
What Are We Optimizing?
I often ask this first question in the planning workshops for my clients which are about to implement a TPO solution. The typical answers include:
- Increased sales volumes
- Increased revenues
- Increased profitability
- Increased ROI
OK, all of these are true for sure, but when the hard work of developing the complex algorithms needed to deliver the optimized figure, the answers are not always clear, or worse—unbelievable. Technology continues to grow in TPO functionality, but we have a long way to go.
RGM leadership will tell you that generating profitable revenue is the ultimate goal. So, delivering the specific objective against the forecast and, more importantly, controlling the outcomes will be the significant benchmark of good TPO.
Invariably, “ROI” is going to be mentioned as a primary, if not THE, reason for a TPO in the first place. But then I ask my second question: What IS ROI?
If this was a simple answer, the corporate integrated annual business planning process would take place in a single afternoon! However, there is a complex interrelationship between so many elements of the forecast, promotion plan and consumer engagement that has to be considered. The data required for the preparation and ultimate approval of a corporate annual forecast and operational plan are the “building blocks” of this mission critical effort.
These “building blocks” of data are intended to work in concert with each other to ensure the most effective outcomes. In my book, “The Invisible Economy of Consumer Engagement,” I spend a lot of time discussing the individual fiefdoms of a typical CPG company and how each leverage these building blocks of data to define their specific performance objectives.
The sales organization continues to concentrate on volume and profitability, of course, but historically, they have been less concerned with the actual performance of the promotion than they are the incentive they provide to their account buyer to agree to a specific volume of product purchased. 
The supply chain organization sees the achievement of the forecast from the perspective of the physical storage, transportation, and safety of the products from the plant to the retailer’s loading docks. They look at time, cost of transportation and warehousing, and reliability of production.
Marketing tracks the consumer demand, determines product messaging, and competitive positioning.
Finance and IT all have their roles and requirements, primary among them being keeping costs down and developing sophisticated technologies to ensure efficient and effective operations.
So much focus and attention is now given to TPO technology as millions of dollars are invested in building an efficient optimization engine. Add to that the frustration in finding and securing the right data and you have a perfect storm that most CPG companies want to avoid.
This is all too often the case as I speak and work with CPG companies, helping to coordinate annual business planning processes. The number one reason for the length of time it takes for integrated corporate business planning is the time it takes to gather, harmonize, align, and analyze all the data. If you have ever been involved in this process, you know how difficult, frustrating, and often emotionally draining it is on all the participants.
TRADE PROMOTION OPTIMIZATION Data Integration
In a typical TPO integration, most companies will include data from historical trade promotions, point-of-sale (POS) data directly from the retailer (typically accounts such as Walmart, Target, Kroger, Home Depot, etc.), as well as indirectly from syndicated data providers. Also, more sophisticated TPO solutions will include baseline, demand plans, and other S&OP (sales and operations planning) data sources.
However, while these integrations provide a set of data for basic “what-if” scenarios, data is often inaccurate or incomplete—rendering output that is flawed and dangerous to fully depend upon. An example of this is the lack of details within a typical three-to-five-year historical accounting of trade promotions.
The old saying, “Garbage in, garbage out” definitely applies here.
In the following graphic, the red dotted lines represent an ideal picture of the data necessary for true optimization—the eventual type of predicted outcome we will all see in the future. Some leading CPG companies already integrate data from marketing and logistics (supply chain, shipping/receiving, and warehousing), but the application of that data to a true projected outcome remains very rare.
If the TPO technology is sophisticated enough to provide a large enough array of scenarios based on the current scope of data, there should be enough data to populate those scenarios. At a minimum, the data represented in the graphic would be required.
What Is the Right Data for tpo?
Most CPG companies lack detailed historical data on prior promotions. Increased sophistication with TPM solutions has helped with this issue, but there are still too many great companies that continue to rely upon spreadsheets where, even if the data is there, it is virtually impossible to extract.
This is especially true for promotions that actually deviate from their originally planned content. Post promotion functionality such as claim and deduction management does not usually allow for a detailed update of the changed terms and content. If the TPO runs predictive scenarios based on only originally planned promotional data instead of the actual performance, your performance analytics and future optimization is bad.
Let’s take a closer look at what is considered a strong database for minimally effective promotion optimization:
Historical Recordkeeping
It is necessary to maintain three (3) years of data in trade promotion history. This ensures you can create a full 12-quarter cycle of promotional performance for the account.
Promotion Detail
The more detailed the promotional data is, the more accurate the projected values from an optimization algorithm.
- Account Hierarchy – The designated level at which the promotion is planned and executed. It makes a difference if it is an account-wide promotion versus a regional or substrata of markets and store locations.
- Product Hierarchy – Ditto the above – keeping track of how the account’s product mix is sold and maintained impacts how the products are promoted.
- Promotion Timing – The full “from-to” dates the promotion is active.
- Promoted Products – This is going to include BOTH the individual SKU-level product data as well as the various identified combinations or groupings of products that are promoted together. The key is that these “groups” must enable breakdowns of each SKU within the group for future product promotion performance analysis.
- Tactical Activities – This is where historical records typically fail. Too many TPM systems only capture a minimum amount of data, such as “Display.” Without sufficient detail of the type of display, it is impossible to create effective predictions of promotional performance. As we all know, a major end-cap display is far more consumer-appealing than a simple sign or shelf talker. Detail the tactics and establish their own hierarchy within the TPM promotion entry fields so that proper analysis can be done and smarter, more accurate projection of outcomes are possible.
- Cost per Tactic – Another failure in many TPM entry fields is the lack of applying a specific promotional allocation or cost (amount to be paid or deducted) to each type of tactic employed. Most systems simply show a single amount of the promotion, even when multiple tactics are planned in the promotion. But remember, many retailers will deduct portions of promotions at times, and this will grind your deduction management or claims audit and payment process to a halt trying to reconcile partial promotion line items.
- Total Cost – As mentioned above, this would be the final expected payout for the entire promotion—calculated as a roll-up from each individual tactical line item.
POS Data
This is the single most important data in the process of effective promotion optimization. It is, after all, the final result of the promotion—what crosses the scanner at checkout. If this data is not exact, you cannot generate a true picture of the end result of the promotion. POS data has been provided to the manufacturer/supplier by two distinct routes: (1) direct from the retailer, or (2) through promotional analyses delivered by syndicated data providers.
The syndicated data providers will base their analysis of promotional performance on sampling methodologies of POS data which, over the years, has been marginally sufficient to understand the causes and effects of promotion results. Today, however, the more sophisticated AI-driven technologies are proving that sampled models are failing to adequately capture the total picture of how a promotion truly performs across all of the account’s locations and markets. The quality of the analysis of the consumers’ reactions to promotions, pricing, and positioning is high, to be clear. With today’s real-time consumer engagement, too much intelligence is lost in not being able to capture and include every sale at every store in every market. 
Otherwise, how else does a brick-and-mortar retailer compete with e-commerce where every single piece of data connected to the sale, including looking but not buying, is available? The past rationale for not providing every single location in the POS data uploads has always been cost and size of the potential database. Today’s big data technology can handle the volumes of data, and AI enables rapid and accurate decision making which is critical to effective trade promotion optimization.
Smart CPG companies are beginning to go direct to the retailers, especially larger regional chains like HEB, Wegman’s, Meijer, and Publix for POS data. The value is that it will be complete—100% coverage of all locations and markets—and also timelier (yesterday’s data). Knowing what happens in the past 24 hours is today’s mandate for precision and trust.
Finally, the timing of POS data is more important now than ever before. Again, syndicated data providers are generally anywhere from a week to a month latent in analyses of promotional performance. Online shopping is forcing more CPG companies to look at promotional performance on a weekly basis—even considering daily analysis for the very near future.
Baseline Forecasts
We are seeing more frequent updating of baselines which enables more accurate and timely predictive planning of promotions. Optimization engines must be able to manage and maintain current baseline data with historical tracking and the ability to integrate the POS data on a daily basis to keep track of near real-time comparisons in volume, revenues, and profitability.
Logistics
The need to keep track of available inventory at both the manufacturer’s warehouse and the retailer’s distribution centers is critical. We saw in the pandemic how the supply chain disruption is affected and how difficult it is to optimize promotions without full and complete understanding of where the product inventory is at each point in the chain.
Retail Execution
We are seeing a renaissance in the technology and processes used to manage in-store execution of promotions. Many of the world’s leading CPG companies and brokerage firms are doing a good job of integrating this data with trade promotions to get a full picture of compliance. 
Key data elements for retail execution include, but are certainly not limited, to the following:
- Inventory – On shelf and in the back room. Field merchandising teams should be equipped with the right technology to capture, store, and distribute this data in real time. It can mean the difference between a promotion with fully stocked shelves and an out-of-stock condition—guaranteed promotion failure!
- Product Assortment – This data would be tracked against the product assortment plan to ensure the store is carrying and promoting the right products. It also enables comparisons and compliance with product plan-o-gram plans as well.
- Pricing – Checking prices for both promoted and non-promoted price, as well as comparisons with competitive prices for the same categories. Both sets of data would enable the optimization engine to create scenarios where expected competitive pressure could be countered.
- Promotion Display Compliance – Compliance across the board is already being tracked, but to be able to capture and integrate non-compliance—especially for continued violators—gives the ability to project scenarios where focus on the noncompliant retailers can activate the field merchandising teams or eliminate the continual violators from future promotions.
Marketing Data
We are seeing more collaboration and sharing of data between sales and marketing. Increased visual alignment between trade and corporate marketing event calendars is helpful, but we need to take the next step to integrate the data into the scenarios and algorithms that drive optimization. Some examples of this type of data include:
- Coupon Drop Schedule and Content – This ensures that a good account will not run into a conflicting situation where a trade promotion runs at the same time as a corporate coupon drop, but with different terms for pricing, packaging, etc.
- National and Regional Ad Schedules – Knowing when and where the ads will run, in what media and markets, will help the key account managers to effectively plan supporting trade promotions. The retail accounts will love it too.
- New Product Introductions – Marketing tests new products all the time and drives new products to production earlier than they are announced in the product list.
“Kicking the Can” on TRADE PROMOTION OPTIMIZATION
A variation on the old childhood game of “hide and seek” is called “Kick the Can.” It’s the same as hide and seek with the exception of a single empty can placed out in the open where the poor soul having to seek the hidden players has to protect the can from being kicked by someone that both frees the ones already captured and extends the anguish of the person having to hunt the ones hiding.
Yet it is also something many CPG companies are doing on TPO. Once they begin to see the critical importance of ensuring the data is available only to learn how difficult and costly it often is to have it in place, they simply “kick the can” on implementation of a TPO—delaying the value proposition for effective optimization.
But it need not be the case.
More often than not, strong investigation into the multiple siloes of data within the company and a careful examination of where and how the data can be sourced leads to proper building of the right database for effective TPO. Combined with the cleansing and harmonization of historical trade promotion data, this process is a mandate to going forward with accurate and trustworthy optimization and predictive and prescriptive promotional outcomes.
Having a partner doing that for you that is not the one providing the TPO technology is the best way I have found to ensure success in that endeavor. Working in collaboration with the TPO solution platform vendor and an independent data collection and management provider is the most efficient way to make this happen.
Once the historical trade promotion data is reasonably accurate and comprehensive, start with the POS data collector and provider. My work and experience with Retail Velocity, for example, has taught me that making sure the data is clean, harmonized, and aligned with the CPG’s own master data ensures that the quality of the data trumps any level of sophistication in the TPO solution itself.
The key to successful promotion optimization is the quality and integrity of the data. There is no compromise when it comes to making sure your second largest line item in the corporate financials, trade spending, can deliver the highest ROI—no matter what the metric is.
About the Author
Rob Hand is the founder and CEO of Hand Promotion Management, a consulting company providing domain expertise across more than 25 consumer-facing industries. Focused on consumer products and retail, he has a 45-year history of helping consumer goods manufacturers improve their trade promotion spending ROI, performance analytics, retail execution, consumer engagement efficiency, and marketing effectiveness.
Rob is a frequent podcaster, blogger, and speaker on the topics of revenue growth management and business process innovation. He is the author of the recently published and popular book “The Invisible Economy of Consumer Engagement,” which outlines how consumer products goods companies can achieve the highest value in both promotion ROI and consumer engagement.
