Best Practices in Impact Measurement

Justin Tischler

Justin Tischler

Best Practices in Impact Measurement

A critical part of a comprehensive site selection strategy is understanding how future pipeline locations will impact the existing portfolio.  Particularly helpful in market infill planning, cannibalization analysis and resultant transfer studies help decision-makers better understand the ramifications of future deals and their potentially negative impact on the profitability of existing locations.

While the performance of an existing location is easy to measure (with some exceptions), measuring the impact that a newly opened location has had on the sales of surrounding locations requires a considerably more nuanced approach.

Impact Measurement – An Attribution Problem

Consider this scenario: a brand decides to open a new location several miles away from the only other location they operate in that market; one year later, the original location reports sales that are 30% lower than the prior year.  This negative comp sales number is easy to calculate, and it is tempting to ascribe the sales decline as the amount of sales cannibalized by the new location.  But there are a multitude of factors beyond the opening of a new sister location that could drive shifts in performance, including:

  • Sales lift/drag for the full chain caused by macroeconomic trends, brand marketing, introduction of new product/service lines, etc.
  • Changes to nearby demand such as population growth/decline, employment growth/decline, tourism effects, traffic increases/decreases, shifting demographics, etc.
  • Changes in unit operations such as management hires, staff training, product/service quality, hours of operation, etc.
  • Market-wide adverse operating conditions such as harsh weather events, the loss of a major employer(s), etc.
  • Opening/closing of competitors in the trade area

This confluence of many possible factors that could be causing the original location’s sales to decline creates an attribution problem – how much of that 30% sales reduction should be attributed to the new location opening vs. all of the other things that could drive performance changes?

To isolate the cannibalization inflicted by the new location, Intalytics relies on control group comparisons of comp sales.  More specifically, Intalytics indexes the year-over-year performance changes by month for the impacted location against the comparable data for a group of “control” locations, making sure that the control  group is comprised of locations operating in situations comparable to the impacted location with the notable exception that none have had new sister locations open in their vicinity within the past two years.  One such example is illustrated below.

Graph-showing-Control-Group-Average-vs-Impacted-Unit-Cannibalization

In this example, the impacted location has experienced comp sales that are stable to slightly declining year-over-year after the new location opens.  However, the control group locations experienced considerable growth over the same period of time; growth that the impacted location would likely otherwise have achieved were it not for the opening of the sister location.  In this example, the impacted location’s performance is clearly lagging relative to the control group locations.  The shaded gray area represents the aggregate cannibalization estimated using this approach.

But what can be done in situations such as the one created by the coronavirus pandemic, where all locations experience drastic shocks at staggered times?  Consider the chart below, which illustrates for one national operator the amplified volatility in comp sales experienced as the coronavirus pandemic began to take hold in the U.S.  Values below the black line indicate less stability in monthly comp sales rates across locations (higher rates of variance).

Stability-of-month-over-month-comp-sales

In light of the challenges presented by this volatility, we offer several suggestions to anyone seeking to more accurately estimate retrospective impacts:

  1. Use a rigorous control group selection process.
    Sample sizes permitting, try to ensure the control group locations are analogous to the analysis location in as many dimensions as possible.  Ideally, control group locations should have a similar geographic/regional disposition, be located in similarly dense areas, share prototypes and location features, etc.  This will ensure that as many of the other possible drivers of performance are sufficiently controlled for.
  1. Screen the control group for outliers (locations and sales).
    The control group should be a stable set of locations used as a benchmark.  Including anomalous locations in this mix – or including drastic sales swings – could undermine the accuracy of any estimated impact.  Additionally, it may be appropriate to carve out anomalous time periods that impact locations at different levels from consideration entirely, such as holiday periods or COVID-affected date ranges.
  1. Evaluate the analysis location for other potential external shocks.
    Even with a perfect control group, the impact estimate may still be off the mark if some other event coincided with the new location opening.  Review the impacted location’s performance drivers to better understand whether something other than the new location likely caused any observed sales fluctuation.

To learn more about how Intalytics can incorporate impact estimation and forecasting into your enterprise site selection strategies, please contact us.


Sign up for our newsletter by filling out the form below.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Related News

Claire’s Stores Expands Partnership with Intalytics to Guide European Expansion

Claire’s Stores Expands Partnership with Intalytics to Guide European Expansion

ANN ARBOR, MI (Aug 4, 2021) - Claire’s Stores, Inc. has expanded its partnership with Intalytics to include analytics in support of driving new store..

barre3 Selects Intalytics to Define and Execute Growth Strategy

barre3 Selects Intalytics to Define and Execute Growth Strategy

ANN ARBOR, MI (July 7, 2021) - barre3, a Portland-based premium fitness brand, has chosen Intalytics to serve as their predictive analytics partner..

Citi Trends Selects Intalytics for Real Estate Analytics Solutions

Citi Trends Selects Intalytics for Real Estate Analytics Solutions

ANN ARBOR, MI (June 23, 2021) - Citi Trends, Inc. (NASDAQ: CTRN),a growing specialty value retailer of apparel, accessories and home trends for way..

Meet the Intalytics Team: Bill McKeogh, Director of Client Services

Meet the Intalytics Team: Bill McKeogh, Director of Client Services

At Intalytics, a Kalibrate company, we believe our success is driven as much by our people as our technology. So, as part of our new meet the..

SiteIntel Client Q&A: Advance Auto Parts

SiteIntel Client Q&A: Advance Auto Parts

Location analytics are a critical factor in driving strategic growth for retail brands. As tenants become more sophisticated in the use of data..

SiteIntel Client Q&A: Goldfish Swim School

SiteIntel Client Q&A: Goldfish Swim School

Location analytics are a powerful foundation upon which tenants can derive data-driven answers to business questions such as where to open new..

Meet the Kalibrate Team: Oliver Shaw, Chief Executive Officer

Meet the Kalibrate Team: Oliver Shaw, Chief Executive Officer

At Intalytics, a Kalibrate company, we believe our success is driven as much by our people as our technology. So, as part of our new meet the..

Intalytics Commentary: Tough Times for Apparel Retailers

Intalytics Commentary: Tough Times for Apparel Retailers

Apparel retailers have been buffeted over the past year. Comp sales growth was nominal leading into the pandemic – clothing stores in the U.S. comped..

Leverage the Link: Best Practices for Marketers to Capture Value from Real Estate Site Selection Analytics

Leverage the Link: Best Practices for Marketers to Capture Value from Real Estate Site Selection Analytics

Marketers should pay attention when their firms have contracted for a real estate site selection model. There are multiple benefits to the savvy..