By: Steven Norton

It used to take Dickey’s Barbecue Pit about a month to figure out if a new menu item met the five requirements needed to become a permanent fixture. With data from its 500 stores now in a

standard format and location, the process now takes a single day.

<img alt="" class="size-full wp-image-5" src="http://si.wsj.net/public/resources/images/BN-IQ595_0528_c_E_20150528181301.jpg" style="border:none; float:right; height:239px; margin:10px; width:359px">

The barbecue chain, which features on-site hickory wood burning pits, sees data as a way to engineer efficiency and profitability into its stores as it goes up against fast-casual competitors like Chipotle.

The new system, called Smoke Stack, is an amalgamation of cloud and Big Data tools hosted and delivered by consulting firm iOLAP Inc. Among them are integration software from Syncsort, a business intelligence platform from Yellowfin and data warehousing from Amazon Redshift, part of Amazon Web Services. The system pulls data from point-of-sale systems and combines it with information about promotions, media buys, food purchasing patterns and inventory data, among other things. Smoke Stack has 175 corporate users and beta testers now, and Dickey’s plans to roll it out to 350 restaurant operators next week.

A primary element of Smoke Stack is the agent that pulls in data from each store’s point-of-sale systems, many of which have different software versions or data extraction formats. iOLAP first had to ensure that data could be uploaded on a schedule, then had to manage it for consistency. That included adding location, date and time stamps on every record.  Smoke Stack also brought system-level data concepts to Dickey’s, allowing each restaurant’s individual product codes and other information to map to a common place.

Being able to quickly manipulate varied datasets helps Dickey’s keep stock keeping units (SKUs) low, CIO Laura Rea Dickey said. With eight proteins served only two ways, plus side dishes and desserts, Dickey’s tries to cap its SKUs at about 125. Exceptions are made for “local favorites,” dishes that sell well at a certain store or in a particular region, and test items hit the menu from time to time.

“Data has driven us to validate that the tighter the inventory, the better,” Ms. Dickey said. “We’re continuing to mine the menu to reduce SKUs as much as possible.”

The chain evaluates menu items and SKUs using five different metrics: brand, quality, profitability, guest response/sales and complexity of preparation. Menu items must meet all five criteria to become permanent. The criteria are a mix of qualitative and quantitative metrics, such as hard sales figures, customer surveys and 1-to-5 ratings to measure preparation difficulty.

Chili, for example, fit the Dickey’s brand concept, but it required too many ingredients and too much prep time. It also sold poorly and was eventually pulled from the menu. Before Smoke Stack, measuring chili’s performance against the five metrics would have required weeks of loading and querying data followed by a laborious analysis process. Now, those metrics can be cross-referenced in a single report. Measurements can be made daily, and Dickey’s can make changes to menu items on the fly to see how they affect performance.

Dickey’s uses Hadoop when it needs to do large manipulations of information not usually needed in the database. In one instance, Dickey’s needed information about what time each POS system completed its end-of-day processes, data that’s stored in a configuration file not easily readable by a relational database. Using Map Reduce and a temporary Hadoop cluster on AWS, Dickey’s was able to parse and read millions of records in minutes.

Sales data could be viewed on a daily basis before, but now it refreshes every 15 to 20 minutes. That information can now be cross-correlated with marketing data and other metrics daily, Ms. Dickey said. Key performance indicators are now reviewed by the corporate team the day before, and the findings drive the next day’s business meetings, operations, marketing initiatives and training sessions.