How a Flexible Data Architecture Allowed an Employee Engagement and Rewards Company to Complete a Market Basket Analysis
Overview
An employee engagement and rewards company consults with clients to identify ways to drive customer and employee loyalty through planning, organizing, and executing incentive programs, travel, and events. This company looked to expand service offerings to include performing data analysis on their clients’ data, providing actionable insights they could not previously obtain to speed the client’s return on investment and strengthen the relationship.
The Challenge
One particular client provided this company with a large amount of point-of-sale data. The rewards company proposed performing a market basket analysis on the data to identify purchasing trends, allowing the client to better understand consumers and influence purchasing decisions. The rewards company’s previous attempts to analyze the 185 million rows of sales data were cumbersome and limited:
Only a subset of the data could be analyzed, which required the assumption that the sampling was a proper representation of the dataset as a whole.
It was extremely time-intensive. Even from the limited subset, an analyst could only process a further limited number of rows at a time, so the analysis required multiple iterations, reviews, and deduplication to actually derive the baskets needed to attempt to identify trends.
The Solution
This employee engagement and rewards company partnered with Zirous to evaluate current limitations and determine how to quickly deliver a custom solution that would provide new analytic possibilities and operational insights.
The teams worked together to:
- Understand business goals and current architecture
- Identify all data sources and formats
- Assist and empower the company’s in-house resources
- Implement a sustainable data analytics solution
The forward-thinking development resulted in a flexible design that allows for additional sources to be added in the future with minimal impact on existing processes.
The Impact
Increased Resource Availability
Data analysis previously took hours, required a lot of manual effort by data analysts, and was only performed on a subset of the data. By utilizing the processing power a big data platform offers, the entirety of the 185+ million rows of point-of-sale data was analyzed in under 10 minutes.
Additionally, filters now quickly recalculate market baskets on segments of the data based on date range, store number, product type, and more.
Increased Revenue
This company’s new data analytics architecture opens up new revenue streams and enables them to offer value to clients they were unable to provide before.
It also provides the company with a sustainable architecture to perform their own internal analytics that can tangibly prove the cost benefits of their services. All of this additionally contributes to strengthening their client relationships.
Flexible Architecture
The development of this platform was done in modular form to allow the addition of future data sources without disrupting the current design.
Quick Facts
Company
- Employee engagement and rewards company
- 600 employees and annual revenue of over $125 million
- Based in the Midwest
Technologies Involved
- Hortonworks
- HDFS
- NiFi
- Hive
- Zeppelin
Highlights
- An employee engagement and rewards company needed to analyze 185 million rows of sales data.
- Analysis that large proved challenging for the company; they could only analyze a subset of the data and had to parse information back together.
- Zirous helped the company create a flexible architecture able to analyze the data set in under 10 minutes and perform a market basket analysis.
- New revenue doors are opened to the company with the ability to offer large-scale data analysis to their clients.
