min read

Taking Control of Customer Data in Retail

Framework to power customer loyalty with AI-driven Customer Profiles, Behavioral Analytics, and Predictive insights

Taking Control of Customer Data in Retail

Knowing the customer and building Customer loyalty are the two most important success factors for a retail business. Both these ideals can only be executed well with the technology and data infrastructure that supports a deep understanding of the customer combined with responsive communications, personalized experiences, seamless multi-channel interactions, and effective returns services.

While data technology is expensive to create and maintain, modern AI/ML and Big Data technologies are making it easier to organize data, discover patterns, and derive intelligence that can revolutionize operations and sales.

However, harnessing AI/ML, Big Data and Cloud computing can be complicated. The technologies and tools are rapidly evolving and skills shortage can hamstring your organization’s ability to develop and maintain analytical and operational systems.

Enter Datamorph - Kwartile’s self-serve data transformation platform. Datamorph is a suite of tools that optimizes development of modern data architectures. It shields developers from the complexities of the implementation, reduces development time, and allows engineers to focus on uncovering intelligence within the data. The platform abstracts out the complexities of two critical areas – distributed computing and deployment environments – that represent the most complex and time-consuming tasks in the development pipeline.

Datamorph makes it vastly more convenient to develop and maintain powerful data and analytical applications such as Customer Identity and Profile Management systems, Omnichannel Behavior Analytics, Helpdesk Analytics, and Returns Analytics. Additionally, the platform is customizable and extensible, allowing engineers to implement any kind of complex business logic required for building custom business applications. With Datamorph, a retail business can create streamlined applications to obtain deeper insights into customer behavior, create highly relevant product lines, optimize customer service, vastly improve customer experience, and achieve high-ROI on marketing campaigns – all with significantly reduced spending on IT skills and infrastructure.


The retail industry is no stranger to headwinds. Even then, the year 2020 witnessed previously unseen challenges with the near collapse of the restaurant and travel industries caused by theCovid-19 pandemic. The retail industry experienced an unprecedented dichotomy as some retailers went bankrupt and others struggled to meet demand. As consumers stayed indoors, online commerce surged and convenience became the mantra of retail.

There is no question that in terms of knowing the customer and anticipating customer needs, the retail industry has crossed the Rubicon. The gap between leaders who understand how consumer expectations are evolving, and the rest of the industry has continued to grow.

Consider the following statistics:

  • Poor personalization and lack of trust cost businesses $756 B in 2018 alone.
  • 71% of shoppers express frustration when their shopping experience is impersonal.
  • 49% of shoppers purchase a product they did not initially intend to buy after receiving a personalized promo.
  • 40% have purchased something more expensive than intended because of a personalized offer.
  • According to a study done by TrueShip, over 60% of customers review a Returns Policy before they make a purchasing decision.
  • As eCommerce grows, the practice of "bracketing," (buying multiple versions of an item to determine the one a shopper likes best and returning the rest), is becoming a common practice. Because of the nature of eCommerce shopping, bracketing is hereto stay. Thus, returns policies have to adapt to this new paradigm. According a study byNarvar, 96% of bracketers said they would give repeat business if the returns process and policies are satisfactory. The biggest turn-offs are return shipping fees (69%),restocking fees (67%), and difficulty finding the returns policy (33%).

Predictably, leaders are pulling away from the pack with increasing revenues while laggards are going out of business. Further strengthening this trend is the rise of digital adoption. Since the Pandemic, digital sales have skyrocketed and retailers that doubled down on digital have done much better than their peers.

Capability Requirements

So, what should a retail business do in order to survive and outperform in the near future as retail is turned on its head?

Apart from the core supply-chain capabilities and operational systems, the four fundamental capabilities that need to be built are:

  • Understanding and anticipating the customer’s needs. This requires a sub-capability –systems that can provide “One view of the customer”
  • Creating frictionless multi-channel experiences for superior customer service
  • Implementing an easy and efficient returns process
  • Making critical investments in e-commerce capabilities

From a systems standpoint, all of the above capabilities require mature data integration and management practices supported by the latest data processing software and infrastructure. These include:

  • Cloud Datawarehouses
  • Next-Gen Data Lakes/Delta Lakes
  • ELT Systems
  • Dataflow Automation
  • Self-Serve Analytical Systems
  • AI/ML Frameworks

More specifically, retailers need to embrace the paradigm shifts happening in data processing technologies, as illustrated below:

System Complexity

The above defined technology stack has great data manipulation and processing capabilities, but requires a highly skilled data team to organize the data, create processing workflows, and build operational data systems.

Add to this, advances in AI/ML systems which require complex training systems and a multitude of data stores for scenarios, and IT development operations can become unwieldy.

It is well known that about 85% of a data scientist’s time is spent on just identifying data sources and pre-processing steps. Similarly, application developers spend much of their time in dealing with the complexities of distributed computing environments (i.e., the multi-cloud),and the complexities of deployment environments.

The Solution

Kwartile’s DataMorph platform is a Low/No-Code Data transformation platform that reduces the hurdles to developing and maintaining modern data applications. The platform offers a powerful set of tools to create and maintain a modern data processing framework (see Figure 2):

Challenges of Modern Data Architecture

While the Modern Data Architecture solves many problems of data organization, data processing, data accessibility, and data governance, it creates a new set of challenges for developing data applications, including the following:

  • The complexities of
    - managing cloud-based data warehouses
    - maintaining deployment environments
    - implementing an optimized DevOps framework
  • Additional complexities of ingesting and transforming both real-time and batch data simultaneously
  • Inflexibility of Visual Tool Environments
  • Non-standard coding methodologies (SQL mixed with polyglot coding)
  • Non-standard logging environments

Due to all these challenges, the data applications programmer is faced with a steep learning curve. The underlying technologies are also advancing rapidly, making it even more challenging for developers to keep up.

What is needed is a way of reducing the quantity of coding required for developing and maintaining data applications, and managing DevOps, cloud deployments, and security frameworks.

The Datamorph Platform

Datamorph is a low/no-code platform that is optimized for developing data applications. By using the platform, developers can improve their productivity by 50-90% while reducing operational costs between 20-60%. In the long term, Datamorph provides many other benefits for software maintenance including automated version control, dependency management, and application security management. This leads to high code maintainability and improves the long-term economics of application development even further for companies.

At a high-level, the specific features and benefits of the Datamorph platform include the following:

  • Abstracts out the complexities of distributed computing
  • Abstracts out the complexities of deployment environments
  • Provides Visual tools for no code development but also supports custom code development
  • Fully extensible and flexible – Developers can add their own custom templates and code libraries
  • Includes deployment components for all leading cloud environments as well as on-prem environments
  • Sophisticated management of Type 2 Slowly Changing Dimensions into the warehouse
  • Data Skew analysis, which often becomes the nemesis of distributed systems
  • Built-in support for regulatory data compliance requirements including GDPR and CCPA deletes

Also, the platform provides many productivity enhancing tools including the following:

  • Design templates and architectural components for common use cases
  • Fast deduplication of data in streaming and batch mode
  • Built-in monitoring tools with unified interfaces
  • Rich data-quality detection library based on configurations
  • Provides pre-defined components and the ability to create customized component libraries

Case Studies

While the Datamorph platform is appropriate for almost any possible data application development scenario, some more representative retail use cases include the following:

  • Customer Profile Management
  • Customer Behavior Analytics
  • In-Store Experience Personalization
  • E-Commerce Customer Journey Analytics
  • Supply-Chain Analytics
  • Returns Analytics
  • Help Desk Performance Analytics

Below are three representative case studies that the Datamorph platform has been implemented at retail customer sites.

  • Customer Profile Management
    The lynchpin of retail data is the Customer Id. However, the lack of a single consistent view of the customer prevents the ability to setup and orchestrate customer journeys across all the marketing channels and businesses. There are significant challenges in unifying the various sources of data and create usable versions of the truth. IT spends an enormous amount of time and effort in data wrangling and stitching together multiple internal and external systems in an effort to obtain a single, “best version of the truth”, and even then, only with poor results.
    The emerging category of Customer Profile Management applications use AI/ML techniques to determine the identity of the customer without having to build traditional and difficult-to-maintain data models. Kwartile’s Datamorph platform makes it easier to utilize AI/ML libraries and build the data sets necessary for model training, versioning, and model implementation.
  • Returns Analytics
    Good returns policies combined with an efficient process vastly improves customer experience and loyalty, and improves the rate of repeat purchases. Creating an efficient returns process requires a robust analytics application that can measure how well the process is functioning. By measuring and monitoring key returns metrics at a product, customer, and help desk agent level, companies can significantly reduce the cost of returns.
    The Datamorph platform enables IT departments to democratize data and make it easily available to business analysts and data scientists. The platform comes with built in tools to develop analytics applications and operational systems for managing, analyzing, and improving the returns process.
  • Customer Behavior Analytics
    The core marketing functions for a retail business are improving customer conversion rates, reducing churn, and personalizing campaigns to increase revenue. A good customer behavior analytics system is a required to measure and improve the effectiveness of the core marketing programs.
    The Datamorph platform enables integration of multiple data sources, near real-time processing of customer interaction data, and development of predictive analytics and recommendation engines, in a fraction of the time required with conventional software development methodologies. Built-in templates and algorithm libraries enable significantly faster implementations.

Benefits Achieved

The rollout of the projects provided multiple benefits and capabilities to the marketing, sales, and supply-chain business units, and operational organizations including IT and Customer Service.

  • At a macro level, one of our customers saved over $2 M in just the first phase of the project. The ROI of marketing campaigns increased by 40% and the time to perform Segmentation analysis to create new target lists decreased by more than 4weeks. Most importantly, the rate of positive customer identification increased from single digits to over 80%.
  • On the Data Science front, another customer developed a predictive analytics solution in only40% of the time previously required. With the democratization of data, post-implementation, data scientists obtained easy access to profile data in near real-time and could perform analysis to determine what aspects of deeper inquiry to conduct next.
  • For a large retail apparel customer, IT now spends 85% less time in data unification, and50% less time in developing and deploying new applications. IT is now off-loaded and has successfully shifted focus to collaboratively creating better data science capabilities.

Support and Services

Kwartile offers a robust set of solution and support services for the Datamorph platform. Our Product Support services offer multiple plans for Enterprises, Software Development professionals, and Partners. The support services include technical help during the warranty period, bug fixes, and regular version upgrades.

In addition, our professional services team offers data strategy, design, and application development services. Kwartile Consultants and Data Engineers possess a combination of deep and practical technology expertise, industry expertise, and strategic product management experience.