CXO’s Guide to Marketing and Sales Data Cleansing and Enrichment

“The average financial impact of poor data on businesses is $9.7 million per year.”- Gartner

We all know that for data to be relevant, it first needs to be cleansed in a method that includes identifying, correcting, and eliminating inaccuracies or corrupted records from a table or database. It’s important to locate inaccurate, incomplete, and irrelevant data components, and then alter, replace, or remove the offending “bad” data.

When it comes to Marketing or Sales, a lot depends on the information about the influencers or prospects. Hence, data mining, cleansing, and enrichment have always been an integral part of organizational outreach process.

Ideal Approach to Data Cleansing

Data cleansing and enrichment can be conducted with scripting as batch processing or with specific data wrangling tools.

The process of data cleansing manually maps or converts data from an original raw form into a more convenient format, where data is easily transformed with semi-automated tools:

• Data cleansing begins with preparation or data parsing. Each observation is removed from its data file, and each independent component is extracted.
• Then, data profiling occurs, which analyses the data for consistency and enables you to delve deeper into the data to observe individual field distribution, identifying outliers and data that doesn’t align with the global dataset.

Methods to Address Missing Values

At times, the data sets have a lot of empty fields and to cleanse the data; it is needed to identify and scope them. In case of missing values, the following methods can be applied:

• Adding the attribute mean, median, and mode for all missing values.
• Dropping instances and attributes.
• Using regression to input attribute missing values.

Data Cleansing and Enrichment Lifecycle

A data cleansing cycle should increase data quality and standardize the datasets, including merging, migration, rebuilding, deduplication, normalization, verifying, enriching, and appending.

Diagram 1 shows the typical data cleansing and enrichment lifecycle, which is an integral brick in the larger building blocks of the entire data processing cycle, as seen in Diagram 2.

Process Outline for Data Cleansing and Enrichment

Awareness of Bad Data

A study of 1000 IT decision makers revealed a unanimous agreement that legacy systems impede digital strategies.

Usage of Legacy Systems and Processes

In the last decade, companies have been using legacy systems such as Wang, AIX, IBM SNA, IBM’s WebsphereMQ, VMS, AS/400, and Unix. 93% of surveyed businesses are planning to upgrade to modern infrastructure and will implement next-gen SD-WAN networking technology within the next four years so they can adopt a complete cloud-first model.

New Technologies for Efficient Data Analysis and Management

Businesses are migrating to platforms that allow real-time capabilities and faster data analysis:

Tools to Predict the Quality of Current Data

Businesses have to use the right tools to predict the quality of their current data. Data Cleaner, CloudDingo, IBM Infosphere Quality Stage, and Paxata, help businesses on-board, profile, assess, and create quality data.

Privacy Concerns and Ethical Practices

Companies are under scrutiny to apply best practices when handling customer data. Facebook and Google are deploying teams to investigate data privacy regulations and transform how users can take control over their privacy settings.

GDPR Implementations 2018

The upcoming GDPR is at the crux of this attention to creating new guidelines and frameworks for managing customer information. In May 2018, the European Union (EU) must follow privacy and governance standards for customer data that include obtaining explicit consent before using, sharing, or storing customer information.

Companies can reassure consumers by being transparent about how they intend to use their data, enable customers to easily opt out of data sharing, and offer clearly written privacy policies.

Data Overload

Massive quantities of data from multivariate sources are leading to data overload. Data unification is critical if businesses want to develop their business strategy. Data must be merged from multiple sources and made useable by cleaning, de-duplicating, and exporting in a proper methodology.

The Digital Analytics Association (DAA) reveals that data accessibility, quality, and integration, present the most significant barriers to efficiency and analytics confidence across all industries.

Trill AI is addressing data overload through efficient data analysis and augmented decision making in financial asset management. All of the incoming and available data in decades-old legacy storage systems are not typically the cleanest format for processing and analysis.

Why Data Cleansing Is Important

The industry’s top experts agree that data cleansing and enrichment is critical to using your data, especially considering that only 3% of companies’ data meet basic quality standards.

Top 3 Global Machine Learning and Big Data Influencer, Ronald van Loon: “The point with data is that it needs to be regularly maintained to ensure that data remains clean and crystal clear.”

“Much of the data may be unstructured, noisy and in need of thorough cleansing and preparation before it is ready to yield working insights,” Big Data expert, Bernard Marr, founder of Intelligent Business Performance Institute.

Why Is Data Cleansing a Major Issue in Businesses?

Too much data that’s being generated by organizations is going unused. One case demonstrates how problematic data cleansing is for businesses in lead scoring processes, which can only be categorized according to real-time enriched data. If a lead enters the database with only a first/last name/email address, your lead scoring system may see the personal email address and automatically apply a low score.

With data cleansing and enrichment, the link can lead to more information, including a business email address, which helps target the visitor as a key decision maker, prioritizing the lead and passing it along to sales.

Data Cleansing vs. Data Scrubbing

Data cleansing is a method of identifying and removing inaccurate and corrupt records from data sets and replacing, modifying, and deleting messy or dirty data. Whereas data scrubbing incorporates a more complicated process that includes merging, decoding, translating, and filtering the data inconsistencies.

Data Cleansing vs. Data Validation

Similarly, data cleansing is distinct from data validation because the latter identifies, removes, and flags anomalous and incorrect information within a dataset leaving clean data for the end user.

Impact of Bad Data

Bad data has a severe effect on everything from customer service, compliance, marketing, ROI, and revenue for all companies.

Forrester Report: $65 Million Additional Income by 10% Increase in Data Quality

A 2017 Forrester report found that for a typical fortune 1000 company, just a 10% increase in data accessibility results in more than $65 million additional net income.

Cork University Study: ROI of Digital Analytics Can be Diagnosed from Quality Data

A study conducted at Cork University Business School evaluated data quality levels in actual practice. 75 surveyed executives identified the last 100 units of work their department had completed, comprised of 100 data records, and measured the work quality. Only 3% resided within an acceptable error range, and upwards of 50% of newly created data records had critical errors.

The above diagram demonstrates how various high-value opportunities can be identified through high-quality data, and the ROI of digital analytics can be more easily diagnosed.

Impact of Data on Marketing

The need of Cultural Shift

There’s a cultural shift occurring where experienced based decision-making is being replaced in favor of data backed decisions. A study performed by the MIT Center for Digital Business discovered that among survey companies, those who were mostly data driven had 4% higher productivity and 6% higher profits than the average.

E-Consultancy partnered with Google to conduct research comparing data-based decisions vs. gut instinct. Findings indicated that executive support for data backed decision making helps create a data-centric culture and nearly ⅔ of leading marketers claim that data supported decisions are far superior to decisions made on gut instinct. 

Achieving Marketing KPIs and Targets

Overconfidence in data accuracy can result in an unfounded sense of security that diminishes overall strategies, including decisions about pursuing micro-segmentation and micro-targeting marketing. 

Marketing CXO’s can utilize digital KPIs that are measurable values for assessing the performance of business initiatives.

• First set of KPIs Evaluates company progress in digitizing a current business model through calculating goals in marketing operations, supply chains, sales, products/services, and customer relations.
• The second set of KPIs Assesses new revenue sources from digital business models. KPI’s should present growth, market share and margin, revenue, and metrics that are distinct from physical assets.
• Traditional KPIs Some KPIs are transitional, while some are perpetual metrics for performance as digital transformation is established.
• Ideal Metrics Should support C-Suite decision making like business process improvements, budget allocations, and cultural modifications.

A Study by Deloitte on Data Quality

Deloitte’s conducted an anonymous survey of commercial data brokers to determine the accuracy of the data these firms depend on for marketing, research and development, product management, and other tasks:

Survey Report by Integrate: Leads Failed Due to Bad Data

According to a recent survey report from Integrate, from around 775,000 B2B leads, only 60% were excellent and whopping 40% of the leads did not materialize because of invalid emails, missing/incorrect filed entries, or duplicate data.

Sales Cycle and Pipeline Visibility

According to a study by Sale Management Association and Vantage Point Performance, 44% of sales executives think that their organization isn’t effective at managing its sales pipeline. In the majority of occasions, this is due to lack of either adequate information or quality data.

Increasing Recurring Billing

Companies can build a sales pipeline and increase recurring billing through quality data. The process outline could be similar to the following:

1. Define the customer.
2. Clean up existing prospect databases.
3. Send personalized messages with a clear view.
4. Create a strategy to handle low scoring leads.
5. Pursue high scoring leads. 

Sales Pipeline with Location Data

Location data can enhance sales pipelines and provide a competitive edge. Conversely, bad location data can:

• Frustrate 93% of consumers.
• Cause 80% of customers to stop trusting a business.
• Cause 60% of customers not to patronize a local business.

Location intelligence arising from IoT and smartphone usage provides a correlation between data sets and helps businesses arrive at insights that positively impact their bottom line, as demonstrated above. 

Bringing in Large Deals

Sales and marketing departments lose about 550 hours and $32,000 per sales representative from using bad data.

Study on Lead Generation and Impact of Bad Data

A study by LeadJen evaluated 12 different lead generation campaigns across different industries that represented over 100K phone and email connections. They found that implementing a lead generation program without investing in cleaning the data first results in a waste of 23.3% of each sales representatives’ time or 546 hours on average a year per representative.

Customer Insight on New Lead or Opportunity

Despite customer data being increasingly crucial, according to Informatica, only 17% of companies integrate insights across their organization, and poor data quality costs businesses 30% of revenues.

Loss of $1 Billion by United Airlines Due to Bad Data

United Airlines makes 8 million price predictions a day on more than 4,500 flights per day with varying pricing combinations and were not relying on their data to accurately forecast demand. This lead to a loss of $1 billion annually in revenue and misplaced assumptions about how much travelers would pay for a seat.

The accuracy of Sales Contact Data Management

Contact Data Management (CDM) helps businesses develop, maintain, and support a customer data strategy and offer elevated customer service and experience.

With the continued proliferation of data, maintaining your database integrity with clean data helps align data entering your database through increased channels, which can quickly become out of date, incomplete, and duplicated.

Opportunity Lost Due to Poor Data Quality

Poor data quality is costing businesses a significant amount of their revenues because data scientists, managers, knowledge workers, and decision makers have to incorporate it into daily work. Improving data quality equates to a more substantial data opportunity.

The above diagram features the lengthy and complex nature of correcting data errors contained in “hidden data factories,” or the repositories of information that businesses must deal with.

• Knowledge workers waste 50% of their time locating and amending inaccurate data and searching for confirmation for untrustworthy data.
• Data scientists spend 60% of their time cleaning and organizing data
• The total costs of hidden data factories in operational processes are 75%.

Logical Solution to These Challenges

Solutions for Marketing Worries

Mapping the customer journey can help businesses redirect their investment to customer innovations and competitive differentiators, promoting a customer culture while improving operations.

Aberdeen Group Inc. says that companies with the strongest omnichannel customer engagement strategies retain 89% of their customers as opposed to 33% of businesses with weak omnichannel strategies.

Customer Profiling to Avoid Duplication of Leads

Businesses also need to profile customer data and avoid duplication of leads. Duplicate data can affect your organization in the following ways:

• Higher churn rates, spam counts, maintenance costs, consumption of resources.
• Lower customer satisfaction and retention.
• Decreased productivity and revenue loss. 

According to Ringlead, strong organizations stand out with 66% more revenue than an average organization simply based on data quality.

Customer Prospect Profile Segmentation

Defining ideal customer prospects with the biggest potential, together with your existing customer base, helps create the highest ROI for sales and marketing campaigns and increases relevancy and communication to a targeted customer group.

If using an e-commerce platform such as Hybris, Drupal, or Magneto, its possible to link marketing software to this platform to store all the customer data in one place for more effective marketing database segmentation.

Achieving Digital Advertising Goals

Top level marketing professionals have revealed their biggest digital advertising goals for 2018:

Channels for Executing Digital Marketing Plans

The most effective channel for a digital marketing plan:

Data Cleansing Tools and Techniques

Popular data cleansing and data enrichment tools:

• Datacleaner Easy to use and allows database profiling and analyzation. Features data health monitoring to manage data quality and duplicate detection (contact for pricing.)
• Cloudingo This SalesForce App removes duplicates, clean records, and maintains data quality. Businesses can set specific rules for its automated scanning capabilities (3 plans: Basic- $1,096 annually; Standard- $2,146 annually; Enterprise-Starting at $10k).
• IBM InfoSphere QualityStage This popular tool allows full control over data quality, in addition to investigative, cleansing, and management capabilities towards your database (contact for pricing).

Data Cleansing Process

Dos and Don’ts of Data Cleansing

Data Cleansing Cycle

Data cleansing processes should be performed at least once a month so that your business can work with high-quality, accurate data. Otherwise, too much messy data accumulates to cause complications that are difficult to correct. 

Solutions for Sales Problems

Customers interact with businesses across numerous devices and mediums: social media, email, websites, mobile, desktop. Effective cross-promotion campaigns of services and products help businesses compete by customer experience, provide consistent communication across channels, and minimize churn.

Multivariate Sales Approach

Data cleansing and customer profile segmentation and investing in CRM software allow businesses to engage with customers from different angles, whether it’s by email or through social media for example.

• Track customers to gauge the effectiveness of your approach.
• Use analytics and data integration to manage the customer relationship.
• Coordinate different channel strategies to reach your customer and align your marketing for maximum benefit.

Proof that the Solution Works

Customers interact with businesses across numerous devices and mediums like social media, email, websites, TV, etc. Cleansing databases can result in improved and increased customer engagement and a way to more directly and consistently contact customers across these mediums because of quality and reliable data. 

Data cleansing and enrichment is a proven method of unlocking the power of data of sales, marketing, training, performance management, and basically of every department. This is why business leaders usually suggest structured marketing activities that eventually result in increased ROI.

Ageas UK, an insurance solution provider with 9 million customers, was able to detect and reject false insurance claims through the integration of an expansive data cleansing protocol. There are many such instances where organizations have utilized the data to protect their business interests in a positive way.

Collaborate with Sales to Ensure Alignment

Organizations have to establish a collaborative relationship with sales to align their program strategy, execution, and key KPIs, focusing on cleaning up legacy data and implementing future capabilities based on comprehensive quality data supports:

• Cost
• IT efficiency and processes
• Business needs and insights
• Legal and regulatory considerations, risk, and audit

A case study involving BMW shows that integrating a data cleansing tool into their call center, website, and front-end systems improved quality control within their expansive customer database and resulted in cost reduction across multiple areas.

Data Cleansing Is the Solution

A KPMG case study reveals that Philips began an organizational transformation using three SAP systems. They focused on cleaning up legacy data and instilling future capabilities. Below are some of the approach and outcome:

• Risk, audit, legal and regulatory considerations:
  Medical regulations like those by the FDA, come out on top of tax, privacy, and other regulations if it’s a healthcare company.
• Cost:
  Decommissioning IT systems saves costs but is only attainable once an information retention program is implemented.
• IT efficiency and legacy:
  Systems and processes become slower, the more data is stored in them.
• Businesses needs and insights:
  A complete set of historical data is necessary to create actionable insights for companies.

Don’t Let Bad Data Cost Your Business

Greater investment in data quality through data cleansing and enrichment is the only way to drive action, efficiency, and insight. Capturing the full benefit of business-critical data doesn’t have to be deterred by lack of prioritization of data cleansing initiatives.

Data cleansing services can help businesses identify and convert opportunities into successful strategies while overcoming obstacles in their data enrichment protocols. Introducing data cleansing techniques in a timely and consistent fashion, managed by the master data management specialists at Data Entry Outsourced, can help businesses generate more revenue and leverage insights to improve sales, marketing, and decision-making.

– DataEntryOutsourced