Tag: bad data

Categories
erwin Expert Blog

Data Governance Helps Build a Solid Foundation for Analytics

If your business is like many, it’s heavily invested in analytics. We’re living in a data-driven world. Data drives the recommendations we get from retailers, the coupons we get from grocers, and the decisions behind the products and services we’ll build and support at work.

None of the insights we draw from data are possible without analytics. We routinely slice, dice, measure and (try to) predict almost everything today because data is available to be analyzed. In theory, all this analysis should be helping the business. It should ensure we’re creating the right products and services, marketing them to the right people, and charging the right price. It should build a loyal base of customers who become brand ambassadors, amplifying existing marketing efforts to fuel more sales.

We hope all these things happen because all this analysis is expensive. It’s not just the cost of software licenses for the analytics software, but it’s also the people. Estimates for the average salary of data scientists, for example, can be upwards of $118,000 (Glassdoor) to $131,000 (Indeed). Many businesses also are exploring or already use next-generation analytics technology like predictive analytics or analytics supported by artificial intelligence or machine learning, which require even more investment.

If the underlying data your business is analyzing is bad, you’re throwing all this investment away. There’s a saying that scares everyone involved in analytics today: “Garbage in, garbage out.” When bad data is used to drive your strategic and operational decisions, your bad data suddenly becomes a huge problem for the business.

The goal, when it comes to the data you feed your analytics platforms, is what’s often referred to as the “single source of truth,” otherwise known as the data you can trust to analyze and create conclusions that drive your business forward.

“One source of truth means serving up consistent, high-quality data,” says Danny Sandwell, director of product marketing at erwin, Inc.

Despite all of the talk in the industry about data and analytics in recent years, many businesses still fail to reap the rewards of their analytics investments. In fact, Gartner reports that more than 60 percent of data and analytics projects fail. As with any software deployment, there are a number of reasons these projects don’t turn out the way they were planned. Among analytics, however, bad data can turn even a smooth deployment on the technology side into a disaster for the business.

What is bad data? It’s data that isn’t helping your business make the right decisions because it is:

  • Poor quality
  • Misunderstood
  • Incomplete
  • Misused

How Data Governance Helps Organizations Improve Their Analytics

More than one-quarter of the respondents to a November 2017 survey by erwin Inc. and UBM said analytics was one of the factors driving their data governance initiatives.

Reputation Management - What's Driving Data Governance

Data governance helps businesses understand what data they have, how good it is, where it is, and how it’s used. A lot of people are talking about data governance today, and some are putting that talk into action. The erwin-UBM survey found that 52 percent of respondents say data is critically important to their organization and they have a formal data governance strategy in place. But almost as many respondents (46 percent) say they recognize the value of data to their organizations but don’t have a formal governance strategy.

Data-driven Analytics: How Important is Data Governance

When data governance helps your organization develop high-quality data with demonstrated value, your IT organizations can build better analytics platforms for the business. Data governance helps enable self-service, which is an important part of analytics for many businesses today because it puts the power of data and analysis into the hands of the people who use the data on a daily basis. A well-functioning data governance program creates that single version of the truth by helping IT organizations identify and present the right data to users and eliminate confusion about the source or quality of the data.

Data governance also enables a system of best practices, subject matter experts, and collaboration that are the hallmarks of today’s analytics-driven businesses.

Like analytics, many early attempts at instituting data governance failed to deliver the expected results. They were narrowly focused, and their advocates often had difficulty articulating the value of data governance to the organization, which made it difficult to secure budget. Some organizations even viewed data governance as part of data security, securing their data to the point where the people who wanted to use it had trouble getting access.

Issues of ownership also hurt early data governance efforts, as IT and the business couldn’t agree on which side was responsible for a process that affects both on a regular basis. Today, organizations are better equipped to resolve these issues of ownership because many are adopting a new corporate structure that recognizes how important data is to modern businesses. Roles like chief data officer (CDO), which increasingly sits on the business side, and the data protection officer (DPO), are more common than they were a few years ago.

A modern data governance strategy weaves itself into the business and its infrastructure. It is present in the enterprise architecture, the business processes, and it helps organizations better understand the relationships between data assets using techniques like visualization. Perhaps most important, a modern approach to data governance is ongoing because organizations and their data are constantly changing and transforming, so their approach to data governance needs to adjust as they go.

When it comes to analytics, data governance is the best way to ensure you’re using the right data to drive your strategic and operational decisions. It’s easier said than done, especially when you consider all the data that’s flowing into a modern organization and how you’re going to sort through it all to find the good, the bad, and the ugly. But once you do, you’re on the way to using analytics to draw conclusions you can trust.

Previous posts:

You can determine how effective your current data governance initiative is by taking erwin’s DG RediChek.

Categories
erwin Expert Blog

Data Governance Tackles the Top Three Reasons for Bad Data

In modern, data-driven busienss, it’s integral that organizations understand the reasons for bad data and how best to address them. Data has revolutionized how organizations operate, from customer relationships to strategic decision-making and everything in between. And with more emphasis on automation and artificial intelligence, the need for data/digital trust also has risen. Even minor errors in an organization’s data can cause massive headaches because the inaccuracies don’t involve just one corrupt data unit.

Inaccurate or “bad” data also affects relationships to other units of data, making the business context difficult or impossible to determine. For example, are data units tagged according to their sensitivity [i.e., personally identifiable information subject to the General Data Protection Regulation (GDPR)], and is data ownership and lineage discernable (i.e., who has access, where did it originate)?

Relying on inaccurate data will hamper decisions, decrease productivity, and yield suboptimal results. Given these risks, organizations must increase their data’s integrity. But how?

Integrated Data Governance

Modern, data-driven organizations are essentially data production lines. And like physical production lines, their associated systems and processes must run smoothly to produce the desired results. Sound data governance provides the framework to address data quality at its source, ensuring any data recorded and stored is done so correctly, securely and in line with organizational requirements. But it needs to integrate all the data disciplines.

By integrating data governance with enterprise architecture, businesses can define application capabilities and interdependencies within the context of their connection to enterprise strategy to prioritize technology investments so they align with business goals and strategies to produce the desired outcomes. A business process and analysis component enables an organization to clearly define, map and analyze workflows and build models to drive process improvement, as well as identify business practices susceptible to the greatest security, compliance or other risks and where controls are most needed to mitigate exposures.

And data modeling remains the best way to design and deploy new relational databases with high-quality data sources and support application development. Being able to cost-effectively and efficiently discover, visualize and analyze “any data” from “anywhere” underpins large-scale data integration, master data management, Big Data and business intelligence/analytics with the ability to synthesize, standardize and store data sources from a single design, as well as reuse artifacts across projects.

Let’s look at some of the main reasons for bad data and how data governance helps confront these issues …

Reasons for Bad Data

Reasons for Bad Data: Data Entry

The concept of “garbage in, garbage out” explains the most common cause of inaccurate data: mistakes made at data entry. While this concept is easy to understand, totally eliminating errors isn’t feasible so organizations need standards and systems to limit the extent of their damage.

With the right data governance approach, organizations can ensure the right people aren’t left out of the cataloging process, so the right context is applied. Plus you can ensure critical fields are not left blank, so data is recorded with as much context as possible.

With the business process integration discussed above, you’ll also have a single metadata repository.

All of this ensures sensitive data doesn’t fall through the cracks.

Reasons for Bad Data: Data Migration

Data migration is another key reason for bad data. Modern organizations often juggle a plethora of data systems that process data from an abundance of disparate sources, creating a melting pot for potential issues as data moves through the pipeline, from tool to tool and system to system.

The solution is to introduce a predetermined standard of accuracy through a centralized metadata repository with data governance at the helm. In essence, metadata describes data about data, ensuring that no matter where data is in relation to the pipeline, it still has the necessary context to be deciphered, analyzed and then used strategically.

The potential fallout of using inaccurate data has become even more severe with the GDPR’s implementation. A simple case of tagging and subsequently storing personally identifiable information incorrectly could lead to a serious breach in compliance and significant fines.

Such fines must be considered along with the costs resulting from any PR fallout.

Reasons for Bad Data: Data Integration

The proliferation of data sources, types, and stores increases the challenge of combining data into meaningful, valuable information. While companies are investing heavily in initiatives to increase the amount of data at their disposal, most information workers are spending more time finding the data they need rather than putting it to work, according to Database Trends and Applications (DBTA). erwin is co-sponsoring a DBTA webinar on this topic on July 17. To register, click here.

The need for faster and smarter data integration capabilities is growing. At the same time, to deliver business value, people need information they can trust to act on, so balancing governance is absolutely critical, especially with new regulations.

Organizations often invest heavily in individual software development tools for managing projects, requirements, designs, development, testing, deployment, releases, etc. Tools lacking inter-operability often result in cumbersome manual processes and heavy time investments to synchronize data or processes between these disparate tools.

Data integration combines data from several various sources into a unified view, making it more actionable and valuable to those accessing it.

Getting the Data Governance “EDGE”

The benefits of integrated data governance discussed above won’t be realized if it is isolated within IT with no input from other stakeholders, the day-to-day data users – from sales and customer service to the C-suite. Every data citizen has DG roles and responsibilities to ensure data units have context, meaning they are labeled, cataloged and secured correctly so they can be analyzed and used properly. In other words, the data can be trusted.

Once an organization understands that IT and the business are both responsible for data, it can develop comprehensive, holistic data governance capable of:

  • Reaching every stakeholder in the process
  • Providing a platform for understanding and governing trusted data assets
  • Delivering the greatest benefit from data wherever it lives, while minimizing risk
  • Helping users understand the impact of changes made to a specific data element across the enterprise.

To reduce the risks of and tackle the reasons for bad data and realize larger organizational objectives, organizations must make data governance everyone’s business.

To learn more about the collaborative approach to data governance and how it helps compliance in addition to adding value and reducing costs, get the free e-book here.

Data governance is everyone's business