Tag: data intelligence

Categories
erwin Expert Blog Data Governance

The What & Why of Data Governance

Modern data governance is a strategic, ongoing and collaborative practice that enables organizations to discover and track their data, understand what it means within a business context, and maximize its security, quality and value.

It is the foundation for regulatory compliance and de-risking operations for competitive differentiation and growth.

However, while digital transformation and other data-driven initiatives are desired outcomes, few organizations know what data they have or where it is, and they struggle to integrate known data in various formats and numerous systems – especially if they don’t have a way to automate those processes.

But when IT-driven data management and business-oriented data governance work together in terms of both personnel, processes and technology, decisions can be made and their impacts determined based on a full inventory of reliable information.

Recently, erwin held the first in a six-part webinar series on the practice of data governance and how to proactively deal with its complexities. Led by Frank Pörschmann of iDIGMA GmbH, an IT industry veteran and data governance strategist, it examined “The What & Why of Data Governance.”

The What: Data Governance Defined

Data governance has no standard definition. However, Dataversity defines it as “the practices and processes which help to ensure the formal management of data assets within an organization.”

At erwin by Quest, we further break down this definition by viewing data governance as a strategic, continuous commitment to ensuring organizations are able to discover and track data, accurately place it within the appropriate business context(s), and maximize its security, quality and value.

Mr. Pörschmann asked webinar attendees to stop trying to explain what data governance is to executives and clients. Instead, he suggests they put data governance in real-world scenarios to answer these questions: “What is the problem you believe data governance is the answer to?” Or “How would you recognize having effective data governance in place?”

In essence, Mr. Pörschmann laid out the “enterprise data dilemma,” which stems from three important but difficult questions for an enterprise to answer: What data do we have? Where is it? And how do we get value from it?

Asking how you recognize having effective data governance in place is quite helpful in executive discussions, according to Mr. Pörschmann. And when you talk about that question at a high level, he says, you get a very “simple answer,”– which is ‘the only thing we want to have is the right data with the right quality to the right person at the right time at the right cost.’

The Why: Data Governance Drivers

Why should companies care about data governance?

erwin’s 2020 State of Data Governance and Automation report found that better decision-making is the primary driver for data governance (62 percent), with analytics secondary (51 percent), and regulatory compliance coming in third (48 percent).

In the webinar, Mr. Pörschmann called out that the drivers of data governance are the same as those for digital transformation initiatives. “This is not surprising at all,” he said. “Because data is one of the success elements of a digital agenda or digital transformation agenda. So without having data governance and data management in place, no full digital transformation will be possible.”

Drivers of data governance

Data Privacy Regulations

While compliance is not the No. 1 driver for data governance, it’s still a major factor – especially since the rollout of the European Union’s General Data Protection Regulation (GDPR) in 2018.

According to Mr. Pörschmann, many decision-makers believe that if they get GDPR right, they’ll be fine and can move onto other projects. But he cautions “this [notion] is something which is not really likely to happen.”

For the EU, he warned, organizations need to prepare for the Digital Single Market, agreed on last year by the European Parliament and commission. With it comes clear definitions or rules on data access and exchange, especially across digital platforms, as well as clear regulations and also instruments to execute on data ownership. He noted, “Companies will be forced to share some specific data which is relevant for public security, i.e., reduction of carbon dioxide. So companies will be forced to classify their data and to find mechanisms to share it with such platforms.”

GDPR is also proving to be the de facto model for data privacy across the United States. The new Virginia Consumer Data Privacy Act, which was modeled on the California Consumer Privacy Act (CCPA), and the California Privacy Rights Act (CPRA), all share many of the same requirements as GDPR.

Like CCPA, the Virginia bill would give consumers the right to access their data, correct inaccuracies, and request the deletion of information. Virginia residents also would be able to opt out of data collection.

Nevada, Vermont, Maine, New York, Washington, Oklahoma and Utah also are leading the way with some type of consumer privacy regulation. Several other bills are on the legislative docket in Alabama, Arizona, Florida, Connecticut and Kentucky, all of which follow a similar format to the CCPA.

Stop Wasting Time

In addition to drivers like digital transformation and compliance, it’s really important to look at the effect of poor data on enterprise efficiency/productivity.

Respondents to McKinsey’s 2019 Global Data Transformation Survey reported that an average of 30 percent of their total enterprise time was spent on non-value-added tasks because of poor data quality and availability.

Wasted time is also an unfortunate reality for many data stewards, who spend 80 percent of their time finding, cleaning and reorganizing huge amounts of data, and only 20 percent of their time on actual data analysis.

According to erwin’s 2020 report, about 70 percent of respondents – a combination of roles from data architects to executive managers – said they spent an average of 10 or more hours per week on data-related activities.

The Benefits of erwin Data Intelligence

erwin Data Intelligence by Quest supports enterprise data governance, digital transformation and any effort that relies on data for favorable outcomes.

The software suite combines data catalog and data literacy capabilities for greater awareness of and access to available data assets, guidance on their use, and guardrails to ensure data policies and best practices are followed.

erwin Data Intelligence automatically harvests, transforms and feeds metadata from a wide array of data sources, operational processes, business applications and data models into a central catalog. Then it is accessible and understandable via role-based, contextual views so stakeholders can make strategic decisions based on accurate insights.

You can request a demo of erwin Data Intelligence here.

[blog-cta header=”Webinar: The Value of Data Governance & How to Quantify It” body=”Join us March 15 at 10 a.m. ET for the second webinar in this series, “The Value of Data Governance & How to Quantify It.” Mr. Pörschmann will discuss how justifying a data governance program requires building a solid business case in which you can prove its value.” button=”Register Now” button_link=”https://attendee.gotowebinar.com/register/5489626673791671307″ image=”https://s38605.p1254.sites.pressdns.com/wp-content/uploads/2018/11/iStock-914789708.jpg” ]

Categories
erwin Expert Blog Metadata Management

7 Benefits of Metadata Management

Metadata management is key to wringing all the value possible from data assets.

However, most organizations don’t use all the data at their disposal to reach deeper conclusions about how to drive revenue, achieve regulatory compliance or accomplish other strategic objectives.

What Is Metadata?

Analyst firm Gartner defines metadata as “information that describes various facets of an information asset to improve its usability throughout its life cycle. It is metadata that turns information into an asset.”

Quite simply, metadata is data about data. It’s generated every time data is captured at a source, accessed by users, moved through an organization, integrated or augmented with other data from other sources, profiled, cleansed and analyzed.

It’s valuable because it provides information about the attributes of data elements that can be used to guide strategic and operational decision-making. Metadata management is the administration of data that describes other data, with an emphasis on associations and lineage. It involves establishing policies and processes to ensure information can be integrated, accessed, shared, linked, analyzed and maintained across an organization.

Metadata Answers Key Questions

A strong data management strategy and supporting technology enables the data quality the business requires, including data cataloging (integration of data sets from various sources), mapping, versioning, business rules and glossaries maintenance and metadata management (associations and lineage).

Metadata answers a lot of important questions:

  • What data do we have?
  • Where did it come from?
  • Where is it now?
  • How has it changed since it was originally created or captured?
  • Who is authorized to use it and how?
  • Is it sensitive or are there any risks associated with it?

Metadata also helps your organization to:

  • Discover data. Identify and interrogate metadata from various data management silos.
  • Harvest data. Automate the collection of metadata from various data management silos and consolidate it into a single source.
  • Structure and deploy data sources. Connect physical metadata to specific data models, business terms, definitions and reusable design standards.
  • Analyze metadata. Understand how data relates to the business and what attributes it has.
  • Map data flows. Identify where to integrate data and track how it moves and transforms.
  • Govern data. Develop a governance model to manage standards, policies and best practices and associate them with physical assets.
  • Socialize data. Empower stakeholders to see data in one place and in the context of their roles.

Metadata management

The Benefits of Metadata Management

1. Better data quality. With automation, data quality is systemically assured with the data pipeline seamlessly governed and operationalized to the benefit of all stakeholders. Data issues and inconsistencies within integrated data sources or targets are identified in real time to improve overall data quality by increasing time to insights and/or repair. It’s easier to map, move and test data for regular maintenance of existing structures, movement from legacy systems to new systems during a merger or acquisition or a modernization effort.

2. Quicker project delivery. Automated enterprise metadata management provides greater accuracy and up to 70 percent acceleration in project delivery for data movement and/or deployment projects. It harvests metadata from various data sources and maps any data element from source to target and harmonize data integration across platforms. With this accurate picture of your metadata landscape, you can accelerate Big Data deployments, Data Vaults, data warehouse modernization, cloud migration, etc.

3. Faster speed to insights. High-paid knowledge workers like data scientists spend up to 80 percent of their time finding and understanding source data and resolving errors or inconsistencies, rather than analyzing it for real value. That equation can be reversed with stronger data operations and analytics leading to insights more quickly, with access/connectivity to underlying metadata and its lineage. Technical resources are free to concentrate on the highest-value projects, while business analysts, data architects, ETL developers, testers and project managers can collaborate more easily for faster decision-making.

4. Greater productivity & reduced costs. Being able to rely on automated and repeatable metadata management processes results in greater productivity. For example, one erwin DI customer has experienced a steep improvement in productivity – more than 85 percent – because manually intensive and complex coding efforts have been automated and 70+ percent because of seamless access to and visibility of all metadata, including end-to-end lineage. Significant data design and conversion savings, up to 50 percent and 70 percent respectively, also are possible with data mapping costs going down as much as 80 percent.

5. Regulatory compliance. Regulations such as the General Data Protection Regulation (GDPR), Health Insurance and Portability Accountability Act (HIPAA), Basel Committee on Banking Supervision (BCBS) and The California Consumer Privacy Act (CCPA) particularly affect sectors such as finance, retail, healthcare and pharmaceutical/life sciences. When key data isn’t discovered, harvested, cataloged, defined and standardized as part of integration processes, audits may be flawed. Sensitive data is automatically tagged, its lineage automatically documented, and its flows depicted so that it is easily found and its use across workflows easily traced.

6. Digital transformation. Knowing what data exists and its value potential promotes digital transformation by 1) improving digital experiences because you understand how the organization interacts with and supports customers, 2) enhancing digital operations because data preparation and analysis projects happen faster, 3) driving digital innovation because data can be used to deliver new products and services, and 4) building digital ecosystems because organizations need to establish platforms and partnerships to scale and grow.

7. An enterprise data governance experience. Stakeholders include both IT and business users in collaborative relationships, so that makes data governance everyone’s business. Modern, strategic data governance must be an ongoing initiative, and it requires everyone from executives on down to rethink their data duties and assume new levels of cooperation and accountability. With business data stakeholders driving alignment between data governance and strategic enterprise goals and IT handling the technical mechanics of data management, the door opens to finding, trusting and using data to effectively meet any organizational objective.

An Automated Solution

When approached manually, metadata management is expensive, time-consuming, error-prone and can’t keep pace with a dynamic enterprise data management infrastructure.

And while integrating and automating data management and data governance is still a new concept for many organizations, its advantages are clear.

erwin’s metadata management offering, the erwin Data Intelligence Suite (erwin DI), includes data catalog, data literacy and automation capabilities for greater awareness of and access to data assets, guidance on their use, and guardrails to ensure data policies and best practices are followed. Its automated, metadata-driven framework gives organizations visibility and control over their disparate data streams – from harvesting to aggregation and integration, including transformation with complete upstream and downstream lineage and all the associated documentation.

erwin has been named a leader in the Gartner 2020 “Magic Quadrant for Metadata Management Solutions” for two consecutive years. Click here to download the full Gartner 2020 “Magic Quadrant for Metadata Management Solutions” report.

Categories
erwin Expert Blog

Data Intelligence in the Next Normal; Why, Who and When?

While many believe that the dawn of a new year represents a clean slate or a blank canvas, we simply don’t leave the past behind by merely flipping over a page in the calendar.

As we enter 2021, we will also be building off the events of  2020 – both positive and negative – including the acceleration of digital transformation as the next normal begins to be defined.

data intelligence

As the pandemic took hold, IDC surveyed technology users and decision makers around the globe, reaching out every two weeks until September, when the survey frequency shifted to monthly. These surveys helped IDC develop a model that describes the five stages of enterprise recovery, aligning business focus with the economic situation:

  • When the COVID-19 crisis hit, organizations focused on business continuity.
  • As the economy slowed, they focused on cost optimization.
  • In the recession period, their focus turned to business resiliency.
  • As the economy returns to growth, organizations are making targeted investments.
  • When we enter into the next normal, the future enterprise will emerge.

The IDC surveys explored how the crisis impacted budgets across different areas of IT, from hardware and networking, to software and professional services. When the pandemic first hit, there was some negative impact on big data and analytics spending.

However, the economic situation changed as time went on. Digital transformation was accelerated, and budgets for spending on big data and analytics increased. This spending has continued during the return to growth, with more organizations moving toward becoming the future enterprise.

I have long stated that data is the lifeblood of digital transformation, and if the pandemic really has accelerated digital transformation, then the trends reported in IDC’s worldwide surveys make sense.

But data without intelligence is just data, and this is WHY data intelligence is required.

Data intelligence is a key input to data enablement in the digital enterprise, both by improving data literacy among data-native workers and by assuring the right data is being used at the right time, and for the right reason(s).

WHO needs to be involved in implementing and using data intelligence in the digital enterprise?

There is an ever-growing number of roles that work with data daily to complete tasks, make decisions, and affect business outcomes. These roles range from technical to business, from operations to strategy, and from the back office to the front office.

IDC has defined people in these roles as a generation: “Generation Data,” or “Gen-D” for short. Gen-D workers are data-natives — data is what they work in and work with to complete their tasks, tactical and/or strategic.

You may be part of Gen-D if “data” is in your job title, you are expected to make data-driven decisions, and you are able to use data to communicate with others. Gen-D workers also contribute to the overall data knowledge in the organization by participating in data intelligence and data literacy efforts and promoting good data culture.

WHEN do you need to gather intelligence about your data?

Now is the time.

The next or new normal has already begun and the more you know about your data, the better your digital business outcomes will be. It has been said that while it can take a long time to gain a customer’s trust, it only takes one bad experience to lose it.

Personally, I have had several instances of poor digital experiences such as items sent to the wrong address or orders (including mobile food orders) being fulfilled incorrectly.

Each represents a data problem: incorrect data, incorrect data interpretation, or a complete disconnect between the virtual and physical world. In these cases, better data intelligence could have helped in assuring the correct address, enabling correct order fulfillment, and assisting with interpretation through better data definition and description.

Even if you don’t have a formal data intelligence program in place, there is a good possibility your organization has intelligence about its data, because it is difficult for data to exist without some form of associated metadata.

Technical metadata is what makes up database schema and table definitions. Logical and physical data models may exist in data modeling or general-purpose diagraming software.

There is also a high likelihood that data models, data dictionaries, and data catalogs exist in the ubiquitous spreadsheet, or in centralized document repositories. However, just having metadata isn’t the same as managing and leveraging it as intelligence. Data in modern business environments is very dynamic, constantly moving, drifting, and shifting – requiring automated collection, management, and analytics to extract and leverage intelligence about it.

In many English-speaking countries, “Auld Lang Syne,” a Scots-language poem written by Robbie Burns and set to a common folk song tune, is often sung as the clock strikes midnight on the first day of the new year.

The phrase “auld lang syne” has several interpretations, but it can loosely be translated as “for the sake of old times.” As we move into 2021, we need to forget the negatives of 2020, and build on the positives to help define the next normal.

Categories
Data Intelligence erwin Expert Blog

From Chaos to Control with Data Intelligence

As the amount of data grows exponentially, organizations turn to data intelligence to reach deeper conclusions about driving revenue, achieving regulatory compliance and accomplishing other strategic objectives.

It’s no secret that data has grown in volume, variety and velocity, with 2.5 quintillion bytes generated every day and 90 percent of the world’s data volume created just in the last two years. This data explosion has overwhelmed most organizations, making it nearly impossible for them to manage much less put to smart, strategic use. How do you identify the time-sensitive, relevant insights that could mean the difference between the life and death of your business?

Data Intelligence

Time sensitivity in data management and analytics is a massive issue. Data needs to fuel rapid decisions that make your organization more effective, customer-centric and competitive. That was true before COVID 19, and it’s even more important in the face of the radical disruption it’s caused. The answer is radical transformation, made possible by an intelligent, data-driven approach to:

  • New business models
  • New products and services
  • Hyper-competition
  • Market expansion

One Customer’s Journey to Controlling Data Chaos

Ultra Mobile recently shared how it uses erwin Data Intelligence (erwin DI) as part of a modern, ongoing approach to data governance and therefore control versus chaos.

To manage not only risk but also to grow and compete effectively requires the ability to deal with both planned and unplanned change.

With erwin DI as its data governance platform, Ultra Mobile has a one-stop shop to see all of its data – and data changes – in one place thanks to forward and reverse lineage. Now, questions about the health of the business can be answered, including how best to retain customers, and it can explore ways to grow the subscriber base.

Being able to integrate all data touchpoints, including erwin DM for data modeling, Denodo for data visualization, and Jira for ticketing, has been key. This metadata is ingested into the data catalog, definitions are added within a business glossary, and the searchable repository enables users to understand how data is used and stored.

erwin DI’s mind map also has proved helpful with being able to see associations and entity relationships, especially in terms of impact analysis for evaluating planned changes and their downstream effects.

Watch the full webinar.

Data Intelligence Just Got Smarter

erwin just released a new version of erwin DI. The enhancements include improvements to the user interface (UI), plus new artificial intelligence (AI) and self-service data discovery capabilities.

 

The new erwin DI makes it easier for organizations to tailor the solution to meet the unique needs of their data governance frameworks, identify and socialize the most valuable data assets, and expand metadata scanning and sensitive data tracking.

Using erwin DI, customers are powering comprehensive data governance initiatives, cloud migration and other massive digital transformation projects.

It facilitates both IT- and business-friendly discovery, navigation and understanding of data assets within context and in line with governance protocols.

And it provides organizations with even more flexibility to ensure the software fits their unique frameworks and workflows because one size does not fit all when it comes to data governance.

Backed by a flexible metamodel and deep metadata-driven automation, the updated erwin DI uniquely addresses both IT and business data governance needs to safeguard against risks and harness opportunities.

It combines and then raises the visibility of business and physical data assets in a framework that is flexible but always in sync and therefore sustainable. Then stakeholders from across the enterprise can discover, manage and collaborate on the most relevant and valuable data assets.

The latest erwin DI release builds on prior 2020 updates with:

  • New role-based and governance assignment capabilities, making it easier for an organization to tailor erwin DI to its data governance needs and framework
  • Enhanced UI, workflow and search to speed navigation, asset discovery, contextual understanding and data governance management
  • Expanded AI capabilities to enrich metadata scanning and speed the handling of sensitive data for automated GDPR and CCPA compliance programs
  • Greater visibility into business and data lineage through new vantage points, filters and drilldowns
  • Improved socialization and collaboration features to increase business user engagement and capitalize on organizational data quality knowledge
  • More administrative tools to efficiently onboard new users and roles, manage access rights, and address audit requests

Additionally, erwin DI was recently evaluated by Gartner for the 2020 “Metadata Management Solutions Magic Quadrant,” which named erwin as a “Leader” for the second consecutive year. Click here to download a copy of the Gartner Magic Quadrant Report.

You also can request a free demo of erwin DI here.

Gartner Magic Quadrant

Categories
Data Governance erwin Expert Blog

erwin Positioned as a Leader in Gartner’s 2020 Magic Quadrant for Metadata Management Solutions for Second Year in a Row

erwin has once again been positioned as a Leader in the Gartner “2020 Magic Quadrant for Metadata Management Solutions.”

This year, erwin had the largest move of any player on the Quadrant and moved up significantly in terms of “Ability to Execute” and also in “Vision.”

This recognition affirms our efforts in developing an integrated platform for enterprise modeling and data intelligence to support data governance, digital transformation and any other effort that relies on data for favorable outcomes.

erwin’s metadata management offering, the erwin Data Intelligence Suite (erwin DI), includes data catalog, data literacy and automation capabilities for greater awareness of and access to data assets, guidance on their use, and guardrails to ensure data policies and best practices are followed.

With erwin DI’s automated, metadata-driven framework, organizations have visibility and control over their disparate data streams – from harvesting to aggregation and integration, including transformation with complete upstream and downstream lineage and all the associated documentation.

We’re super proud of this achievement and the value erwin DI provides.

We invite you to download the report and quadrant graphic.

Categories
erwin Expert Blog Data Intelligence

Surviving Radical Disruption with Data Intelligence

It’s certainly no secret that data has been growing in volume, variety and velocity, and most companies are overwhelmed by managing it, let alone harnessing it to put it to work.

We’re now generating 2.5 quintillion bytes of data every day, and 90% of the world’s data volume has been created in the past two years alone. With this absolute data explosion, it’s nearly impossible to filter out the time-sensitive data, the information that has immediate relevance and impact on your business.

And this time sensitivity is a massive issue, as taking a proactive and data-driven approach can literally mean life or death to your business or to your customers. And that’s where data analytics can play a huge role.

By leveraging the power of the cloud, harnessing data from the Internet of Things (IoT) and other events, and processing this data in near-real time, analytics helps to effectively process the relentless incoming data feed.

Without automation and the development of a governed data pipeline, you’ll never have enough data scientists in the front office to put the data to work. The benefits of fast time to insights is clear, regardless of the industry you’re in.

Think about these examples: a communications agency that needs to get out in front of a difficult message, a retailer driving sales based on real-time customer behavior, a logistics and delivery company needing to understand road conditions, stoppages and up-to-the-minute weather, or a hospital that needs to tailor patient care based on the latest public health findings.

Your data needs to fuel rapid decisions that make your organization more effective, customer-centric and competitive. This was true before the world changed.

COVID-19 Changed Everything

COVID changed everything. It’s a radical disruptor the likes of which we’ve never seen.

As a CEO, a husband and a father, I’ve made decisions during the past seven months that I never dreamed possible, and I’m sure this is true for you and your family – and business – as well.

Now to survive and thrive in the face of radical disruption requires radical transformation and new business models. Reimagining business, like moving fitness centers outdoors, or developing new products and services, such as restaurants packaging fruits and vegetables to sell as food bundles, or market expansion, like traditional grocers that are becoming online shopping hubs.

The companies that come out of this historic period of global uncertainty and change are those who’ve taken intelligent and data-driven approaches to their businesses.

What holds most companies back from faster time to insights and leveraging radical transformation? I think those answers can be found by asking these core questions:

  1. What data do I have?
  2. Where is the data?
  3. What people and systems are using that data and for what purposes?
  4. What processes should governance use?
  5. How is this data relevant and accessible to the business?  

Data Intelligence Provides an EDGE

There’s a common denominator in what they’re all missing, and that is data intelligence.

IDC defines data intelligence as business, technical, relational, and operational metadata that provides transparency of data profiles, classification, quality, location, context, and lineage, providing people, processes, and technology with trustworthy, reliable data.

In a new IDC Solution Brief, “The Value of Robust Data Intelligence to Enable Data Governance with erwin,” its authors state:

Data is the lifeblood of the digital economy — it is what is driving new business models, better customer experiences, better decision-making, and artificially intelligent automation. The global pandemic in 2020 has accelerated digital transformation and amplified the value of data in what will become the next normal as the global economy struggles through recovery. In a world where market conditions, supply chains, work locations, and communication methods are constantly changing, data is a constant source that can be used to inform decisions from crisis to recovery. To use data effectively, it needs to be trusted, understood, and used appropriately, and herein lies many problems that organizations face in the digital economy.

The IDC authors also interviewed erwin customers who described the erwin Data Intelligence Suite, part of the erwin EDGE platform, as a fundamental component of their efforts to generate more value from data while minimizing data-related risk.

The erwin EDGE helps organizations unlock their potential by maximizing the security, quality and value of their data assets, and it operationalizes these steps by connecting enterprise architecture, business process and data modeling with data intelligence software.

The result is an automated, real-time, high-quality data pipeline from which accurate insights can be derived.

The erwin EDGE enables organizations to see how data flows through and impacts all their business, technology and data architectures. Then all stakeholders within a company, those in IT as well as the larger enterprise, can collaborate to make better decisions based upon data truth, not just gut instinct.

Parts of this blog are excerpted from my keynote on day No. 1 of erwin Insights 2020, our virtual conference on enterprise modeling and data governance/intelligence.

You can view the entire keynote and all other sessions of the conference by registering here.

erwin Insights 2020

Subscribe to the erwin Expert Blog

Once you submit the trial request form, an erwin representative will be in touch to verify your request and help you start data modeling.

Categories
erwin Expert Blog Data Intelligence

Why You Need End-to-End Data Lineage

Not Documenting End-to-End Data Lineage Is Risky Business – Understanding your data’s origins is key to successful data governance.

Not everyone understands what end-to-end data lineage is or why it is important. In a previous blog, I explained that data lineage is basically the history of data, including a data set’s origin, characteristics, quality and movement over time.

This information is critical to regulatory compliance, change management and data governance not to mention delivering an optimal customer experience. But given the volume, velocity and variety of data (the three Vs of data) we generate today, producing and keeping up with end-to-end data linage is complex and time-consuming.

Yet given this era of digital transformation and fierce competition, understanding what data you have, where it came from, how it’s changed since creation or acquisition, and whether it poses any risks is paramount to optimizing its value. Furthermore, faulty decision-making based on inconsistent analytics and inaccurate reporting can cost millions.

Data Lineage

Data Lineage Tells an Important Origin Story

End-to-end data lineage explains how information flows into, across and outside an organization. And knowing how information was created, its origin and quality may have greater value than a given data set’s current state.

For example, data lineage provides a way to determine which downstream applications and processes are affected by a change in data expectations and helps in planning for application updates.

As I mentioned above, the three Vs of data and the integration of systems makes it difficult to understand the resulting data web much less capture a simple visual of that flow. Yet a consistent view of data and how it flows is paramount to the success of enterprise data governance and any data-driven initiative.

Whether you need to drill down for a granular view of a particular data set or create a high-level summary to describe a particular system and the data it relies on, end-to-end data lineage must be documented and tracked, with an emphasis on the dynamics of data processing and movement as opposed to data structures. Data lineage helps answer questions about the origin of data in key performance indicator (KPI) reports, including:

  • How are the report tables and columns defined in the metadata?
  • Who are the data owners?
  • What are the transformation rules?

Five Consequences of Ignoring Data Lineage

Why do so many organizations struggle with end-to-end data lineage?

The struggle is real for a number of reasons. At the top of the list, organizations are dealing with more data than ever before using systems that weren’t designed to communicate effectively with one another.

Next, their IT and business stakeholders have a difficult time collaborating. And, for a lot of organizations, they’ve relied mostly on manual processes – if data lineage documentation has been attempted at all.

The risks of ignoring end-to-end data lineage are just too great. Let’s look at some of those consequences:

  1. Derailed Projects

Effectively managing business operations is a key factor to success– especially for organizations that are in the midst of digital transformation. Failures in business processes attributed to errors can be a big problem.

For example, in a typical business scenario where an incorrect data set is discovered within a report, the length of time (on average) that it takes a team to find the source of the error can take days or sometimes weeks – derailing the project and costing time and money.

  1. Policy Bloat and Unruly Rules

The business glossary environment must represent the actual environment, e.g., be refreshed and synched, otherwise it becomes obsolete. You need real collaboration.

Data dictionaries, glossaries and policies can’t live in different formats and in different places. It is common for these to be expressed in different ways, depending on the database and underlying storage technology, but this causes policy bloat and rules that no organization, team or employee will understand, let alone realistically manage.

Effective data governance requires that business glossaries, data dictionaries and data privacy policies live in one central location, so they can be easily tracked, monitored and updated over time.

  1. Major Inefficiencies

Successful data migration and upgrades rely on seamless integration of tools and processes with coordinated efforts of people/resources. A passive approach frequently relies on creating new copies of data, usually with sensitive identifiers removed or obscured.

Not only does this passive approach create inefficiencies between determining what data to copy, how to copy it, and where to store the copy, it also creates new volumes of data that become harder to track over time. Yet again, a passive approach to data cannot scale. Direct access to the same live data across the organization is required.

  1. Not Knowing Where Your Data Is

Metadata management and manual mapping are a challenge to most organizations. Data comes in all shapes, sizes and formats, and there is no way to know what type of data a project will need – or even where that data will sit.

Some data might be in the cloud, some on premise, and sometimes projects will require a hybrid approach. All data must be governed, regardless of where it is located.

  1. Privacy and Compliance Challenges

Privacy and compliance personnel know the rules that must be applied to data, but may not necessarily know the technology. Instead, automated data governance requires that anyone, with any level of expertise, can understand what rules (e.g. privacy policies) are applied to enterprise data.

Organizations with established data governance must empower both those with technical skill sets and those with privacy and compliance knowledge, so all teams can play a meaningful role controlling how data is used.

For more information on data lineage, get the free white paper, Tech Brief: Data Lineage.

End-to-End Data Lineage

 

Categories
erwin Expert Blog Data Intelligence

What Is Data Literacy?

Today, data literacy is more important than ever.

Data is now being used to support business decisions few executives thought they’d be making even six months ago.

With your employees connected and armed with data that paints a clear picture of the business, your organization is better prepared to turn its attention to whatever your strategic priority may be – i.e. digital transformation, customer experience, or withstanding this current (or future) crisis.

So, what is data literacy?

Data Literacy

Data Literacy Definition

Gartner defines data literacy as the ability to read, write and communicate data in context, including an understanding of data sources and constructs, analytical methods and techniques applied — and the ability to describe the use case, application and resulting value.

Organizations use data literacy tools to improve data literacy across the organization. A good data literacy tool will include functionality such as business glossary management and self-service data discovery. The end result is an organization that’s more data fluent and efficient in how they store, discover and use their data.

What Is Data Literacy For?

For years, we’ve been saying that “we’re all data people.” When all stakeholders in an organization can effectively “speak data” they can:

  • Better understand and identify the data they require
  • Be more self-sufficient in accessing and preparing the data
  • Better articulate the gaps that exist in the data landscape
  • Share their knowledge and experience with data with other consumers to contribute to the greater good
  • Collaborate more effectively with their partners in data (management and governance) for greater efficiency and higher quality outcomes

Why is Data Literacy Important?

Without good data, it’s difficult to make good decisions.

Data access, literacy and knowledge leads to sound decision-making and that’s key to data governance and any other data-driven effort.

Data literacy enables collaboration and innovation. To determine if your organization is data literate you need to ask two questions:  

  1. Can your employees use data to effectively communicate with each other?
  2. Can you develop and circulate ideas that will help the business move forward?

data literacy and data intelligence

The Data Literacy and Data Intelligence Connection

Businesses that invest in data intelligence and data literacy are better positioned to weather any storm and chart a path forward because they have accurate, trusted data at their disposal.

erwin helps customers turn their data from a burden into a benefit by fueling an accurate, real-time, high-quality data pipeline they can mine for insights that lead to smart decisions for operational excellence.

erwin Data Intelligence (erwin DI) combines data catalog and data literacy capabilities for greater awareness of and access to available data assets, guidance on their use, and guardrails to ensure data policies and best practices are followed.

erwin Data Literacy (DL) is founded on enriched business glossaries and socializing data so all stakeholders can view and understand it within the context of their roles.

It allows both IT and business users to discover the data available to them and understand what it means in common, standardized terms, and automates common data curation processes, such as name matching, categorization and association, to optimize governance of the data pipeline including preparation processes.

erwin DL provides self-service, role-based, contextual data views. It also provides a business glossary for the collaborative definition of enterprise data in business terms.

It also includes built-in accountability and workflows to enable data consumers to define and discover data relevant to their roles, facilitate the understanding and use of data within a business context, and ensure the organization is data literate.

With erwin DL, your organization can build glossaries of terms in taxonomies with descriptions, synonyms, acronyms and their associations to data policies, rules and other critical governance artifacts. Other advantages are:

  • Data Visibility & Governance: Visualize and navigate any data from anywhere within a business-centric data asset framework that provides organizational alignment and robust, sustainable data governance.
  • Data Context & Enrichment: Put data in business context and enable stakeholders to share best practices and build communities by tagging/commenting on data assets, enriching the metadata.
  • Enterprise Collaboration & Empowerment: Break down IT and business silos to provide broad access to approved organizational information.
  • Greater Productivity: Reduce the time it takes to find data assets and therefore reliance on technical resources, plus streamline workflows for faster analysis and decision-making.
  • Accountability & Regulatory Peace of Mind: Create an integrated ecosystem of people, processes and technology to manage and protect data, mitigating a wide range of data-related risks and improving compliance.
  • Effective Change Management: Better manage change with the ability to identify data linkages, implications and impacts across the enterprise.
  • Data Literacy, Fluency & Knowledge: Enhance stakeholder discovery and understanding of and trust in data assets to underpin analysis leading to actionable insights.

Learn more about the importance of data literacy by requesting a free demo of erwin Data Intelligence.

erwin Data Intelligence

 

Categories
Data Intelligence erwin Expert Blog

Four Steps to Building a Data-Driven Culture

data-driven culture

Fostering organizational support for a data-driven culture might require a change in the organization’s culture. But how?

Recently, I co-hosted a webinar with our client E.ON, a global energy company that reinvented how it conducts business from branding to customer engagement – with data as the conduit.

There’s no doubt E.ON, based in Essen, Germany, has established one of the most comprehensive and successful data governance programs in modern business.

For E.ON, data governance is not just about data management but also about using information to increase efficiencies. The company needed to help its data scientists and engineers improve their knowledge of the data, find the best data for use at the best time, and put the data in the most appropriate business context.

As an example, E.ON was able to improve data quality, detect redundancies, and create a needs-based, data-use environment by applying a common set of business terms across the enterprise.

Avoiding Hurdles

Businesses have not been able to get as much mileage out of their data governance efforts as hoped, chiefly because of how it’s been handled. And data governance initiatives sometimes fail because organizations tend to treat them as siloed IT programs rather than multi-stakeholder imperatives.

Even when business groups recognize the value of a data governance program and the potential benefits to be derived from it, the IT group traditionally has owned the effort and paid for it.

Despite enterprise-wide awareness of the importance of data governance, a troublingly large number of organizations continue to stumble because of a lack of executive support.

IT and the business will need to take responsibility for selling the benefits of data governance across the enterprise and ensure all stakeholders are properly educated about it.

IT may have to go it alone, at least initially, educating the business on the risks and rewards of data governance and the expectations and accountabilities in implementing it. The business needs to have a role in the justification.

Being a Change Agent

Becoming a data-driven enterprise means making decisions based on facts. It requires a clear vision, strategy and disciplined execution. It also must be well thought out, understood and communicated to others – from the C-suite on down.

For E.ON, the board supported and drove a lot of the thinking that data has to be at the center of everything to reimagine the company. But the data team still needed to convince the head of every one of the company’s hundreds of legal entities to support the digital transformation journey. As a result, the team went on a mission to spread the message.

“The biggest challenge was change management — convincing people to be part of the journey. It is very often underestimated,” said Romina Medici, E.ON’s Program Manager for Data Management and Governance. “Technology is logical, so you can always understand it. Culture is more complex and more diverse.”

She said that ultimately the “communication (across the organization) was bottom up and top down.”

Four Steps to Building a Data-Driven Culture

1. Accelerate Time to Value: Data governance isn’t a one-off project with a defined endpoint. It’s an on-going initiative that requires active engagement from executives and business leaders. The ability to make faster decisions based on data is one way to make the organization pay attention.

2. Ensure Company-Wide Compliance: Compliance isn’t just about government regulations. In today’s business environment, we’re all data people. Everyone in the organization needs to commit to data compliance to ensure high-quality data.

3. Demand Trusted Insights Based on Data Truths: To make smart decisions, you can’t have multiple sets of numbers. Everyone needs to be in lockstep, using and basing decisions on the same data.

4. Foster Data-Driven Collaboration: We call this “social data governance,” meaning you foster collaboration across the business, all the time. 

A data-driven approach has never been more valuable to addressing the complex yet foundational questions enterprises must answer. Organizations that have their data management, data governance and data intelligence houses in order are much better positioned to respond to challenges and thrive moving forward.

As demonstrated by E.ON, data-driven cultures start at the top – but need to proliferate up and down, even sideways.

Business transformation has to be based on accurate data assets within the right context, so organizations have a reliable source of truth on which to base their decisions.

erwin provides a with the data catalog, lineage, glossary and visualization capabilities needed to evaluate the business in its current state and then evolve it to serve new objectives.

Request a demo of the erwin Data Intelligence Suite.

Data Intelligence Solution: Data Catalog, Data Literacy and Automation Tools

Categories
Data Intelligence Data Modeling Business Process Enterprise Architecture erwin Expert Blog

Benefits of Enterprise Modeling and Data Intelligence Solutions

Users discuss how they are putting erwin’s data modeling, enterprise architecture, business process modeling, and data intelligences solutions to work

IT Central Station members using erwin solutions are realizing the benefits of enterprise modeling and data intelligence. This article highlights some specific use cases and the results they’re experiencing within the organizations.

Enterprise Architecture & Business Process Modeling with erwin Evolve

An enterprise architect uses erwin Evolve at an aerospace/defense firm with more than 10,000 employees. His team is “doing business process modeling and high-level strategic modeling with its capabilities.” Others in his company are using it for IT infrastructure, such as aligning requirements to system solutions.

For Matthieu G., a senior business process management architect at a pharma/biotech company with more than 5,000 employees, erwin Evolve was useful for enterprise architecture reference. As he put it, “We are describing our business process and we are trying to describe our data catalog. We are describing our complete applications assets, and we are interfacing to the CMDB of our providers.”

His team also is using the software to manage roadmaps in their main transformation programs. He added, “We have also linked it to our documentation repository, so we have a description of our data documents.” They have documented 200 business processes in this way. In particular, the tool helped them to design their qualification review, which is necessary in a pharmaceutical business.

erwin Evolve users are experiencing numerous benefits. According to the aerospace enterprise architect, “It’s helped us advance in our capabilities to perform model-based systems engineering, and also model-based enterprise architecture.”

This matters because, as he said, “By placing the data and the metadata into a model, which is what the tool does, you gain the abilities for linkages between different objects in the model, linkages that you cannot get on paper or with Visio or PowerPoint.” That is a huge differentiator for this user.

This user also noted, “I use the automatic diagramming features a lot. When one of erwin’s company reps showed that to me a couple of years ago, I was stunned. That saves hours of work in diagramming. That capability is something I have not seen in other suppliers’ tools.”

He further explained “that really helps too with when your data is up to date. The tool will then automatically generate the updated diagram based on the data, so you know it’s always the most current version. You can’t do that in things like Visio and PowerPoint. They’re static snapshots of a diagram at some point in time. This is live and dynamic.”

erwin DM

Data Modeling with erwin Data Modeler

George H., a technology manager, uses erwin Data Modeler (erwin DM) at a pharma/biotech company with more than 10,000 employees for their enterprise data warehouse.

He elaborated by saying, “We have an enterprise model being maintained and we have about 11 business-capability models being maintained. Examples of business capabilities would be finance, human resources, supply-chain, sales and marketing, and procurement. We maintain business domain models in addition to the enterprise model.”

Roshan H., an EDW architect/data modeler who uses erwin DM at Royal Bank of Canada, works on diverse platforms, including Microsoft SQL Server, Oracle, DB2, Teradata and NoSQL. After gathering requirements and mapping data on Excel, they start building the conceptual model and then the logical model with erwin DM.

He said, “When we have these data models built in the erwin DM, we generate the PDF data model diagrams and take it to the team (DBA, BSAs, QA and others) to explain the model diagram. Once everything is reviewed, then we go on to discuss the physical data model.”

“We use erwin DM to do all of the levels of analysis that a data architect does,” said Sharon A., a senior manager, data governance at an insurance company with over 500 employees. She added, “erwin DM does conceptual, logical and physical database or data structure capture and design, and creates a library of such things.

We do conceptual data modeling, which is very high-level and doesn’t have columns and tables. It’s more concepts that the business described to us in words. We can then use the graphic interface to create boxes that contain descriptions of things and connect things together. It helps us to do a scope statement at the beginning of a project to corral what the area is that the data is going to be using.”

Data Governance with erwin Data Intelligence

IT Central Station members are seeing benefits from using erwin Data Intelligence (erwin DI) for data governance use cases. For Rick D., a data architect at NAMM, a small healthcare company, erwin DI “enabled us to centralize a tremendous amount of data into a common standard, and uniform reporting has decreased report requests.”

As a medical company, they receive data from 17 different health plans. Before adopting erwin DI, they didn’t have a centralized data dictionary of their data. The benefit of data governance, as he saw it, was that “everybody in our organization knows what we are talking about. Whether it is an institutional claim, a professional claim, Blue Cross or Blue Shield, health plan payer, group titles, names, etc.”

A solution architect at a pharma/biotech company with more than 10,000 employees used erwin DI for metadata management, versioning of metadata and metadata mappings and automation. In his experience, applying governance to metadata and creating mappings has helped different stakeholders gain a good understanding of the data they use to do their work.

Sharon A. had a comparable use case. She said, “You can map the business understanding in your glossary back to your physical so you can see it both ways. With erwin DI, I can have a full library of physical data there or logical data sets, publish it out through the portal, and then the business can do self-service. The DBAs can use it for all different types of value-add from their side of the house. They have the ability to see particular aspects, such as RPII, and there are some neat reports which show that. They are able manage who can look at these different pieces of information.”

For more real erwin user experiences, visit IT Central Station.