Tag: data governance

Categories
erwin Expert Blog

Data Governance and Metadata Management: You Can’t Have One Without the Other

When an organization’s data governance and metadata management programs work in harmony, then everything is easier.

Data governance is a complex but critical practice. There’s always more data to handle, much of it unstructured; more data sources, like IoT, more points of integration, and more regulatory compliance requirements.

Creating and sustaining an enterprise-wide view of and easy access to underlying metadata is also a tall order.

The numerous data types and data sources that exist today weren’t designed to work together, and data infrastructures have been cobbled together over time with disparate technologies, poor documentation and little thought for downstream integration.

Therefore, most enterprises have encountered difficulty trying to master data governance and metadata management, but they need a solid data infrastructure on which to build their applications and initiatives.

Without it, they risk faulty analyses and insights that effect not only revenue generation but regulatory compliance and any number of other organizational objectives.

Data Governance Predictions

Data Governance Attitudes Are Shifting

The 2020 State of Data Governance and Automation (DGA) shows that attitudes about data governance and the drivers behind it are changing – arguably for the better.

Regulatory compliance was the biggest driver for data governance implementation, according to the 2018 report. That’s not surprising given the General Data Protection Regulation (GDPR) was going into effect just six months after the survey.

Now better decision-making is the primary reason to implement data governance, cited by 60 percent of survey participants. This shift suggests organizations are using data to improve their overall performance, rather than just trying to tick off a compliance checkbox.

We’re pleased to see this because we’ve always believed that IT-siloed data governance has limited value. Instead, data governance has to be an enterprise initiative with IT and the wider business collaborating to limit data-related risks and determine where greater potential and value can be unleashed.

Metadata Management Takes Time

About 70 percent of DGA report respondents – a combination of roles from data architects to executive managers – say they spend an average of 10 or more hours per week on data-related activities.

Most of that time is spent on data analysis – but only after searching for and preparing data.

A separate study by IDC indicates data professionals actually spend 80 percent of their time on data discovery, preparation and protection and only 20 percent on analysis.

Why such a heavy lift? Finding metadata, “the data about the data,” isn’t easy.

When asked about the most significant bottlenecks in the data value chain, documenting complete data lineage leads with 62 percent followed by understanding the quality of the source data (58 percent), discovery, identification and harvesting (55 percent), and curating data assets with business context (52%.)

So it make sense that the data operations deemed most valuable in terms of automation are:

  • Data Lineage (65%)
  • Data Cataloging (61%)
  • Data Mapping (53%)
  • Impact Analysis (48%)
  • Data Harvesting (38%)
  • Code Generation (21%)

But as suspected, most data operations are still manual and largely dependent on technical resources. They aren’t taking advantage of repeatable, sustainable practices – also known as automation.

The Benefits of Automating Data Governance and Metadata Management Processes

Availability, quality, consistency, usability and reduced latency are requirements at the heart of successful data governance.

And with a solid framework for automation, organizations can generate metadata every time data is captured at a source, accessed by users, moved through an organization, integrated or augmented with other data from other sources, profiled, cleansed and analyzed.

Other benefits of automating data governance and metadata management processes include:

  • Better Data Quality – Identification and repair of data issues and inconsistencies within integrated data sources in real time
  • Quicker Project Delivery – Acceleration of Big Data deployments, Data Vaults, data warehouse modernization, cloud migration, etc.
  • Faster Speed to Insights – Reversing the 80/20 rule that keeps high-paid knowledge workers too busy finding, understanding and resolving errors or inconsistencies to actually analyze source data
  • Greater Productivity & Reduced Costs – Use of automated, repeatable processes to for metadata discovery, data design, data conversion, data mapping and code generation
  • Digital Transformation – Better understanding of what data exists and its potential value to improve digital experiences, enhance digital operations, drive digital innovation and build digital ecosystems
  • Enterprise Collaboration – The ability for IT and the wider business to find, trust and use data to effectively meet organizational objectives

To learn more about the information we’ve covered in today’s blog, please join us for our webinar with Dataversity on Feb. 18.

Data Governance Webinar

Categories
erwin Expert Blog

Data Governance Automation: What’s the Current State of Data Governance and Automation?

A new study into data governance automation indicates organizations are prioritising value-adding use cases, over efforts concerning regulatory compliance.

Continue reading “Data Governance Automation: What’s the Current State of Data Governance and Automation?”
Categories
erwin Expert Blog

5 Ways Data Modeling Is Critical to Data Governance

Enterprises are trying to manage data chaos. They might have 300 applications, with 50 different databases and a different schema for each one.

They also face increasing regulatory pressure because of global data regulations, such as the European Union’s General Data Protection Regulation (GDPR) and the new California Consumer Privacy Act (CCPA), that went into effect last week on Jan. 1.

Then there’s unstructured data with no contextual framework to govern data flows across the enterprise not to mention time-consuming manual data preparation and limited views of data lineage.

For decades, data modeling has been the optimal way to design and deploy new relational databases with high-quality data sources and support application development. It is a tried-and-true practice for lowering data management costs, reducing data-related risks, and improving the quality and agility of an organization’s overall data capability.

And the good news is that it just keeps getting better. Today’s data modeling is not your father’s data modeling software.

While it’s always been the best way to understand complex data sources and automate design standards and integrity rules, the role of data modeling continues to expand as the fulcrum of collaboration between data generators, stewards and consumers.

That’s because it’s the best way to visualize metadata, and metadata is now the heart of enterprise data management and data governance/ intelligence efforts.

So here’s why data modeling is so critical to data governance.

1. Uncovering the connections between disparate data elements: Visualize metadata and schema to mitigate complexity and increase data literacy and collaboration across a broad range of data stakeholders. Because data modeling reduces complexity, all members of the team can work around a data model to better understand and contribute to the project.

2. Capturing and sharing how the business describes and uses data: Create and integrate business and semantic metadata to augment and accelerate data intelligence and governance efforts. Data modeling captures how the business uses data and provides context to the data source.

3. Deploying higher quality data sources with the appropriate structural veracity: Automate and enforce data model design tasks to ensure data integrity. From regulatory compliance and business intelligence to target marketing, data modeling maintains an automated connection back to the source.

4. Building a more agile and governable data architecture: Create and implement common data design standards from the start. Data modeling standardizes design tasks to improve business alignment and simplify integration.

5. Governing the design and deployment of data across the enterprise: Manage the design and maintenance lifecycle for data sources. Data modeling provides visibility, management and full version control over the lifecycle for data design, definition and deployment.

Data Modeling Tool

erwin Data Modeler: Where the Magic Happens

erwin has just released a new version of erwin DM, the world’s No. 1 data modeling software for designing, deploying and understanding data sources to meet modern business needs. erwin DM 2020 is an essential source of metadata and a critical enabler of data governance and intelligence efforts.

The new version of erwin DM includes these features:

  • A modern, configurable workspace so users can customize the modeling canvas and optimize access to features and functionality that best support their workflows
  • Support for and model integration from major databases to work effectively across platforms and reuse work product, including native support for Amazon Redshift and updated support for the latest DB2 releases and certification for the latest MS SQL Server releases
  • Model exchange (import/export) to/from a wide variety of data management environments
  • Modeling task automation that saves modelers time, reduces errors and increases work product quality and speed, including a new scheduler to automate the offline reverse-engineering of databases into data models
  • New Quick Compare templates as part of the Complete Compare feature to compare and synchronize data models and sources
  • New ODBC query tool for creating and running custom model and metadata reports
  • Design transformations to customize and automate super-type/sub-type relationships between logical and physical models

erwin DM also integrates with the erwin Data Intelligence Suite (erwin DI) to automatically harvest the metadata in erwin data models for ingestion into the data catalog for better analytics, governance and overall data intelligence.

The role of data modeling in the modern data-driven business continues to expand with the benefits long-realized by database professionals and developers now experienced by a wider range of architects, business analysts and data administrators in a variety of data-centric initiatives.

Click here to test drive of the new erwin DM.

Categories
erwin Expert Blog

Top 7 Data Governance and Metadata Management Blog Posts of 2019

Data has been the driving force of the decade. Digital pioneers like Amazon, Netflix and Uber account for some of the most extreme market disruption their respective industries have faced.

But such success cannot be attributed soley to head-starts.

Many organizations have tried and failed to become truly “data-driven,” and many organizations will continue to do so.

The difference between success and failure is often a deeper understanding of the data behind an organization’s operational decisions.

With a deeper understanding of their data assets, organizations can realize more trustworthy and effective analysis.

Such understanding also equips organizations  to meet customer demands as well as deal more effectively with the regulatory landscape – which is evolving at a fast rate.

To help you prepare for 2020, we’ve compiled some of the most popular data governance and metadata management blog posts from the erwin Experts from this year.

Data Governance and Metadata Management Blog Posts

The Best Data Governance and Metadata Management Blog Posts of 2019

Data Governance Framework: Three Steps to Successful and Sustainable Implementation

A strong data governance framework is central to the success of any data-driven organization because it ensures this valuable asset is properly maintained, protected and maximized.

But despite this fact, enterprises often face push back when implementing a new data governance initiative or trying to improve an existing one:

Four Use Cases Proving the Benefits of Metadata-Driven Automation

Data scientists and other data professionals can spend up to 80 percent of their time bogged down trying to understand source data or address errors and inconsistencies.

That’s time needed and better used for data analysis.

In this metadata management blog, the erwin Experts assess four use cases that demonstrate exactly how metadata-driven automation increases productivity:

Data Mapping Tools: What Are the Key Differentiators

Data mapping tools help organizations discover important insights.

They provide context to what otherwise would be isolated units of meaningless data.

Now with the General Data Protection Regulation (GDPR) in effect, data mapping has become even more significant:

The Unified Data Platform – Connecting Everything That Matters

Businesses stand to gain a lot from unifying their data platforms.

Data-driven leaders dominate their respective markets and inspire other organizations across the board to use data to fuel their businesses.

It was even dubbed “the new oil at one point” but data is arguably far more valuable than that analogy suggests:

A Guide to CCPA Compliance and How the California Consumer Privacy Act Compares to GDPR

The California Consumer Privacy Act (CCPA) and GDPR share many of the same data privacy and security requirements.

While the CCPA has been signed into law, organizations have until Jan. 1, 2020, to enact its mandates. Luckily, many organizations have already laid the regulatory groundwork for it because of their efforts to comply with GDPR.

However, there are some key differences, which the erwin Experts explore in a Q&A format:

Business Architecture and Process Modeling for Digital Transformation

Digital transformation involves synthesizing an organization’s people, processes and technologies, so involving business architecture and process modeling is a best practice organizations can’t ignore.

The following post outlines how business architecture and process modeling work in tandem to facilitate efficient and successful digital transformation efforts:

Constructing a Digital Transformation Strategy: Putting the Data in Digital Transformation

Having a clearly defined digital transformation strategy is an essential best practice to ensure success.

But what makes a digital transformation strategy viable? Learn here:

Gartner Magic Quadrant Metadata Management

Categories
erwin Expert Blog

How Metadata Makes Data Meaningful

Metadata is an important part of data governance, and as a result, most nascent data governance programs are rife with project plans for assessing and documenting metadata. But in many scenarios, it seems that the underlying driver of metadata collection projects is that it’s just something you do for data governance.

So most early-stage data governance managers kick off a series of projects to profile data, make inferences about data element structure and format, and store the presumptive metadata in some metadata repository. But are these rampant and often uncontrolled projects to collect metadata properly motivated?

There is rarely a clear directive about how metadata is used. Therefore prior to launching metadata collection tasks, it is important to specifically direct how the knowledge embedded within the corporate metadata should be used.

Managing metadata should not be a sub-goal of data governance. Today, metadata is the heart of enterprise data management and governance/ intelligence efforts and should have a clear strategy – rather than just something you do.

metadata data governance

What Is Metadata?

Quite simply, metadata is data about data. It’s generated every time data is captured at a source, accessed by users, moved through an organization, integrated or augmented with other data from other sources, profiled, cleansed and analyzed. Metadata is valuable because it provides information about the attributes of data elements that can be used to guide strategic and operational decision-making. It answers these important questions:

  • What data do we have?
  • Where did it come from?
  • Where is it now?
  • How has it changed since it was originally created or captured?
  • Who is authorized to use it and how?
  • Is it sensitive or are there any risks associated with it?

The Role of Metadata in Data Governance

Organizations don’t know what they don’t know, and this problem is only getting worse. As data continues to proliferate, so does the need for data and analytics initiatives to make sense of it all. Here are some benefits of metadata management for data governance use cases:

  • Better Data Quality: Data issues and inconsistencies within integrated data sources or targets are identified in real time to improve overall data quality by increasing time to insights and/or repair.
  • Quicker Project Delivery: Accelerate Big Data deployments, Data Vaults, data warehouse modernization, cloud migration, etc., by up to 70 percent.
  • Faster Speed to Insights: Reverse the current 80/20 rule that keeps high-paid knowledge workers too busy finding, understanding and resolving errors or inconsistencies to actually analyze source data.
  • Greater Productivity & Reduced Costs: Being able to rely on automated and repeatable metadata management processes results in greater productivity. Some erwin customers report productivity gains of 85+% for coding, 70+% for metadata discovery, up to 50% for data design, up to 70% for data conversion, and up to 80% for data mapping.
  • Regulatory Compliance: Regulations such as GDPR, HIPAA, PII, BCBS and CCPA have data privacy and security mandates, so sensitive data needs to be tagged, its lineage documented, and its flows depicted for traceability.
  • Digital Transformation: Knowing what data exists and its value potential promotes digital transformation by improving digital experiences, enhancing digital operations, driving digital innovation and building digital ecosystems.
  • Enterprise Collaboration: With the business driving alignment between data governance and strategic enterprise goals and IT handling the technical mechanics of data management, the door opens to finding, trusting and using data to effectively meet organizational objectives.

Giving Metadata Meaning

So how do you give metadata meaning? While this sounds like a deep philosophical question, the reality is the right tools can make all the difference.

erwin Data Intelligence (erwin DI) combines data management and data governance processes in an automated flow.

It’s unique in its ability to automatically harvest, transform and feed metadata from a wide array of data sources, operational processes, business applications and data models into a central data catalog and then make it accessible and understandable within the context of role-based views.

erwin DI sits on a common metamodel that is open, extensible and comes with a full set of APIs. A comprehensive list of erwin-owned standard data connectors are included for automated harvesting, refreshing and version-controlled metadata management. Optional erwin Smart Data Connectors reverse-engineer ETL code of all types and connect bi-directionally with reporting and other ecosystem tools. These connectors offer the fastest and most accurate path to data lineage, impact analysis and other detailed graphical relationships.

Additionally, erwin DI is part of the larger erwin EDGE platform that integrates data modeling, enterprise architecture, business process modeling, data cataloging and data literacy. We know our customers need an active metadata-driven approach to:

  • Understand their business, technology and data architectures and the relationships between them
  • Create an automate a curated enterprise data catalog, complete with physical assets, data models, data movement, data quality and on-demand lineage
  • Activate their metadata to drive agile and well-governed data preparation with integrated business glossaries and data dictionaries that provide business context for stakeholder data literacy

erwin was named a Leader in Gartner’s “2019 Magic Quadrant for Metadata Management Solutions.”

Click here to get a free copy of the report.

Click here to request a demo of erwin DI.

Gartner Magic Quadrant Metadata Management

 

Categories
erwin Expert Blog

Data Governance 2.0: The CIO’s Guide to Collaborative Data Governance

In the data-driven era, CIO’s need a solid understanding of data governance 2.0 …

Data governance (DG) is no longer about just compliance or relegated to the confines of IT. Today, data governance needs to be a ubiquitous part of your organization’s culture.

As the CIO, your stakeholders include both IT and business users in collaborative relationships, which means data governance is not only your business, it’s everyone’s business.

The ability to quickly collect vast amounts of data, analyze it and then use what you’ve learned to help foster better decision-making is the dream of business executives. But that vision is more difficult to execute than it might first appear.

While many organizations are aware of the need to implement a formal data governance initiative, many have faced obstacles getting started.

A lack of resources, difficulties in proving the business case, and challenges in getting senior management to see the importance of such an effort rank among the biggest obstacles facing DG initiatives, according to a recent survey by UBM.

Common Data Governance Challenges - Data Governance 2.0

Despite such hurdles, organizations are committed to trying to get data governance right. The same UBM study found that 98% of respondents considered data governance either important, or critically important to their organization.

And it’s unsurprising too. Considering that the unprecedented levels of digital transformation, with rapidly changing and evolving technology, mean data governance is not just an option, but rather a necessity.

Recognizing this, the IDC DX Awards recently resurfaced to give proper recognition and distinction to organizations who have successfully digitized their systems and business processes.

Creating a Culture of Data Governance

The right data of the right quality, regardless of where it is stored or what format it is stored in, must be available for use only by the right people for the right purpose. This is the promise of a formal data governance practice.

However, to create a culture of data governance requires buy-in from the top down, and the appropriate systems, tools and frameworks to ensure its continued success.

This take on data governance is often dubbed as Data Governance 2.0.

At erwin, we’ve identified what we believe to be the five pillars of data governance readiness:

  1. Initiative Sponsorship: Without executive sponsorship, you’ll have difficulty obtaining the funding, resources, support and alignment necessary for successful DG.
  2. Organizational Support: DG needs to be integrated into the data stewardship teams and wider culture. It also requires funding.
  3. Team Resources: Most successful organizations have established a formal data management group at the enterprise level. As a foundational component of enterprise data management, DG would reside in such a group.
  4. Enterprise Data Management Methodology: DG is foundational to enterprise data management. Without the other essential components (e.g., metadata management, enterprise data architecture, data quality management), DG will be a struggle.
  5. Delivery Capability: Successful and sustainable DG initiatives are supported by specialized tools, which are scoped as part of the DG initiative’s technical requirements.

Data Security

Data is becoming increasingly difficult to manage, control and secure as evidenced by the uptick in data breaches in almost every industry.

Therefore companies must work to secure intellectual property (IPs), client information and so much more.

So CIOs have to come up with appropriate plans to restrict certain people from accessing this information and allow only a small, relevant circle to view it when necessary.

However, this job isn’t as easy as you think it is. Organizations must walk the line between ease of access/data discoverability and security.

It’s the CIO’s responsibility to keep the balance, and data governance tools with role-based access can help maintain that balance.

Data Storage

The amount of data modern organizations have to manage means CIOs have to rethink data storage, as well as security.

This includes considerations as to what data should be stored and where, as well as understanding what data the organization – and the stakeholders within it – is responsible for.

This knowledge will enable better analysis, and the data used for such analysis more easily accessed when required and by approved parties. This is especially crucial for compliance with government regulations like the General Data Protection Regulation (GDPR), as well as other data regulations.

Defining the Right Audience

It’s a CIO’s responsibility to oversee the organization’s data governance systems. Of course, this means the implementation and upkeep of such systems, but it also includes creating the policies that will inform the data governance program itself.

Nowadays, lots of employees think they need access to all of an organization’s data to help them make better decisions for the company.

However, this can possibly expose company data to numerous threats and cyber attacks as well as intellectual property infringement.

So data governance that ensures only the right audience can access specific company information can come in handy, especially during a company’s brainstorming seasons, new products and services releases, and so much more.

Data governance is to be tailored by CIOs to meet their organizations’ specific needs (and wants). This is to ensure an efficient and effective way of utilizing data while also enabling employees to make better and wiser business decisions.

The Right Tools Help Solve the Enterprise Data Dilemma

What data do we have, where is it and what does it mean? This is the data dilemma that plagues most organizations.

The right tools can make or break your data governance initiatives. They encompass a number of different technologies, including data cataloging, data literacy, business process modeling, enterprise architecture and data modeling.

Each of these tools separately contribute to better data governance, however, increasingly, organizations are realizing the benefits of interconnectivity between them. This interconnectivity can be achieved through centralizing data-driven projects around metadata.

This means data professionals and their work benefits from a single source of truth, making analysis faster, more trustworthy and far easier to collaborate on.

With the erwin EDGE, an “enterprise data governance experience” is created to underpin Data Governance 2.0.

It unifies data and business architectures so all IT and business stakeholders can access relevant data in the context of their roles, supporting a culture committed to using data as a mission-critical asset and orchestrating the key mechanisms required to discover, fully understand, actively govern and effectively socialize and align data to the business.

You can learn more about data governance by reading our whitepaper: Examining the Data Trinity: Governance, Security and Privacy.

Examining the Data Trinity - Governance, Security and Privacy

Categories
erwin Expert Blog

Data Governance Tools: What Are They? Are They Optional?

Data governance tools used to occupy a niche in an organization’s tech stack, but those days are gone.

The rise of data-driven business and the complexities that come with it ushered in a soft mandate for data governance and data governance tools.

Data governance refers to the strategic and ongoing efforts by an organization to ensure that data is discoverable and its quality is good. It is also used to make data more easily understood and secure.

The technology that makes end-to-end data governance possible includes data cataloging, data literacy, business process modeling, enterprise architecture and data modeling.

Research indicates business leaders recognize the need for data governance tools. In fact, 98 percent of participants in erwin’s “2018 State of Data Governance Report” consider data governance either “important” or “critically important” to their organizations.

Over the years, organizations have faced a number of challenges pointing to the need for data governance tools, including:

  • the increasing volume, variety and velocity of data (the “three Vs”)
  • the potential revenue that well-governed data can drive
  • the employees and systems responsible for data, diversifying (or, data democratization)

Additionally, the unprecedented industry disruption of such data-driven companies as Airbnb, Netflix and Uber demonstrates the benefits of well-governed data.

Such examples were persuasive and pervasive, leading to the rise of data governance adoption.

Data Governance Tools

Data Governance Tools for Regulatory Compliance

In recent years, hard mandates for data governance also have increased.

In the United States, the Health Insurance Portability and Accountability Act (HIPAA) requires organizations in the healthcare space to protect the privacy and security of certain health information.

Other highly regulated industries, like financial services, also face strict data privacy mandates, including those from the Basel Committee on Banking Supervision (BCBS) and the Financial Industry Regulatory Authority (FINRA).

Now new, industry-agnostic regulations such as the General Data Protection Regulation (GDPR) and the forthcoming California Consumer Protection Act (CCPA) leave little room for data-driven businesses to operate without data governance.

So it’s not surprising that the “2018 State of Data Governance Report” revealed regulatory compliance to be the leading driver in data governance adoption.

Data Governance Tools and Data Ethics

Customer trust/satisfaction is also a key driver for data governance. Given the landscape of modern business – in which data breaches make big and lasting news – this this is also not a surprise.

Fines levied against both Facebook and Google earlier this year are a reminder that regulators are serious, and the fines can be serious also.

But even without penalties from regulatory bodies, the cost of poor data governance is still huge.

IBM’s annual “Cost of a Data Breach” report found that the biggest cost of a data breach to an organization is a loss of business. It also found that, on average, a data breach can cost a business a staggering $3.9 million.

And perhaps more worrisome is that those figures are increasing. Costs have risen by 12 percent during the last five years.

It’s not just breaches. The prominence of data-related stories in the news is leading more and more people to be skeptical of how their personal data is handled.

Because of this, organizations with good data governance can make data ethics part of their brand. Some organizations are even beginning to hire “data ethicists” – employees dedicated to overseeing data ethics.

As the use cases for data-driven tech, such as AI, grow, you can expect the calls for ethical data practices to grow too.

Data Governance Tools Aren’t Optional

Considering the revenue potential, regulatory mandates and data-conscious consumers, a comprehensive data governance practice supported by robust data governance tools should no longer be seen as optional.

But what’s the best way to set up and sustain a data governance program?

Data Governance 1.0 was an isolated domain, managed by IT so it largely disconnected from the wider enterprise.

As data and the responsibilities for discovering, understanding and using it for strategic decision-making have become more democratized, a new approach for IT and business collaboration has taken hold.

Data Governance 2.0, as defined by Forrester, is as “an agile approach to data governance focused on just enough controls for managing risk, which enables broader and more insightful use of data required by the evolving needs of an expanding business ecosystem.”

At erwin, we believe in this approach and have incorporated it into what we refer to as the erwin Enterprise Data Governance Experience – or the erwin EDGE, for short.

The erwin EDGE empowers organizations with visibility and control over their data, both at rest and in motion.

It enables enterprises to plan and document how they will discover and understand their data within context, track its physical existence and lineage, and maximize its security, quality and value. It also helps organizations operationalize these steps.

Therefore the speed and quality of the data pipeline increases. Of course, metadata management is at the heart of any data governance initiative.

Gartner Magic Quadrant Metadata Management

Categories
erwin Expert Blog

Metadata Management, Data Governance and Automation

Can the 80/20 Rule Be Reversed?

erwin released its State of Data Governance Report in February 2018, just a few months before the General Data Protection Regulation (GDPR) took effect.

This research showed that the majority of responding organizations weren’t actually prepared for GDPR, nor did they have the understanding, executive support and budget for data governance – although they recognized the importance of it.

Of course, data governance has evolved with astonishing speed, both in response to data privacy and security regulations and because organizations see the potential for using it to accomplish other organizational objectives.

But many of the world’s top brands still seem to be challenged in implementing and sustaining effective data governance programs (hello, Facebook).

We wonder why.

Too Much Time, Too Few Insights

According to IDC’s “Data Intelligence in Context” Technology Spotlight sponsored by erwin, “professionals who work with data spend 80 percent of their time looking for and preparing data and only 20 percent of their time on analytics.”

Specifically, 80 percent of data professionals’ time is spent on data discovery, preparation and protection, and only 20 percent on analysis leading to insights.

In most companies, an incredible amount of data flows from multiple sources in a variety of formats and is constantly being moved and federated across a changing system landscape.

Often these enterprises are heavily regulated, so they need a well-defined data integration model that will help avoid data discrepancies and remove barriers to enterprise business intelligence and other meaningful use.

IT teams need the ability to smoothly generate hundreds of mappings and ETL jobs. They need their data mappings to fall under governance and audit controls, with instant access to dynamic impact analysis and data lineage.

But most organizations, especially those competing in the digital economy, don’t have enough time or money for data management using manual processes. Outsourcing is also expensive, with inevitable delays because these vendors are dependent on manual processes too.

The Role of Data Automation

Data governance maturity includes the ability to rely on automated and repeatable processes.

For example, automatically importing mappings from developers’ Excel sheets, flat files, Access and ETL tools into a comprehensive mappings inventory, complete with automatically generated and meaningful documentation of the mappings, is a powerful way to support governance while providing real insight into data movement — for data lineage and impact analysis — without interrupting system developers’ normal work methods.

GDPR compliance, for instance, requires a business to discover source-to-target mappings with all accompanying transactions, such as what business rules in the repository are applied to it, to comply with audits.

When data movement has been tracked and version-controlled, it’s possible to conduct data archeology — that is, reverse-engineering code from existing XML within the ETL layer — to uncover what has happened in the past and incorporating it into a mapping manager for fast and accurate recovery.

With automation, data professionals can meet the above needs at a fraction of the cost of the traditional, manual way. To summarize, just some of the benefits of data automation are:

• Centralized and standardized code management with all automation templates stored in a governed repository
• Better quality code and minimized rework
• Business-driven data movement and transformation specifications
• Superior data movement job designs based on best practices
• Greater agility and faster time-to-value in data preparation, deployment and governance
• Cross-platform support of scripting languages and data movement technologies

One global pharmaceutical giant reduced costs by 70 percent and generated 95 percent of production code with “zero touch.” With automation, the company improved the time to business value and significantly reduced the costly re-work associated with error-prone manual processes.

Gartner Magic Quadrant Metadata Management

Help Us Help You by Taking a Brief Survey

With 2020 just around the corner and another data regulation about to take effect, the California Consumer Privacy Act (CCPA), we’re working with Dataversity on another research project.

And this time, you guessed it – we’re focusing on data automation and how it could impact metadata management and data governance.

We would appreciate your input and will release the findings in January 2020.

Click here to take the brief survey

Categories
erwin Expert Blog

Data Governance Makes Data Security Less Scary

Happy Halloween!

Do you know where your data is? What data you have? Who has had access to it?

These can be frightening questions for an organization to answer.

Add to the mix the potential for a data breach followed by non-compliance, reputational damage and financial penalties and a real horror story could unfold.

In fact, we’ve seen some frightening ones play out already:

  1. Google’s record GDPR fine – France’s data privacy enforcement agency hit the tech giant with a $57 million penalty in early 2019 – more than 80 times the steepest fine the U.K.’s Information Commissioner’s Office had levied against both Facebook and Equifax for their data breaches.
  2. In July 2019, British Airways received the biggest GDPR fine to date ($229 million) because the data of more than 500,000 customers was compromised.
  3. Marriot International was fined $123 million, or 1.5 percent of its global annual revenue, because 330 million hotel guests were affected by a breach in 2018.

Now, as Cybersecurity Awareness Month comes to a close – and ghosts and goblins roam the streets – we thought it a good time to resurrect some guidance on how data governance can make data security less scary.

We don’t want you to be caught off guard when it comes to protecting sensitive data and staying compliant with data regulations.

Data Governance Makes Data Security Less Scary

Don’t Scream; You Can Protect Your Sensitive Data

It’s easier to protect sensitive data when you know what it is, where it’s stored and how it needs to be governed.

Data security incidents may be the result of not having a true data governance foundation that makes it possible to understand the context of data – what assets exist and where, the relationship between them and enterprise systems and processes, and how and by what authorized parties data is used.

That knowledge is critical to supporting efforts to keep relevant data secure and private.

Without data governance, organizations don’t have visibility of the full data landscape – linkages, processes, people and so on – to propel more context-sensitive security architectures that can better assure expectations around user and corporate data privacy. In sum, they lack the ability to connect the dots across governance, security and privacy – and to act accordingly.

This addresses these fundamental questions:

  1. What private data do we store and how is it used?
  2. Who has access and permissions to the data?
  3. What data do we have and where is it?

Where Are the Skeletons?

Data is a critical asset used to operate, manage and grow a business. While sometimes at rest in databases, data lakes and data warehouses; a large percentage is federated and integrated across the enterprise, introducing governance, manageability and risk issues that must be managed.

Knowing where sensitive data is located and properly governing it with policy rules, impact analysis and lineage views is critical for risk management, data audits and regulatory compliance.

However, when key data isn’t discovered, harvested, cataloged, defined and standardized as part of integration processes, audits may be flawed and therefore your organization is at risk.

Sensitive data – at rest or in motion – that exists in various forms across multiple systems must be automatically tagged, its lineage automatically documented, and its flows depicted so that it is easily found and its usage across workflows easily traced.

Thankfully, tools are available to help automate the scanning, detection and tagging of sensitive data by:

  • Monitoring and controlling sensitive data: Better visibility and control across the enterprise to identify data security threats and reduce associated risks
  • Enriching business data elements for sensitive data discovery: Comprehensively defining business data element for PII, PHI and PCI across database systems, cloud and Big Data stores to easily identify sensitive data based on a set of algorithms and data patterns
  • Providing metadata and value-based analysis: Discovery and classification of sensitive data based on metadata and data value patterns and algorithms. Organizations can define business data elements and rules to identify and locate sensitive data including PII, PHI, PCI and other sensitive information.

No Hocus Pocus

Truly understanding an organization’s data, including its value and quality, requires a harmonized approach embedded in business processes and enterprise architecture.

Such an integrated enterprise data governance experience helps organizations understand what data they have, where it is, where it came from, its value, its quality and how it’s used and accessed by people and applications.

An ounce of prevention is worth a pound of cure  – from the painstaking process of identifying what happened and why to notifying customers their data and thus their trust in your organization has been compromised.

A well-formed security architecture that is driven by and aligned by data intelligence is your best defense. However, if there is nefarious intent, a hacker will find a way. So being prepared means you can minimize your risk exposure and the damage to your reputation.

Multiple components must be considered to effectively support a data governance, security and privacy trinity. They are:

  1. Data models
  2. Enterprise architecture
  3. Business process models

Creating policies for data handling and accountability and driving culture change so people understand how to properly work with data are two important components of a data governance initiative, as is the technology for proactively managing data assets.

Without the ability to harvest metadata schemas and business terms; analyze data attributes and relationships; impose structure on definitions; and view all data in one place according to each user’s role within the enterprise, businesses will be hard pressed to stay in step with governance standards and best practices around security and privacy.

As a consequence, the private information held within organizations will continue to be at risk.

Organizations suffering data breaches will be deprived of the benefits they had hoped to realize from the money spent on security technologies and the time invested in developing data privacy classifications.

They also may face heavy fines and other financial, not to mention PR, penalties.

Gartner Magic Quadrant Metadata Management

Categories
erwin Expert Blog

Very Meta … Unlocking Data’s Potential with Metadata Management Solutions

Untapped data, if mined, represents tremendous potential for your organization. While there has been a lot of talk about big data over the years, the real hero in unlocking the value of enterprise data is metadata, or the data about the data.

However, most organizations don’t use all the data they’re flooded with to reach deeper conclusions about how to drive revenue, achieve regulatory compliance or make other strategic decisions. They don’t know exactly what data they have or even where some of it is.

Quite honestly, knowing what data you have and where it lives is complicated. And to truly understand it, you need to be able to create and sustain an enterprise-wide view of and easy access to underlying metadata.

This isn’t an easy task. Organizations are dealing with numerous data types and data sources that were never designed to work together and data infrastructures that have been cobbled together over time with disparate technologies, poor documentation and with little thought for downstream integration.

As a result, the applications and initiatives that depend on a solid data infrastructure may be compromised, leading to faulty analysis and insights.

Metadata Is the Heart of Data Intelligence

A recent IDC Innovators: Data Intelligence Report says that getting answers to such questions as “where is my data, where has it been, and who has access to it” requires harnessing the power of metadata.

Metadata is generated every time data is captured at a source, accessed by users, moves through an organization, and then is profiled, cleansed, aggregated, augmented and used for analytics to guide operational or strategic decision-making.

In fact, data professionals spend 80 percent of their time looking for and preparing data and only 20 percent of their time on analysis, according to IDC.

To flip this 80/20 rule, they need an automated metadata management solution for:

• Discovering data – Identify and interrogate metadata from various data management silos.
• Harvesting data – Automate the collection of metadata from various data management silos and consolidate it into a single source.
• Structuring and deploying data sources – Connect physical metadata to specific data models, business terms, definitions and reusable design standards.
• Analyzing metadata – Understand how data relates to the business and what attributes it has.
• Mapping data flows – Identify where to integrate data and track how it moves and transforms.
• Governing data – Develop a governance model to manage standards, policies and best practices and associate them with physical assets.
• Socializing data – Empower stakeholders to see data in one place and in the context of their roles.

Addressing the Complexities of Metadata Management

The complexities of metadata management can be addressed with a strong data management strategy coupled with metadata management software to enable the data quality the business requires.

This encompasses data cataloging (integration of data sets from various sources), mapping, versioning, business rules and glossary maintenance, and metadata management (associations and lineage).

erwin has developed the only data intelligence platform that provides organizations with a complete and contextual depiction of the entire metadata landscape.

It is the only solution that can automatically harvest, transform and feed metadata from operational processes, business applications and data models into a central data catalog and then made accessible and understandable within the context of role-based views.

erwin’s ability to integrate and continuously refresh metadata from an organization’s entire data ecosystem, including business processes, enterprise architecture and data architecture, forms the foundation for enterprise-wide data discovery, literacy, governance and strategic usage.

Organizations then can take a data-driven approach to business transformation, speed to insights, and risk management.
With erwin, organizations can:

1. Deliver a trusted metadata foundation through automated metadata harvesting and cataloging
2. Standardize data management processes through a metadata-driven approach
3. Centralize data-driven projects around centralized metadata for planning and visibility
4. Accelerate data preparation and delivery through metadata-driven automation
5. Master data management platforms through metadata abstraction
6. Accelerate data literacy through contextual metadata enrichment and integration
7. Leverage a metadata repository to derive lineage, impact analysis and enable audit/oversight ability

With erwin Data Intelligence as part of the erwin EDGE platform, you know what data you have, where it is, where it’s been and how it transformed along the way, plus you can understand sensitivities and risks.

With an automated, real-time, high-quality data pipeline, enterprise stakeholders can base strategic decisions on a full inventory of reliable information.

Many of our customers are hard at work addressing metadata management challenges, and that’s why erwin was Named a Leader in Gartner’s “2019 Magic Quadrant for Metadata Management Solutions.”

Gartner Magic Quadrant Metadata Management