data governance research Archives

How to Automate Data Management

Here are our eight recommendations for how to transition from manual to automated data management:

1) Put Data Quality First:
Automating and matching business terms with data assets and documenting lineage down to the column level are critical to good decision making.

2) Don’t Ignore Data Lineage Complexity:
It’s a risky endeavor to support data lineage using a manual approach, and businesses that attempt it that way will find that it’s not sustainable given data’s constant movement from one place to another via multiple routes- and doing it correctly down to the column level.

3) Automate Code Generation:
Mapping data elements to their sources within a single repository to determine data lineage and harmonize data integration across platforms reduces the need for specialized, technical resources with knowledge of ETL and database procedural code.

4) Use Integrated Impact Analysis to Automate Data Due Diligence:
This helps IT deliver operational intelligence to the business. Business users benefit from automating impact analysis to better examine value and prioritize individual data sets.

5) Catalog Data:
Catalog data using a solution with a broad set of metadata connectors so all data sources can be leveraged.

6) Stress Data Literacy Across the Organization:
There’s a clear connection to data literacy because of its foundation in business glossaries and socializing data so that all stakeholders can view and understand it within the context of their roles.

7) Make Automation Standard Practice:
Too many companies are still living in a world where data governance is a high-level mandate and not a practically implemented one.

8) Create a Solid Data Governance Strategy:
Craft your data governance strategy before making any investments. Gather multiple stakeholders—both business and IT—with multiple viewpoints to discover where their needs mesh and where they diverge and what represents the greatest pain points to the business.

The Benefits of Data Management Automation

With data management automation, data professionals can meet their organization’s data needs at a fraction of the cost of the traditional, manual way.

Some of the benefits of data management automation are:

Centralized and standardized code management with all automation templates stored in a governed repository

Better quality code and minimized rework

Business-driven data movement and transformation specifications

Superior data movement job designs based on best practices

Greater agility and faster time-to-value in data preparation, deployment and governance

Cross-platform support of scripting languages and data movement technologies

For a deeper dive on how to automate data management and to view the full research, download a copy of erwin’s 2020 State of Data Governance and Automation report.

Defining Data Governance: What Is Data Governance?

Data governance (DG) is one of the fastest growing disciplines, yet when it comes to defining data governance many organizations struggle.

Dataversity says DG is “the practices and processes which help to ensure the formal management of data assets within an organization.” These practices and processes can vary, depending on an organization’s needs. Therefore, when defining data governance for your organization, it’s important to consider the factors driving its adoption.

The General Data Protection Regulation (GDPR) has contributed significantly to data governance’s escalating prominence. In fact, erwin’s 2018 State of Data Governance Report found that 60% of organizations consider regulatory compliance to be their biggest driver of data governance.

Other significant drivers include improving customer trust/satisfaction and encouraging better decision-making, but they trail behind regulatory compliance at 49% and 45% respectively. Reputation management (30%), analytics (27%) and Big Data (21%) also are factors.

But data governance’s adoption is of little benefit without understanding how DG should be applied within these contexts. This is arguably one of the issues that’s held data governance back in the past.

With no set definition, and the historical practice of isolating data governance within IT, organizations often have had different ideas of what data governance is, even between departments. With this inter-departmental disconnect, it’s not hard to imagine why data governance has historically left a lot to be desired.

However, with the mandate for DG within GDPR, organizations must work on defining data governance organization-wide to manage its successful implementation, or face GDPR’s penalties.

Defining Data Governance: Desired Outcomes

A great place to start when defining an organization-wide DG initiative is to consider the desired business outcomes. This approach ensures that all parties involved have a common goal.

Past examples of Data Governance 1.0 were mainly concerned with cataloging data to support search and discovery. The nature of this approach, coupled with the fact that DG initiatives were typically siloed within IT departments without input from the wider business, meant the practice often struggled to add value.

Without input from the wider business, the data cataloging process suffered from a lack of context. By neglecting to include the organization’s primary data citizens – those that manage and or leverage data on a day-to-day basis for analysis and insight – organizational data was often plagued by duplications, inconsistencies and poor quality.

The nature of modern data-driven business means that such data citizens are spread throughout the organization. Furthermore, many of the key data citizens (think value-adding approaches to data use such as data-driven marketing) aren’t actively involved with IT departments.

Because of this, Data Governance 1.0 initiatives fizzled out at discouraging frequencies.

This is, of course, problematic for organizations that identify regulatory compliance as a driver of data governance. Considering the nature of data-driven business – with new data being constantly captured, stored and leveraged – meeting compliance standards can’t be viewed as a one-time fix, so data governance can’t be de-prioritized and left to fizzle out.

Even those businesses that manage to maintain the level of input data governance needs on an indefinite basis, will find the Data Governance 1.0 approach wanting. In terms of regulatory compliance, the lack of context associated with data governance 1.0, and the inaccuracies it leads to mean that potentially serious data governance issues could go unfounded and result in repercussions for non-compliance.

We recommend organizations look beyond just data cataloging and compliance as desired outcomes when implementing DG. In the data-driven business landscape, data governance finds its true potential as a value-added initiative.

Organizations that identify the desired business outcome of data governance as a value-added initiative should also consider data governance 1.0’s shortcomings and any organizations that hasn’t identified value-adding as a business outcome, should ask themselves, “why?”

Many of the biggest market disruptors of the 21^st Century have been digital savvy start-ups with robust data strategies – think Airbnb, Amazon and Netflix. Without high data governance standards, such companies would not have the level of trust in their data to confidently action such digital-first strategies, making them difficult to manage.

Therefore, in the data-driven business era, organizations should consider a Data Governance 2.0 strategy, with DG becoming an organization-wide, strategic initiative that de-silos the practice from the confines of IT.

This collaborative take on data governance intrinsically involves data’s biggest beneficiaries and users in the governance process, meaning functions like data cataloging benefit from greater context, accuracy and consistency.

It also means that organizations can have greater trust in their data and be more assured of meeting the standards set for regulatory compliance. It means that organizations can better respond to customer needs through more accurate methods of profiling and analysis, improving rates of satisfaction. And it means that organizations are less likely to suffer data breaches and their associated damages.

Defining Data Governance: The Enterprise Data Governance Experience (EDGE)

The EDGE is the erwin approach to Data Governance 2.0, empowering an organization to:

Manage any data, anywhere (Any²)
Instil a culture of collaboration and organizational empowerment
Introduce an integrated ecosystem for data management that draws from one central repository and ensures data (including real-time changes) is consistent throughout the organization
Have visibility across domains by breaking down silos between business and IT and introducing a common data vocabulary
Have regulatory peace of mind through mitigation of a wide range of risks, from GDPR to cybersecurity.

To learn more about implementing data governance, click here.