A new study into data governance automation indicates organizations are prioritising value-adding use cases, over efforts concerning regulatory compliance.
I’m excited to share the results of our new study with Dataversity that examines how data governance attitudes and practices continue to evolve.
The 2020 State of Data Governance and Automation (DGA) report is a follow-up to an initial survey we commissioned two years ago to explore data governance ahead of the European Union’s General Data Protection Regulation (GDPR) going into effect.
Not surprisingly, the respondents that shaped the 2018 report ranked regulatory compliance as the No. 1 reason to implement data governance.
However, the latest group of survey participants say better decision-making is their primary driver (62 percent), with analytics secondary (51 percent), and regulatory compliance coming in third (48 percent).
Digital transformation and data standards/uniformity round out the top five data governance drivers, with 37 and 36 percent, respectively.
I’m encouraged by these results as it tells us that enterprises are really beginning to embrace the power of data to shape their organizations.
The results of our new research show that organizations are still trying to master data governance, including adjusting their strategies to address changing priorities and overcoming challenges related to data discovery, preparation, quality and traceability.
However, more than 50 percent say they have deployed metadata management, data analytics, and data quality solutions. And close to 50 percent have deployed data catalogs and business glossaries.
But having piecemeal data governance elements in place doesn’t constitute a strategy or a sustainable program. And as we suspected, organizations seem to be procrastinating in automating their processes and therefore aren’t positioned to achieve data governance and intelligence at speed or scale. Most have only data governance operations.
Without a comprehensive and automated data governance framework, enterprises put themselves at high risk of conducting business based on poor metadata management and data intelligence processes, introducing unnecessary slowdowns and inaccuracies in their analytics.
Organizations also are experiencing multiple bottlenecks in their data value chains, including documenting complete data lineage, understanding the quality of source data, and finding, identifying and harvesting data assets and curating assets with business context. All of these factors have an impact on a well-defined data integration model.
Stop Wasting Your Time
Organizations still depend too much on manual data management.
The data from our latest survey suggests that companies are still grappling with the challenges of data governance, challenges that will only get worse as companies collect increasing amounts of data.
Unless data is well-governed, downstream data analysts and data scientists will not be able to generate significant value from it.
Availability, quality, consistency, usability and reduced latency are all requirements at the heart of successful data governance, as well as the provisioning of a strategic data pipeline.
These are also the benefits that can be realized through automation, whether it’s rules-based or steeped in artificial intelligence and machine learning.
Data governance maturity includes the ability to rely on automated and repeatable processes.
The Benefits of Data Governance Automation
One powerful way to support data governance while providing real insight into data movement is to automatically import mappings from developers’ Excel sheets, flat files, Access and ETL tools into a comprehensive mappings inventory, complete with automatically generated and meaningful documentation of the mappings.
Such automation provides data lineage and impact analysis — without interrupting system developers’ normal work methods.
Another example involves GDPR compliance, which requires a business to discover source-to-target mappings with all accompanying transactions, such as what business rules in the repository are applied to it, to comply with audits.
When data movement has been tracked and version-controlled, it’s possible to conduct data archeology — that is, reverse-engineering code from existing XML within the ETL layer — to uncover what has happened in the past and incorporating it into a mapping manager for fast and accurate recovery.
With automation, data professionals can meet the above needs at a fraction of the cost of the traditional, manual way. To summarize, just some of the benefits of data operations automation are:
- Centralized and standardized code management with all automation templates stored in a governed repository
- Better quality code and minimized rework
- Business-driven data movement and transformation specifications
- Superior data movement job designs based on best practices
- Greater agility and faster time-to-value in data preparation, deployment and governance
- Cross-platform support of scripting languages and data movement technologies
Click here to read the full 2020 State of Data Governance and Automation report, which includes our recommendations on how to implement a data governance program that harmonizes data management and data governance processes in an automated flow.
