Categories
erwin Expert Blog

The Unified Data Platform – Connecting Everything That Matters

Businesses stand to gain a lot from a unified data platform.

This decade has seen data-driven leaders dominate their respective markets and inspire other organizations across the board to use data to fuel their businesses, leveraging this strategic asset to create more value below the surface. It’s even been dubbed “the new oil,” but data is arguably more valuable than the analogy suggests.

Data governance (DG) is a key component of the data value chain because it connects people, processes and technology as they relate to the creation and use of data. It equips organizations to better deal with  increasing data volumes, the variety of data sources, and the speed in which data is processed.

But for an organization to realize and maximize its true data-driven potential, a unified data platform is required. Only then can all data assets be discovered, understood, governed and socialized to produce the desired business outcomes while also reducing data-related risks.

Benefits of a Unified Data Platform

Data governance can’t succeed in a bubble; it has to be connected to the rest of the enterprise. Whether strategic, such as risk and compliance management, or operational, like a centralized help desk, your data governance framework should span and support the entire enterprise and its objectives, which it can’t do from a silo.

Let’s look at some of the benefits of a unified data platform with data governance as the key connection point.

Understand current and future state architecture with business-focused outcomes:

A unified data platform with a single metadata repository connects data governance to the roles, goals strategies and KPIs of the enterprise. Through integrated enterprise architecture modeling, organizations can capture, analyze and incorporate the structure and priorities of the enterprise and related initiatives.

This capability allows you to plan, align, deploy and communicate a high-impact data governance framework and roadmap that sets manageable expectations and measures success with metrics important to the business.

Document capabilities and processes and understand critical paths:

A unified data platform connects data governance to what you do as a business and the details of how you do it. It enables organizations to document and integrate their business capabilities and operational processes with the critical data that serves them.

It also provides visibility and control by identifying the critical paths that will have the greatest impacts on the business.

Realize the value of your organization’s data:

A unified data platform connects data governance to specific business use cases. The value of data is realized by combining different elements to answer a business question or meet a specific requirement. Conceptual and logical schemas and models provide a much richer understanding of how data is related and combined to drive business value.

2020 Data Governance and Automation Report

Harmonize data governance and data management to drive high-quality deliverables:

A unified data platform connects data governance to the orchestration and preparation of data to drive the business, governing data throughout the entire lifecycle – from creation to consumption.

Governing the data management processes that make data available is of equal importance. By harmonizing the data governance and data management lifecycles, organizations can drive high-quality deliverables that are governed from day one.

Promote a business glossary for unanimous understanding of data terminology:

A unified data platform connects data governance to the language of the business when discussing and describing data. Understanding the terminology and semantic meaning of data from a business perspective is imperative, but most business consumers of data don’t have technical backgrounds.

A business glossary promotes data fluency across the organization and vital collaboration between different stakeholders within the data value chain, ensuring all data-related initiatives are aligned and business-driven.

Instill a culture of personal responsibility for data governance:

A unified data platform is inherently connected to the policies, procedures and business rules that inform and govern the data lifecycle. The centralized management and visibility afforded by linking policies and business rules at every level of the data lifecycle will improve data quality, reduce expensive re-work, and improve the ideation and consumption of data by the business.

Business users will know how to use (and how not to use) data, while technical practitioners will have a clear view of the controls and mechanisms required when building the infrastructure that serves up that data.

Better understand the impact of change:

Data governance should be connected to the use of data across roles, organizations, processes, capabilities, dashboards and applications. Proactive impact analysis is key to efficient and effective data strategy. However, most solutions don’t tell the whole story when it comes to data’s business impact.

By adopting a unified data platform, organizations can extend impact analysis well beyond data stores and data lineage for true visibility into who, what, where and how the impact will be felt, breaking down organizational silos.

Getting the Competitive “EDGE”

The erwin EDGE delivers an “enterprise data governance experience” in which every component of the data value chain is connected.

Now with data mapping, it unifies data preparation, enterprise modeling and data governance to simplify the entire data management and governance lifecycle.

Both IT and the business have access to an accurate, high-quality and real-time data pipeline that fuels regulatory compliance, innovation and transformation initiatives with accurate and actionable insights.

Categories
erwin Expert Blog

Massive Marriott Data Breach: Data Governance for Data Security

Organizations have been served yet another reminder of the value of data governance for data security.

Hotel and hospitality powerhouse Marriott recently revealed a massive data breach that led to the theft of personal data for an astonishing 500 million customers of its Starwood hotels. This is the second largest data breach in recent history, surpassed only by Yahoo’s breach of 3 billion accounts in 2013 for which it has agreed to pay a $50 million settlement to more than 200 million customers.

Now that Marriott has taken a major hit to its corporate reputation, it has two moves:

  1. Respond: Marriott’s response to its data breach so far has not received glowing reviews. But beyond how it communicates to effected customers, the company must examine how the breach occurred in the first place. This means understanding the context of its data – what assets exist and where, the relationship between them and enterprise systems and processes, and how and by what parties the data is used – to determine the specific vulnerability.
  2. Fix it: Marriott must fix the problem, and quickly, to ensure it doesn’t happen again. This step involves a lot of analysis. A data governance solution would make it a lot less painful by providing visibility into the full data landscape – linkages, processes, people and so on. Then more context-sensitive data security architectures can put in place to for corporate and consumer data privacy.

The GDPR Factor

It’s been six months since the General Data Protection Regulation (GDPR) took effect. While fines for noncompliance have been minimal to date, we anticipate them to dramatically increase in the coming year. Marriott’s bad situation could potentially worsen in this regard, without holistic data governance in place to identify whose and what data was taken.

Data management and data governance, together, play a vital role in compliance, including GDPR. It’s easier to protect sensitive data when you know what it is, where it’s stored and how it needs to be governed.

FREE GUIDE: THE REGULATORY RATIONALE FOR INTEGRATING DATA MANAGEMENT & DATA GOVERNANCE 

Truly understanding an organization’s data, including the data’s value and quality, requires a harmonized approach embedded in business processes and enterprise architecture. Such an integrated enterprise data governance experience helps organizations understand what data they have, where it is, where it came from, its value, its quality and how it’s used and accessed by people and applications.

Data Governance for Data Security

Data Governance for Data Security: Lessons Learned

Other companies should learn (like pronto) that they need to be prepared. At this point it’s not if, but when, a data breach will rear its ugly head. Preparation is your best bet for avoiding the entire fiasco – from the painstaking process of identifying what happened and why to notifying customers their data and trust in your organization have been compromised.

A well-formed security architecture that is driven by and aligned by data intelligence is your best defense. However, if there is nefarious intent, a hacker will find a way. So being prepared means you can minimize your risk exposure and the damage to your reputation.

Multiple components must be considered to effectively support a data governance, security and privacy trinity. They are:

  1. Data models
  2. Enterprise architecture
  3. Business process models

What’s key to remember is that these components act as links in the data governance chain by making it possible to understand what data serves the organization, its connection to the enterprise architecture, and all the business processes it touches.

THE EXPERT GUIDE TO DATA GOVERNANCE, SECURITY AND PRIVACY

Creating policies for data handling and accountability and driving culture change so people understand how to properly work with data are two important components of a data governance initiative, as is the technology for proactively managing data assets.

Without the ability to harvest metadata schemas and business terms; analyze data attributes and relationships; impose structure on definitions; and view all data in one place according to each user’s role within the enterprise, businesses will be hard pressed to stay in step with governance standards and best practices around security and privacy.

As a consequence, the private information held within organizations will continue to be at risk. Organizations suffering data breaches will be deprived of the benefits they had hoped to realize from the money spent on security technologies and the time invested in developing data privacy classifications. They also may face heavy fines and other financial, not to mention PR, penalties.

Less Pain, More Gain

Most organizations don’t have enough time or money for data management using manual processes. And outsourcing is also expensive, with inevitable delays because these vendors are dependent on manual processes too. Furthermore, manual processes require manual analysis and auditing, which is always more expensive and time consuming.

So the more processes an organization can automate, the less risk of human error, which is actually the primary cause of most data breaches. And automated processes are much easier to analyze and audit because everything is captured, versioned and available for review in a log somewhere. You can read more about automation in our 10 Reasons to Automate Data Mapping and Data Preparation.

And to learn more about how data governance underpins data security and privacy, click here.

Automate Data Mapping

Categories
erwin Expert Blog Data Governance Data Intelligence

Demystifying Data Lineage: Tracking Your Data’s DNA

Getting the most out of your data requires getting a handle on data lineage. That’s knowing what data you have, where it is, and where it came from – plus understanding its quality and value to the organization.

But you can’t understand your data in a business context much less track data lineage, its physical existence and maximize its security, quality and value if it’s scattered across different silos in numerous applications.

Data lineage provides a way of tracking data from its origin to destination across its lifespan and all the processes it’s involved in. It also plays a vital role in data governance. Beyond the simple ability to know where the data came from and whether or not it can be trusted, there’s an element of statutory reporting and compliance that often requires a knowledge of how that same data (known or unknown, governed or not) has changed over time.

A platform that provides insights like data lineage, impact analysis, full-history capture, and other data management features serves as a central hub from which everything can be learned and discovered about the data – whether a data lake, a data vault or a traditional data warehouse.

In a traditional data management organization, Excel spreadsheets are used to manage the incoming data design, what’s known as the “pre-ETL” mapping documentation, but this does not provide any sort of visibility or auditability. In fact, each unit of work represented in these ‘mapping documents’ becomes an independent variable in the overall system development lifecycle, and therefore nearly impossible to learn from much less standardize.

The key to accuracy and integrity in any exercise is to eliminate the opportunity for human error – which does not mean eliminating humans from the process but incorporating the right tools to reduce the likelihood of error as the human beings apply their thought processes to the work.

Data Lineage

Data Lineage: A Crucial First Step for Data Governance

Knowing what data you have and where it lives and where it came from is complicated. The lack of visibility and control around “data at rest” combined with “data in motion,” as well as difficulties with legacy architectures, means organizations spend more time finding the data they need rather than using it to produce meaningful business outcomes.

Organizations need to create and sustain an enterprise-wide view of and easy access to underlying metadata, but that’s a tall order with numerous data types and data sources that were never designed to work together and data infrastructures that have been cobbled together over time with disparate technologies, poor documentation and little thought for downstream integration. So the applications and initiatives that depend on a solid data infrastructure may be compromised, resulting in faulty analyses.

These issues can be addressed with a strong data management strategy underpinned by technology that enables the data quality the business requires, which encompasses data cataloging (integration of data sets from various sources), mapping, versioning, business rules and glossaries maintenance and metadata management (associations and lineage).

An automated, metadata-driven framework for cataloging data assets and their flows across the business provides an efficient, agile and dynamic way to generate data lineage from operational source systems (databases, data models, file-based systems, unstructured files and more) across the information management architecture; construct business glossaries; assess what data aligns with specific business rules and policies; and inform how that data is transformed, integrated and federated throughout business processes – complete with full documentation.

Centralized design, immediate lineage and impact analysis, and change-activity logging means you will always have answers readily available, or just a few clicks away. Subsets of data can be identified and generated via predefined templates, generic designs generated from standard mapping documents, and pushed via ETL process for faster processing via automation templates.

With automation, data quality is systemically assured and the data pipeline is seamlessly governed and operationalized to the benefit of all stakeholders. Without such automation, business transformation will be stymied. Companies, especially large ones with thousands of systems, files and processes, will be particularly challenged by a manual approach. And outsourcing these data management efforts to professional services firms only increases costs and schedule delays.

With erwin Mapping Manager, organizations can automate enterprise data mapping and code generation for faster time-to-value and greater accuracy when it comes to data movement projects, as well as synchronize “data in motion” with data management and governance efforts.

Map data elements to their sources within a single repository to determine data lineage, deploy data warehouses and other Big Data solutions, and harmonize data integration across platforms. The web-based solution reduces the need for specialized, technical resources with knowledge of ETL and database procedural code, while making it easy for business analysts, data architects, ETL developers, testers and project managers to collaborate for faster decision-making.

Data Lineage

Categories
erwin Expert Blog

The Role of Data Modeling in Data Governance

Greetings and Happy New Year to you all!

Over the past 9 months, the erwin Modeling team has been busy shouting from the mountaintops about our entry in the Data Governance space.  In April of 2015, we released a new edition of our modeling portal, erwin® Web Portal Data Governance Edition, and we have focused on explaining the key capabilities and value that this solution delivers in support of an organization’s data governance efforts.  Built on a foundation of erwin DM models, we naturally refer to our approach as model-driven data governance.  This is an apt description in that we support data governance controls and processes through and by leveraging data models.  However, I think in our natural inclination to focus on what’s “new,” we are relegating a key element of our data governance solution to the shadows.

The act of data modeling, when done correctly, is by definition a data governance activity and a key enabler for success for any data governance initiative.  A high-value output of any well-executed data modeling process is a set of standardized, business-aligned, and multi-contextual data definitions: “standardized” in that they conform to a reusable set of data definition requirements; “business-aligned” in that they capture business rules and regulatory requirements; “multi-contextual” in that they encapsulate metadata that represents both a business and a technical or infrastructure perspective.  They are also generally derived from a collaboration of business and IT stakeholders.  If the data modeling practice is mature, the models and the definitions they contain were created under some form of versioning control, review, and approval processes and industry-standard notations and methodologies.  In other words, the data modeling process is governed in its own right which naturally dovetails into and supports data governance at the enterprise level.  So it’s not good enough to say that data modeling supports data governance because, truth be told, data modeling and data definition through modeling is a key pillar of data governance.

Being the leader in data modeling, erwin Modeling has been delivering valuable capabilities in support of data governance for years.  It’s in the form of reusable templates, naming standards, use-defined properties, and support for other standards. It’s the relationships between the entities that capture the business rules and constraints inherent in our businesses and reflected in our data assets.  It’s the sound model management practices that promote collaboration and enable traceability.  Similarly, as data modelers, you have been living, breathing, and evangelizing data governance within your organization with every model you create.

As a heads up, I will be participating in a panel discussion on this very topic at DATAVERSITY’s EDGO conference on January 27th.  I would recommend attending this valuable virtual conference for anyone interested in this topic or data governance in general.

As an old grizzled data modeler (who asked to remain nameless) asserted in a conversation on the rise of data governance as a topic of interest and inquiry, we have been practicing data governance for decades.  We didn’t need a fancy name, we just called it good data administration.  Just another example of data modelers being ahead of the curve and leading the charge.

In this new year of 2016, make sure you resolve to hug a data modeler.  They deserve it more that you realize.

Have a great year!!!

Data Governance & Data Modeling White Paper Download

Categories
erwin Expert Blog

It’s Not Big. It’s Not Small. It’s Both.

Back in March at our erworld virtual conference, I participated in a DM Radio session that discussed the reality of living in a hybrid data world. It was interesting and eye-opening in many ways, yet confirmed what I always believed – Big Data is a reality (no surprise) but companies are still living in a “Small Data” world from an operational perspective (still no surprise, but it’s good to have it confirmed).

One of the analogies I used at the time leverages my own experiences over the years in IT. In the late 80’s, the rise of distributed computing threatened to take over from the mainframe. Over 25 years later, it still is “threatening,” but the mainframe is alive and well. Why? Because both have their purposes, and every large IT shop these days lives in a hybrid world.

Skip forward a number of years to the start of virtual computing. I recall speaking with a large customer who loved using the leading virtualization tool for testing and QA duties but “never for production.” A chance follow-up meeting a couple of years later revealed that, indeed, they were now putting VMs into production “where and when it made sense.” They’re living in a hybrid world.

Finally, I thought about cloud computing and how things were a few years back. Everyone was setting up private clouds and using SaaS applications (i.e. public cloud); I was leading an initiative at erwin around assessment and management of hybrid cloud environments – whether workloads should be placed in private or public clouds. Today, customers that I speak with on this topic confirm that they look at both options. They live in a hybrid world.

This leads us back to the topic at hand – the hybrid nature of data.

I’ve spoken with data architects, data modelers, and other database professionals who have built their careers with SQL & structured databases. It’s clear by looking at market trends that there will still be an increasing demand for these skills and products to help them in the future. Companies, however, are getting interested in Big Data and what value it has to bring business insights into seemingly unrelated (and previously unavailable) data.

One of the major analyst firms publishes an annual graphic showing emerging technologies and where they fit with respect to perception vs. reality. For the first time in 2015, Big Data is no longer present on this list, which proves that it’s no longer a promise. It’s reality. By my estimate, somewhere between 10 and 15% of large organizations have some sort of NoSQL/unstructured initiative in full production. Many more are in various stages of exploration and exploitation of Big Data.

The challenge that we at erwin see is that the practices, procedures and levels of governance with “small” data don’t yet fully exist with “big” data. I saw this same thing with distributed vs. mainframe computing, virtual vs. physical computing, and cloud vs. on-premise computing. Similar challenge, similar problem.

Big Data initiatives to date have been largely focused on the “why” and “what” and not so much the “how” – how they will manage them. Govern them. That’s why we released our Data Governance edition of the erwin Web Portal; you can look at all sources regardless of the modeling tool used. Ultimately it’s not about “big” data vs. “small” data, it’s just data.

There is a French saying: “plus ça change, plus c’est la même chose.” It means “the more things change, the more things stay the same.” I’ve found this to be true of IT as well. Rather than Big Data becoming a disruptive force and a replacement for small data, it’s a complement to small data. There’s ample room for both.

So, it’s time to stop questioning where we’ll end up. We’re living in a hybrid world with a lot of opportunity for data management professionals. Good times are ahead!

Download the White Paper The Business Value of Data Modeling for Data Governance

Categories
erwin Expert Blog

erwin Modeling Letters from the Frontline

In my role as Product Marketing Manager for erwin Modeling, I have the pleasure of attending many industry and vendor-sponsored events related to the practice of data management. However, as a representative for a software solution vendor, I often feel like my fellow attendees are taking me “with a grain of salt” because I am perceived as being there to move product.  While understandable, it still hurts, as our philosophy within the erwin Modeling team has always been that if you focus on delivering true value, the product will move itself.

The 10th Annual MDM & Data Governance Summit, held in NYC earlier this week, was somehow different. It was cathartic and in many ways an eye opener in terms of how MDM/Data Governance and, by extension, the practice of Data Management has matured and morphed over these many years. Before I begin “waxing philosophical,” I would like to thank Information Management and the MDM Institute for a thoroughly enjoyable, well-run and germane event.

My colleagues and I have always been consistent in our belief in the following: Data represents the true value of what IT delivers to the business and is truly a strategic asset. As such, it is imperative that data be managed and measured just like any other strategic asset. While IT professionals are critical in this endeavor, the business needs to be an equal and active partner in data management. Data models (metadata visualization) add significant value to most if not all data management initiatives and are critical to enabling business stakeholders to take their place at the table.

I am being brutally honest when I say that, over the years, there have been times where I felt like I may have been barking at the moon.

Which brings me to my experiences at the summit this week in New York. The event was not an awakening so much as a confirmation to the changes taking place in this space.  There is plenty of research, anecdotal evidence, and success stories from analysts and pundits available out in the ether.  But nothing takes the place of seeing, hearing, and feeling and seeing the excitement and satisfaction of real people, who are making real progress, confident in their ability to make a real impact on the success of the organizations they serve.

The value and importance of data is now accepted. It no longer takes the back seat to the infrastructure and “cool” apps. Data management is moving from an IT-driven infrastructure management exercise to a business imperative that is owned, driven, and funded by the business. Attendees in my session (65+ ☺) and the event in general were, for the most part, business people. They are sponsored, funded, and focused on what’s needed to achieve their goals. Additionally, the perceived value of data models and metadata is on the rise, not as a tool for IT to answer the annoying questions of the business, but as a true enabler for and by the business.

A perfect example of this shift in approach is “Big Data” and how it’s being adopted. In the past, new technologies have been driven by IT as an answer to a physical problem (i.e. performance, scale) or because it is a “cool” new toy. While elements of a technology driven approach still exist around “Big Data,” the majority of attendees I spoke to were taking a much more mature and business-oriented approach, focused on managing and integrating this new technology out of the gate to ensure they can realize the true “return on opportunity.”

Finally, as the leading data modeling provider, we are no longer trying to convince people that we can add value to these strategic data management initiatives and non-traditional (from a data modeling perspective) roles. Organizations are demanding it.

As a part-time musician and child of the 1960’s and 70’s, I should probably quote Bob Dylan and say “the times they are a-changin’.” Instead, I will break with convention and quote a band from the (yikes!!!) 80’s, Timbuk3: “the future’s so bright, I gotta wear shades!”