Tag: data modeler

Categories
erwin Expert Blog

5 Ways Data Modeling Is Critical to Data Governance

Enterprises are trying to manage data chaos. They might have 300 applications, with 50 different databases and a different schema for each one.

They also face increasing regulatory pressure because of global data regulations, such as the European Union’s General Data Protection Regulation (GDPR) and the new California Consumer Privacy Act (CCPA), that went into effect last week on Jan. 1.

Then there’s unstructured data with no contextual framework to govern data flows across the enterprise not to mention time-consuming manual data preparation and limited views of data lineage.

For decades, data modeling has been the optimal way to design and deploy new relational databases with high-quality data sources and support application development. It is a tried-and-true practice for lowering data management costs, reducing data-related risks, and improving the quality and agility of an organization’s overall data capability.

And the good news is that it just keeps getting better. Today’s data modeling is not your father’s data modeling software.

While it’s always been the best way to understand complex data sources and automate design standards and integrity rules, the role of data modeling continues to expand as the fulcrum of collaboration between data generators, stewards and consumers.

That’s because it’s the best way to visualize metadata, and metadata is now the heart of enterprise data management and data governance/ intelligence efforts.

So here’s why data modeling is so critical to data governance.

1. Uncovering the connections between disparate data elements: Visualize metadata and schema to mitigate complexity and increase data literacy and collaboration across a broad range of data stakeholders. Because data modeling reduces complexity, all members of the team can work around a data model to better understand and contribute to the project.

2. Capturing and sharing how the business describes and uses data: Create and integrate business and semantic metadata to augment and accelerate data intelligence and governance efforts. Data modeling captures how the business uses data and provides context to the data source.

3. Deploying higher quality data sources with the appropriate structural veracity: Automate and enforce data model design tasks to ensure data integrity. From regulatory compliance and business intelligence to target marketing, data modeling maintains an automated connection back to the source.

4. Building a more agile and governable data architecture: Create and implement common data design standards from the start. Data modeling standardizes design tasks to improve business alignment and simplify integration.

5. Governing the design and deployment of data across the enterprise: Manage the design and maintenance lifecycle for data sources. Data modeling provides visibility, management and full version control over the lifecycle for data design, definition and deployment.

Data Modeling Tool

erwin Data Modeler: Where the Magic Happens

erwin has just released a new version of erwin DM, the world’s No. 1 data modeling software for designing, deploying and understanding data sources to meet modern business needs. erwin DM 2020 is an essential source of metadata and a critical enabler of data governance and intelligence efforts.

The new version of erwin DM includes these features:

  • A modern, configurable workspace so users can customize the modeling canvas and optimize access to features and functionality that best support their workflows
  • Support for and model integration from major databases to work effectively across platforms and reuse work product, including native support for Amazon Redshift and updated support for the latest DB2 releases and certification for the latest MS SQL Server releases
  • Model exchange (import/export) to/from a wide variety of data management environments
  • Modeling task automation that saves modelers time, reduces errors and increases work product quality and speed, including a new scheduler to automate the offline reverse-engineering of databases into data models
  • New Quick Compare templates as part of the Complete Compare feature to compare and synchronize data models and sources
  • New ODBC query tool for creating and running custom model and metadata reports
  • Design transformations to customize and automate super-type/sub-type relationships between logical and physical models

erwin DM also integrates with the erwin Data Intelligence Suite (erwin DI) to automatically harvest the metadata in erwin data models for ingestion into the data catalog for better analytics, governance and overall data intelligence.

The role of data modeling in the modern data-driven business continues to expand with the benefits long-realized by database professionals and developers now experienced by a wider range of architects, business analysts and data administrators in a variety of data-centric initiatives.

Click here to test drive of the new erwin DM.

Categories
erwin Expert Blog

Benefits of Data Vault Automation

The benefits of Data Vault automation from the more abstract – like improving data integrity – to the tangible – such as clearly identifiable savings in cost and time.

So Seriously … You Should Automate Your Data Vault

 By Danny Sandwell

Data Vault is a methodology for architecting and managing data warehouses in complex data environments where new data types and structures are constantly introduced.

Without Data Vault, data warehouses are difficult and time consuming to change causing latency issues and slowing time to value. In addition, the queries required to maintain historical integrity are complex to design and run slow causing performance issues and potentially incorrect results because the ability to understand relationships between historical snap shots of data is lacking.

In his blog, Dan Linstedt, the creator of Data Vault methodology, explains that Data Vaults “are extremely scalable, flexible architectures” enabling the business to grow and change without “the agony and pain of high costs, long implementation and test cycles, and long lists of impacts across the enterprise warehouse.”

With a Data Vault, new functional areas typically are added quickly and easily, with changes to existing architecture taking less than half the traditional time with much less impact on the downstream systems, he notes.

Astonishingly, nearly 20 years since the methodology’s creation, most Data Vault design, development and deployment phases are still handled manually. But why?

Traditional manual efforts to define the Data Vault population and create ETL code from scratch can take weeks or even months. The entire process is time consuming slowing down the data pipeline and often riddled with human errors.

On the flipside, automating the development and deployment of design changes and the resulting data movement processing code ensures companies can accelerate dev and deployment in a timely and cost-effective manner.

Benefits of Data Vault Automation

Benefits of Data Vault Automation – A Case Study …

Global Pharma Company Saves Considerable Time and Money with Data Vault Automation

Let’s take a look at a large global pharmaceutical company that switched to Data Vault automation with staggering results.

Like many pharmaceutical companies, it manages a massive data warehouse combining clinical trial, supply chain and other mission-critical data. They had chosen a Data Vault schema for its flexibility in handling change but found creating the hubs and satellite structure incredibly laborious.

They needed to accelerate development, as well as aggregate data from different systems for internal customers to access and share. Additionally, the company needed lineage and traceability for regulatory compliance efforts.

With this ability, they can identify data sources, transformations and usage to safeguard protected health information (PHI) for clinical trials.

After an initial proof of concept, they deployed erwin Data Vault Automation and generated more than 200 tables, jobs and processes with 10 to 12 scripts. The highly schematic structure of the models enabled large portions of the modeling process to be automated, dramatically accelerating Data Vault projects and optimizing data warehouse management.

erwin Data Vault Automation helped this pharma customer automate the complete lifecycle – accelerating development while increasing consistency, simplicity and flexibility – to save considerable time and money.

For this customer the benefits of data vault automation were as such:

  • Saving an estimated 70% of the costs of manual development
  • Generating 95% of the production code with “zero touch,” improving the time to business value and significantly reduced costly re-work associated with error-prone manual processes
  • Increasing data integrity, including for new requirements and use cases regardless of changes to the warehouse structure because legacy source data doesn’t degrade
  • Creating a sustainable approach to Data Vault deployment, ensuring the agile, adaptable and timely delivery of actionable insights to the business in a well-governed facility for regulatory compliance, including full transparency and ease of auditability

Homegrown Tools Never Provide True Data Vault Automation

Many organizations use some form of homegrown tool or standalone applications. However, they don’t integrate with other tools and components of the architecture, they’re expensive, and quite frankly, they make it difficult to derive any meaningful results.

erwin Data Vault Automation centralizes the specification and deployment of Data Vault architectures for better control and visibility of the software development lifecycle. erwin Data Catalog makes it easy to discover, organize, curate and govern data being sourced for and managed in the warehouse.

With this solution, users select data sets to be included in the warehouse and fully automate the loading of Data Vault structures and ETL operations.

erwin Data Vault Smart Connectors eliminate the need for a business analyst and ETL developers to repeat mundane tasks, so they can focus on choosing and using the desired data instead. This saves considerable development time and effort plus delivers a high level of standardization and reuse.

After the Data Vault processes have been automated, the warehouse is well documented with traceability from the marts back to the operational data to speed the investigation of issues and analyze the impact of changes.

Bottom line: if your Data Vault integration is not automated, you’re already behind.

If you’d like to get started with erwin Data Vault Automation or request a quote, you can email consulting@erwin.com.

Data Modeling Drives Business Value

Categories
erwin Expert Blog

Data Modeling in a Jargon-filled World – Big Data & MPP

Big Data & MPP

By now, you’ve likely heard a lot about Big Data. You may have even heard about “the three Vs” of Big Data. Originally defined by Gartner, “Big Data is “high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision-making, insight discovery and process optimization.”

Continue reading “Data Modeling in a Jargon-filled World – Big Data & MPP”