benefits of Archives - erwin, Inc.

Replace manual and recurring tasks for fast, reliable data lineage and overall data governance

It’s paramount that organizations understand the benefits of automating end-to-end data lineage. Critically, it makes it easier to get a clear view of how information is created and flows into, across and outside an enterprise.

The importance of end-to-end data lineage is widely understood and ignoring it is risky business. But it’s also important to understand why and how automation plays a critical role.

Benjamin Franklin said, “Lost time is never found again.” According to erwin’s “2020 State of Data Governance and Automation” report, close to 70 percent of data professional respondents say they spend an average of 10 or more hours per week on data-related activities, and most of that time is spent searching for and preparing data.

Data automation reduces the loss of time in collecting, processing and storing large chunks of data because it replaces manual processes (and human errors) with intelligent processes, software and artificial intelligence (AI).

Automating end-to-end data lineage helps organizations further focus their available resources on more important and strategic tasks, which ultimately provides greater value.

For example, automatically importing mappings from developers’ Excel sheets, flat files, Access and ETL tools into a comprehensive mappings inventory, complete with auto generated and meaningful documentation of the mappings, is a powerful way to support overall data governance.

According to the erwin report, documenting complete data lineage is currently the data operation with the largest percentage spread between its current level of automation (25%) and being seen as the most valuable operation to automate (65%).

Doing Data Lineage Right

Eliminating manual tasks is not the only reason to adopt automated data lineage. Replacing recurring tasks that don’t rely on human intelligence for completion is where automation makes an even bigger difference. Here are six benefits of automating end-to-end data lineage:

Reduced Errors and Operational Costs

Data quality is crucial to every organization. Automated data capture can significantly reduce errors when compared to manual entry. Company documents can be filled out, stored, retrieved, and used more accurately and this, in turn, can save organizations a significant amount of money.

The 1-10-100 rule, commonly used in business circles, states that preventing an error will cost an organization $1, correcting an error already made will cost $10, and allowing an error to stand will cost $100.

Ratios will vary depending on the magnitude of the mistake and the company involved, of course, but the point remains that adopting the most reliable means of preventing a mistake, is the best approach to take in the long run.

Faster Business Turnaround

Speed and faster time to market is a driving force behind most organizations’ efforts with data lineage automation. More work can be done when you are not waiting on someone to manually process data or forms.

For example, when everything can be scanned using RFID technology, it can be documented and confirmed instantaneously, cutting hours of work down to seconds.

This opens opportunities for employees to train for more profitable roles, allowing organizations to reinvest in their employees. With complex data architectures and systems within so many organizations, tracking data in motion and data at rest is daunting to say the least.

Harvesting the data through automation seamlessly removes ambiguity and speeds up the processing time-to-market capabilities.

Compliance and Auditability

Regulatory compliance places greater transparency demands on firms when it comes to tracing and auditing data.

For example, capital markets trading firms must implement data lineage to support risk management, data governance and reporting for various regulations such as the Basel Committee on Banking Supervision’s standard number 239 (BCBS 239) and Markets in Financial Instruments Directive (MiFID II).

Business terms and data policies should be implemented through standardized and documented business rules. Compliance with these business rules can be tracked through data lineage, incorporating auditability and validation controls across data transformations and pipelines to generate alerts when there are non-compliant data instances.

Also, different organizational stakeholders (customers, employees and auditors) need to understand and trust reported data. Automated data lineage ensures captured data is accurate and consistent across its trajectory.

Consistency, Clarity and Greater Efficiency

Data lineage automation can help improve efficiency and ensure accuracy. The more streamlined your processes, the more efficient your business. The more efficient your business, the more money you save on daily operations.

For example, backing up your data effectively and routinely is important. Data is one of the most important assets for any business.

However, different types of data need to be treated differently. Some data needs to be backed up daily while some types of data demand weekly or monthly backups.

With automation in place, you just need to develop backup strategies for your data with a consistent scheduling process. The actual job of backing things up will be managed by the system processes you set up for consistency and clarity.

Improved Customer and Employee Satisfaction

Customer disengagement is a more severe problem than you might think. A recent study has shown that it costs U.S. businesses around $300 billion annually, nearly equal to the U.S. defense budget. When the employees are disengaged, they consistently give you their time but do not put the best of their efforts.

With data lineage automation, employers can automate such tasks and free up time for high-value work. According to a smartsheet report, 69% of employees thought that automation would reduce wasting time during their workday and 59% thought that they would have more than six spare hours per week if repetitive jobs were automated.

Governance Enforcement

Data lineage automation is a great way to implement governance in any business. Any task that an automated process completes is always documented and has traceability.

For every task, you get clear logs that tell you what was done, who did it and when it was done. As stated before, automation plays a major role in reducing human errors and speeds up tasks that need to be performed repeatedly.

If you have not made the jump to digital yet, you are probably wading through high volumes of resources and manual processes daily. There is no denying the fact that automating business processes contributes immensely to an organization’s success.

Automated Data Lineage in Action

Automated data lineage tools document the flow of data into and out of an organization’s systems. They capture end-to-end lineage and ensure proper impact analysis can be performed in the event of problems or changes to data assets as they move across pipelines.

erwin Data Intelligence (erwin DI) helps bind business terms to technical data assets with a complete data lineage of scanned metadata assets. Automating data capture frees up resources to focus on more strategic and useful tasks.

It automatically generates end-to-end data lineage, down to the column level and between repositories. You can view data flows from source systems to the reporting layers, including intermediate transformation and business logic.

Request your own demo of erwin DI to see metadata-driven, automated data lineage in action.

Users discuss how they are putting erwin’s data modeling, enterprise architecture, business process modeling, and data intelligences solutions to work

IT Central Station members using erwin solutions are realizing the benefits of enterprise modeling and data intelligence. This article highlights some specific use cases and the results they’re experiencing within the organizations.

Enterprise Architecture & Business Process Modeling with erwin Evolve

An enterprise architect uses erwin Evolve at an aerospace/defense firm with more than 10,000 employees. His team is “doing business process modeling and high-level strategic modeling with its capabilities.” Others in his company are using it for IT infrastructure, such as aligning requirements to system solutions.

For Matthieu G., a senior business process management architect at a pharma/biotech company with more than 5,000 employees, erwin Evolve was useful for enterprise architecture reference. As he put it, “We are describing our business process and we are trying to describe our data catalog. We are describing our complete applications assets, and we are interfacing to the CMDB of our providers.”

His team also is using the software to manage roadmaps in their main transformation programs. He added, “We have also linked it to our documentation repository, so we have a description of our data documents.” They have documented 200 business processes in this way. In particular, the tool helped them to design their qualification review, which is necessary in a pharmaceutical business.

erwin Evolve users are experiencing numerous benefits. According to the aerospace enterprise architect, “It’s helped us advance in our capabilities to perform model-based systems engineering, and also model-based enterprise architecture.”

This matters because, as he said, “By placing the data and the metadata into a model, which is what the tool does, you gain the abilities for linkages between different objects in the model, linkages that you cannot get on paper or with Visio or PowerPoint.” That is a huge differentiator for this user.

This user also noted, “I use the automatic diagramming features a lot. When one of erwin’s company reps showed that to me a couple of years ago, I was stunned. That saves hours of work in diagramming. That capability is something I have not seen in other suppliers’ tools.”

He further explained “that really helps too with when your data is up to date. The tool will then automatically generate the updated diagram based on the data, so you know it’s always the most current version. You can’t do that in things like Visio and PowerPoint. They’re static snapshots of a diagram at some point in time. This is live and dynamic.”

Data Modeling with erwin Data Modeler

George H., a technology manager, uses erwin Data Modeler (erwin DM) at a pharma/biotech company with more than 10,000 employees for their enterprise data warehouse.

He elaborated by saying, “We have an enterprise model being maintained and we have about 11 business-capability models being maintained. Examples of business capabilities would be finance, human resources, supply-chain, sales and marketing, and procurement. We maintain business domain models in addition to the enterprise model.”

Roshan H., an EDW architect/data modeler who uses erwin DM at Royal Bank of Canada, works on diverse platforms, including Microsoft SQL Server, Oracle, DB2, Teradata and NoSQL. After gathering requirements and mapping data on Excel, they start building the conceptual model and then the logical model with erwin DM.

He said, “When we have these data models built in the erwin DM, we generate the PDF data model diagrams and take it to the team (DBA, BSAs, QA and others) to explain the model diagram. Once everything is reviewed, then we go on to discuss the physical data model.”

“We use erwin DM to do all of the levels of analysis that a data architect does,” said Sharon A., a senior manager, data governance at an insurance company with over 500 employees. She added, “erwin DM does conceptual, logical and physical database or data structure capture and design, and creates a library of such things.

We do conceptual data modeling, which is very high-level and doesn’t have columns and tables. It’s more concepts that the business described to us in words. We can then use the graphic interface to create boxes that contain descriptions of things and connect things together. It helps us to do a scope statement at the beginning of a project to corral what the area is that the data is going to be using.”

Data Governance with erwin Data Intelligence

IT Central Station members are seeing benefits from using erwin Data Intelligence (erwin DI) for data governance use cases. For Rick D., a data architect at NAMM, a small healthcare company, erwin DI “enabled us to centralize a tremendous amount of data into a common standard, and uniform reporting has decreased report requests.”

As a medical company, they receive data from 17 different health plans. Before adopting erwin DI, they didn’t have a centralized data dictionary of their data. The benefit of data governance, as he saw it, was that “everybody in our organization knows what we are talking about. Whether it is an institutional claim, a professional claim, Blue Cross or Blue Shield, health plan payer, group titles, names, etc.”

A solution architect at a pharma/biotech company with more than 10,000 employees used erwin DI for metadata management, versioning of metadata and metadata mappings and automation. In his experience, applying governance to metadata and creating mappings has helped different stakeholders gain a good understanding of the data they use to do their work.

Sharon A. had a comparable use case. She said, “You can map the business understanding in your glossary back to your physical so you can see it both ways. With erwin DI, I can have a full library of physical data there or logical data sets, publish it out through the portal, and then the business can do self-service. The DBAs can use it for all different types of value-add from their side of the house. They have the ability to see particular aspects, such as RPII, and there are some neat reports which show that. They are able manage who can look at these different pieces of information.”

For more real erwin user experiences, visit IT Central Station.