Categories
erwin Expert Blog

Big Data Posing Challenges? Data Governance Offers Solutions

Big Data is causing complexity for many organizations, not just because of the volume of data they’re collecting, but because of the variety of data they’re collecting.

Big Data often consists of unstructured data that streams into businesses from social media networks, internet-connected sensors, and more. But the data operations at many organizations were not designed to handle this flood of unstructured data.

Dealing with the volume, velocity and variety of Big Data is causing many organizations to re-think how they store and govern their data. A perfect example is the data warehouse. The people who built and manage the data warehouse at your organization built something that made sense to them at the time. They understood what data was stored where and why, as well how it was used by business units and applications.

The era of Big Data introduced inexpensive data lakes to some organizations’ data operations, but as vast amounts of data pour into these lakes, many IT departments found themselves managing a data swamp instead.

In a perfect world, your organization would treat Big Data like any other type of data. But, alas, the world is not perfect. In reality, practicality and human nature intervene. Many new technologies, when first adopted, are separated from the rest of the infrastructure.

“New technologies are often looked at in a vacuum, and then built in a silo,” says Danny Sandwell, director of product marketing for erwin, Inc.

That leaves many organizations with parallel collections of data: one for so-called “traditional” data and one for the Big Data.

There are a few problems with this outcome. For one, silos in IT have a long history of keeping organizations from understanding what they have, where it is, why they need it, and whether it’s of any value. They also have a tendency to increase costs because they don’t share common IT resources, leading to redundant infrastructure and complexity. Finally, silos usually mean increased risk.

But there’s another reason why parallel operations for Big Data and traditional data don’t make much sense: The users simply don’t care.

At the end of the day, your users want access to the data they need to do their jobs, and whether IT considers it Big Data, little data, or medium-sized data isn’t important. What’s most important is that the data is the right data – meaning it’s accurate, relevant and can be used to support or oppose a decision.

Reputation Management - What's Driving Data Governance

How Data Governance Turns Big Data into Just Plain Data

According to a November 2017 survey by erwin and UBM, 21 percent of respondents cited Big Data as a driver of their data governance initiatives.

In today’s data-driven world, data governance can help your business understand what data it has, how good it is, where it is, and how it’s used. The erwin/UBM survey found that 52 percent of respondents said data is critically important to their organization and they have a formal data governance strategy in place. But almost as many respondents (46 percent) said they recognize the value of data to their organization but don’t have a formal governance strategy.

A holistic approach to data governance includes thesekey components.

  • An enterprise architecture component is important because it aligns IT and the business, mapping a company’s applications and the associated technologies and data to the business functions they enable. By integrating data governance with enterprise architecture, businesses can define application capabilities and interdependencies within the context of their connection to enterprise strategy to prioritize technology investments so they align with business goals and strategies to produce the desired outcomes.
  • A business process and analysis component defines how the business operates and ensures employees understand and are accountable for carrying out the processes for which they are responsible. Enterprises can clearly define, map and analyze workflows and build models to drive process improvements, as well as identify business practices susceptible to the greatest security, compliance or other risks and where controls are most needed to mitigate exposures.
  • A data modeling component is the best way to design and deploy new databases with high-quality data sources and support application development. Being able to cost-effectively and efficiently discover, visualize and analyze “any data” from “anywhere” underpins large-scale data integration, master data management, Big Data and business intelligence/analytics with the ability to synthesize, standardize and store data sources from a single design, as well as reuse artifacts across projects.

When data governance is done right, and it’s woven into the structure and architecture of your business, it helps your organization accept new technologies and the new sources of data they provide as they come along. This makes it easier to see ROI and ROO from your Big Data initiatives by managing Big Data in the same manner your organization treats all of its data – by understanding its metadata, defining its relationships, and defining its quality.

Furthermore, businesses that apply sound data governance will find themselves with a template or roadmap they can use to integrate Big Data throughout their organizations.

If your business isn’t capitalizing on the Big Data it’s collecting, then it’s throwing away dollars spent on data collection, storage and analysis. Just as bad, however, is a situation where all of that data and analysis is leading to the wrong decisions and poor business outcomes because the data isn’t properly governed.

Previous posts:

You can determine how effective your current data governance initiative is by taking erwin’s DG RediChek.

Categories
erwin Expert Blog

Managing Any Data, Anywhere with Any2

The amount of data in the world is staggering. And as more and more organizations adopt digitally orientated business strategies the total keeps climbing. Modern organizations need to be equipped to manage Any2 – any data, anywhere.

Analysts predict that the total amount of data in the world will reach 44 zettabytes by 2020 – one zettabyte = 44 trillion gigabytes. That’s an incredible feat in and of itself. But considering the fact that the total had only reached 4.4 zettabytes in 2013, the rate at which data is collected and stored becomes even more astonishing.

However, it is equally incredible that less than 0.5% of that data is currently analyzed and/or utilized effectively by the business.

What does this mean for business?

Perhaps the most obvious answer is opportunity. You likely wouldn’t be reading this blog if you weren’t at least passively aware of the potential insight that can be derived from a series of ones and zeros.

Start-ups such as Uber, Netflix and Airbnb are perhaps some of the best examples of data’s potential being realized. It’s even more apparent when you consider these three organizations refer to themselves as technology companies, as opposed to the fields their services fall under.

But with data’s potential, potentially open for any business to invest in, action, and benefit from, competition is more fierce than ever, which brings us to what else this new wave of data means for business. That being effective data management.

All of this new data is being created, or even stored, under one manageable umbrella. It’s disparate, it’s noisy, and in its raw form it’s often useless. So to uncover data’s aforementioned potential, businesses must take the necessary steps to “clean it up”.

That’s what the Any2 concept is all about. Allowing businesses to manage, govern and analyse any data, anywhere.

Any2 - Data Management Platform

Any2 – Any Data

The first part of the Any2 equation, pertains to Any Data.

Managing data requires facing the challenges that come with the ‘three Vs of data’: volume, variety and velocity, with volume referring the amount of data, variety to its different sources, and velocity the speed in which it must be processed.

We can stretch these three Vs to five when we include veracity (confidence in the accuracy of the data), and value.

Generally, any data concerns the variety ‘V’, referring to the numbered and disparate potential sources data can be derived from. But as we need to be able to incorporate all of the varying forms of data to accurately analyze it, we can also say any data concerns the volume, and velocity too – especially where Big Data is considered.

Big Data initiatives increase the volume of data businesses have to manage exponentially, and to achieve desired time to market, it must be processed quickly (albeit thoroughly), too.

Additionally, data can be represented as either structured or unstructured.

Traditionally, most data fell under the structured label. Data including business data, relational data, and operational data, for example. And although the different types of data were still disparate, being inherently structured within their own vertical still made them far easier to manage, define, and analyze.

Unstructured data, however, is the polar opposite. It’s inherently messy and it’s hard to define, making both reporting and analysis potentially problematic. This is an issue many businesses face when transitioning to a more data-centric approach to operations.

Big data sources such as click stream data, IoT data, machine data and social media data all fall under this banner. All of these sources need to be rationalized and correlated so they can be analyzed more effectively, and in the same vain as the aforementioned structured data.

Any2 – Anywhere

The anywhere half of the equation is arguably also predominantly focused on the variety ‘V’ – but from a different angle. Anywhere is more concerned with the differing and disparate ways and places in which data can be securely stored, rather than the variety in the data itself.

Although an understanding of where your data is has always been a necessity, it’s now become more relevant than ever. Prior to the adoption of cloud storage and services, data would have to have been managed locally, within the “firewall”.

Businesses would still have to know where the data was saved, and how it could be accessed.

However, the advantages of storing data outside of the business have become more apparent and more widely accepted. This has seen many businesses take the leap and invest in varying capacities, into-cloud based storage and software-as-a-service (SaaS).

Take SAP, for example. SAP provides one solution and one collated database, in favour of a business paying installation and upkeep fees for multiple softwares and databases.

And we still need to consider the uptick in the amount of businesses that buy customer data.

All of this data still has to be integrated, documented and understood in order for it to be useful, as poor management of data can lead to poor results – or, garbage in, garbage out for short.

Therefore, the key focus of the anywhere part of the equation is granting businesses the ability to manage external data at the same level as internal.

Effectively managing data anywhere, requires data modeling, business process and enterprise architecture.

Data modeling is needed to establish what you have whether internal or external, and to identify what that data is.

Business Processes is required to understand how the data should be used and how it best drives the business.

Enterprise Architecture is useful as it allows a business to determine how best to leverage the data to drive value. It’s also needed to ensure the business has a solid enough architecture to allow for this value to come to fruition, and in analyzing/predicting the impact of change, so that value isn’t adversely affected.

So how do we manage Any Data, Anywhere?

The best way to effectively manage Any Data, Anywhere, so that we can ensure investing in data management and analysis adds value, is to consider the ‘3Vs’ in relation to the data timeline. You should also consider the various initiatives (Data Modeling, Enterprise Architecture and Business Process) that can be actioned at each stage to ensure the data is properly processed and understood.

Any2 - Data management platform

Any2 approach helps you:

  • Effectively manage and govern massive volumes of data
  • Consolidate and build applications with hybrid data architectures – traditional + Big Data, cloud and on-premise
  • Support expanding regulatory and legislative requirements: GDPR etc
  • Simplify collaboration and improve alignment with accurate financial and operation information
  • Improve business processes for operational efficiency and compliance standards
  • Empower your people with self-service data access: The right information at the right time to improve corporate decision-making

 

 

For more Data Modeling, Enterprise Architecture, and Business Process advice follow us on Twitter and Linkedin to stay updated with the new posts!

Importance of Governing Data

Categories
erwin Expert Blog

The Rise of NoSQL and NoSQL Data Modeling

With NoSQL data modeling gaining traction, data governance isn’t the only data shakeup organizations are currently facing.