Create a Customized Big Data Architecture and Implementation Plan
- Big
data architecture is different from traditional data for several key reasons,
including:
- Big data architecture starts with the data itself, taking a bottom-up approach. Decisions about data influence decisions about components that use data.
- Big data introduces new data sources such as social media content and streaming data.
- The enterprise data warehouse (EDW) becomes a source for big data.
- Master data management (MDM) is used as an index to content in big data about the people, places, and things the organization cares about.
- The variety of big data and unstructured data requires a new type of persistence.
- Many data architects have no experience with big data and feel overwhelmed by the number of options available to them (including vendor options, storage options, etc.). They often have little to no comfort with new big data management technologies.
- If
organizations do not architect for big data, there are a couple of main risks:
- The existing data architecture is unable to handle big data, which will eventually result in a failure that could compromise the entire data environment.
- Solutions will be selected in an ad hoc manner, which can cause incompatibility issues down the road.
Our Advice
Critical Insight
- Before beginning to make technology decisions regarding the big data architecture, make sure a strategy is in place to document architecture principles and guidelines, the organization’s big data business pattern, and high-level functional and quality of service requirements.
- The big data business pattern can be used to determine what data sources should be used in your architecture, which will then dictate the data integration capabilities required. By documenting current technologies, and determining what technologies are required, you can uncover gaps to be addressed in an implementation plan.
- Once you have identified and filled technology gaps, perform an architectural walkthrough to pull decisions and gaps together and provide a fuller picture. After the architectural walkthrough, fill in any uncovered gaps. A proof-of-technology project can be started as soon as you have evaluation copies (or OSS) products and at least one person who understands the technology.
Impact and Result
- Save time and energy trying to fix incompatibilities between technology and data.
- Allow the Data Architect to respond to big data requests from the business more quickly.
- Provide the organization with valuable insights through the analytics and visualization technologies that are integrated with the other building blocks.
Create a Customized Big Data Architecture and Implementation Plan Research & Tools
Besides the small introduction, subscribers and consulting clients within this management domain have access to:
1. Recognize the importance of big data architecture
Big data is centered on the volume, variety, velocity, veracity, and value of data. Achieve a data architecture that can support big data.
- Storyboard: Create a Customized Big Data Architecture and Implementation Plan
2. Define architectural principles and guidelines while taking into consideration maturity
Understand the importance of a big data architecture strategy. Assess big data maturity to assist with creation of your architectural principles.
- Big Data Maturity Assessment Tool
- Big Data Architecture Principles & Guidelines Template
3. Build the big data architecture
Come to accurate big data architecture decisions.
- Big Data Architecture Decision Making Tool
4. Determine common services needs
What are common services?
5. Plan a big data architecture implementation
Gain business satisfaction with big data requests. Determine what steps need to be taken to achieve your big data architecture.
- Big Data Architecture Initiative Definition Tool
- Big Data Architecture Initiative Planning Tool
Infographic
Workshop: Create a Customized Big Data Architecture and Implementation Plan
Workshops offer an easy way to accelerate your project. If you are unable to do the project yourself, and a Guided Implementation isn't enough, we offer low-cost delivery of our project workshops. We take you through every phase of your project and ensure that you have a roadmap in place to complete your project successfully.
1 Recognize the Importance of Big Data Architecture
The Purpose
Set expectations for the workshop.
Recognize the importance of doing big data architecture when dealing with big data.
Key Benefits Achieved
Big data defined.
Understanding of why big data architecture is necessary.
Activities
1.1 Define the corporate strategy.
1.2 Define big data and what it means to the organization.
1.3 Understand why doing big data architecture is necessary.
1.4 Examine Info-Tech’s Big Data Reference Architecture.
Outputs
Defined Corporate Strategy
Defined Big Data
Reference Architecture
2 Design a Big Data Architecture Strategy
The Purpose
Identification of architectural principles and guidelines to assist with decisions.
Identification of big data business pattern to choose required data sources.
Definition of high-level functional and quality of service requirements to adhere architecture to.
Key Benefits Achieved
Key Architectural Principles and Guidelines defined.
Big data business pattern determined.
High-level requirements documented.
Activities
2.1 Discuss how maturity will influence architectural principles.
2.2 Determine which solution type is best suited to the organization.
2.3 Define the business pattern driving big data.
2.4 Define high-level requirements.
Outputs
Architectural Principles & Guidelines
Big Data Business Pattern
High-Level Functional and Quality of Service Requirements Exercise
3 Build a Big Data Architecture
The Purpose
Establishment of existing and required data sources to uncover any gaps.
Identification of necessary data integration requirements to uncover gaps.
Determination of the best suited data persistence model to the organization’s needs.
Key Benefits Achieved
Defined gaps for Data Sources
Defined gaps for Data Integration capabilities
Optimal Data Persistence technology determined
Activities
3.1 Establish required data sources.
3.2 Determine data integration requirements.
3.3 Learn which data persistence model is best suited.
3.4 Discuss analytics requirements.
Outputs
Data Sources Exercise
Data Integration Exercise
Data Persistence Decision Making Tool
4 Plan a Big Data Architecture Implementation
The Purpose
Identification of common service needs and how they differ for big data.
Performance of an architectural walkthrough to test decisions made.
Group gaps to form initiatives to develop an Initiative Roadmap.
Key Benefits Achieved
Common service needs identified.
Architectural walkthrough completed.
Initiative Roadmap completed.
Activities
4.1 Identify common service needs.
4.2 Conduct an architectural walkthrough.
4.3 Group gaps together into initiatives.
4.4 Document initiatives on an initiative roadmap.
Outputs
Architectural Walkthrough
Initiative Roadmap