Skip to main content

Data Warehouse Architecture in detail


 Data Warehouse

  • A data warehouse is a centralized repository of structured, organized, and historical data that is used for reporting, analysis, and business intelligence purposes.
  • It is a historical collection of data from different sources that is integrated, cleansed, and organized for analysis.
  •  Data warehouses are used to support business intelligence (BI) and decision-making.

Characteristics of a Data Warehouse:

  • Centralized repository: A data warehouse is a centralized repository of data, which means that it is a single source of truth for all of the data that is stored in it. 
  • Historical data: Data warehouses typically contain historical data, which means that they store data that has been collected over a period of time. 
  • Integrated data: The data in a data warehouse is integrated, which means that it has been combined from different sources. 
  • Cleansed data: The data in a data warehouse is cleansed, which means that it has been cleaned of errors and inconsistencies. .
  • Organized data: The data in a data warehouse is organized in a way that makes it easy to access and analyze. 
Data warehouses are an essential tool for businesses that want to make better decisions based on data. 

Architecture of Data Warehouse:


Detailed Description of data warehouse Architecture

  •  ETL: Extract, transform, load. This process is used to load data from operational databases and external sources into the data warehouse.
  • Refresh: This process is used to update the data warehouse data on a regular basis, such as daily, weekly, or monthly.
  • Data marts typically contain a subset of the data in the data warehouse, and they are optimized for specific types of analysis.
  • The metadata repository is a database that stores information about the data warehouse data. This information includes the schema of the data warehouse, the lineage of the data, and the quality of the data. 
  • OLAP servers can be used to process queries from the front-end tools. They can also be used to generate reports.
  • Query/report: This process is used to retrieve data from the data warehouse and to generate reports. 
  • Analysis: This process is used to analyze the data warehouse data. Users can use analysis tools to perform various analysis tasks, such as trend analysis, forecasting, and clustering.
  • Data mining: This process is used to discover patterns in the data warehouse data. Users can use data mining tools to discover patterns that can be used to make predictions or to identify new opportunities.

Advantages of data warehouses:

  • Improved business intelligence: Data warehouses can help businesses to gain a better understanding of their data by providing a centralized repository for historical data.
  • Better decision-making: By providing access to a wide range of data, data warehouses can help businesses to make better decisions.
  • Improved data quality: Data warehouses can help to improve data quality by providing a single source of truth for data.

Disadvantages of data warehouses:

  • Costly setup and maintenance: Data warehouses can be expensive to set up and maintain.
  • Limited flexibility: Data warehouses can be limited in their flexibility.
  • Data silos: Data warehouses can create data silos.

Used for 

  • Track sales trends
  • Analyze customer behavior.
  • Identify new market opportunities.
  • Make better strategic decisions.
  • Here are some examples of data warehouses:

Example Of Data Warehouse

  • Amazon Redshift
  • Google Big Query
  • Microsoft Azure SQL Data Warehouse

Labels:

Popular posts from this blog

HOW TO BUILD LOGIN PAGE AND SIGN UP PAGE IN REACT JS AND AWS AMPLIFY

  STEPS TO BUILD LOGIN & SIGNUP PAGE IN REACTJS,NODEJS USING AWS AMPLIFY IN VS CODE SOFTWARE REQUIREMENTS NODE JS  AWS ACCOUNT VISUAL STUDIO CODE PLEASE BE FOLLOW THESE STEPS 1.      INSTALL  NODEJS   2.      CHECK VERSION OF NODEJS IN CMD - node --version 3.      CREATE AWS ACCOUNT(IF NOT HAVE OR TEMPORARY NEEDED,Dm me  BUT,ITS COST) 4.      INSTALL VISUAL STUDIO CODE ( install Extension-Live server,AWS Toolkit) 5.      RUN THESE COMMAND IN VS CODE TERMINAL-NPM UPDATE-npm install npm -g 6.      INSTALL - AWSTOOLKIT  EXTENSION IN VSCODE 7.      ENABLE SCRIPTING IN VS CODE - Set-ExecutionPolicy -Scope CurrentUser Unrestricted (paste these higlighted cmd in vs code Terminal) 8.      RUN THESE CMD IN TERMINAL- npx create-react-app projectname( Procedure ) 9.      RUN IN VSCODE TERMINAL(INSIDE PROJECTNAME DIRECTORY)- amplify configure (AWS ACCOUNT NECESSARY)TO PERFORM THESE CMD 10. RUN IN VSCODE TERMINAL(INSIDE PROJECTNAME DIRECTORY)- amplify init 11. RUN IN VS CO
READ

AWS Toolkit Extension for Visual Studio Code (VS Code)

  AWS Toolkit Extension for Visual Studio Code (VS Code) is a plugin that enables developers to work with Amazon Web Services (AWS) services directly from within the VS Code editor. This extension provides several features and functionalities that can be used to develop, deploy, and debug applications on AWS. The AWS Toolkit Extension provides a set of tools and functionalities that make it easier for developers to build, test, and deploy serverless applications on AWS. It also provides an integrated development environment (IDE) for developing applications with AWS services, which includes support for AWS Lambda, AWS Step Functions, Amazon API Gateway, Amazon S3, and other AWS services. Some of the key features and functionalities of the AWS Toolkit Extension for VS Code include: Ø   AWS Explorer: A graphical user interface (GUI) that enables developers to view and manage their AWS resources from within VS Code. Developers can browse and navigate through their AWS accounts, c
READ