Consolidation: Data extracted are consolidated in the required format before loading it into the data warehouses.4.For example, for generating sales reports one only needs sales records for that specific year. Filtering: Here we filter out the required data out of a large amount of data present according to business requirements.Data Cleansing or Enrichment: It refers to cleaning the undesired data from the staging area so that wrong data doesn’t get loaded from the data warehouses.Basic Transformations: These transformations are applied in every scenario as they are basic need while loading the data that has been extracted from various sources, in the data warehouses. In this step many transformations are performed to make data ready for load in data warehouses by applying the below transformations:Ī.
This step is the most important step of ETL.
#Etl processes update#
Partial Extraction (without update notification): This strategy refers to extract specific required data from sources according to load in the data warehouses instead of extracting whole data.Partial Extraction (with update notification): This strategy is also known as delta, where only the data being changed is extracted and update data warehouses.
#Etl processes full#
Full Extraction: This is followed when whole data from sources get loaded into the data warehouses that show either the data warehouse is being populated the first time or no strategy has been made for data extraction.while fetching data from these systems.īut one should take care that these systems must remain unaffected while extraction. A proper data map is required between source and target before data extraction occurs as the ETL process needs to interact with various systems such as Oracle, Hardware, Mainframe, real-time systems such as ATM, Hadoop, etc. Thus, data is checked thoroughly before moving it to data warehouses otherwise it will become a challenge to revert the changes in data warehouses. The extracted data is stored in the staging area where further transformations are being performed.
This step refers to fetching the required data from various sources that are present in different formats such as XML, Hadoop files, Flat Files, JSON, etc. The ETL process is a 3-step process that starts with extracting the data from various data sources and then raw data undergoes various transformations to make it suitable for storing in data warehouse and load it in data warehouses in the required format and make it ready for analysis. Hadoop, Data Science, Statistics & others
0 kommentar(er)
