IJCATR Volume 11 Issue 8

Developing an ETL Pipeline for Data Analysis

A S Prajwal Babu, Prof. Suma B
10.7753/IJCATR1108.1004
keywords : Data pipeline,ETL pipeline Cloud, Data Warehouse, Data Analytics

PDF
The world's most valuable resource these days is the expanding data. Large organisations continuously produce data about their clients, consumers, and employees in real time. This data cannot be easily interpreted in its raw form, but after being processed and changed, it can be widely used for analytics. This improves a number of the aforementioned business entity's existential traits, including organisational management, market capabilities, and consumer feedback.Given the volume of data that a corporation generates, it is obvious that it will need a significant investment of money, time, talent, and resources to achieve the goal of in-house data processing, calibration, and storage. The goal is to overcome the obstacles businesses present for data-pipelining technology and get processed data directly at the conclusion of the data sync cycle. One sync cycle is the continuous fetching of data created or altered over the course of a given time frame, such as a fortnight or a month.
@artical{1182022ijcatr11081004,
Title = "Developing an ETL Pipeline for Data Analysis",
Journal ="International Journal of Computer Applications Technology and Research(IJCATR)",
Volume = "11",
Issue ="8",
Pages ="315 - 319",
Year = "2022",
Authors =" A S Prajwal Babu, Prof. Suma B"}
  • The paper proposes a efficient way to build ETL pipeline.
  • Compare two frameworks used for building ETL pipeline.
  • A detailed methodology of how to build the pipeline is provided.
  • Results of the framework is discussed.