Master ETL in cloud with AWS glue & Azure data factory lab
In this course, we would learn the following:
What you’ll learn
- Confidently work with AWS Serverless services to develop Data Catalogue, ETL, Analytics on a Data Lake.
- Build a serverless data lake on AWS using structured and unstructured data.
- By end of this course you will be able to develop pipeline in azure data factory.
- how to perform transformation in azure data factory.
- you will learn how to develop incremental load pipeline.
- you will be able to understand about databricks architecture and how to use.
- Industry level SCD type-1,type-2 projects.
- Failure handling and debug pipeline and learn about email alerts and all.
Course Content
- Introduction –> 2 lectures • 3min.
- DEMO PROJECT –> 2 lectures • 1hr 8min.
- AWS GLUE –> 11 lectures • 1hr 45min.
- AWS (ATHENA) –> 8 lectures • 41min.
- ADF (COPY & DELETE activities) –> 8 lectures • 41min.
- ADF ( SQL TO BLOB) –> 6 lectures • 29min.
- ADF (CSV file to SQL table data pipeline) –> 3 lectures • 17min.
- ADF (TRANSFORMATION) –> 7 lectures • 51min.
- ADF (INCREMENTAL LOAD) –> 3 lectures • 18min.
- ADF (Parameterization ) –> 4 lectures • 26min.
- ADF( SCD TYPE-1 & SCD TYPE 2) –> 6 lectures • 1hr 4min.
- ADF (FAILURE) –> 2 lectures • 9min.
- SQL –> 22 lectures • 1hr 56min.
Requirements
In this course, we would learn the following:
1) We will start with Basics on Serverless Computing .
2) We will learn Schema Discovery, ETL, Scheduling, and Tools integration using Serverless AWS Glue Engine built on Spark environment.
3) We will learn to develop a centralized Data Catalogue too using Serverless AWS Glue Engine.
Businesses have always wanted to manage less infrastructure and more solutions. Big data challenges are continuously challenging the infrastructure boundaries. Having Serverless Storage, Serverless ETL, Serverless Analytics, and Serverless Reporting, all on one cloud platform had sounded too good to be true for a very long time. But now its a reality on AWS platform. AWS is the only cloud provider that has all the native serverless components for a true Serverless Data Lake Analytics solution.
This course understands your time is important, and so the course is designed to be laser-sharp on lecture timings, where all the trivial details are kept at a minimum and focus is kept on core content for experienced AWS Developers / Architects / Administrators. By the end of this course, you can feel assured and confident that you are future-proof for the next change and disruption sweeping the cloud industry.
I am very passionate about AWS Serverless computing on Data and Analytics platform, and am covering A-to-Z of all the topics discussed in this course.
This course will target anyone who likes to learn azure data factory and databricks . This course will cover all ADF components from the Model and View layer. In this course I have selected 15+ real time industry level project learning which will cover all the topic which is necessary to learn azure data factory and databricks and will be able to work on industry . This is completely hands on learning tutorial where we will do practical’s based learning and by end of this course we will develop a complete ADF application together (step by step) and By the end of this course, you should be able to develop a complete ADF application by yourself.
If you will be able to practical’s with me you will be able to get complete picture of data factory
I have also included lessons on the storage blob Storage, amazon s3 , Azure SQL Database etc. Also, there are lessons on Azure Databricks. I have even included lessons on building reports using Power BI on the data processed by the Azure Data Factory data pipelines.