Creating ETL workflows for Snowflake.
What you'll learn
- Build workflows for snowflake.
- Talend basics.
- Airflow basics.
- Building airflow Dags.
- Connect snowflake with python.
- Auditing snowflake commands.
- Capturing cost and performance metrics.
- Virtual machine with preconfigured airflow.
- End to End project to load Nyc traffic data.
- This Snowflake cloud database with ETL(Airflow+Python+Talend) course will use virtual box software.
- Disk space of ~30 Gb is required to run the virtual machine.
Through our previous Snowflake cloud database with ETL(Airflow+Python+Talend) course, Snowflake Masterclass [Real-time demos+Best practices+Labs], we learned the fundamentals of Snowflake, solved many assignments, and learned best practices for loading and unloading data.
Also, we carefully considered each snowflake feature, understanding how they operate underneath the hood. You realized that you could use Snowflake efficiently because of these discussions.
Snowflake's ETL workflow implementation was missing one key element, How to build and orchestrate ETL workflows. That is the topic of this Snowflake cloud database with ETL(Airflow+Python+Talend) course.
In this Snowflake cloud database with ETL(Airflow+Python+Talend) course, we are going to learn,
- Build workflows in Airflow.
- We will leverage Talend capabilities to build generic code to ingest data and process data in snowflake.
- We will build audit tables and record every command we fire on the snowflake. We will record the time consumed for each task and capture snowflake credits.
- Once we build the framework we will build a workflow to process and transform 250 + GB volume of NYC traffic data.
- At last, we will connect the Snowflake with python and write code to capture stats of data we loaded to the snowflake.
- you will also get access to preconfigured Jupyter notebook to run your python code on the Snowflake.
Please do not be anxious if you have never worked with Talend, Airflow, or Python – these platforms are very simple and I can teach you everything you need to know.
I am sure you will learn a lot from this journey. See you in the Snowflake cloud database with ETL(Airflow+Python+Talend) course!!
Who this course is for:
- Developers interested to know how to build workflows for snowflake.
Created by Pradeep H C
Last updated 7/2021
Size: 3.4 GB