Follow Us On:
Facebook-f Linkedin-in Youtube Instagram
Online Courses

Azure Data Engineering

  • Home
  • Azure Data Engineering
Data Engineering and Azure Cloud
Download Course Brochure
  • What is On-prem?
  • Why do we go for cloud? What is cloud?
  • Difference between On Prem vs Azure Cloud?
  • Setting up Azure subscription
  • Overview of Azure services for data engineering
  • IAAS, PAAS, SAAS services
  • Resource groups, Virtual networks, Virtual Machines,
  • Storage accounts, blob storage, Azure data lake gen2, key vault, logic apps, azure monitor.
  • Introduction to Azure Data Factory (ADF).
  • ADF architecture and components.
  • Integration runtimes
  • Linked services.
  • Data sets
  • Pipelines
  • Activities (Lookup, foreach, get metadata,
  • Copy, Stored Procedure, web switch)
  • Triggers (Schedule, Thumbling window, Event based)
  • What is RDBMS?
  • Types of DBMS
  • Introduction to Azure SQL & Creation
  • SSMS download and setup into local.
  • Overview of servers, databases, Tables,
  • Constraints, normalization, operators, data types, Queries, joins, stored procedures, windows functions, other important functions, temp tables, common table expression, performance tuning.
  • Advanced SQL Concepts.
  • Azure SQL Database and migration strategies.
  • Scenario 03: Copy Activity Behavior
  • Scenario 04: Validate copied data between source and sink in ADF
  • Scenario 05: Load data from on premise SQL server to Azure SQL DB in Azure Data Factory
  • Scenario 06: Full load and
  • Scenario 07: Incremental load
  • Scenario 08: How to load REST API data into Azure Cloud by using ADF
  • Scenario 09: Copy Data from on premise to Azure SQL DW with polybase_with Bulk insert
  • Scenario 10: Copy Data from on premise to Azure SQL DW with polybase_with Bulk insert
  • Data Flows Overview
  • Types of Transformations
  • What are Azure data bricks & purpose & advantages
  • What is Pyspark
  • Diff types of formats
  • Types of data
  • Real time streaming data handling vs batch processing data handling
  • Difference between Hadoop MapReduce and bricks spark
  • Clusters & types, DBFS, Unity catalog
  • Data frames vs RDD
  • Pyspark syntax
  • Db utilities & widgets
  • How to read the data from Azure blob
  • How to read the data from ADLS gen2
  • Mount point purpose
  • Create secrete scope
  • How to call one notebook to another notebook
  • How to Integrate notebook into Azure data factory piplines
  • Repo setups in Azure databricks
  • How to run the jobs
  • Python Introduction, Installation and setup
  • Python Data Types & Deep dive into String Data
  • Deep dive into python collection list and tuple and dict
  • Python Functions and Arguments, Lambda Functions
  • Python Modules & Packages
  • Python Flow Control and Python Logging Module
  • Python File Handling and Exception Handling
  • Why snowflake
  • Snowflake trial account creation
  • Snowflake architecture
  • Snowflake virtual warehouses
  • Snowflake micro partitioning & data clustering
  • Snowflake data loading and transformation while loading.
  • Snowflake external stages – working sessions.
  • Snowflake how to login to snow sql
  • Internal stages & snow sql
  • Copy command options.
  • Loading semi structured data
  • Snowake loading data from azure
  • Caching
  • Zero copy cloning
  • Table types
  • External tables
  • RBAC
  • Roles & users
  • Dynamic data masking
  • Data sharing
  • User dened functions & stored procedures
  • Spark SQL Introduction & Different ways to create DF
  • Spark SQL Basics
  • Joins in Spark SQL & Joins scenarios
  • Spark SQL Functions
  • Implement SCD Type1 & Apache Spark Databricks Delta
  • Delta Lake in Azure Databricks
  • Azure Data Factory Integration with Azure Databricks
  • Best practices and optimization.
  • Monitoring and troubleshooting.
  • Azure devops Introduction.
  • Boards Agile and Scrum, kanban process
  • Repos
  • Branching strategies, pull requests, approvals
  • CI and CD process in Azure devops
  • ADF and ADE code setup into classic pipelines under azure devops
  • Mid-term project incorporating ADF,ADB,SQL,and Python.
  • Advanced project incorporating Snowflake, Synapese Analytics,and optimization.
  • Work on a comprehensive end-to-end data engineering project.
  • Project completion,presentation,and feedback.
AZURE DATA ENGINEERING COURSE
(Data Factory + Databricks with PySpark + Synapse Analytics)

WEEK – 12:

Cert ification preparation:
  • Conducting workshops for specifc technologies and tools.

Continuous Learning and Updates:
  • Implementation of industry best practices for data engineering.
  • Real-World case studies.

Guest Lectures and Workshops:
  • Inviting industry experts for guest lecture.
  • Conducting workshops for specic technologies and tools.

Continuous Learning and Updates:
  • Encourage continuous learning by staying updated on new features and services in Azure.

All Courses

Have Any Query?

Get A Quote

    Experience our courses and enhance your skills

    User can click on get started by entering there user details like currently we have

    Frequently Asked Question

    AI in data science automates complex tasks like pattern recognition and predictive modeling. Machine learning algorithms analyze vast datasets to uncover insights, make predictions, and optimize decision-making. AI enhances data processing efficiency, enables advanced analytics, and supports data-driven decision-making, revolutionizing the field of data science.

    In general programming, we have the data and the logic by using these two we create the answers. But in machine learning, we have the data and the answers and we let the machine learn the logic from them so, that the same logic can be used to answer the questions which will be faced in the future. Also, there are times when writing logic in codes is not possible so, at those times machine learning becomes a saviour and learns the logic itself

    "With our data science course, you'll master essential skills in Python, power BI and machine learning. Analyse data, create insightful visualizations, and apply statistical techniques. Gain hands-on experience with real-world projects, fostering critical problem-solving skills. Acquire the knowledge to excel in diverse data-driven roles and drive innovation in any industry.

    Python is known for its simplicity, readability, and versatility. It supports object-oriented, imperative, and functional programming styles. Its extensive standard library facilitates diverse tasks. Dynamic typing and automatic memory management enhance development speed. Python is widely used for web development, data science, machine learning, and automation.

    content 11