This is your job:

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate should have extensive experience in Azure Databricks, Data Lakehouse concepts, Python, SQL, and a strong background in working with various data sources and platforms.

Azure Databricks:

Design, develop, and maintain data pipelines using Azure Databricks.
Optimize and tune Spark jobs for performance and scalability.
Implement Spark DataFrames and Pandas DataFrames for efficient data processin

Data Lakehouse Concept:

Implement and manage data lakehouse concepts for effective data storage and processing.
Work with Delta Tables and Delta Live Tables to maintain versioned and transactional data.

Azure Data Factory:

Design and implement data workflows using Azure Data Factory.
Handle data integration from various sources, both on-premises and in the cloud.

Data Sources:

Integrate data from diverse sources such as Azure Data Lake, Azure SQL, Azure Synapse, APIs, and other relevant sources.
Expertise in Copy Activity, Mapping Data Flow, and handling various data formats.

API Integration:

Experience with making API calls and processing JSON responses.
Ability to handle pagination and complex API interactions.

Azure Analysis Services (SSAS)

Experience with Azure SSAS for creating and managing tabular models.

Azure Key Vault and Data Lake Concepts:

Familiarity with Azure Key Vault for secure storage of sensitive information.
Knowledge of Azure Data Lake concepts and best practices.

Additional Skills:

The candidate should be proficient in SQL and understand the usage of functions and stored procedures for efficient data handling. He or she should possess a deep understanding of database design and normalization and have experience in database and query performance tuning and optimization.
Familiarity with dbt (Data Build Tool) for analytics engineering (optional but advantageous).

Your Qualifications:

Bachelor's or higher degree in Computer Science, Data Science, or related field.
Proven experience as a Data Engineer with a focus on Azure technologies.
Strong proficiency in Python, SQL, and Spark DataFrames.
Excellent understanding of data integration, data processing, and data modeling concepts.
Experience with cloud and on-premises data sources and platforms.

All applications will be treated in strict confidentiality

Please note that only shortlisted candidates will be invited to an interview

DIGITALL Nature jobs

Cloud Data Engineer