Position Summary:
We are seeking a passionate and experienced data professional with a strong understanding of DevOps principles and practices to join our team. You will play a key role in building, deploying, and maintaining our data infrastructure, ensuring data quality, consistency, and performance, while also contributing to the automation and optimization of our data pipelines.
Key Responsibilities:
Data Engineering:
- Data Pipeline Development: Design, develop, and maintain data pipelines using tools like AWS Glue, Kinesis, S3, and Redshift, to efficiently load, transform, and analyze data.
- Data Modeling & Transformations: Design and implement robust data models that effectively capture and represent our business data.
- Data Integration: Integrate data from various sources, including SQL databases, cloud platforms, and APIs, into our data platform.
- Data Quality & Governance: Establish and enforce data governance policies and procedures, ensuring data quality, consistency, and compliance with regulations.
- Data Security: Implement and maintain data security measures, ensuring the confidentiality, integrity, and availability of our data.
DevOps:
- Infrastructure Management: Manage and maintain our cloud infrastructure, including AWS services like EC2, S3, and Redshift.
- Deployment & Automation: Automate the deployment and configuration of data pipelines and infrastructure using tools like Terraform, Ansible, or CloudFormation.
- Monitoring & Optimization: Monitor the performance of data pipelines and infrastructure, identifying and resolving bottlenecks and performance issues.
- Continuous Integration & Delivery (CI/CD): Implement CI/CD pipelines to automate the build, test, and deployment of data solutions.
- Collaboration & Communication: Effectively collaborate with Data Architects, Data Analysts, Power Developers, and other stakeholders to ensure alignment and successful project delivery.
Qualifications:
- Experience: 7+ years of experience in a data engineering role, with a strong understanding of DevOps principles and practices.
- Technical Skills: Proficiency in SQL, data warehousing concepts, and data modeling methodologies. Experience with AWS technologies (Redshift, S3, EMR, etc.) is essential.
- DevOps Tools: Experience with automation tools like Terraform, Ansible, or CloudFormation, and CI/CD pipelines.
- Cloud Experience: Extensive experience in designing and implementing data solutions in the AWS cloud environment.
- Data Governance & Security: Strong understanding of data governance principles and experience implementing data security measures.
- Communication Skills: Excellent written and verbal communication skills with the ability to clearly explain technical concepts to both technical and non-technical audiences.
Desired Skills (Not Required):
- Data Visualization: Experience with BI tools like PowerBI, Tableau, Looker, or Sigma.
- Agile Methodology: Experience with Agile development methodologies.
- ELT Expertise: Solid understanding of ELT processes and experience with DBT Cloud.
