We are seeking an experienced Data Engineer to join our team and help us build and maintain highly reliable data pipelines and data platforms. You will work on a wide range of data engineering tasks, from data discovery and profiling to data governance and compliance. The ideal candidate will have strong technical skills, particularly in Python, Java, SQL, and AWS Data and Analytics Tools.
Key Responsibilities:
Data Pipeline and Platform Development:
- Develop highly reliable data pipelines and data platforms with comprehensive test coverage under limited supervision.
- Ensure the solutions are scalable, secure, and aligned with industry best practices.
Technical Design and Enhancements:
- Contribute to the technical design for system enhancements and new features.
- Ensure designs meet business requirements and adhere to architectural principles.
CI/CD Pipelines and Release Management:
- Implement and manage CI/CD pipelines for automated deployment and release management on data platforms.
- Apply deployment mechanisms that ensure smooth and efficient release processes.
Technical Support and Collaboration:
- Provide technical assistance and guidance to other team members.
- Collaborate with stakeholders to deliver robust, secure, and highly available data solutions.
Peer Reviews and Continuous Improvement:
- Actively participate in peer reviews to maintain high code quality.
- Identify and drive opportunities for continuous improvement within the team and in the delivery of products.
Data Engineering Duties:
- Engage in core data engineering tasks including data design, preparation, transformation, and load processes.
- Perform data discovery, profiling, analysis, and impact assessments.
- Design, build, and maintain data solutions that integrate data from various sources, including third-party APIs and transactional databases.
- Ensure data governance, security, and compliance across all platforms.
- Create and maintain comprehensive data documentation and reporting.
Requirements
Here's a detailed job description based on the provided requirements:
Job Title: Data Engineer
Location: [Insert Location]
Job Type: Full-time
About the Role:
We are seeking an experienced Data Engineer to join our team and help us build and maintain highly reliable data pipelines and data platforms. You will work on a wide range of data engineering tasks, from data discovery and profiling to data governance and compliance. The ideal candidate will have strong technical skills, particularly in Python, Java, SQL, and AWS Data and Analytics Tools.
Key Responsibilities:
Data Pipeline and Platform Development:
- Develop highly reliable data pipelines and data platforms with comprehensive test coverage under limited supervision.
- Ensure the solutions are scalable, secure, and aligned with industry best practices.
Technical Design and Enhancements:
- Contribute to the technical design for system enhancements and new features.
- Ensure designs meet business requirements and adhere to architectural principles.
CI/CD Pipelines and Release Management:
- Implement and manage CI/CD pipelines for automated deployment and release management on data platforms.
- Apply deployment mechanisms that ensure smooth and efficient release processes.
Technical Support and Collaboration:
- Provide technical assistance and guidance to other team members.
- Collaborate with stakeholders to deliver robust, secure, and highly available data solutions.
Peer Reviews and Continuous Improvement:
- Actively participate in peer reviews to maintain high code quality.
- Identify and drive opportunities for continuous improvement within the team and in the delivery of products.
Data Engineering Duties:
- Engage in core data engineering tasks including data design, preparation, transformation, and load processes.
- Perform data discovery, profiling, analysis, and impact assessments.
- Design, build, and maintain data solutions that integrate data from various sources, including third-party APIs and transactional databases.
- Ensure data governance, security, and compliance across all platforms.
- Create and maintain comprehensive data documentation and reporting.
Technical Skills:
Required Qualifications:
- Bachelor’s degree in Computer Science, Data Science, or a related field.
- 5+ years of experience in data engineering or a related role.
- Strong experience with data pipeline and platform development.
- Proven experience in deploying and managing CI/CD pipelines.
- Solid understanding of data governance, security, and compliance requirements.
- Excellent problem-solving skills and attention to detail.
- Strong communication skills and ability to work collaboratively in a team environment.
