The Office of Technology and Innovation (OTI) oversees all Citywide technology, privacy, cybersecurity, infrastructure, and telecommunications to ensure the security of, and enhance, City operations and service delivery to New York City's residents, businesses, employees, and visitors. As the City's technology and innovation leader, OTI is responsible for operating, maintaining and securing IT infrastructure and systems that touch every aspect of City life from public safety to human services, from education to economic development crossing the full spectrum of governmental operations.
At OTI, we offer great benefits, and the chance to work on projects that have a meaningful impact on millions of people. You'll have the opportunity to work with cutting-edge technology, transition our existing data infrastructure to a cloud-centric platform, and collaborate with other passionate professionals who share your drive and commitment to making a difference through data technology.
Join our dynamic DataOps team at OTI as a Senior Data Engineer! Work with talented professionals to build foundational solutions that provide core services to New York City's residents, businesses, and visitors. As a key player in defining our data strategy, you'll have the opportunity to develop innovative solutions that improve the quality of life for all New Yorkers.
As a successful candidate, you'll work with a wide range of datasets and build data assets for city agencies and divisions, including 311, Public Safety (911), Sanitation, Health, Buildings, Transportation, and more.
Responsibilities will include:
- Collaborate with stakeholders to define and understand data needs;
- Design and develop efficient data architectures that can support large-scale data processing and storage requirements;
- Develop and maintain data pipelines, data models, and ETL processes that align with business requirements, data quality standards, and industry best practices;
- Work closely with other data engineering teams to build and maintain reusable data pipelines and tools, enabling faster time-to-market for data-driven solutions;
- Monitor, troubleshoot, and optimize data pipelines and processes for performance, reliability, and scalability;
- Ensure the quality and integrity of various datasets across different platforms and data sources;
- Continuously evaluate and recommend emerging technologies and methodologies to improve data engineering processes, workflows, and performance;
- Mentor and guide junior data engineers on technical best practices, code reviews, and design patterns to ensure high-quality, scalable, and maintainable data engineering solutions;
- Perform special technology-related projects, as assigned.
Minimum Qualifications
- A baccalaureate degree from an accredited college including or supplemented by 24 credits in the field of voice and/or data telecommunications or in a pertinent scientific, technical, electronic or related area, and four years of satisfactory fulltime experience in the performance of analytical, planning, operational, technical, or administrative duties in a voice and/or data telecommunications or closely related electronics planning, management, and/or service organization, one year of which must have been in a highly specialized capacity and 18 months must have been in an executive, managerial, or administrative capacity or in the supervision of staff performing work in the voice and/or data telecommunications field; or
- An associate degree from an accredited college including or supplemented by 12 credits in the field of voice and/or data telecommunications or in a pertinent, scientific, technical, electronic or related area and five years of experience as described in "1" above; or
- Education and/or experience equivalent to "1" above. However, all candidates must have at least a four-year high school diploma or its educational equivalent and one year of the specialized experience as described in "1" above and must possess the 18 months of executive, managerial, administrative or supervisory experience as described in "1" above.
Preferred Skills
Who are you? - A passionate data engineer with a desire to make a positive impact in the public sector - Excited about tackling tough challenges that don't have a defined playbook - A highly motivated team player, self-starter, and problem solver - Familiar with cloud computing platforms (Azure, AWS, GCP, etc.) The preferred candidate should possess the following: - 5+ years of experience in data engineering with a focus on designing and building scalable data platforms using cloud technologies - Expert proficiency in SQL and one or more programming languages such as Python, Java, or Scala - Knowledge of cloud-based data platforms such as AWS, Azure, or GCP, including experience with cloud-based storage, compute, and data processing services - Experience designing and implementing ETL and data integration pipelines, and familiarity with data modeling concepts and database design principles - Excellent communication skills and the ability to collaborate effectively with cross-functional teams - Experience with Agile methodologies and working in an Agile team environment - Experience developing production-grade, large-scale data solutions using cloud technologies - Experience managing data orchestration at scale using tools such as Airflow and Dagster - Familiarity with version control systems (e.g., Git) and CI/CD principles - Experience developing dashboards and reports in applications such as Oracle Analytics Server (OAS), Microsoft Power BI, and Google Looker - Thorough experience with data integration tools such as Informatica Intelligent Cloud Services and Mulesoft.
55a Program
This position is also open to qualified persons with a disability who are eligible for the 55-a Program. Please indicate at the top of your resume and cover letter that you would like to be considered for the position through the 55-a Program.
Public Service Loan Forgiveness
As a prospective employee of the City of New York, you may be eligible for federal loan forgiveness programs and state repayment assistance programs. For more information, please visit the U.S. Department of Education’s website at https://studentaid.gov/pslf/
Residency Requirement
New York City residency is generally required within 90 days of appointment. However, City Employees in certain titles who have worked for the City for 2 continuous years may also be eligible to reside in Nassau, Suffolk, Putnam, Westchester, Rockland, or Orange County. To determine if the residency requirement applies to you, please discuss with the agency representative at the time of interview.
Additional Information
The City of New York is an inclusive equal opportunity employer committed to recruiting and retaining a diverse workforce and providing a work environment that is free from discrimination and harassment based upon any legally protected status or protected characteristic, including but not limited to an individual's sex, race, color, ethnicity, national origin, age, religion, disability, sexual orientation, veteran status, gender identity, or pregnancy.
