Role Summary:
As the Data Engineer Team Lead, you will be responsible for leading a team of data engineers to design, build, and maintain scalable data infrastructure and pipelines. You will collaborate closely with cross-functional teams to ensure data quality, optimize performance, and enable data-driven decision-making across the organization.
Key Responsibilities:
Team Leadership
- Lead and mentor a team of 5-8 data engineer
- Recruit, hire, and onboard new team members
- Conduct performance reviews and career development planning
- Foster a culture of engineering excellence and innovation
Technical Leadership
- Architect and oversee implementation of scalable data pipelines and ETL/ELT processes
- Define data engineering standards, best practices, and governance policies
- Lead technical design reviews and ensure code quality
- Drive adoption of modern data engineering tools and technologies
Project Management
- Partner with product, analytics, and engineering teams to understand data requirements
- Prioritize and plan data engineering roadmap and sprint activities
- Ensure timely delivery of data infrastructure projects
- Manage technical debt and system reliability
Strategic Initiatives
- Design and implement real-time and batch data processing systems
- Build self-service data platforms for internal stakeholders
- Establish data quality monitoring and alerting frameworks
- Optimize data infrastructure costs and performance
Required Qualifications:
- Strong expertise in SQL and Python/Scala/Java
- Extensive experience with big data technologies (Spark, Hadoop, Kafka)
- Proficiency with cloud platforms (AWS/GCP/Azure) and their data services
- Experience with workflow orchestration tools (Airflow, Dagster, Prefect)
- Knowledge of data modeling, data warehousing, and data lake architectures
- Familiarity with containerization and infrastructure as code (Docker, Kubernetes, Terraform)
- 3+ years leading technical teams
- Proven track record of building and scaling data engineering teams
- Experience with agile methodologies and project management
- Strong communication skills with ability to translate technical concepts to non-technical stakeholders
- Bachelor's degree in Computer Science, Engineering, or related field (Master's preferred)
Preferred Qualifications
- Experience with streaming architectures and real-time processing
- Knowledge of machine learning pipelines and MLOps
- Experience with data mesh or similar distributed data architectures
- Certifications in cloud platforms (AWS, GCP, Azure)
- Experience in [specific industry relevant to company]