Job Title: API Developer
Location: Chennai, Bangalore, Gurugram
Employment Type: [Full-time / Hybrid ]
Experience Level: [Mid / Senior]
Start Date: Immediate
About the Role
We are seeking a highly skilled Data Engineer with expertise in Python and PySpark, and a strong background in designing, building, and maintaining scalable data pipelines in a cloud-native environment. This role will focus on building robust data solutions, enabling analytics, and ensuring data quality and integrity across systems.
Key Responsibilities
- Design, develop, and maintain data pipelines using Python and PySpark
- Build and manage services deployed on AWS, including hands-on experience with EKS, EMR, S3, and Lambda
- Develop and manage Docker images and ensure smooth deployments within Kubernetes (EKS) environments
- Develop and orchestrate Airflow DAGs for workflow management and automation
- Work with Kafka for data ingestion and real-time data streaming
- Write and optimize complex SQL queries for data extraction, transformation, and analysis
- Build and maintain APIs for data services and integrations (bonus)
- Implement robust data quality checks, reconciliation processes, and data controls to ensure accuracy and reliability of data pipelines
Requirements
- Strong proficiency in Python and PySpark
- Proven experience in AWS cloud services and distributed computing
- Experience with Docker, Kubernetes (EKS), and EMR
- Hands-on development experience with Apache Airflow
- Proficiency in SQL and working with large datasets
- Experience with Kafka for real-time data processing (preferred)
- Exposure to API development and integration (desirable)
- Background in implementing data validation, controls, and reconciliation checks is a plus
- Excellent problem-solving skills and attention to detail
- Strong communication skills and ability to work in a collaborative environment
Preferred Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
- Certification(s) in AWS or related cloud technologies
- Previous experience in a data-centric or analytics-driven organization
