Job Summary
Synechron is seeking a skilled Python Spark Developer to design and optimize large-scale data pipelines and processing systems. The successful candidate will leverage expertise in Python and Apache Spark to build scalable, high-performance data workflows, supporting enterprise analytics, fraud detection, and real-time data applications. This role is instrumental in driving data architecture advancements, operational excellence, and delivering solutions aligned with business and technical standards.
Software Requirements
Required Skills:
3+ years of professional experience in Python development with a focus on data engineering and Big Data processing
Hands-on expertise with Apache Spark (preferably Spark 2.x or 3.x) in batch and streaming environments
Strong SQL skills with experience working with relational and distributed data systems (e.g., Hive, Snowflake, NoSQL databases)
Experience with data pipeline orchestration and management tools (e.g., Airflow, Jenkins, Git)
Solid understanding of software engineering principles, clean code practices, and design patterns
Familiarity with system design for scalable, data-intensive applications
Preferred Skills:
Exposure to cloud data platforms such as Snowflake, Databricks, AWS Glue, or GCP DataProc
Experience working with Kafka, Redis, or similar messaging systems
Knowledge of observability tools like OpenTelemetry, Grafana, Loki, Tempo
Understanding of containerization using Docker, orchestration with Kubernetes, and GitOps workflows
Overall Responsibilities
Design, develop, and optimize scalable data pipelines and workflows utilizing Python and Apache Spark
Build high-performance data processing applications emphasizing pushdown optimization, partitioning, clustering, and streaming
Integrate modern data platforms and tools into existing enterprise architectures for improved data accessibility and security
Engineer feature pipelines to support real-time fraud detection and other critical analytics systems
Define data models and processing strategies aligned with distributed architecture principles to ensure scalability and consistency
Develop solutions that are production-ready, maintainable, and feature observability and operational monitoring capabilities
Adhere to clean code standards, SOLID principles, and architecture best practices to enable extensibility and robustness
Participate in code reviews, testing, deployment, and performance tuning activities
Contribute to architectural governance, innovation initiatives, and continuous improvement efforts
Technical Skills (By Category)
Programming Languages:
Essential: Python (version 3.7+)
Preferred: Scala, Java for integration purposes
Frameworks & Libraries:
Essential: Apache Spark, Spark Streaming, Spark SQL, PySpark
Preferred: Kafka clients, Flink, or other streaming frameworks
Data & Databases:
Essential: SQL (PostgreSQL, MySQL), Spark dataframes, Hive, or similar distributed storage
Preferred: NoSQL databases (MongoDB, Cassandra), Data Lake architectures
Cloud & Infrastructure:
Preferred: Cloud platforms such as Snowflake, Databricks, AWS, or GCP
Experience with containerization: Docker, Kubernetes, Helm
Infrastructure automation: Terraform, CloudFormation (desirable)
DevOps & Monitoring:
Essential: CI/CD (Jenkins, GitHub Actions), observability tools (OpenTelemetry, Prometheus, Grafana)
Preferred: Log aggregation tools like Loki, Tempo; metrics collection
Experience Requirements
3+ years of hands-on experience developing data pipelines in Python with Apache Spark
Proven experience designing scalable, reliable ETL/ELT workflows in enterprise environments
Demonstrated ability to optimize Spark jobs for performance in batch and streaming scenarios
Experience working in distributed system architectures with a focus on data security and compliance
Background in financial, fraud detection, or data-intensive environments is preferred; relevant industry experience is desirable
Proven ability to collaborate across cross-functional teams and influence technical decision-making
Day-to-Day Activities
Develop and maintain large-scale data pipelines supporting enterprise analytics and real-time applications
Optimize Spark jobs and workflows for throughput, latency, and resource utilization
Implement pushdown optimizations, partitioning strategies, and clustering techniques to improve data processing efficiency
Collaborate with data architects, platform teams, and stakeholders to evaluate new tools and platforms for data solutions
Troubleshoot technical issues, resolve data pipeline failures, and improve system observability
Conduct code reviews and participate in agile planning, deployment, and operational activities
Document architecture, processes, and best practices to facilitate knowledge sharing and operational excellence
Stay current with industry trends, emerging tools, and best practices in big data engineering
Qualifications
Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Science, or related field
Additional certifications in Big Data, Spark, or cloud data services are a plus
Extensive hands-on experience developing large-scale data pipelines and processing solutions with Python and Apache Spark
Professional Competencies
Strong analytical and problem-solving skills for complex data workflows
Excellent collaboration and communication skills with technical and non-technical stakeholders
Ability to lead initiatives, influence best practices, and mentor junior engineers
Adaptability to evolving technologies and organizational needs
Focus on operational excellence, observability, and sustained performance
Commitment to continuous learning and process improvement
SYNECHRON’S DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.
All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.
Candidate Application Notice
