Must-Have Skills: 6–9 years of experience with PySpark, Apache Spark, Flink, NiFi, Kafka, and other data engineering technologies. Strong proficiency in SQL, RDBMS, and Hadoop technologies. Familiarity with the Cloudera Tech Stack, including Hive, Impala, and HDFS. Good-to-Have Skills: Hands-on experience with Flink and Kudu streaming. Knowledge of Informatica workflow migration. Automation of workflows using CI/CD pipelines. Experience with Airflow or similar workflow engines. BFSI industry experience, especially with cloud technologies (Azure, AWS, Google Cloud). Key Responsibilities: Develop and optimize Spark pipelines for batch and streaming data. Design and implement ETL pipelines using PySpark and other tools. Migrate existing implementations to Spark-based solutions. Work with programming languages like Scala, Java, and Python. Utilize tools like Kafka and Jupyter Notebooks for streaming and development.
Perks & Benefits: Competitive salary package Career growth opportunities Comprehensive health insurance Hybrid work model Performance-based incentives Learning and development programs
