DE&A -Data Engineering Apache

Design, develop, and implement scalable, resilient, and high-performance data pipelines and data architecturesfor enterprise data platforms. · Build and optimize ETL/ELT pipelines supporting batch, near-real-time, and streaming data processing requirements. · Design and implement enterprise data ingestion and integration workflows using Apache NiFi, including data routing, transformation, orchestration, monitoring, and integration across heterogeneous data sources and platforms. · Develop distributed data processing solutions using Apache Spark with Scala and/or Python (PySpark). · Design and implement data solutions across relational database management systems (RDBMS) and modern data lakehouse technologies such as Apache Iceberg. · Build and optimize federated and distributed SQL capabilities using Starburst and Trino query engines. · Develop event-driven and real-time data processing solutions using Apache Kafka and Spark Streaming / Structured Streaming. · Lead end-to-end delivery of enterprise-scale data engineering initiatives involving multiple engineering, product, architecture, and business teams. · Strengthen platform reliability through automation, CI/CD, testing, monitoring, observability, documentation, and operational excellence. · Improve data quality, performance, scalability, resiliency, and cost efficiency across data platforms and pipelines. · Support enterprise data governance by maintaining metadata, technical and business documentation, data lineage, data quality controls, and secure access to data assets. · Partner with security and governance teams to ensure data solutions meet enterprise requirements for privacy, security, access control, compliance, and data lifecycle management. · Troubleshoot complex production issues, perform root-cause analysis, and implement sustainable solutions to improve platform stability. · Collaborate effectively with stakeholders across engineering, architecture, analytics, AI/ML, governance, product, and business organizations. We are seeking an experienced Data Engineer to join our Data Management team and contribute to the design, development, and evolution of enterprise-scale data platforms and capabilities. In this role, you will be responsible for the design, development, and implementation of scalable data pipelines and data architecture that enable reliable, secure, governed, and high-performance access to enterprise data. You will work with modern data engineering technologies including Apache NIFI, ETL/ELT frameworks, Apache Spark, Scala/Python, RDBMS, Apache Iceberg, Starburst/Trino, Kafka, and Spark Streaming.