Senior Data Engineer

Senior Data Engineer Location - India (Coimbatore preferred) Work Mode - WFO preferred, hybrid or remote will also be considered for the right candidate Experience - 5+ years in Data Engineering, including hands-on cloud data platform delivery Employment Type - Full-Time ROLE OVERVIEW Vserve is looking for a Senior Data Engineer to design, build, and own scalable data platforms on Microsoft Azure ecosystems. This role will lead the development of Enterprise Data Warehousing, ETL/ELT pipelines, and analytics- ready data models that power reporting and decision-making for our retail, wholesale, and distribution clients. The ideal candidate is equally comfortable architecting cloud-native pipelines and partnering directly with business stakeholders to translate requirements into reliable, production-grade data solutions. KEY RESPONSIBILITIES • Design, build, and maintain scalable Enterprise Data Warehouse (EDW) solutions on Azure (ADF, Databricks, Azure SQL, Microsoft Fabric). • Develop and orchestrate end-to-end ETL/ELT pipelines using Apache Airflow, Azure Data Factory, and Databricks, ensuring reliability and minimal manual intervention. • Build and optimize PySpark transformations and Delta Lake pipelines to process large-scale datasets into curated, analytics-ready data products. • Design dimensional data models (star/snowflake schemas), fact and dimension tables, and data marts to support sales, inventory, demand, and profitability reporting. • Develop Power BI semantic models, datasets, and interactive dashboards enabling self-service analytics and KPI reporting for business stakeholders. • Implement data quality and validation frameworks, including reconciliation checks, exception reporting, and automated alerting. • Own data reconciliation between ERP/business systems (e.g., Microsoft Dynamics AX / D365 F&O, POS platforms) and the enterprise data warehouse. • Implement complex business logic for margin calculations, cost attribution, and promotional/discount allocation to support accurate financial reporting. • Build automated data ingestion frameworks using Python and REST APIs, and implement incremental loading strategies for efficient, near real-time reporting. • Tune and optimize SQL/PL-SQL queries and data workflows for performance across high-volume, regulated, or compliance-sensitive datasets. • Implement CI/CD for data pipelines (GitHub Actions, Git) to improve release consistency and reduce deployment effort. • Partner with business users, analysts, and SMEs to gather requirements and translate them into scalable, well- documented data engineering solutions. • Mentor junior data engineers and contribute to data architecture and engineering best practices across the team. CANDIDATE PROFILE — SKILLS & EXPERIENCE Must-Have • 5+ years of hands-on Data Engineering experience, with demonstrated ownership of production data pipelines end to end. • Strong experience with Microsoft Azure data services: Azure Data Factory (ADF), Databricks, Azure Data Lake, Azure SQL. • Hands-on expertise in Apache Spark / PySpark and Delta Lake for large-scale data processing. • Proven experience building and orchestrating pipelines with Apache Airflow or equivalent workflow orchestration tools. • Advanced SQL skills and solid Python (Pandas, NumPy) programming ability. • Strong grounding in dimensional data modeling (star/snowflake schemas, normalization, ER-diagram design). • Experience with Power BI (DAX, Power Query) and/or Microsoft Fabric for reporting and dashboard delivery. • Experience implementing data quality, validation, and reconciliation frameworks in a production environment. Good-to-Have • Exposure to Microsoft Dynamics AX / Dynamics 365 Finance & Operations (D365 F&O) or similar ERP/POS platforms (e.g., Aptos POS). • Experience with Informatica PowerCenter or other enterprise ETL tools. • Familiarity with MongoDB, PostgreSQL, or DynamoDB alongside relational/warehouse platforms. • Experience in retail, wholesale distribution, or financial services data domains. • Working knowledge of CI/CD practices (GitHub Actions, Git) and Agile delivery methodologies. • Prior experience mentoring engineers or leading a small data engineering pod. Education • Bachelor degree in Computer Science, Information Technology, or a related field is required. • Master degree is a plus. Requirements Azure, ADF, Databricks, PySpark, Python, SQL, Airflow, Delta Lake, ETL/ELT, Data Warehousing, Data Modeling, Power BI, DAX, Data Quality, REST API, Git/CI-CD, D365 F&O