Azure Data Engineer - Assistant Manager

Role overview

Azure Data Engineer - Assistant Manager at KPMG

Required qualifications

• Mandatory Skills Bachelor’s or higher degree in Computer Science or a related discipline; or equivalent (minimum 4+ years work experience).
• At least 6+ years of consulting or client service delivery experience on Azure Microsoft data engineering .
• At least 4+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Synapse/Azure Databricks, Microsoft Fabric Hands-on experience implementing data ingestion, ETL and data processing using Azure services: Fabric, onelake, ADLS, Azure Data Factory, Azure Functions, services in Microsoft Fabric etc.
• Minimum of 5+ years of hands-on experience in Azure and Big Data technologies such as Fabric, databricks, Python, SQL, ADLS/Blob, pyspark/SparkSQL.
• Minimum of 3+ years of RDBMS experience Experience in using Big Data File Formats and compression techniques.

Preferred qualifications and skills

• Technical Leadership & Demo Delivery: Provide technical leadership to the data engineering team, guiding the design and implementation of data solutions.
• Deliver compelling and clear demonstrations of data engineering solutions to stakeholders and clients, showcasing functionality and business value.
• Communicate fluently in English with clients, translating complex technical concepts into business-friendly language during presentations, meetings, and consultations.
• ETL Development & Deployment on Azure Cloud: Design, develop, and deploy robust ETL (Extract, Transform, Load) pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, Azure Notebooks, Azure Functions, and other Azure services.
• Ensure scalable, efficient, and secure data integration workflows that meet business requirements.
• Microsoft Fabric Expertise: Develop and deploy ETL solutions using Microsoft Fabric, leveraging its capabilities for end-to-end data integration and analytics.
• Build and maintain medallion architecture (Bronze, Silver, Gold layers) in Microsoft Fabric to ensure data quality, scalability, and performance.
• Implement metadata-driven ingestion frameworks to streamline data ingestion processes.
• Design and develop data quality frameworks to validate, cleanse, and monitor data integrity.
• Perform advanced data transformations, including Slowly Changing Dimensions (SCD Type 1 and Type 2), using Fabric Notebooks or Databricks.
• Generative AI & Microsoft Copilot: Exposure to Microsoft Copilot and Generative AI fundamentals, with the ability to identify and implement industry-relevant use cases (e.g., automated insights generation, data summarization, or predictive analytics).
• Leverage Gen AI tools to enhance data engineering workflows, such as automating data pipeline creation or improving data quality checks.
• Microsoft Certifications: Hold relevant role-based Microsoft certifications, such as: DP-600: Implementing Analytics Solutions Using Microsoft Fabric.
• DP-203: Data Engineering on Microsoft Azure.
• DP-900: Microsoft Azure Data Fundamentals.
• AI-102: Designing and Implementing a Microsoft Azure AI Solution.
• AI-900: Microsoft Azure AI Fundamentals.
• Additional certifications in related areas (e.g., PL-300 for Power BI) are a plus.
• Power BI & Business Insights: Proficient in Microsoft Power BI for designing and developing interactive reports and dashboards.
• Generate actionable insights for business users by analyzing data trends and patterns, ensuring alignment with business objectives.
• Collaborate with stakeholders to define KPIs and visualize data effectively.
• Azure Security & Access Management: Strong knowledge of Azure Role-Based Access Control (RBAC) and Identity and Access Management (IAM).
• Implement and manage access controls, ensuring data security and compliance with organizational and regulatory standards on Azure Cloud.
• Alignment with Microsoft Vision: Stay updated with Microsoft’s vision, roadmap, and latest tools/technologies in the data and AI ecosystem (e.g., Azure Synapse, Fabric, Power Platform, and AI advancements).
• Proactively explore and adopt emerging tools to enhance team capabilities and deliver cutting-edge solutions.
• Additional Responsibilities & Skills: Team Collaboration: Mentor junior engineers, fostering a culture of continuous learning and knowledge sharing within the team.
• Project Management: Oversee data engineering projects, ensuring timely delivery within scope and budget, while coordinating with cross-functional teams.
• Data Governance: Implement data governance practices, including data lineage, cataloging, and compliance with standards like GDPR or CCPA.
• Performance Optimization: Optimize ETL pipelines and data workflows for performance, cost-efficiency, and scalability on Azure platforms.
• Cross-Platform Knowledge: Familiarity with integrating Azure services with other cloud platforms (e.g., AWS, GCP) or hybrid environments is an added advantage.
• Soft Skills & Client Engagement: Exceptional problem-solving skills with a proactive approach to addressing technical challenges.
• Strong interpersonal skills to build trusted relationships with clients and stakeholders.
• Ability to manage multiple priorities in a fast-paced environment, ensuring high-quality deliverables.

Employment details

• Location: Bangalore [Karnataka] [India]

Additional role details

• Experience working with Developer tools such as Azure DevOps, Visual Studio Team Server, Git, etc.
• Primary Roles and Responsibilities An Azure Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using Azure cloud services.
• This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures.
• The Azure Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency.