Data Quality Engineer

<div class="gflanj gflg8a gflh8y gflgxu gflha6 gflhbm gflgm7 gfllgi"> <h2 class="gflkzy gflkxx gfljx0 gflkxc gflkyo gflkyu gflkyx gflkzf gflfc9 gflkzw gflk7l"><strong>About the job</strong></h2> </div> <p><strong>Company:</strong> Finarb Analytics Consulting Pvt. Ltd.</p> <p><strong>Job Title:</strong> Data Quality Engineer – SQL / ETL / Microsoft Fabric</p> <p><strong>Location:</strong> Hyderabad or Kolkata, India</p> <p><strong>Experience:</strong> 3–6 Years (Suggested)</p> <p><strong>Employment Type:</strong> Full-Time</p> <p><strong>Application Link:</strong> https://zurl.to/KmZM?source=CareerSite</p> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">About Finarb</span></strong></p> <p>Finarb Analytics Consulting is a data, analytics, and technology consulting firm helping organizations transform complex data into actionable business insights through advanced analytics, business intelligence, artificial intelligence, and modern data platforms.</p> <p><strong>Website:</strong> https://www.finarb.ai</p> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">About the Role</span></strong></p> <p>We are seeking a detail-oriented, technically strong <strong>Data Quality Engineer</strong> to ensure the accuracy, completeness, consistency, and reliability of enterprise data across multiple platforms.</p> <p>The ideal candidate will have strong hands-on experience with <strong>SQL, data validation, ETL testing, data reconciliation, and data quality automation</strong>, along with a good understanding of data warehousing and modern cloud data platforms.</p> <p>This role will focus on validating data flowing through SQL Server, Azure Data Factory, and Microsoft Fabric environments, with particular emphasis on ensuring data integrity during platform migration and ongoing production operations.</p> <p>The successful candidate will collaborate closely with data engineers, BI developers, senior data quality professionals, and data architects to identify data discrepancies, investigate root causes, automate validation processes, and establish reliable data quality controls.</p> <p>We are looking for engineers who can independently analyze complex datasets, understand transformation logic, and develop scalable, repeatable data quality testing solutions.</p> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Key Responsibilities</span></strong></p> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">1. Data Quality Validation and Testing</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Develop and execute comprehensive data quality checks across enterprise data platforms.</li> <li>Validate data accuracy, completeness, consistency, uniqueness, and integrity.</li> <li>Perform source-to-target validation across ingestion, staging, transformation, and reporting layers.</li> <li>Identify missing records, duplicates, null values, invalid mappings, and inconsistent business calculations.</li> <li>Validate data against defined business rules and transformation specifications.</li> <li>Investigate discrepancies and collaborate with engineering teams to resolve issues.</li> <li>Develop reusable test cases and maintain data quality documentation.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">2. SQL-Based Data Testing and Reconciliation</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Develop complex SQL queries to validate data across source and target systems.</li> <li>Perform record-count reconciliation, aggregate reconciliation, and detailed field-level comparisons.</li> <li>Validate joins, transformations, aggregations, and business calculations.</li> <li>Test stored procedures, SQL transformations, views, and data warehouse tables.</li> <li>Analyze large datasets to identify inconsistencies and anomalies.</li> <li>Investigate root causes of data mismatches and document findings.</li> <li>Optimize validation queries for performance and scalability.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">3. ETL / ELT Pipeline Testing</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Test data ingestion and transformation pipelines built using Azure Data Factory and Microsoft Fabric.</li> <li>Validate pipeline execution results, data transformations, and target data structures.</li> <li>Verify incremental loads, full loads, upserts, and change data capture processes.</li> <li>Test data handling for missing values, duplicate records, invalid formats, and unexpected source changes.</li> <li>Validate pipeline error handling, retry mechanisms, and exception processing.</li> <li>Perform regression testing following ETL enhancements and production releases.</li> <li>Collaborate with data engineers to investigate and resolve pipeline-related data issues.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">4. Microsoft Fabric Migration Validation</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Support data quality assurance during migration from SQL Server and Azure Data Factory to Microsoft Fabric.</li> <li>Develop source-to-target reconciliation processes for migrated datasets.</li> <li>Validate data transformations across Fabric Lakehouse, Warehouse, and data pipelines.</li> <li>Perform row-count, checksum, aggregate, and field-level reconciliation where appropriate.</li> <li>Verify historical data completeness and incremental data consistency.</li> <li>Identify discrepancies between legacy and Fabric-based processing.</li> <li>Support parallel-run validation and post-migration stabilization.</li> <li>Maintain migration validation reports and evidence of data quality checks.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">5. Data Quality Automation</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Develop reusable SQL-based validation scripts and automated reconciliation processes.</li> <li>Build automated data quality checks using Python, SQL, or other appropriate tools.</li> <li>Integrate validation processes into ETL/ELT workflows where applicable.</li> <li>Automate recurring checks for data completeness, uniqueness, referential integrity, and business rule compliance.</li> <li>Implement exception reporting and automated identification of data quality failures.</li> <li>Support continuous improvement of testing efficiency and coverage.</li> <li>Reduce manual validation effort through repeatable and maintainable testing frameworks.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">6. Production Data Quality Monitoring</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Monitor data quality across production data pipelines and reporting datasets.</li> <li>Investigate data-related incidents reported by business users or BI teams.</li> <li>Perform root cause analysis for recurring data quality defects.</li> <li>Track data quality issues and coordinate resolution with engineering teams.</li> <li>Validate corrective fixes and perform regression testing.</li> <li>Support production release validation and ongoing operational checks.</li> <li>Maintain traceability of data quality defects and their resolutions.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">7. Collaboration and Documentation</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Work closely with data engineers, Power BI developers, and data architects.</li> <li>Understand business rules, data transformations, and reporting requirements.</li> <li>Translate data specifications into comprehensive validation scenarios.</li> <li>Document test cases, reconciliation logic, test results, and identified defects.</li> <li>Participate in requirement reviews, technical discussions, and defect triage.</li> <li>Contribute to establishing standardized data quality processes and best practices.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Required Technical Skills</span></strong></p> <p>Technology / SkillExpected ProficiencySQL / T-SQLAdvanced querying and validationData Quality TestingAccuracy, completeness, consistency, uniquenessETL / ELT TestingSource-to-target and transformation validationData ReconciliationRecord-level, aggregate, and field-level comparisonsSQL ServerStrong hands-on experienceData WarehousingFact/dimension tables and transformation logicAzure Data FactoryUnderstanding of pipelines and data flowsData Quality AutomationSQL scripts and automated validationDefect InvestigationRoot cause analysis and issue trackingMicrosoft FabricBasic knowledge or practical exposure</p> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Required Qualifications</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline.</li> <li>3–6 years of experience in data quality engineering, ETL testing, data validation, or data warehouse testing.</li> <li>Strong hands-on SQL skills, including complex joins, CTEs, aggregations, window functions, and subqueries.</li> <li>Proven experience performing source-to-target data validation and reconciliation.</li> <li>Experience testing ETL/ELT pipelines and complex data transformations.</li> <li>Good understanding of relational databases and data warehousing concepts.</li> <li>Experience identifying data discrepancies and independently investigating root causes.</li> <li>Familiarity with SQL Server and Azure-based data environments.</li> <li>Experience developing and maintaining test cases, validation scripts, and defect reports.</li> <li>Strong analytical skills and attention to detail.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Preferred / Good-to-Have Skills</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Experience with Microsoft Fabric, including Lakehouse, Warehouse, OneLake, and Fabric Data Factory.</li> <li>Knowledge of Python or PySpark for automated data validation.</li> <li>Familiarity with data quality frameworks such as Great Expectations, Soda, or equivalent tools.</li> <li>Experience with Azure Data Lake Storage Gen2.</li> <li>Understanding of Delta Lake and medallion architecture.</li> <li>Experience validating large-scale cloud data migration projects.</li> <li>Familiarity with Power BI reports and semantic model validation.</li> <li>Knowledge of Git, Azure DevOps, and CI/CD processes.</li> <li>Exposure to data profiling, data observability, and data governance concepts.</li> <li>Familiarity with Microsoft Purview or equivalent metadata management tools.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Key Competencies</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Strong analytical thinking and problem-solving capabilities.</li> <li>Excellent attention to detail and data accuracy.</li> <li>Ability to independently investigate complex data discrepancies.</li> <li>Strong understanding of business rules and transformation logic.</li> <li>Ability to communicate technical defects clearly and effectively.</li> <li>Collaborative approach to working with engineering and BI teams.</li> <li>Ownership of testing deliverables and data quality outcomes.</li> <li>Ability to manage multiple testing priorities in a dynamic environment.</li> </ul> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">What Success Looks Like</span></strong></p> <p><strong>The successful candidate will be expected to:</strong></p> <ol class="gflkrc gflkz4 gfllgi"> <li>Ensure high levels of data accuracy, completeness, and consistency across enterprise datasets.</li> <li>Identify and resolve data quality issues before they impact downstream reporting.</li> <li>Develop reusable SQL-based reconciliation and validation processes.</li> <li>Automate recurring data quality checks to improve testing efficiency.</li> <li>Validate data integrity throughout Microsoft Fabric migration activities.</li> <li>Reduce production data discrepancies and recurring data quality defects.</li> <li>Maintain clear documentation and traceability of data quality testing.</li> <li>Contribute to establishing reliable, scalable data quality practices across the organization.</li> </ol> <p class="gflkzy gflkxo gflkxc gflkyo gflkyr gflky0 gflkzf gflfc9 gflkzw gflk7l gflk7p"><strong><span class="gfllfj gflam3 gfliiy gfllgi" data-testid="expandable-text-box">Why Join Finarb?</span></strong></p> <ul class="gflkrc gflkz5 gfllgi"> <li>Work on enterprise-scale data engineering and analytics initiatives.</li> <li>Gain exposure to Microsoft Fabric and modern cloud data platforms.</li> <li>Participate in complex data migration and modernization programs.</li> <li>Collaborate with experienced data engineers, architects, and BI professionals.</li> <li>Develop expertise in data quality automation and advanced data validation.</li> <li>Contribute to delivering reliable and trusted enterprise data solutions.</li> </ul>