Browse current jobs

DataOps Consultant, Lakehouse Support

Luxoft · Remote

Apply on employer website

Collected role description

Role overview

DataOps Consultant, Lakehouse Support at Luxoft

Project description We are hiring a Support Engineer who combines production support expertise with DevOps engineering capability and AI fluency.

Responsibilities

  • You will ensure operational stability while continuously improving reliability through automation, observability, and modern DevOps practices.
  • You will leverage AI tools daily to accelerate triage, automate runbooks, and proactively detect issues before they impact users.
  • Lead incident triage, diagnosis, and resolution with structured root cause analysis (RCA) and post-incident reviews.
  • Manage incident, problem, and change management processes aligned with ITIL best practices.
  • Coordinate with development teams, infrastructure, and vendors for escalation and resolution of complex production issues.
  • Maintain and improve production verification suites, canary checks, and reliability playbooks.
  • Manage cloud resource health, cost optimization, and security posture.
  • Support microservice architectures: service discovery, load balancing, scaling, and resilience patterns.
  • Build and maintain alerting rules, dashboards, and SLO/SLI tracking for critical services and data pipelines.
  • Build self-healing mechanisms and automated remediation where feasible.

Required qualifications

  • Responsibilities Perform deep operational diagnostics across data pipelines (batch/stream); Coordinate resolution across platform engineering, Data Engineers and data domain teams (Databricks, Snowflake, Palantir); Ensure correct usage of ITSM tools (e.g., ServiceNow) for incident tracking and lifecycle management; Monitor critical workflows using Dynatrace; Workflows orchestration using StoneBranch; Leverage metadata and documentation for troubleshooting and root cause analysis; Maintain clear ownership, escalation paths and communication during incidents; Reduce incident noise and recurring issues through structured problem management Skills Must have Production Support and Incident Management: Own L2/L3 production support across in-house applications (.NET/C#, Python), APIs, event-driven systems, and data platforms.

Preferred qualifications and skills

  • Nice to have Palantir (Foundry) — DQC setup in workshop, diagnostics of build failures and data pipelines Monte Carlo — data quality alert management and observability Azure DevOps — repository management, CI/CD Working knowledge of investment management data: portfolio terminology (maturity date, value date), trade lifecycle, market and risk data; familiarity with domain systems (Optima, Cardia) Understanding of data lineage, observability and metadata-driven operations concepts Exposure to enterprise-scale data platforms and batch orchestration / data workflow tools Other Languages English: C1 Advanced Seniority Senior

Employment details

  • Location: Remote India [India]

Application details

  • DevOps and CI/CD: Own and maintain CI/CD pipelines in Azure DevOps: build, test, deploy, and release automation for both application and data platform projects.
  • Application and Data Platform Support: Applications: Support .NET/C# backend services, REST/GraphQL APIs, event-driven systems (Solace, Kafka, RabbitMQ), and WPF/web front-ends.

Additional role details

  • This role spans the full D&E portfolio: in-house .NET/C# applications, APIs, event-driven systems, data platforms, and BI tools.
  • You will work directly alongside Software Engineers as the bridge between development and operations, owning incident resolution, CI/CD pipeline health, infrastructure automation, and production readiness.
  • We are an AI-first engineering team: you will use AI-powered observability, automated diagnostics, and intelligent incident management to deliver faster resolution and higher system reliability.
  • Infrastructure as Code: Terraform, Bicep, or ARM templates for provisioning and managing Azure cloud resources.
  • Container orchestration: Docker and Kubernetes (AKS) for deploying, scaling, and troubleshooting containerized services.
  • Environment management: provision, configure, and maintain development, staging, and production environments.
  • Implement and maintain deployment strategies (blue-green, canary, rolling) to minimize production risk.
  • Cloud and Infrastructure: Azure: Practical experience with App Services, AKS, Functions, Event Hubs, Storage, Key Vault, Azure SQL, and Azure Monitor.
  • Networking fundamentals: DNS, load balancers, firewalls, VNets, and private endpoints in Azure.
  • Observability and Monitoring: Implement and manage monitoring stacks: Dynatrace, Azure Monitor or equivalent.
  • Distributed tracing with OpenTelemetry: correlate trace IDs across services, diagnose latency bottlenecks, and validate signal fidelity.
  • Data observability: monitor data pipeline health, detect anomalies, latency issues, and data loss using platforms like Monte Carlo Data or equivalent.
  • Log aggregation and analysis for rapid incident diagnosis.
  • Data Platforms: Support ETL/ELT pipelines (PySpark, Spark), big data platforms (Databricks, Palantir Foundry), and data warehouses.
  • Databases: Advanced SQL for diagnosing data issues, validating transformations, and supporting migrations (SQL Server, MongoDB).
  • Scheduling and Orchestration: Manage batch workloads in StoneBranch, Airflow, or equivalent.
  • AI-First Engineering (How We Work) AI-powered incident triage: use AI tools to cluster alerts, suggest root causes, and recommend resolution steps from historical incident data.
  • Automated runbooks: build and maintain AI-assisted diagnostic scripts that accelerate common troubleshooting workflows.
  • AIOps: leverage AI/ML for anomaly detection, predictive alerting, and capacity forecasting across applications and data pipelines.
  • AI coding assistants (GitHub Copilot, Cursor) used daily for scripting, automation, configuration, and documentation.
  • Support AI-based applications and services deployed by D&E teams; understand model serving, inference pipelines, and their operational requirements.
  • Responsible AI: follow engineering playbooks for data handling (no sensitive data in prompts), security, and observability of AI-generated changes.
  • Scripting and Automation: Python scripting for operational automation, data validation, and diagnostic tooling.
  • PowerShell for Windows administration and Azure resource management.