Senior Staff - IT Data Engineering and Analytics

Role overview

Senior Staff - IT Data Engineering and Analytics at StoneX India

Responsibilities

• You will embed with trading, operations, finance and risk teams to understand how they work, turn their questions into precise source-to-target mappings, and ship production Databricks pipelines on the StoneX Unified Data Platform in days and weeks, not quarters.

Required qualifications

• To land this role you will need: 6-9 years of hands-on data engineering, including 3+ years building production pipelines on Spark / Databricks. 2+ years in a business-facing delivery role (forward-deployed, consulting, solutions or embedded analytics engineering) where you owned outcomes with non-technical stakeholders.
• Proven ability to write source-to-target mappings and dimensional / medallion data models from ambiguous requirements.
• Expert Python, PySpark and SQL; strong Delta Lake, Unity Catalog and Databricks Workflows / Lakeflow experience on Microsoft Azure.
• Recent, demonstrable use of AI coding agents to deliver production work, with the judgment to verify, test and reject what they produce.
• Excellent verbal and written communication; confident running workshops and explaining trade-offs to senior business stakeholders.
• Thrives at pace: comfortable with shifting priorities, tight deadlines and overlap with US / UK business hours when needed.
• Bachelor's degree in Computer Science, Engineering, Mathematics or a related field, or equivalent demonstrable experience.
• Azure Data Engineer Associate or equivalent.
• Generative AI certifications (e.g., Databricks Generative AI Engineer Associate) are a plus.
• Working environment: Hybrid (as per RTO policy), Bengaluru.
• Embedded with business and technology teams across the US, UK, India and Poland; regular overlap with US / UK hours.
• Travel: occasional, for discovery workshops and go-lives.

Preferred qualifications and skills

• Capital markets, commodities or trade-lifecycle data experience (trades, positions, postings, balances, reference data).
• You have built a reusable framework or internal tool, ideally AI-enabled, that other engineers adopted.
• Streaming with Kafka / RedPanda, and APIs (OpenAPI / REST) as data sources.
• MDM (Profisee) and CRM (Microsoft Dynamics) data integration.
• Power BI semantic models or Databricks AI/BI (Genie) spaces that business users rely on.
• Time in a start-up, consulting firm or forward-deployed engineering team.
• Education / Certification
• Databricks Certified Data Engineer Professional (strongly preferred) or Associate.

Employment details

• Location: Pune [India]

Additional role details

• Job Purpose and Responsibilities Position Purpose: The Senior Data Engineer (Forward Deployed) is a hands-on engineer who works alongside the business, not behind a ticket queue.
• AI is a core part of how you work: you use it to analyse sources, draft mappings, generate and test code, and build reusable frameworks that make the whole Global Data Services team faster.
• The role suits an engineer who enjoys a live problem, a demanding stakeholder and a fast pace, and who can explain a technical trade-off as clearly to a trader as to an engineer.
• What 'Forward Deployed' means here: Embedded: you spend a large share of your time with business users - in their meetings, their tools and their data - and you own the outcome, not just the code.
• End to end: discovery → mapping → pipeline → validation → adoption, delivered by one accountable engineer.
• Fast and iterative: working increments in front of users early, with short feedback loops.
• Leverage: every engagement leaves something reusable behind (a framework, a template, an agent) so the next one is faster.
• Technology Ecosystem: Data platform: Databricks (Delta Lake, Unity Catalog, Lakeflow / Workflows, Databricks SQL), Python, PySpark, SQL, SQL Server Integration: RedPanda (Kafka), OpenAPI / REST APIs, Airflow, Profisee MDM, Microsoft Dynamics AI-assisted delivery: Claude Code, GitHub Copilot, Databricks Assistant and AI/BI Genie Engineering: medallion architecture (Bronze / Silver / Gold), data warehousing, CI/CD (GitHub Actions, Jenkins), Docker, Kubernetes Consumption and cloud: Power BI, React.js; Microsoft Azure Primary duties will include: Discover and translate: run discovery sessions with business stakeholders; turn loosely defined asks into clear requirements, data contracts and acceptance criteria; play your understanding back in plain language.
• Map: produce rigorous source-to-target mappings (STTM) covering field-level lineage, transformation logic, keys, defaults, SCD handling and data quality rules, signed off by engineers and business users alike.
• Build: design and build batch and streaming pipelines on Databricks (PySpark, SQL, Delta Lake, Unity Catalog, Lakeflow) that follow the medallion patterns of the Unified Data Platform.
• Accelerate with AI: use AI coding agents every day to profile sources, draft STTMs, and generate pipelines and tests; build reusable AI-enabled frameworks such as metadata-driven ingestion, STTM-to-code generation and automated data quality and reconciliation checks.
• Validate and land: reconcile outputs with the business, own data quality and monitoring, and drive adoption in Power BI and downstream systems.
• Operate: own production reliability for what you build (CI/CD, alerting, incident response, root-cause analysis), including cost and performance tuning on shared Databricks compute.
• Communicate: give crisp written and verbal updates to business and technology leaders, record decisions in Confluence and Jira, and reset expectations early when scope, dates or data change.
• Raise the bar: coach engineers on business engagement and AI-assisted delivery; review others' AI-generated code for correctness, security and performance.