Senior Data Engineer.

Primesoft Enterprise · Hyderabad

Collected role description

<h2>Designation: Senior Data Engineer </h2> Experience : 6 to 10 years Qualification : BE/B tech, ME/M tech, Bsc, Msc. Location : Hyderabad, Bangalore and Mumbai <h2>Job Description </h2> As a Senior Data Engineer, you will own the end-to-end data platform, from raw data ingestion through transformation to trusted, business-ready datasets. You will be responsible for designing, building, and maintaining scalable, reliable, and high-quality data pipelines that power analytics and decision-making across the organization. This role requires someone who can make sound architectural decisions, balance trade-offs, and deliver production-grade solutions in a fast-paced environment. You'll partner closely with analysts, engineers, and business stakeholders, ensuring data solutions are reliable, maintainable, and aligned with business needs. A strong bias for action, systems thinking, and expertise in modern data engineering practices are essential.Key Responsibilities <ul><li>Design, build, and maintain end-to-end data pipelines from ingestion to curated, trusted data models.</li><li>Develop scalable and maintainable data transformation layers using modern transformation frameworks.</li><li>Design and optimize data warehouse schemas and data models to support reporting and analytics.</li><li>Write highly efficient, production-grade SQL and optimize complex queries for performance.</li><li>Build, maintain, and monitor orchestration workflows, ensuring proper dependency management and data lineage.</li><li>Implement data quality, validation, monitoring, and reliability practices to ensure trusted datasets.</li><li>Make architecture and design decisions while balancing scalability, maintainability, and delivery speed.</li><li>Collaborate closely with analysts, engineers, and business stakeholders to understand requirements and translate them into robust data solutions.</li><li>Proactively identify improvements in data architecture, pipeline performance, and engineering practices.</li><li>Leverage Large Language Models (LLMs) and AI-assisted development tools to improve engineering productivity and accelerate delivery.</li><li>Build solutions that are production-ready, scalable, and resilient while maintaining high development velocity.</li></ul> <h2>Must-Have Skills</h2> <h3>Technical Skills</h3> <ul><li>Expert-level proficiency in SQL, including complex query optimization, performance tuning, and advanced data modeling.</li><li>Strong expertise in data warehousing concepts, dimensional modeling, and analytical database design.</li><li>Hands-on experience with modern data transformation frameworks such as:<ul><li>dbt</li><li>SQLMesh</li></ul></li><li>Strong understanding of workflow orchestration, DAG design, dependency management, and data lineage.</li><li>Experience designing and maintaining reliable, production-grade data pipelines.</li><li>Strong knowledge of data quality frameworks, testing, validation, and monitoring.</li><li>Excellent systems thinking with the ability to design scalable and maintainable data architectures.</li><li>Proven ability to deliver high-quality solutions in fast-paced, agile environments.</li></ul> <h3>AI & LLM Skills (Mandatory)</h3> Demonstrated hands-on experience using Large Language Models (LLMs) as a core part of the software development lifecycle, including: <ul><li>Prompt engineering</li><li>Context engineering</li><li>Retrieval-Augmented Generation (RAG)</li><li>Vector embeddings and vector search</li><li>Token optimization and context management</li><li>LLM evaluations and quality assessment</li><li>Guardrails and hallucination mitigation</li><li>Function calling, tool use, and AI workflow orchestration</li></ul> Candidates should have practical experience integrating AI into engineering workflows rather than limited exposure or experimentation. <h3>Professional Skills</h3> <ul><li>Strong communication and stakeholder management skills.</li><li>Ability to challenge requirements constructively and provide technical guidance.</li><li>Strong ownership mindset with the ability to make informed technical decisions independently.</li><li>Bias toward execution, continuous improvement, and delivering business value.</li></ul> <hr /> <h2>Nice-to-Have Skills</h2> <ul><li>Hands-on experience building and operating data platforms on Databricks.</li><li>Experience with AI-assisted or "vibe coding" approaches to software development.</li><li>Exposure to fintech, banking, payments, or financial services data platforms.</li><li>Experience designing scalable data architectures in cloud-native environments.</li><li>Familiarity with modern DevOps, CI/CD, Infrastructure as Code (IaC), and observability practices.</li><li>Experience working in high-growth startups or small engineering teams with broad ownership.</li></ul>

Collected experience requirement

6 - 10 Years