Senior Databricks Engineer

prodapt | Experience: 12+ Years | Bangalore | Posted 6 hours ago

Job Description

  • Design and architect scalable, secure, and highly available enterprise data platforms using Databricks, Snowflake, and AWS Cloud.
  • Lead the development and optimization of large-scale ETL/ELT pipelines, supporting both batch and real-time data processing workloads.Design and implement modern data lakehouse architectures and cloud-native data ecosystems.
  • Architect and deploy AWS-based data platform components while ensuring compliance with enterprise security, IAM, governance, and networking standards.Build and optimize data ingestion frameworks integrating Databricks APIs, REST APIs, GraphQL APIs, and third-party data sources.
  • Develop real-time data processing solutions for business-critical, low-latency use cases using modern streaming technologies.
  • Design and implement scalable data models, incorporating both traditional data warehousing methodologies (Kimball, Inmon, Star Schema, Snowflake Schema) and modern semi-structured/unstructured data patterns.
  • Collaborate with business stakeholders, architects, data scientists, and engineering teams to define and execute data strategies.Drive performance optimization, cost management, scalability, and reliability initiatives across the data platform landscape.
  • Provide technical leadership, architectural governance, code reviews, and mentorship to engineering teams.
  • Support AI and Machine Learning initiatives by designing data foundations that enable LLMs, Generative AI, Machine Learning pipelines, and advanced analytics solutions.
    • Establish best practices for data engineering, architecture, governance, observability, and platform operations.Prepare and maintain architecture documentation, design standards, and technical roadmaps.

⚡ Hot DevOps & Cloud Jobs Trending Now

SRE
Sr. SRE Engineer
Stripe | Bangalore
DEV
Backend Developer
Coinbase | Remote
Infra
Cloud Infra Lead
Datadog | Pune
ML
MLOps Architect
Anthropic | Hyderabad
SRE
Sr. SRE Engineer
Stripe | Bangalore
DEV
Backend Developer
Coinbase | Remote
Infra
Cloud Infra Lead
Datadog | Pune
ML
MLOps Architect
Anthropic | Hyderabad