LEAD DATA ENGINEER
happiestminds
|
Experience: 8+ Years
|
Bengaluru
|
Posted 23 minutes ago
Job Description
- Design, develop, and maintain scalable data pipelines using Databricks and Azure Data Factory.
- Implement data processing solutions using Spark and Delta Tables to optimize performance and reliability.
- Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
- Manage and optimize data storage solutions in Azure Data Lake and ensure data accessibility through Unity Catalog.
- Monitor and troubleshoot data pipeline performance and implement improvements as necessary.
- Document data processes and maintain clear communication with stakeholders regarding project status and challenges.
- Stay updated with the latest industry trends and technologies to continuously improve data engineering practices.