Data Engineer
tieto
|
Experience: 5+ Years
|
Pune
|
Posted 3 hours ago
Job Description
- Design, develop and maintain scalable data pipelines using Azure Databricks, PySpark, Python, SQL and Delta Lake.
- Implement Bronze, Silver and Gold Lakehouse data layers and reusable data-engineering frameworks.
- Develop complex ETL/ELT solutions involving SAP and other enterprise source systems.
- Ingest and transform data originating from SAP environments, applying appropriate business and transformation logic.
- Build data solutions supporting manufacturing business processes, reporting and analytics.
- Implement batch and incremental data-loading patterns, schema evolution, historical data handling and pipeline recovery.
- Develop and optimize complex Spark/PySpark workloads and SQL transformations for scalability and performance.
- Implement data-quality checks, validation, reconciliation, monitoring and exception handling.
- Develop analytical and dimensional data models, including Fact/Dimension and Star Schema models.
- Follow engineering best practices for Git, CI/CD, automated deployment, coding standards and documentation.
- Work with Data Architects and business stakeholders to translate technical and business requirements into scalable data solutions.
- Participate in design reviews, code reviews and continuous improvement of Databricks engineering practices.