Data Engineer

hire22 | Experience: 3+ Years | Kerala | Posted 14 hours ago

Job Description

 Pipelines & Ingestion: Independently build database, file, and API ingestion into ADLS Gen2 and Azure Databricks; implement incremental loads, change data capture where appropriate, and safe reprocessing.

• Transformations & Modeling: Develop SQL and Python/PySpark transformations and Delta Lake tables; build bronze, silver, and gold layers and analytics-ready models with clear data contracts.

• Reliability & Orchestration: Schedule dependencies through Databricks Jobs or Azure Data Factory; implement validation, monitoring, alerts, retries, and recovery procedures to meet agreed data freshness targets.

• Security & Delivery: Apply Unity Catalog access controls and lineage, use Azure Key Vault or approved secret management, and deliver automated tests and CI/CD releases through Git-based review.

• Performance & Partnership: Tune Spark jobs, queries, and table layout for performance and cost; gather requirements with U.S.-based stakeholders and document mappings, quality rules, and operational runbooks.

• AI-Assisted Engineering: Use approved AI tools to assist with SQL/Python development, testing, troubleshooting, and documentation; validate generated code and outputs and protect client data.

⚡ Hot DevOps & Cloud Jobs Trending Now

SRE
Sr. SRE Engineer
Stripe | Bangalore
DEV
Backend Developer
Coinbase | Remote
Infra
Cloud Infra Lead
Datadog | Pune
ML
MLOps Architect
Anthropic | Hyderabad
SRE
Sr. SRE Engineer
Stripe | Bangalore
DEV
Backend Developer
Coinbase | Remote
Infra
Cloud Infra Lead
Datadog | Pune
ML
MLOps Architect
Anthropic | Hyderabad