Data Science ML/Gen AI Engineer
Job Description
- Contribute to medallion architecture pipelines (Bronze → Silver → Gold) using Databricks, and support column-level lineage and governance initiatives (targeting at least 95% lineage coverage).
- Explore, prototype, evaluate, and productionize machine learning and GenAI solutions across use cases including forecasting, anomaly detection, NLP, Retrieval-Augmented Generation (RAG), LLM-powered assistants/copilots, and predictive analytics.
- Package and manage models using Unity Catalog model management/registries, and design batch and streaming inference architectures where appropriate.
- Partner with Product and business stakeholders to define success metrics, KPIs, and A/B testing strategies, and move successful experiments from prototype to production with clear SLAs, monitoring, and documentation.
- Build production workflows, jobs, and notebooks as infrastructure-as-code using Databricks Asset Bundles (DABs), with CI/CD via GitHub Actions.
- Contribute business metrics, definitions, and semantic models to Unity Catalog, supporting consumption through Power BI and Databricks AI/BI.
- Implement secure data and ML architectures using RBAC/ABAC within Unity Catalog, and support compliance requirements (SOC 2, ISO 27001, GDPR, PIPEDA).