Lead Python and Spark Data Engineer
citi
|
Experience: NM+ Years
|
Pune
|
Posted 3 hours ago
Job Description
- Own a large-scale data warehouse; develop data products; and lead the architecture, design, development, and maintenance of scalable and secure data architectures to support downstream reporting applications, data integration, and simplified complex data processes — utilizing best engineering practices and data lifecycle management principles.
- Identify, evaluate, implement improvements to legacy systems, ensure alignment with target architecture design
- Provide strong leadership to engineering teams including mentoring, code review and enforcing engineering best practices.
- Own and define data models, taxonomies, metadata, data mappings, data standards, versioning, and semantic layers for domains and data products and support data consumption, AI enablement, and regulatory requirements.
- Build automated data integration solutions and data pipelines including reference data to meet business and reporting requirements
- Exhibit strong leadership qualities, fostering a positive and collaborative work environment, providing guidance and mentorship to team members, and promoting a culture of continuous learning.
- Implement data governance and management frameworks, quality standards, and data controls to ensure data accuracy and security, enabling downstream applications to consume data with confidence. Manage and maintain data process documentation
- Define target-state data architecture and drive delivery against the strategy, manage the execution to delivery
- Contribute to technical direction and enhancement as a technical expert in data development and implementation phases, providing in-depth and sophisticated analyses to define problems and develop innovative solutions
- Work closely with business users, stakeholders, BAs, product managers and other development teams to understand problem space, brainstorm ideas, approaches, and solutions.
- Contribute to CI/CD implementation using DevOps best practices and ensure robust testing framework is utilized for functional and nonfunctional testing for the release cycles based on industry and Citi standards and work to implement them.
- Work closely with business users, stakeholders, BAs, product managers and other development teams to understand problem space, brainstorm ideas, approaches, and solutions.
- Contribute to technical direction and strategic decisions as a technical expert in all development and implementation phases, providing in-depth and sophisticated analyses to define problems and develop innovative solutions
- Demonstrate a data-driven approach to problem-solving and decision-making, utilizing metrics, and performance indicators to assess team efficiency and identify improvement opportunities.
- Manage and maintain data processes and jobs. Partner with the Support team to ensure the stability of the production environment through proactive monitoring and health checks
- Partner with other Citi development teams to build APIs, integrations and share components that are critical to our platform’s success.