Data Engineer II
thermofisher
|
Experience: 4+ Years
|
Bengalor
|
Posted 12 days ago
Job Description
- Perform source to target data analysis and mappings.
- Build, test, and optimize data pipelines for various use-cases, including real-time and batch processing, based on specific requirements.
- Collaborate with admins, developers, data engineers and analysts for successful functionality delivery
- Perform requirement analysis and co-ordinate with project managers and development team to drive the delivery cycle.
- Collaborate with the scrum master on product backlogs and helps with sprint planning.
- Supports the evolution of EDP architecture and takes part in roadmap activities around data platform architecture initiatives or changes.
- Collaborate with leadership and stakeholders to ensure data quality and integrity in DWH & AWS platforms for BI/Analytical reporting.
- Pre-meditate any risks well in advance and communicate to stakeholders to formulate the plan of action to mitigate any risks.
- Follow agile development methodologies to deliver solutions and product features by following DevOps practices.
- Ensure the teams follow the prescribed development processes and approaches.
- Knowledge of Large Language models and proven experience in developing LLM applications to improve the productivity of data engineering.
- Build most efficient Pyspark pipelines ensuring the operational costs are kept to a minimum.