Software Engineer II

• Design, develop, and maintain scalable data processing pipelines using the Spark ecosystem, including
PySpark, Databricks, and Synapse Notebooks.
• Develop and manage data pipelines using Azure Data Factory, Azure Synapse, and Azure Data Lake to
ingest, process, and integrate data from Azure and on-premises sources.
• Implement full and incremental data load strategies, ensuring efficient, reliable, and timely movement of
data across source systems and target platforms.
• Perform data cleansing, transformation, aggregation, and modeling to prepare high-quality datasets for
analytics, reporting, and downstream consumption.
• Apply Azure security, access control, and data governance practices to ensure secure and compliant data
processing and storage.
• Optimize Spark jobs, data pipelines, and SQL workloads for performance, scalability, reliability, and cost
efficiency.
• Collaborate with BI and analytics teams to support Power BI reports and dashboards, including
datasets/models hosted on Azure Analysis Services (AAS) and Power BI.
• Troubleshoot data pipeline and processing issues, perform root-cause analysis, and implement appropriate
fixes to maintain data availability and quality.
• Develop reusable frameworks, processes, and best practices for data ingestion, transformation, validation,
and monitoring.
• Work closely with business stakeholders, data analysts, and engineering teams to translate data
requirements into scalable and maintainable data solutions.