Data Engineer
Location: 3 days Hybrid in Chicago, IL
Duration: 6+ Month Contract
Interview: 2 video interviews and a final on-site
About:
Data Engineer with hands-on experience in PySpark, Databricks, and Azure data platforms to design, build, and support end-to-end data pipelines. The ideal candidate will develop and optimize data transformations, build production-grade Python components, and maintain cloud-native Azure environments while collaborating with application teams and ensuring high-quality, reliable data delivery. This role offers the opportunity to work with large-scale datasets, implement ETL/ELT best practices, optimize Databricks clusters, and leverage modern cloud technologies to support AI/ML initiatives.
Qualifications
Education: Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent experience).
Required Skills
Experience: 5+ years as a Data Engineer; 3+ years in ETL/ELT concepts, PySpark, and SQL.
SQL: Advanced querying, CTEs, views, joins, complex transformations, and performance tuning.
Python: 2+ years building production-quality modules, unit/integration testing, logging, and CI/CD integration.
Databricks: 1+ year working with notebooks, jobs, workflows, external/managed tables, Delta Lake, and basic cluster configuration.
Azure Data Factory (ADF): 1+ year creating and maintaining pipelines, including triggers, parameterisation, monitoring, and error handling.
Azure Cloud: Hands-on with ADLS Gen2, Azure SQL, Azure App Service, and general Azure portal/resource group operations.
Infrastructure as Code: Experience provisioning Azure resources with Pulumi.
ETL/ELT Concepts: Strong understanding of pipeline patterns, incremental loads, data validation, and troubleshooting.
Preferred Skills
Additional Skills (nice-to-have): R for data validation, TypeScript for Pulumi pipelines, Java/.NET for integration, Angular/Spring Boot for minor troubleshooting.
...choice for awards and gifts. About the Role At Crystal D, we Turn Emotions Into Memories and our IT team helps make that happen. As our IT Help Desk Support Specialist, youll keep our technology running smoothly and our teams supported every day. *This position...
...we've done it with integrity for more than 70 years. Were problem solvers and innovation seekers. Were team players and safety fanatics. And we're proud to be part of Quanta Services (PWR:NYSE), North America's largest employer of skilled craft labor. Because what we...
The Logistics Coordinator, Dispatcher is responsible for coordinating and managing requests from customers, with the goal of meeting and exceeding customer expectations, while accurately tracking various shipment pick-ups to their final deliveries. The ability to work ...
&##127769; NOW HIRING: Compassionate Overnight Caregivers (CNA or HHA) Are you passionate about providing exceptional care that... ...and compassionate Caregivers to support clients with private duty, overnight care in the 19129 area. As a KeystoneCare Caregiver...
...environmental, and national security solutions. The Geospatial Data Modelling Group within the Human Dynamics Section, part of the... ...Human Security Division at ORNL, is seeking a Geospatial Data Engineer to support research and operational workflows focused on...