Posted 3 weeks ago
Senior Data Engineer
Remote · Warsaw, Mazovia, Poland 🇵🇱
- Remote
- Full Time
- AWS
- Data Engineering
- Python
- Apache Spark
Job description
- Design and build scalable cloud-native data platforms from greenfield to production
- Design and implement near-real-time ingestion pipelines using event-driven patterns
- Define and enforce data platform standards including Data Lake/Lakehouse principles, medallion architecture, and data contracts
- Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Drive adoption of AI tooling and agentic workflows within the data engineering team
- Ensure data quality, observability, and reliability across pipelines and platforms
- Design and deliver self-service tooling and microservices that abstract platform complexity
- Collaborate directly with the Customer’s scientific and engineering teams
- Monitor platform performance, reliability, and cloud infrastructure costs
Qualifications
- 5+ years of experience in data engineering
- 3+ years of production experience with Databricks and Delta Lake
- Strong expertise in medallion architecture and event-driven asynchronous pipeline design
- Advanced proficiency in Python, PySpark, and SQL
- Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
- Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
- Practical experience with Git workflows, pull request reviews, and automated testing
- Experience configuring secure CI pipelines using OIDC or service principals
- Practical MLflow experience including experiment tracking, model registration, and reproducibility practices
- Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
- Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX
- Experience with monitoring, observability, and cloud cost optimization
- Upper-Intermediate English level or higher
WILL BE A PLUS
- Experience packaging workloads in Docker containers and running batch jobs on AWS
- Familiarity with experiment optimization tools such as scikit-optimize, BoTorch, Ax, and scipy
- Experience building feature sets and train/test splits that avoid data leakage
- Experience with regulated or high-security environments including data residency requirements
- Experience disabling or restricting AI features in secured environments
Additional Information
PERSONAL PROFILE
- Strong analytical mindset
- Ownership and accountability
- Ability to communicate complex technical concepts clearly
- Proactive approach to problem-solving
- Attention to scalability and maintainability
- Collaborative and team-oriented attitude
More jobs at Sigma Software
Senior Data Engineer (Cybersecurity)
Sigma Software Remote · Warsaw, Mazovia, Poland🇵🇱
Remote Full Time 2 weeks ago
Data Engineer (DATABRICKS / PYSPARK)
Sigma Software Remote · Warsaw, Mazovia, Poland🇵🇱
Remote Full Time 2 weeks ago
Other jobs in Poland
Kierownik Regionalny w Dziale Zwierząt Towarzyszących (k/m/n)
6F9 - Zoetis Polska S.p.z.o.o Warsaw, Mazovia, Poland🇵🇱
Onsite Full Time 54 minutes ago
(Mid, Sr) Systems Development Engineer
BorgWarner Kraków, Lesser Poland, Poland🇵🇱
Hybrid Full Time 53 minutes ago
Team Lead – Systems Engineering
BorgWarner Kraków, Lesser Poland, Poland🇵🇱
Hybrid Full Time 53 minutes ago
Firmware/Software Platform Engineer, Senior
Zebra Technologies Multiple locations🇵🇱
Hybrid Full Time 54 minutes ago
Specjalista / Specjalistka ds. Zakupów (k/m/n)
Unison Engine Components Poland Sp. z o.o. Dzierżoniów, Lower Silesia, Poland🇵🇱
Onsite Full Time 1 hour ago
Inżynier / Inżynierka Konstruktor / Konstruktorka (k/m/n)
Unison Engine Components Poland Sp. z o.o. Dzierżoniów, Lower Silesia, Poland🇵🇱
Onsite Full Time 1 hour ago
Advanced Lead Engineer - Mechanical Component (w/m/x)
GE Aerospace Warsaw, Mazovia, Poland🇵🇱
Hybrid Full Time 1 hour ago
Specjalista ds. Planowania Materiałowego i Logistyki (k/m)
Segula Technologies Poland 🇵🇱
Onsite Full Time 1 hour ago
Facilities Support Specialist
MJA-Jones Lang LaSalle Sp. z o.o. Gdańsk, Pomerania, Poland🇵🇱
Onsite Full Time 1 hour ago