Publisert for 2 dager siden
DATA ENGINEER (Scala/Spark)
Madrid, Madrid, Spania 🇪🇸
- Hybrid
- Heltid
- Airflow
- Data Engineering
- Kafka
- Kubernetes
Stillingsbeskrivelse
Hybrid work mode in Madrid.
The main responsibilities are:
- Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
- Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
- Set up CI/CD pipelines to automate deployments, unit testing and development management.
- Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
- Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
- Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions.
- Foster a culture of continuous learning and improvement within the team.
Qualifications
Good knowledge of:
- Spark on Scala
- CI/CD tools (Gitlab, Jenkins…)
- HDFS and structured databases (SQL)
- Apache Airflow
- S3 storage/COS and parquet (and ORC) format
- Strong knowledge of SQL and NoSQL databases
- Hadoop
Full understanding of:
- Oozie
- Shell script
- Software Development Lifecycle (SDLC) awareness
- Agile principles and ceremonies
- Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.
Some knowledge of:
- Kubernetes containerization
- Dremio as tool to virtualize data
Optionally/ as a plus:
- Elasticsearch and Kibana
- Streaming process (Kafka, event steam…)
- HVault
Additional Information
BUSINESS AND TRANSVERSAL SKILLS
- Knowledge of banking industry and processes
- Problem-Solving and Decision-Making person
- Business / IT relationship (including IT OPS)
- Ability to understand, explain and support change
- Ability to Deliver / Results driven
- Ability to collaborate / Teamwork with data squads and business teams
LANGUAGES
- English level C1
- Optionally Fench
Andre jobber i Spania
2027 Business Development Intern - U4B - Madrid
Uber Madrid, Madrid, Spania🇪🇸
På stedet HeltidPraksisplass for 2 timer siden
1217 | Digital Project Manager
Intetics Fjernarbeid · Flere steder 🇩🇪 🇪🇸 🇱🇹 +3
Fjernarbeid for 3 timer siden
Digestate Strategy Specialist (m/f/d) Spain
VORN Iberia S.L. Spania 🇪🇸
På stedet Heltid for 3 timer siden
Sr Sales Enablement Lead EMEA, Acquisition
Uber Flere steder🇳🇱 🇫🇷 🇬🇧 +2
På stedet Heltid for 3 timer siden
Senior Paralegal
Jobgether Rekrutteringsbyrå Fjernarbeid · Spania 🇪🇸
Fjernarbeid Heltid for 3 timer siden
Staff Product Manager, Mobile
Jobgether Rekrutteringsbyrå Fjernarbeid · Spania 🇪🇸
Fjernarbeid Heltid for 3 timer siden
Excellence Fellowship (AI Product Management)
Prosper AI Flere steder🇪🇸
På stedet Heltid for 3 timer siden
Operario/a Celda de Ensayos de vehículo completo - camara climática (Temporal)
Applus IDIADA Tarragona, Catalonia, Spania🇪🇸
På stedet Heltid for 3 timer siden
Ingeniero/a electrónico/a, automoción. Temporal
Applus IDIADA Tarragona, Catalonia, Spania🇪🇸
På stedet Heltid for 3 timer siden
2D Artist - Puzzle Unit
Voodoo Fjernarbeid · Flere steder 🇦🇩 🇦🇱 🇦🇹 +49
Fjernarbeid Heltid for 3 timer siden
Técnico/a de Proyectos de la Fundación Xcalibur
Xcalibur Madrid, Madrid, Spania🇪🇸
På stedet Heltid for 1 time siden