Đăng 2 ngày trước
Data Engineer
- Tại nơi làm việc
- Toàn thời gian
- Azure
- Data Engineering
- Python
- Apache Spark
Mô tả công việc
Location: UAE-based (Travel required)
Duration: 6-month assignment
Summary (Data Engineer)
Timely, reliable data pipelines are what turn a platform design into business value. The Data Engineer builds and operates the pipelines that move data from source systems into the lakehouse and on to analytics and AI consumers, ensuring that the pipelines are automated, performant, secure and governed. The role works alongside the data architect, data quality engineer, business analysts and AI engineers, delivering production-grade pipelines on Azure / Microsoft Fabric, with metadata and lineage captured in Microsoft Purview.
Responsibilities
- Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer, using Azure Data Factory and Fabric pipelines.
- Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks, including cleansing, conformance, enrichment and business-rule logic.
- Implement dimensional models, SCDs, surrogate keys and data marts according to the architect's designs.
- Integrate DQ checks, validation, quarantine and reconciliation into pipelines in partnership with the Senior Data Quality Engineer.
- Ensure that pipelines publish metadata and lineage to Microsoft Purview.
- Implement scheduling, dependency management, parameterization, error handling and alerting.
- Optimize pipelines, partitioning, file formats (Delta/Parquet) and compute usage for performance and cost efficiency.
- Apply access controls, masking, encryption and secure credential management in line with UAE PDPL, the Dubai Data Law and DESC ISR.
- Maintain code in source control and deploy through CI/CD pipelines, with automated tests and environment promotion.
- Support production operations, incident resolution and runbooks, and coach counterparts so that the pipelines can be sustained after handover.
Must Have
- 4+ years in data engineering, preferably within UAE government or regulated-sector environments.
- Advanced SQL, Python and PySpark/Spark skills.
- Hands-on experience with Azure Data Factory, Microsoft Fabric, Databricks or Synapse.
- Strong understanding of Lakehouse/Medallion architecture, Delta/Parquet formats and dimensional modelling, including SCDs and surrogate keys.
- Experience with CI/CD (Azure DevOps or GitHub), source control and DataOps practices.
- Familiarity with Microsoft Purview metadata and lineage, and with DQ tooling.
- Awareness of UAE data regulation, including UAE PDPL, the Dubai Data Law and DESC ISR.
- Excellent English communication and documentation skills.
Good to Have
- Arabic proficiency is an advantage.
Thêm việc làm tại Datamatics Technologies
Data Engineer
Datamatics Technologies Abu Dhabi, Abu Dhabi, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Data Quality Engineer
Datamatics Technologies Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Việc làm khác tại Các Tiểu Vương quốc Ả Rập Thống nhất
Product Deployment Engineer, Search and Shopping, gTech Ads LCS
Google Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Hotel Cleanliness Expert (Housekeeping Attendant)
Marriott Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Assistant Restaurant Manager - Mina's Kitchen
Marriott Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Manager, Brand Experience and Creative
NYU Abu Dhabi Abu Dhabi, Abu Dhabi, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
(UAE National) Asst Manager - Investor Relations
ENBD Recruitment Team Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Graduate Trainee – Internal Shariah Control (UAEN)
ENBD Recruitment Team Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Cybersecurity OT Sales - Lead Sales Representative
Honeywell Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪 🇸🇦
Groups, Conference & Events Specialist - Planner at Conrad Dubai
Hilton Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Groups, Conference & Events Specialist - Seller at Conrad Dubai
Hilton Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Technical Manager – Political Violence and Terrorism
Marsh Management Services (MENA) Limited Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Lead Control & Protection Application Engineer
GE Vernova Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪
Service Cloud Account Executive - UAE
Salesforce Dubai, Các Tiểu Vương quốc Ả Rập Thống nhất🇦🇪