Posted 1 hour ago

Data Engineer

Side
Braga, Braga, Portugal 🇵🇹
  • Onsite
  • AWS
  • Data Engineering
  • Python
  • Snowflake

Job description

We are looking for a Data Engineer to join our team, helping to design and build the data platform underpinning the company's analytics product. In this role, you will develop reliable ingestion and transformation processes that make source data usable, traceable and suitable for downstream application and BI requirements.

  • · Contribute to data platform design using AWS and Snowflake.
  • · Implement ETL/ELT processes and data pipelines.
  • · Automate ingestion from agreed source systems and document-processing outputs.
  • · Develop transformations and data structures supporting analytical use cases.
  • · Implement data validation, error handling and appropriate recovery or reprocessing processes.
  • · Collaborate with financial-domain stakeholders to understand data definitions and reconciliation rules.
  • · Identify and investigate missing, inconsistent, duplicate or otherwise invalid data.
  • · Support visibility into pipeline execution, failures and data freshness.
  • · Document data flows, transformation logic, dependencies and operating procedures.
  • · Collaborate with back-end engineers and BI specialists to deliver usable datasets and resolve integration issues.
  • · Prepare knowledge transfer for the team responsible for maintaining the platform after the initial engagement.

Requirements

  • Professional experience with AWS and Snowflake.
  • ·Experience using data platform design.
  • ·A strong understanding of ETL/ELT implementation.
  • ·A clear understanding of data pipeline development.
  • ·An in-depth knowledge of automated data ingestion, data validation and error handling.
  • · Strong SQL and practical data modelling skills.
  • · Experience making data pipelines repeatable, testable and maintainable.
  • · Understanding of data quality, lineage and reconciliation.
  • · Ability to investigate failures across ingestion, transformation and storage.
  • · Awareness of access controls and responsible handling of sensitive financial or personal data.
  • · Ability to explain data assumptions and limitations to business stakeholders.

In addition to the above, it would be advantageous to bring a clear understanding of financial or accounting datasets, semi-structured data, document-derived data, Python or another relevant scripting language, workflow orchestration, automated data testing or Snowflake performance and cost optimization.

Other jobs in Portugal