Databricks Engineer

Responsibilities

The client, a large bank, is in the process of centralizing its key reporting and aggregation processes on cloud platforms. In 2024, the intention is to centralize cashflow aggregation onto a platform based on Databricks technologies (Databricks and Deltalake). The project involves collecting both actual and forecast cashflows from both real-time and batch data sources, cleaning and enriching the data, and storing the resulting aggregated data in Deltalake for use in reporting.

Qualifications

• Technical skills: Successful candidates will have played an important role in at least one Databricks platform implementation. Experience in data migration, data integration, and delivery of data warehouses is also advantageous.
  • Databricks: the candidate should be familiar with ETL/ELT in Databricks, with Medallion architecture and with Deltalake.
    • Other relevant technologies include: Streaming technologies such as Kafka, Delta Share and Deltalake Connector, SQL, Python and PySpark
• Domain knowledge: The successful candidate will have a background in financial products and processes. Relevant areas of domain knowledge include: Cashflow forecasting and liquidity management, Financial products including securities such as swaps, and banking products such as loans, Reference data including counterparty and business hierarchy data, Regulatory and risk management reporting needs