Data Engineer

BA Markets wants to professionalise and streamline its activities with regards to data streaming. Therefore, a new agile team has been created to build up a technical platform for BA Markets users to get a hub for all required streaming services. ​

The team will develop the service suite largely internally but will also rely on managed vendor services, such as Databricks. The streaming technology used is Kafka. ​

Many BA Markets users are technically quite advanced so the central data streaming team must constantly keep up to date to deliver state of the art services. Because of that, there will be constant development opportunities for the candidate, and the candidate must be comfortable to work in a fast-changing, international environment. The team consists of four members in Hamburg, one member in Stockholm and should also have three colleagues in Poland. 
 
What will you do? 
Your tasks and responsibilities: 

• Supporting the design of data architecture in Azure in close collaboration and support the decision regarding tools used for monitoring, ETL, archiving and data access. ​
• Supporting the lead architect in designs for e.g.lag monitoring, throughput optimization, fault tolerance.
• Supporting the definition of governance standards (e.g. data formats and schemas and message types) to make data findable and re-usable. ​
• Building ETL or ELT pipelines​.
• Requirement capturing and facilitation and implementation of solutions for analytics, ML,and self-service use cases.
• Setting standards for and support users in self-service via trainings. 

Location  
Katowice or remotely in Poland

Who are you? 

As our Data Engineer you have:

• Interest in and hands-on experience with data streaming. ​
• Interest in domain (=derivative trading) requirements and why data is needed in a certain structure for a certain use case.​
• Hands-on experience with streaming design principles. ​
• A proactive, communicative team player. Fluent in English.

Hard skills (the more the better) ​

• Hands-on experience with general Azure services like Data Lake Storage, Synapse Analytics, Azure​
• Event Hub, Azure SQL, Data Factory, and Databricks.​
• Understanding of data modelling, data warehousing, and lakehouse concepts​
• Involved in Kafka based implementation projects and implemented Kafka based architectures designs in previous projects. ​
  • Hands-on experience with :
    • monitoring tools, such as Azure Stream Analytics, Azure Monitor, Log Analytics, or Grafana ​
    • the creation of long-term storage strategies, APIs, and data cataloging tools, such as Databricks Marketplace​the implementation of data access strategies, e.g. via implementation of client libraries. ​
• Experience with ML requirements (ML OPS Engineer) would be a merit but is not a prerequisite

Our offer 
Good remuneration, a challenging and international work environment, and the possibility to work with some of the best in the field. You will be working in interdisciplinary teams and you can always count on support from committed colleagues. We offer attractive employment conditions and opportunities for personal and professional development. 

More Information 
We welcome your application in English. We kindly request that you do not send applications by any means other than via our website as we cannot guarantee that we will be able to process applications that are not made via our website. 
 
For more information about the recruitment process you are welcome to contact our recruiter Ewa Krajewska at [email protected]

At Vattenfall we are convinced that diversity contributes to build a more profitable and attractive company and we strive to be good role model regarding diversity. Vattenfall works actively for all employees to have the same opportunities and rights regardless of gender, ethnicity, age, transgender identity or expression, religion or other belief, disability or sexual orientation. 

The security of Vattenfall and its employees is essential. For that reason, a pre-employment screening will be part of your recruitment process. The screening is based on the role you will fulfill within Vattenfall. The screening will be performed by a third party, DISA Global Solutions AB.
 
We look forward to receiving your application!