A day in the life of a Data Engineer

What is a Data Engineer?

To start with the question, what exactly is a Data Engineer? The expectations and general views on this are quite different. In essence, a Data Engineer is someone who creates and develops solutions for the storage and analysis of data. For example, think of building data pipelines, moving data from A to B and adjusting (transforming) it if necessary before placing it at B. Or setting up a database, working with distributed systems such as Spark and Databricks, setting up an infrastructure and actually much more.

Once working as a Data Engineer, you will gain specific knowledge in this domain, and you will specialise more. Every Data Engineer has preferences for what she or he likes to work on. And if you look at it even more abstractly, as a Data Engineer you are exactly in between business, IT and analytics. You must have some knowledge of everything and be able to deal with all stakeholders and explain clearly what you are doing.

"A Data Engineer is someone who designs and builds solutions for data storage and data analysis" 

The functions of a Data Engineer and a Data Scientist are further apart than people often think. Oskar explains how this works:

"The tooling we use isn't always very different, but the approach differs. A Data Scientist tends more towards an analyst for whom knowledge of mathematics and statistics play an important role. A Data Engineer will function more as a software programmer or IT person. 

Both profiles work in a position where business, IT and analysis intersect. Thus, depending on your position in the organisation, you will have to serve stakeholders to the same extent. Think of it as a painter touching up your window frame. You wouldn't ask him to fix the crack in your glass either. You have a glazier for that! But they do work together on the same frame." 

Read here where Data Engineers and Data Scientists reinforce each other. 

Different types of Data Engineering assignments

As a Data Engineer at Digital Power you can perform different types of assignments. What does that look like in practice?

"In a more Scrum environment, your day is the same as any other Scrum development team. You start with a cup of coffee and do the daily stand-up. Then you work with a tool on the tasks that you can handle and you discuss the progress with your team. In a typical assignment, you will work a lot with Python and one of the major cloud providers: AWS, Azure or GCP. Stakeholder management mainly involves the interaction with your Product Owner and the demos you give. 

If you are working on a smaller assignment on a project basis, the share of stakeholder management will be bigger. You design and develop the solution yourself, and you also maintain contact with the customer. You often do this in collaboration with a colleague from Digital Power. This way you come to the best solution together, and you learn from each other.

The field of Data Engineering is very broad. For example, a colleague can review your setup for a CI/CD street, while you give your opinion on their application of Kubernetes." 

Learning on the job

The work of the Data Engineer is relatively new and still evolving. This means that as a Data Engineer you learn a lot by simply doing it.

"This is also why I love working as a Data Engineering consultant. I have seen and been able to solve problems for several large clients. Intakes and the influence I have on the customer demand are very important here. This is where you determine the scope of the problem to be solved and find out what knowledge and experience is needed to."

Eventually, I can learn many new techniques or solve new problems during an assignment. In this way, I continue to develop myself and take on new challenges. In the past year, I started working on the realisation of a datalake house architecture for the first time, whereas previously I had mainly worked with (no)SQL databases. And all that from one employer." 

Knowledge sharing at the core

Besides learning by doing, you can of course also learn a lot from your colleagues. Different assignments also present different challenges. And how cool is it if you can then share those learning moments?

"Internally at Digital Power I also continue to develop, for example by transferring knowledge, certainly also gaining new knowledge through internal meet-ups and training courses, and by mentoring colleagues within the Data Engineering domain. This allows me to combine the opportunities and pleasure of a varied job with the benefits of a permanent employer." 

A job as a Data Engineer

Has reading Oskar's experience made you excited? You will find several opportunities to become a Data Engineer with us. If you already have experience with the profession, you can immediately apply for our Data Engineer vacancy. 

Do you already have work experience as a data professional, but no specific knowledge of Data Engineering? Get retrained! We offer a special programme for this. Read more about it here. 

Receive data insights, use cases and behind-the-scenes peeks once a month?

Sign up for our email list and stay 'up to data':