Why do I need Data Engineers when I have Data Scientists?

And then: finally it worked! You have a team of Data Scientists, the first valuable insights about your company are emerging and maybe even the first models have already been developed. Either way, you are now ready to realise that added value to your company or department.

Why your Data Scientists may be dissatisfied 

And then it stops after all. It is not possible to get things into production, things go wrong when maintaining the models or the turnover of your department starts to increase.

It often happens that the work of the Data Scientists after the first models have been put into production mainly consists of maintaining and retraining models. Of course maintenance is necessary, but regular maintenance should be automated. In daily practice, this often involves manual work for the Data Science team, reducing its job satisfaction. The consequence is this... Your Data Scientists are looking for another job and you need to start looking for new people. 

What does a Data Scientist do? 

Retrieving data, cleaning data, creating scripts… this is work that almost anyone who works with data can do. That is why almost every Data Scientist considers it part of his or her job. While it is part of the job, the focus should be on examining data and mapping connections. Data Scientists extract hidden signals from the data that are not easy to find. They get their energy from that research and inventing; not from making software robust or production-worthy.

Why you need Data Scientists AND Data Engineers 

It's like building a car factory. There are designers who develop a prototype of a car. This includes thinking about what the car should look like, what the car should be capable of, in which price range the car should be available, etc. That is a completely different field than ensuring that the same car can also be built in large volume numbers in a factory.

When building in the car factory, the original prototype is once again studied. Then adjustments are made by which the car does not change substantially, but can be built better and cheaper in the factory.

In this example, Data Scientists are the designers of the car prototype, and Data Engineers are the designers and builders of the factory. Both play an important role in eventually getting the cars on the market: they complement each other.

A Data Engineer can help a Data Scientist by addressing the following issues:

• How can the model be further developed so that it becomes a robust solution that requires little maintenance?
• How can it be put into production and what dependencies are there in it?
• In what ways can the model fail, and how can we prevent or detect this?

The prototype is jointly screened and prepared for production work.

From Data Scientist to Data Science Engineer 

Can't Data Scientists do that themselves? Some of them can, but in general Data Scientists focus more on the research and math side of the story rather than the technical side.

Sometimes a workable solution is found, despite the lack of Engineers in the team. Then one of the Data Scientists will become more of a Data Science Engineer or Machine Learning Engineer than a pure Data Scientist. If you don't like that, that could be yet another reason for that Data Scientist explore opportunities.

What does a Data Engineer do? 

Data Engineers* provide reliable data solutions that can take a beating, increasing confidence in the decisions made based on that data. They provide access to new data, which offers countless possibilities for the organisation. In addition, they also help ensure a data infrastructure that meets the requirements and wishes of the organisation, so that data is not only available, but also available on time and of sufficient quality.

That is exactly why an effective Data Science department also needs Data Engineering capacity.

*Read more in the article 'What is a Data Engineer? '

Need a sparring partner for your ideal Data Science and Engineering team? 

We are happy to share our ideas! Contact us directly. 

Receive data insights, use cases and behind-the-scenes peeks once a month?

Sign up for our email list and stay 'up to data':