5 reasons to use Infrastructure as Code (IaC)
When do you use IaC for your new project?
- Article
- Data Engineering
Infrastructure as Code has proven itself as a reliable technique for setting up platforms in the cloud. However, it does require an additional investment of time from the developers involved. In which cases does the extra effort pay off? Find out in this article.
What is Infrastructure as Code?
IaC is starting and managing IT infrastructure with code. There are techniques that are specific to one provider, such as Cloud formation for Amazon Web Services (AWS), Deployment Manager for Google Cloud Platform (GCP) or Bicep for Microsoft Azure. A commonly used technique is Terraform, which offers APIs for all major cloud parties.
Do you want to know more about the meaning of IaC? Read this article by RedHat.
Because Digital Power specialises in data, the content of this article applies to building a data platform. For example, a data warehouse, or a machine learning application of an existing application.
IaC can be used within both on-premises and cloud environments. As specialists in the field of Azure, GCP and AWS, we assume a cloud environment for the context of this article.
Are you considering using IaC? Here are the 5 benefits:
1: Your environments are easy to duplicate
This point is very relevant for two reasons: first of all, you can use IaC to separate your DTAP (development, test, acceptance and production) environments. You use the same template for all environments and use variables for the factors that you want to be different. For example, for your test environment, set a smaller database than for your production environment.
Another application of this point is separating different development environments. When you build a platform with multiple developers, each developer can start their own platform. That way you don't get in each other's way. At the end of each day, any dev platform can be shut down again.
2: You have better insight into platform status and clear documentation
In IaCn does not only name the infrastructure and software itself, but also the configuration. This includes user access and firewalls. Do you want to know which IP address is accessible within the subnet of your Business Intelligence team? An analyst or product owner can also find this in the code!
3: Code checkouts and version control on your infrastructure
This is a big benefit for the development team. Because the configuration of your infrastructure is stored in a Git repository, you can easily collaborate and have all your code reviewed by your teammates. This is also positive for the business stakeholders: mistakes are easier spotted and removed from the codebase. Due to version management, the platform is also easily auditable, which is increasingly becoming a requirement for a new platform.
4: You save costs
Working with IaC ensures that everything which runs in production is also present in the codebase. This makes it less likely that services will continue to run that are no longer used. Because your Infrastructure as Code is stored in Git, unnecessary services are quickly removed by developers during code checkouts. IaC is also often used in the cloud, which makes it easy to scale up and down.
IaC forces you to work in a transparent way, which lowers the risk of shadow IT significantly (non-visible adjustments and configuration).
5: You can quickly go to production
Infrastructure as Code integrated with CI/CD matches the DevOps method. Because your infrastructure is fully automated, it becomes easy to deploy a new module to production. In addition, you can quickly switch back to a previous version using both the IaC technique and the version management of Git.
Disadvantages of Infrastructure as Code
Are there no disadvantages to using IaC? They certainly do exist, although they are relatively small. The disadvantages of IaC at a glance:
- Developers' knowledge is not always sufficient. It is still a relatively new technique that not everyone is familiar with. Data Engineers and application developers are also stepping into the role of Infra Engineers. This requires flexibility in the way IT departments work.
- To be able to use Infrastructure as Code properly, a CI/CD pipeline must also be set up. This increases the initial investment for your project.
- Some minor changes may take longer. For example, if you want to give a Developer a higher level of access to your production platform, you first have to run this through the rest of your DTAP street. With this, it is, incidentally, immediately well-tested and documented.
- Your codebase grows in size. You will now also have to debug your IaC, and practice shows that not all documentation from the major cloud providers is always up to date.
What project do you use IaC for?
Are you considering using IaC? The advantages usually outweigh the disadvantages. But keep in mind: the advantages are not applicable in every context. Therefore, compare the terms of your project to the pros and cons in this article and draw your own conclusions.
Also, know that IaC works especially well for new platforms. Once you've started, it's not a good idea to make changes outside of the code. So you also make a commitment for the future.
Curious how IaC can be applied in daily practice? Read how we helped The Neuron predict the number of expected impressions for digital advertisements on billboards in a scalable and efficient way.
Our Data Engineers are happy to exchange thoughts on using Infrastructure as Code for your project. Contact us for advice or a request to join our Data Engineering team.
Interested in a job as Data Engineer at Digital Power? We have an open place in our Data Engineering team! Check out the vacancy here.
This is an article by Oskar van den Berg, Data Engineering Consultant at Digital Power
Oskar started developing his own websites at the age of 10. During his career, he increasingly focused on Data Engineering. Through Digital Power he works for major clients such as De Nederlandsche Bank and ASML.
Data Engineeroskar.vandenberg@digital-power.com
Receive data insights, use cases and behind-the-scenes peeks once a month?
Sign up for our email list and stay 'up to data':
You may find this interesting too
Implementing a data platform
Based on our know-how, the purpose of this blog is to transmit our knowledge and experience to the community by describing guidelines for implementing a data platform in an organisation. We understand that the specific needs of every organisation are different, that they will have an impact on the technologies used and that a single architecture satisfying all of them makes no sense. So, in this blog we will keep it as general as we can.
Reliable reporting using robust Python code
The National Road Traffic Data Portal (NDW) is a valuable resource for municipalities, provinces, and the national government to gain insight into traffic flows and improve infrastructure efficiency.
Senior Data Engineer
Work on challenging technical assignments with various clients and keep abreast of developments in your field.
5 questions for Data Engineer Oskar
In this video, you will find out what a job as a Data Engineer looks like! What does a working week look like, which clients do our Data Engineers work for and what makes working so much fun? Oskar likes to tell you more about it!
A standardised way of processing data using dbt
One of the largest online shops in the Netherlands wanted to develop a standardised way of data processing within one of its data teams. All data was stored in the scalable cloud data warehouse Google BigQuery. Large amounts of data were available within this platform regarding orders, products, marketing, returns, customer cases and partners.
Setting up a future-proof data infrastructure
Valk Exclusief is a chain of 4-star+ hotels with 43 hotels in the Netherlands. The hotel chain wants to offer guests a personal experience, both in the hotel and online.
