Register for your free TechRepublic membership or if you are already a member, sign in using your preferred method below.
We recently updated our Terms and Conditions for TechRepublic Premium. By clicking continue, you agree to these updated terms.
Invalid email/username and password combination supplied.
An email has been sent to you with instructions on how to reset your password.
By registering, you agree to the Terms of Use and acknowledge the data practices outlined in the Privacy Policy.
You will also receive a complimentary subscription to TechRepublic’s News and Special Offers newsletter and the Top Story of the Day newsletter. You may unsubscribe from these newsletters at any time.
All fields are required. Username must be unique. Password must be a minimum of 6 characters and have any 3 of the 4 items: a number (0 through 9), a special character (such as !, $, #, %), an uppercase character (A through Z) or a lowercase (a through z) character (no spaces).
Dattell report reveals most popular technologies for data engineers
Your email has been sent
Discover some of the most popular tools and technologies desired by employers looking to fill data engineering jobs, according to Dattell.
Demand for technology professionals has been rising after a brief downturn following the start of the COVID-19 pandemic. And one tech job with slots to fill throughout the industry is that of data engineer. Responsible for creating the infrastructure needed to consume, process and store vast amounts of information, data engineers are seeing an increase not just in job openings but in the tools that help them do their work.
A report released June 29, 2022, by data engineering provider Dattell looks at the current market for this career. To compile its findings, Dattell collected and analyzed a huge amount of data itself, matching specific technologies with the number of job openings.
Eyeing the job market for data engineers, Dattell analyzed 340,000 different job postings. Among these, 35% were for positions in data orchestration, 30% for data storage, 29% for data visualization, and 6% for data processing.
Additionally, Dattell analyzed the languages used by data engineers. Python was the most popular, preferred by employers 38% of the time, followed by Java at 33% and SQL at 22%. Combined, the three of them account for 550,000 job openings.
SEE: Hiring Kit: Database engineer (TechRepublic Premium)
Moreover, compensation for data engineering jobs varied from $60,000 as a starting salary to as high as $180,000 for more advanced positions. The highest compensations were offered to those working with Kubernetes, Elasticsearch, PostgreSQL and Terraform, with salaries landing at $140,000 and higher. And while there are a healthy amount of openings for those working with Tableau and Power BI, most of them offer less than $100,000.
In its research, Dattell compiled a list of the top 20 most popular data engineering technologies, pointing to such tools as Microsoft Power BI, Terraform, Chef, Spark, Elasticsearch, Hadoop and Kafka. The products cited offer support in areas spanning data storage, data orchestration, data processing and data visualization, indicating that no single segment of data engineering dominates the rest.
Based on Dattell’s research, the five most popular data engineering tools are MongoDB, Tableau, Kubernetes, PostgreSQL and Ansible, covering data storage, data orchestration and data visualization technologies.
Among the tools themselves, Tableau and Kubernetes took the top spots for the greatest number of job openings. Other tools generating a high number of open jobs were Ansible, Hadoop, Terraform, Splunk, Power BI, MongoDB and PostgreSQL.
And among the data orchestration tools examined, Kubernetes is by far the leader, followed by Ansible. Though both products fall into the same space, each one is used differently. Kubernetes lets professionals manage and maintain container health, while Ansible allows them to deploy configuration changes and manage updates.
With so many free products available, employers don’t see a great need to pay for data processing tools, according to Dattell. As a result, companies are looking for employees and consultants with expertise in free and open-source technologies, such as leading tools like Apache Spark and Apache Kafka.
For example, with data storage technologies, paid tools are preferred by 59% of employers, leaving a still hefty 39% who prefer free tools. Among the most popular data storage technologies analyzed by Dattell, MongoDB was the most popular, followed by PostgreSQL, both of which are free. Some products, such as Elasticsearch, come in both paid and free versions.
Learn the latest news and best practices about data science, big data analytics, and artificial intelligence.
Dattell report reveals most popular technologies for data engineers
Your email has been sent
Your message has been sent
TechRepublic Premium content helps you solve your toughest IT issues and jump-start your career or next project.
Windows 11 gets an annual update on September 20 plus monthly extra features. In enterprises, IT can choose when to roll those out.
Edge AI offers opportunities for multiple applications. See what organizations are doing to incorporate it today and going forward.
This is a complete guide for Apple’s iPadOS. Find out more about iPadOS 16, supported devices, release dates and key features with our cheat sheet.
Discover data intelligence solutions for big data processing and automation. Read more to explore your options.
Whether you are a Microsoft Excel beginner or an advanced user, you’ll benefit from these step-by-step tutorials.
This document helps make sure that you address data governance practices for an efficient, comprehensive approach to data management. This checklist from TechRepublic Premium includes: an introduction to data governance, a data governance checklist and how to manage a data governance checklist. From this checklist’s introduction: Data governance is the process by which an organization …
Recruiting a Scrum Master with the right combination of technical expertise and experience will require a comprehensive screening process. This hiring kit provides a customizable framework your business can use to find, recruit and ultimately hire the right person for the job. This hiring kit from TechRepublic Premium includes a job description, sample interview questions …
Knowing the terminology associated with Web 3.0 is going to be vital to every IT administrator, developer, network engineer, manager and decision maker in business. This quick glossary will introduce and explain concepts and terms vital to understanding Web 3.0 and the technology that drives and supports it.
While the perfect color palette or the most sublime button shading or myriad of other design features play an important role in any product’s success, user interface design is not enough. Customer engagement and retention requires a strategic plan that attempts to measure, quantify and ultimately create a complete satisfying user experience on both an …