Job postings for data engineers often list extra skills next to the ones you learned in the course. We added the most common ones that are not used in the track but still appear in a lot of job ads below.
Power BI is the most used BI tool used in NL simply because Microsoft offers it for free (with paid tiers available). Although technically a BI tool meant for reporting and thus more suited for analysts, it is also popular among Analytics Engineers, which is a hybrid between Analyst and Data Engineer. The demand for Power BI developers has only been growing over the past years, so it is a useful skill to have. Job postings often add DAX (formulas) and Power Query (data preparation). Power BI reports can be developed in the browser using Microsoft's Power BI Service or traditionally in the Power BI Desktop application (Windows-only).
Fabric is Microsoft's all-in-one data platform: storage (OneLake), lakehouse, warehouse, pipelines, notebooks, and Power BI in one place. Many of the ideas match what you did on Databricks in Week 13. Although less mature and less well liked by DE's than Databricks and Snowflake, Fabric is still a relatively new competitor in the market and gaining ground at 'Microsoft-only' companies. And because of less competition (lack of enthusiasm) and growing demand it can be very worthwhile to specialize in Fabric.
Azure Data Factory (ADF) is Azure's low-code tool to copy data between systems and run pipelines on a schedule. It does part of what Airflow did in the track. Fabric has the same tool built in, so learning one helps with the other. At companies fully using Azure, its common to use ADF instead of Airflow.
Most job postings use some kind of SQL dialect like SQL Server, Azure SQL, or the Fabric warehouse, which all use T-SQL. Most of your Postgres-SQL carries over, with small differences (for example TOP in place of LIMIT). Note that not having knowledge in the right SQL dialect is usually not a problem for interviews!
Many Microsoft-based teams use Azure DevOps for Git repositories and CI/CD pipelines in place of GitHub. The ideas match the GitHub Actions workflows you wrote in Week 5.
Snowflake is a cloud data warehouse, similar to the Databricks SQL warehouse you used in Week 13. It came up in about 1 in 6 postings. Databricks and Snowflake are the current market leaders of Cloud Data Warehouses, and although both started off as distinct products, they are becoming more similar each day due to copying each other's features.
The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 https://hackyourfuture.net/
Built with ❤️ by the HackYourFuture community · Thank you, contributors
Found a mistake or have a suggestion? Let us know in the feedback form.