Expert Insights

Q1: As a Microsoft Data Platform MVP, what major shifts have you observed in how organizations approach Azure SQL, Microsoft Fabric, and modern cloud data architecture over the past two to three years?

Well, there are several aspects to discuss.

Firstly, in November 2023, Microsoft Fabric reached GA status. Although parts of it already existed in Azure Synapse, it meant a fundamental shift from SQL first to other languages too. It also meant a shift towards Notebooks. Nevertheless, Microsoft never forgot those people who prefer the no-code/lowcode approach. I however personally by far prefer using notebooks with PySpark or R.

Microsoft also created OneLake. This is another fundamental shift, towards data mesh and as few data copies as possible, and still using the medallion architecture. I personally remain more a SQL Server guy, but those shifts are important to me too, as I agree: Data Warehousing needs flexibility because I’ve seen too many DWHs that became too stiff over the years and as a result, they had to be rebuilt from scratch, hence investments into data and DWHs became very limited, restricting data quality. That again, restricts organizations for their prospects for AI because AI relies on high quality data.

And with that I’m getting to the last point that changed. It’s been a bit more than 3 years since OpenAI launched its ChatGPT-3.5 model on November 30 2022, but the effects of it are still ongoing so, I think it’s fair to count that still in. AI has changed the industry already today. Whether it will change as starkly as they predict it, I have my doubts, but the changes are unmistakably here today, and Microsoft’s products reflect that. I’m not just talking about Copilot, but also about MS Fabric, SQL Server and Azure.

The last point is realizing that data solution architecture is not the same as system architecture. Those are different disciplines, and they have to work together.

Q2: Many organizations are still dealing with fragmented, on-premises data across legacy systems. What does a realistic first step toward Azure-based data modernization look like, and where should data leaders begin?

With strategy. Many leaders think about the budget first and strategy only second. While I get that oftentimes it is a restriction that a company has, but without a clear strategy, you won’t have a clear budget either.

Another pitfall is complexity: I sometimes see companies with a plethora of different tools that may have chain reactions to any tooling that gets an upgrade. While I also understand that every tool has their ups and downs, and nobody likes lock-ins, but with a clear understanding of your tool landscape and strategy, this is something you can plan. Some vendors are doing more lock-ins than others, but probably no vendor today can afford not offering any exit strategy. Microsoft is no exception to that.

In order to answer the question more explicitly towards Azure, I would usually suggest to start with having a data lineage first because if you move data, you don’t want to end up finding out that you moved a database that another system was somehow indirectly using and isn’t working anymore after you did the cutover. Data is surprisingly hard to trace, yet everything is there: SQL or any other code tell you the story of what happens and which calculations are being made, but humans don’t see them.

Secondly, consider moving to Azure SQL Server VMs first unless you have a smaller data landscape that you really want to consider to partly do a refactoring for. After you’ve moved your data to a VM, a lot of things will already be more clear because that will change connection strings and more, and the hard work will begin because you also may consider to change some things of your data before you move, for example, to SQL Server Managed Instance or MS Fabric. If time is of the essence and you absolutely need to move in one go, try to take your time and have a good plan because that will save a lot of time after.

Lastly, some organizations decide to plan a migration over the span of years. If you realize that your planning may take more than 3 years, you may want to consider hiring more staff for this project because within 3 years, computer science will change, too. As a result, you may end up running after a moving target. This is even more true today in the era of AI.

Q3: With generative AI becoming increasingly embedded in Microsoft Fabric, Copilot, and Power BI, how is AI changing the way organizations generate reports, analyze data, and derive insights from sources such as Azure SQL?

Not as much as you may expect. Sure, you now can tell AI to create some more or less simple queries, but the actual hard work is not that unless you are building a DWH from scratch with complex queries to begin with or are doing stochastically complex Machine Learning projects.

The hard work is still to talk with business to get the definitions of what they need and translate that into a data solution architecture. This is not something an AI can do because data is deeply rooted in processes and human interpretation which is different in every company. Also, it is currently impossible to have an AI being able to solve that problem because virtually every database has object names like tbl_first_name and no AI is able to understand why there are two databases with the same table name or even worse, how to work with those. To make this matter worse, an AI would on top of that need to deal with company jargon expressions which often is different for every company, even in the same industry. This is what I mean with human interpretation and processes. For the foreseeable future, I don’t think this is going to change, and maybe they will but I think it’ll require different solutions for this than we are now available.

Q4: As enterprises move beyond traditional BI toward AI-powered analytics, how does Microsoft Fabric help maintain a governed and consistent semantic layer so that AI-generated insights remain contextual, accurate, and trustworthy?

Microsoft offers several solutions towards that. For data government, they offer Microsoft Purview. However, like I stated above, those AI tools have their limits. You still will need to architect your data solution from the ground up, including a consistent semantic layer. With AI you can collect ideas and bounce your ideas to it, and it may help you with coding, but the thinking, that is still what you need to do yourself. Sure, with AI you may be faster, but people tend to overestimate AI’s abilities.

Q5: As tools like Copilot gain the ability to interact directly with enterprise data, what governance and security guardrails have become non-negotiable to prevent sensitive, inaccurate, or misleading information from surfacing in AI-generated responses?

So let me break down this, there are several questions for one umbrella term “data governance”. I think we have to talk about data criticality first. If you have public data, e.g. content that is going to be public for public relations, you probably don’t want to have the same measures as critical data like data that may infringe regulations or end your company if they would get public. So the first step is to catalogize. That means not just to categorize, but also to document where they are, who has access to them and how they flow. With that, companies have to instruct their employees on how and which AI they may use. Some AI models are public, so for public information or very insensitive information, you can use that. For more sensitive information, an internal, not shared model may be an option. Microsoft offers with Azure Foundry a solution that can do just that — your data will stay within your guardrails. So this is one of the reasons why AI will not take our jobs anytime soon because in the end, we still need people to work with AI and make sure that AI does not leak information you don’t want to leak. Because again, this is about human interpretation and processes which an AI, at least today, isn’t capable of doing and won’t be able to do for the foreseeable future.

The next question you were asking is about the hallucination problem. There have been many influential papers that pointed out that an AI as we know it today that isn’t hallucinating doesn’t exist. We can use other AI models to check if an answer is good or not, but in the end, that is also an AI. Ultimately, if we spin this further, we will get down to Gödel’s incompleteness theorem which states that any mathematical system is incomplete. In other words, we will never have a system that will be perfect in terms of its correctness. Or to use a more practical analogy, this theorem is exactly why we still have human testers to test a software product today. You can write a testing program to test a program, but you can’t write a program that tests any software for correctness. This is the same reason why humans will remain responsible for what AI returns. Nobody will want to trust an AI blindly, just as we don’t blindly trust software developers — we still test their software first before we ship them. But the question that remains is: How far will AI be able to go? That will be yet to be seen.

Q6: Looking ahead, what do you see as the biggest opportunity for enterprises as AI becomes more deeply embedded into everyday data platforms? And what is the biggest risk organizations should be preparing for?

“Prediction is very difficult, especially about the future.” This bon mot is often credited to Niels Bohr, and I’d like to use that here. As we probably still see AI in its infancy and it hasn’t really matured yet, it is hard to tell what is going to happen. But there are some patterns that worry me to some extent. Companies using AI are still in the exploration phase of where the limits are. A risk however I see is that some employees write code, let AI write the documentation and nobody really understands the code anymore. What if APIs are wrongly documented? That is when things may become difficult, especially if those people leave the company. But that isn’t really a new problem. New is that the leverage of a person or team has become much bigger, hence the consequences, too.

Q7: Looking across the broader Microsoft ecosystem including Azure, Fabric, Power Platform, and Dynamics 365, where do you see the next major leap in enterprise data and analytics capabilities coming from over the next two to three years?

Again, I don’t have a crystal ball. But I think the industry will continue to change as it always has, but with an unprecedented velocity. Lakehouses may provide an excellent base for what AI will consume for building data warehouses and AI-driven reports. I can imagine that a next generation of data warehouse architectures will come out of that. Power BI may also see some changes due to that, and CRMs will become much more automated than ever before.

Q8: If you could give one piece of advice to a data leader starting their Microsoft Fabric or Power BI journey today, what would it be and what common mistake would you encourage them to avoid?

I think I come back to what I said before that strategy is important, and also data lineage is important. Those are the groundwork. I know, it’s not always easy or feasible to do this for many reasons. But believe me, once you have done that, talking to business leaders and getting them into the boat for a solution they want, need and use, will become much easier because you will know the data you’re having and are advising your stakeholders with the knowledge you need. Make your stakeholders understand that with this, they will gain a solution they can rely on and use for making decisions. The goal of data solutions is always to serve the business for this.