Building a foundation for AI success: Technology and data strategy

This is part three of a six-part blog series. See part one, part two, and download the white paper.

AI is redefining the boundaries of what’s possible, but the unique demands of AI—from vast data volumes and high-speed processing requirements to complex security and compliance challenges—require a strategic approach to data and technology infrastructure.

“It’s been over a year now since generative AI became mainstream. We are through the science experiment phase and leading companies are now putting AI into action,” says Wangui McKelvey, General Manager, Microsoft Azure Data Analytics.

In this post, we’ll focus on the five data and technology fundamentals required to deliver meaningful, sustainable, and responsible value creation with AI:

1. Align technology strategy and business strategy
2. Assess infrastructure needs and goals
3. Prepare your data estate to smooth the path from proof-of-concept to production
4. Consider build-versus-buy decisions
5. Determine a strategy for regulatory compliance and safeguarding AI assets

Align technology strategy and business strategy

Successful AI projects begin with a clear, prioritized, and valued set of business objectives, such as maximizing productivity and efficiency, improving customer experience, or growth-oriented objectives such as revenue acquisition or product innovation. These goals will help you prioritize use cases against likelihood of impact and realistically assess feasibility based on data and infrastructure requirements. It’s also useful to think of your technology investments in aggregate as an investment portfolio, which will enable you to set clear success criteria and reallocate resources across projects as needed.

Assess infrastructure needs and goals

Moving successfully from proof-of-concept (POC) to production with AI depends on a mix of technology and business factors that, ideally, must work together.

“Understanding how your business strategy maps to your product strategy, and then how your product strategy maps to your infrastructure, is key,” says Omar Khan, General Manager, Microsoft Azure Product Marketing. “AI-optimized infrastructure will help accelerate both building AI solutions and integrating AI into applications.” From a technology perspective, the most critical requirement is access to infrastructure that is built for AI—with the ability to run large AI workloads and perform securely and reliably at scale.

• Cloud-based AI deployments leverage third-party cloud service providers to access scalable and flexible computing power without the need for extensive on-premises infrastructure. This option allows for cost efficiency with pay-as-you-go models and access to a variety of AI services.
• Colocation, or the practice of renting space within a third-party data center to house privately-owned servers and networking equipment with data, applications, AI services, and infrastructure all in one place—minimizes transfer times and leads to lower latency, which is crucial for real-time and high-performance AI applications. Colocation also provides cost savings because centralized infrastructure is more efficient to manage, maintain, and scale, reducing overall operational costs.
• On-premises deployment refers to hosting and running applications and infrastructure within an organization’s physical premises or datacenters. While this deployment option gives organizations more control over their IT infrastructure, it can mean high upfront costs for hardware, ongoing maintenance responsibilities, and limited scalability.

Prepare your data estate to smooth the path from proof-of-concept (POC) to production

Data is the fuel that powers AI technology, so planning for any successful AI implementation requires that you identify the right data sources and ensure that the data is complete, of high quality, in the right format, and representative of your target customers and business objectives.

“Organizations continue to see their data as their competitive advantage,” says Wangui McKelvey. “Unlocking insights from their data, across their organization, in a single, integrated platform, empowers businesses to take advantage of AI. If you don’t understand the insights you can deliver from your data with AI, then you’re going to be left behind.”

Consider what to build vs. what to buy

The decisions you make today to buy, build, or modernize your technology infrastructure can significantly influence your ability to execute on future goals, so it’s essential to choose a path that is supportive of your organization’s vision. Here are some of the considerations and trade-offs:

• Build: A “build” strategy enables you to tailor solutions precisely to your needs and implement your strategic business vision. Custom-built AI applications give organizations greater flexibility and control over the development process, facilitating scalability to accommodate future growth and evolving business needs.
• Modernize: In some scenarios, existing AI applications may still deliver value, but need an update to meet current demands. This approach can be beneficial when you have substantial investments in legacy AI infrastructure, but technology has evolved and your applications need to catch up.
• Buy: Some businesses buy prebuilt AI solutions, which may be a viable option if your AI use cases align with existing products. However, prebuilt AI solutions offer limited customization, reducing the ability to support more specific business needs. Other important considerations include integration complexities with existing IT systems, scalability limitations, and cumulative long-term costs, including licensing fees and upgrades.

Determine a strategy for regulatory compliance and safeguarding your AI assets

Regulatory compliance, particularly with the General Data Protection Requirement (GDPR) and the coming EU AI Act, introduces a set of technical, legal, and operational considerations for infrastructure choices.¹ Factors such as data sensitivity, data residency, scalability, and governance all play a part in architecture decisions, whether organizations choose on-premises, cloud, or co-located deployments. One critical factor to consider is access to sophisticated security measures that utilize machine learning, AI, and global threat intelligence databases to contribute to a more robust defense against cyber threats.

“Contrary to common belief, on-premises environments are often less secure than the cloud,” says Omar Khan. “Despite the sense of control associated with in-house infrastructure, cloud-based security solutions have evolved significantly, offering more advanced tools and technologies for threat detection and mitigation.”

Next steps

Stay tuned for the next post in our series, “Building a Foundation for AI Success: AI strategy and experience,” in which we will explore the factors that contribute to a successful AI strategy and experience for customers. We will follow the next entry with dedicated posts focusing on organization and culture, and AI governance.

Download a copy of “Building a Foundation for AI Success: A Leader’s Guide.”

---

¹EU AI Act.