AWS re: Invent 2024: A ‘critical inflexion point’ for AI & data
Suitably named for his deep-tech expertise and expansive knowledge of the Amazon Web Services (AWS) universe is Swami Sivasubramanian in his role as VP for AI and data at AWS.
Hosting the day two mainstage keynote, Sivasubramanian had some big messages to share on where we are today with AI.
He thinks that AI has reached a “tipping point” thanks to the convergence of technological progress and an increased understanding of what it can accomplish.
“Couple that with the massive proliferation of data, the availability of highly scalable compute capacity and the advancement of ML technologies over time… and the focus on generative AI is finally taking shape,” noted Sivasubramanian.
A monumental year
“It’s been a monumental year and disruption is the new normal,” said Sivasubramanian… in an introduction that touched on the innovation progressions that saw the Wright brothers first take flight.
Quickly moving to get technical and talk about the history of backpropagation (a machine learning technique that trains artificial neural networks by adjusting their weights based on the error rate of their predictions) through to the new era of large language models, Sivasubramanian explained how AI is now being used by developers to help them work through end-to-end software creation processes so much faster.
“I have always been driven by innate curiosity and the need to solve real customer problems in my 18 years at AWS,” enthused Sivasubramanian, before then explaining how firms like GE Healthcare are using AI stemming from AWS to provide more intelligent services.
“We can now converge big data analytics and AI and machine learning (with the next generation of Amazon Sagemaker) as a unified centre for data analytics and AI,” he said.
This technology aims to make deep learning more accessible for all users … it builds, trains and deploys machine learning models for any use case with fully managed infrastructure tools and workflows said Sivasubramanian.
Critical inflexion point
Noting that with so many large foundation models out there, firms would need some way to develop massive deep learning expertise… but this is what Sagemaker is designed for (to remove the heavy lifting) as it manages the process of training foundation models and accelerates model development at all levels.
“We’re at a critical inflexion point in model training in terms of data consumption where trillions of parameters need to be handled.. and traditional scaling techniques are reaching the limit of their ability to cope with these issues when it comes to model training and inference at scale,” explained Sivasubramanian.
What all these means is that teams need to define their compute requirements and get tools and workflows working faster. Sagemaker Hyperpod flexible training plans is a new service from AWS designed to address these challenges. This technology dovetails with the (also newly announced) Sagemaker Hyperpod task governance that works to help control the use of these functions.
Amazon Nova
AWS has of course used this event to detail Amazon Nova, a new generation of foundation models (FMs) with the ability to process text, image and video as prompts, data scientists and developers can use Amazon Nova-powered generative AI applications to understand videos, charts and documents, or generate videos and other multimedia content.
“Inside Amazon, we have about 1,000 generative AI applications in motion and we’ve had a bird’s-eye view of what application builders are still grappling with,” said Rohit Prasad, SVP of Amazon Artificial General Intelligence. “Our new Amazon Nova models are intended to help with these challenges for internal and external builders and provide compelling intelligence and content generation while also delivering meaningful progress on latency, cost-effectiveness, customization, information grounding and agentic capabilities.”
Amazon Bedrock is a fully-managed service that offers developers access to high-performing models from leading AI companies through a single API.
The new Amazon Nova models available in Amazon Bedrock include:
- Amazon Nova Micro, a text-only model that delivers the lowest latency responses at very low cost.
- Amazon Nova Lite, a very low-cost multimodal model that is lightning-fast for processing image, video and text inputs.
- Amazon Nova Pro, a capable multimodal model with the best combination of accuracy, speed and cost for a wide range of tasks.
- Amazon Nova Premier, the most capable of Amazon’s multimodal models for complex reasoning tasks and for use as the best teacher for distilling custom models (available in the Q1 2025 timeframe).
- Amazon Nova Canvas, a state-of-the-art image generation model.
- Amazon Nova Reel, a state-of-the-art video generation model.
“Amazon Nova highlights AWS’s commitment to delivering flexibility and choice in AI. By offering a diverse suite of foundation models, AWS empowers organisations to select the tools best suited for their specific workloads and industries. This approach realises that no two businesses are the same and ensures that companies can tailor AI solutions to their unique needs. This exemplifies the shift towards adaptable AI systems, giving enterprises the ability to innovate with precision and confidence while maintaining control over costs and scalability. It’s a critical step in enabling broader adoption across industries.” JB McGinnis, AWS lead at Deloitte Consulting LLP.
Sivasubramanian also spent time during his keynote address covering the need to work with inference at new and more powerful levels. Amazon Bedrock is positioned as the service developers need to use to build and scale generative AI applications and be able to engineer best-in-class technology into every inference-related task.
Selecting and operating the right models at the start is not a given, which is why Amazon Bedrock is engineered to enable developers to work across Mistral, Meta’s Llama, stability.ai’s model (where a new high-quality AI image generation service is now being added) and of course Anthropic as well.
Poolside is also coming to Amazon Bedrock next year… this technology provides software engineering AI for large enterprises. Luma AI (for high-quality video generation) is also being added to Amazon Bedrock.
A dip into poolside
AWS will now make poolside’s generative AI Assistant and foundation models available in Amazon Bedrock. As a result, enterprise customers will soon be able to customise poolside’s generative AI Assistant for software development with their own data, using the security, privacy and performance of AWS.
Bringing poolside’s models to Amazon Bedrock and EC2 provides enterprise customers of both companies with the ability to use poolside FMs, which are designed to be fine-tuned with each business’s code and data, producing a proprietary generative AI model and software engineering assistant for that specific business. poolside in Amazon Bedrock and EC2 will enable enterprise organizations to meet security, privacy and compliance concerns and deploy their custom generative AI models.
AWS customers will be able to deploy poolside models where their data is stored, within their firewalls, on the cloud and with no data shared back with poolside. Today, poolside’s Reinforcement Learning From Code Execution Feedback (RLCEF) technique runs on AWS , powering approximately one million container images, while allowing for 10,000 code executions per minute.
“We’ve been incredibly impressed with the AWS team and are excited to partner on this journey to unlock more of the potential of AI for software development,” said poolside CEO Jason Warner. “Today, the majority of developers sit within an enterprise environment where access to these tools is scarce and their full potential is not yet realised. Companies of this scale need a tailored model that can both capture their proprietary knowledge and learn from their interactions over time. That is what we’ve built and AWS’s reputation and depth within the enterprise is key to accelerating adoption and impact.”
Intelligent prompt routing for Amazon Bedrock was also announced – a technology designed to enable developers with the power to get directed to the most cost-effective model for any given inference job.
Amazon Kendra was also showcased, a tool designed to answer faster with enterprise search powered by machine learning. Amazon Kendra generative AI index enables developers to connect to more than 40 enterprise enterprise data sources.
Overall look & feel
Overall then, what can we say about AWS and its current slew of announcements?
It’s really a question of AWS working to deliver AI-focused cloud computing software engineering at more levels, in more efficient ways, through more granular controls and across a more expansive model universe. Obviously looking to make AWS-hosted services the AI infrastructure backbone of choice for any given use case, the diversity of tools and functions on show is (as always) almost dizzying at times, but for every powerful new channel being brought to the table, AWS is clearly thinking about the spectre of complexity and making sure it engineers enough backbone support and developer self-service management into technologies such as Amazon Bedrock and Amazon Sagemaker, plus of course Amazon Q Developer.
The always-upbeat Swami Sivasubramanian uses his keynote to explain real world AI & ML development and deployment issues to set up his favourite phrase (“That’s why today, I’m excited to announce…”) to explain how the latest services from AWS will make enterprise applications run more effectively.
It’s a lot to ingest in one session, but it’s worth the journey.
