Google Cloud Next ’23 - Day 1 Recap

We were thrilled to be in person at San Francisco's Moscone Center for the highly anticipated Google Cloud Next event, marking its first large-scale in-person gathering since 2019! The event started with a captivating keynote delivered by Google CEO Sundar Pichai, who delved into the remarkable advancements Google has made in its AI technologies to enhance their usefulness and responsibility. Subsequently, Google Cloud CEO Thomas Kurian shed light on how customers are leveraging our AI capabilities, infrastructure, and tools to revolutionize their business operations and provide unparalleled experiences to their customers.

New infrastructure and tools to help customers

Google has made significant investments in our data centers and network over the past 25 years to support the advanced capabilities and wide-ranging applications of gen AI. With a global network of 38 cloud regions, the goal is to operate on carbon-free energy 24/7 by 2030, ensuring the most sophisticated and capable infrastructure for gen AI.

Their AI-optimized infrastructure has become the top choice for training and serving gen AI models. In fact, more than 70% of gen AI unicorns, such as AI21, Anthropic, Cohere, Jasper, MosaicML, Replit, Runway, and Typeface, are Google Cloud customers. Additionally, over half of all funded gen AI startups, including Copy.ai, CoRover, Elemental Cognition, Fiddler AI, Fireworks.ai, PromptlyAI, Quora, Synthesized, Writer, and many others, rely on Google's infrastructure.

To further support their customers, we are excited to announce key infrastructure advancements, including:

• Cloud TPU v5e: Introducing the latest breakthrough in AI acceleration - the Cloud TPU v5e. This cutting-edge platform offers unparalleled cost-efficiency, versatility, and scalability. With the Cloud TPU v5e, customers can now seamlessly conduct large-scale AI training and inference on a single platform. This state-of-the-art technology scales to tens of thousands of chips and is meticulously optimized for maximum efficiency. In comparison to its predecessor, the Cloud TPU v4, the v5e delivers up to a 2x improvement in training performance per dollar and an impressive 2.5x improvement in inference performance per dollar.

• A3 VMs with NVIDIA H100 GPU: Next month, organizations will have access to the powerful A3 VMs, which are equipped with NVIDIA's H100 GPU. These VMs have been specifically designed with advanced networking capabilities to cater to the demands of cutting-edge gen AI and large language model (LLM) innovations. With these high-performance VMs, organizations can achieve a remarkable three-fold improvement in training performance compared to the previous-generation A2.

• GKE Enterprise: With the introduction of multi-cluster horizontal scaling, organizations can now experience remarkable productivity gains of 45% and reduce software deployment times by over 70%. The latest update brings autoscaling, workload orchestration, and automatic upgrades to the Cloud TPU v5e, making it an ideal choice for the most demanding and mission-critical AI/ML workloads.

• Cross-Cloud Network: Cross-Cloud Network is a cutting-edge global networking platform that revolutionizes the way customers connect and secure their applications across multiple clouds. It provides an open, optimized infrastructure with ML-powered security, ensuring a zero trust environment. With its innovative design, Cross-Cloud Network makes it effortless for customers to access Google services seamlessly from any cloud, while also significantly reducing network latency by up to 35%.

• Google Distributed Cloud: To cater to the specific requirements of organizations that aim to run workloads at the edge or within their own data center, Google is introducing enhancements to its GDC portfolio. These enhancements include next-generation hardware, advanced security capabilities, and integrations with Vertex AI. Additionally, Google is offering a new managed solution called AlloyDB Omni on GDC Hosted, bringing the power of AI to the edge as mentioned earlier. 

Enhanced capabilities of Google's Vertex AI platform

On top of our world-class infrastructure, we deliver what we believe is the most comprehensive AI platform — Vertex AI — which enables customers to build, deploy and scale machine learning (ML) models. We have seen tremendous usage, with the number of gen AI customer projects growing more than 150 times from April-July this year. Customers have access to more than 100 foundation models, including third-party and popular open-source versions, in our Model Garden. They are all optimized for different tasks and different sizes, including text, chat, images, speech, software code, and more. We also offer industry specific models like Sec-PaLM 2 for cybersecurity, to empower global security providers like Broadcom and Tenable; and Med-PaLM 2 to assist leading healthcare and life sciences companies including Bayer Pharmaceuticals, HCA Healthcare, and Meditech. 

Vertex AI Search and Conversation are now generally available, enabling organizations to create Search and Chat applications using their data in just minutes, with minimal coding and enterprise-grade management and security built in. In addition, Vertex AI Generative AI Studio provides user-friendly tools to tune and customize models, all with enterprise-grade controls for data security. These include developer tools like Text Embeddings API, which lets developers build sophisticated applications based on semantic understanding of text or images, and Reinforcement Learning from Human Feedback (RLHF), which incorporates human feedback to deeply customize and improve model performance. 

Today, we’re excited to announce several new models and tooling in the Vertex AI platform:

• PaLM 2, Imagen and Codey Upgrades: We’re updating PaLM 2 to 32k context windows so enterprises can easily process longer form documents like research papers and books. We’re also improving Imagen’s visual appeal, and extending support for new languages in Codey.

• Tools for tuning: For PaLM 2 and Codey, we’re making adapter tuning generally available and in preview respectively, which can help improve LLM performance with as few as 100 examples. We're also introducing a new method of tuning for Imagen, called Style Tuning, so enterprises can create images aligned to their specific brand guidelines or other creative needs with a small amount of reference images.

• New models: We’re announcing availability of Llama 2 and Code Llama from Meta, and Technology Innovative Institute’s Falcon LLM, a popular open-source model, as well as pre-announcing Claude 2 from Anthropic. In the case of Llama 2, we will be the only cloud provider offering both adapter tuning and RLHF.

• Vertex AI extensions: Developers can access, build, and manage extensions that deliver real-time information, incorporate company data, and take action on the user's behalf. This opens up endless new possibilities for gen AI applications that can operate as an extension of your enterprise, enabled by the ability to access proprietary information and take action on third-party platforms like your CRM system or email.

• Grounding: We are announcing an enterprise grounding service that works across Vertex AI foundation models, Search and Conversation that gives customers the ability to ground responses in their own enterprise data to deliver more accurate responses. We are also working with a few early customers to test grounding with the technology that powers Google Search.

• Digital Watermarking on Vertex AI: Powered by Google DeepMind SynthID, this offers a state-of-the art technology that embeds the watermark directly into the image of pixels, making it invisible to the human eye and difficult to tamper with. Digital watermarking provides customers with a scalable approach to creating and identifying AI-generated images responsibly. We are the first hyperscale cloud provider to offer this technology for AI-generated images.

• Colab Enterprise: This managed service combines the ease-of-use of Google’s Colab notebooks with enterprise-level security and compliance capabilities. Data scientists can use Colab Enterprise to collaboratively accelerate AI workflows with access to the full range of Vertex AI platform capabilities, integration with BigQuery, and even code completion and generation. 

Equally important to discovering and training the right model is controlling your data. From the beginning, we designed Vertex AI to give you full control and segregation of your data, code, and IP, with zero data leakage. When you customize and train your model with Vertex AI — with private documents and data from your SaaS applications, databases, or other proprietary sources — you are not exposing that data to the foundation model. We take a snapshot of the model, allowing you to train and encapsulate it together in a private configuration, giving you complete control over your data. Your prompts and data, as well as user inputs at inference time, are not used to improve our models and are not accessible to other customers.

Duet AI in Google Cloud

Today, Duet AI in Google Workspace generally available, while expanding the preview capabilities of Duet AI in Google Cloud, with general availability coming later this year. 

In addition to its capabilities within Workspace, Duet AI has expanded its reach to provide AI assistance across various Google Cloud products and services. It serves as a coding assistant for developers, offering faster coding solutions. It also acts as an expert adviser, helping operators troubleshoot application and infrastructure issues swiftly. Furthermore, Duet AI serves as a data analyst, providing quick and insightful data analysis, and as a security adviser, recommending best practices to prevent cyber threats.

The value of Duet AI in Google Cloud has already been realized by customers such as L’Oréal, who have achieved better and faster business decisions using their data. Early testing by Turing has shown engineering productivity gains of one-third.

The latest announcements regarding Duet AI in Google Cloud include advancements in:

• Software development: Duet AI acts as a trusted companion throughout your software development journey, empowering developers to maintain their focus and productivity by reducing the need for context switching. Beyond providing code completion and generation, it accelerates the modernization of applications by offering guidance on code refactoring. Moreover, with Duet AI seamlessly integrated into Apigee, developers can effortlessly create APIs and integrations using simple, natural language prompts. Experience the transformative power of Duet AI in optimizing your software development process.

• Application and infrastructure operations: Operators have the incredible opportunity to engage in natural language conversations with Duet AI across various services, right within the Google Cloud Console. This seamless interaction allows operators to swiftly access valuable "how to" information regarding infrastructure configuration, deployment best practices, and receive expert recommendations for optimizing cost and performance. Say goodbye to time-consuming searches and hello to efficient and effective operations with Duet AI as your trusted guide.

• Data Analytics: Duet AI in BigQuery revolutionizes data analytics by providing contextual support for crafting SQL queries and Python code. It goes beyond simple suggestions, offering comprehensive functions and code blocks, as well as explaining SQL statements in plain language. Moreover, Duet AI leverages your schema and metadata to generate personalized recommendations. These advanced capabilities empower data teams to focus on driving impactful outcomes for their business.

• Accelerating and modernizing databases: Duet AI seamlessly integrates with Cloud Spanner, AlloyDB, and Cloud SQL to empower users in generating code that structures, modifies, or queries data using natural language. Its power extends to the Database Migration Service (DMS), automating the conversion of complex database code, including stored procedures, functions, triggers, and packages, which were previously challenging to convert using traditional translation technologies.

• Security Operations: Moreover, Google is introducing Duet AI to enhance its suite of security products, including Chronicle Security Operations, Mandiant Threat Intelligence, and Security Command Center. This integration empowers security professionals to proactively prevent threats, streamline security workflows, and elevate their expertise.

Duet AI goes beyond merely providing recommendations and expert guidance. It taps into the power of PaLM 2 LLM models and leverages Google Cloud-specific content, including documentation, sample code, and best practices, to deliver contextual insights. With a strong focus on protecting customer security and privacy, as well as adhering to AI principles, Duet AI ensures that your data remains exclusively yours. Rest assured that your code, inputs to Duet AI, and recommendations generated by the system will never be utilized to train shared models or develop any products.

Day 1 in a nutshell

Google is at the forefront of a new era of digital transformation, driven by the power of gen AI. This revolutionary technology is already revolutionizing how businesses operate and how people interact with one another. It is transforming the healthcare industry, enabling doctors to provide better care for their patients. It is revolutionizing communication, making it easier for people to connect and collaborate. And it is even enhancing workplace safety, ensuring that workers are protected on the job. But this is just the beginning. Stay tuned for more updates and don't hesitate to reach out to our consultants to unlock the full potential of Gen AI capabilities.