We are also sharing the frontiers of our agentic research by showcasing prototypes enabled by Gemini 2.0’s native multimodal capabilities.
Gemini 2.0 Flash builds on the success of 1.5 Flash, our most popular model yet for developers, with enhanced performance at similarly fast response times. Notably, 2.0 Flash even outperforms 1.5 Pro on key benchmarks, at twice the speed. 2.0 Flash also comes with new capabilities. In addition to supporting multimodal inputs like images, video and audio, 2.0 Flash now supports multimodal output like natively generated images mixed with text and steerable text-to-speech (TTS) multilingual audio. It can also natively call tools like Google Search, code execution as well as third-party user-defined functions.
Our goal is to get our models into people’s hands safely and quickly. Over the past month, we’ve been sharing early, experimental versions of Gemini 2.0, getting great feedback from developers.
Gemini 2.0 Flash is available now as an experimental model to developers via the Gemini API in Google AI Studio and Vertex AI with multimodal input and text output available to all developers, and text-to-speech and native image generation available to early-access partners. General availability will follow in January, along with more model sizes.
To help developers build dynamic and interactive applications, we’re also releasing a new Multimodal Live API that has real-time audio, video-streaming input and the ability to use multiple, combined tools. More information about 2.0 Flash and the Multimodal Live API can be found in our developer blog.
Also starting today, Gemini users globally can access a chat optimized version of 2.0 Flash Experimental by selecting it in the model drop-down on desktop and mobile web and it will be available in the Gemini mobile app soon. With this new model, users can experience an even more helpful Gemini assistant.
Early next year, we’ll expand Gemini 2.0 to more Google products.
Gemini 2.0 Flash’s native user interface action-capabilities, along with other improvements like multimodal reasoning, long context understanding, complex instruction following and planning, compositional function-calling, native tool use and improved latency, all work in concert to enable a new class of agentic experiences.
The practical application of AI agents is a research area full of exciting possibilities. We’re exploring this new frontier with a series of prototypes that can help people accomplish tasks and get things done. These include an update to Project Astra, our research prototype exploring future capabilities of a universal AI assistant; the new Project Mariner, which explores the future of human-agent interaction, starting with your browser; and Jules, an AI-powered code agent that can help developers.
We’re still in the early stages of development, but we’re excited to see how trusted testers use these new capabilities and what lessons we can learn, so we can make them more widely available in products in the future.
Since we introduced Project Astra at I/O, we’ve been learning from trusted testers using it on Android phones. Their valuable feedback has helped us better understand how a universal AI assistant could work in practice, including implications for safety and ethics. Improvements in the latest version built with Gemini 2.0 include:
Better dialogue: Project Astra now has the ability to converse in multiple languages and in mixed languages, with a better understanding of accents and uncommon words.
New tool use: With Gemini 2.0, Project Astra can use Google Search, Lens and Maps, making it more useful as an assistant in your everyday life.
Better memory: We’ve improved Project Astra’s ability to remember things while keeping you in control. It now has up to 10 minutes of in-session memory and can remember more conversations you had with it in the past, so it is better personalized to you.
Improved latency: With new streaming capabilities and native audio understanding, the agent can understand language at about the latency of human conversation.
We’re working to bring these types of capabilities to Google products like Gemini app, our AI assistant, and to other form factors like glasses. And we’re starting to expand our trusted tester program to more people, including a small group that will soon begin testing Project Astra on prototype glasses.
Project Mariner is an early research prototype built with Gemini 2.0 that explores the future of human-agent interaction, starting with your browser. As a research prototype, it’s able to understand and reason across information in your browser screen, including pixels and web elements like text, code, images and forms, and then uses that information via an experimental Chrome extension to complete tasks for you.
When evaluated against the WebVoyager benchmark, which tests agent performance on end-to-end real world web tasks, Project Mariner achieved a state-of-the-art result of 83.5% working as a single agent setup.
It’s still early, but Project Mariner shows that it’s becoming technically possible to navigate within a browser, even though it’s not always accurate and slow to complete tasks today, which will improve rapidly over time.
To build this safely and responsibly, we’re conducting active research on new types of risks and mitigations, while keeping humans in the loop. For example, Project Mariner can only type, scroll or click in the active tab on your browser and it asks users for final confirmation before taking certain sensitive actions, like purchasing something.
Trusted testers are starting to test Project Mariner using an experimental Chrome extension now, and we’re beginning conversations with the web ecosystem in parallel.
Next, we’re exploring how AI agents can assist developers with Jules — an experimental AI-powered code agent that integrates directly into a GitHub workflow. It can tackle an issue, develop a plan and execute it, all under a developer’s direction and supervision. This effort is part of our long-term goal of building AI agents that are helpful in all domains, including coding.
More information about this ongoing experiment can be found in our developer blog post.
Google DeepMind has a long history of using games to help AI models become better at following rules, planning and logic. Just last week, for example, we introduced Genie 2, our AI model that can create an endless variety of playable 3D worlds — all from a single image. Building on this tradition, we’ve built agents using Gemini 2.0 that can help you navigate the virtual world of video games. It can reason about the game based solely on the action on the screen, and offer up suggestions for what to do next in real time conversation.
We're collaborating with leading game developers like Supercell to explore how these agents work, testing their ability to interpret rules and challenges across a diverse range of games, from strategy titles like “Clash of Clans” to farming simulators like “Hay Day.”
Beyond acting as virtual gaming companions, these agents can even tap into Google Search to connect you with the wealth of gaming knowledge on the web.
In addition to exploring agentic capabilities in the virtual world, we’re experimenting with agents that can help in the physical world by applying Gemini 2.0's spatial reasoning capabilities to robotics. While it’s still early, we’re excited about the potential of agents that can assist in the physical environment.
You can learn more about these research prototypes and experiments at labs.google.
Gemini 2.0 Flash and our research prototypes allow us to test and iterate on new capabilities at the forefront of AI research that will eventually make Google products more helpful.
As we develop these new technologies, we recognize the responsibility it entails, and the many questions AI agents open up for safety and security. That is why we are taking an exploratory and gradual approach to development, conducting research on multiple prototypes, iteratively implementing safety training, working with trusted testers and external experts and performing extensive risk assessments and safety and assurance evaluations.
For example:
As part of our safety process, we’ve worked with our Responsibility and Safety Committee (RSC), our longstanding internal review group, to identify and understand potential risks.
Gemini 2.0's reasoning capabilities have enabled major advancements in our AI-assisted red teaming approach, including the ability to go beyond simply detecting risks to now automatically generating evaluations and training data to mitigate them. This means we can more efficiently optimize the model for safety at scale.
As Gemini 2.0’s multimodality increases the complexity of potential outputs, we’ll continue to evaluate and train the model across image and audio input and output to help improve safety.
With Project Astra, we’re exploring potential mitigations against users unintentionally sharing sensitive information with the agent, and we’ve already built in privacy controls that make it easy for users to delete sessions. We’re also continuing to research ways to ensure AI agents act as reliable sources of information and don’t take unintended actions on your behalf.
With Project Mariner, we’re working to ensure the model learns to prioritize user instructions over 3rd party attempts at prompt injection, so it can identify potentially malicious instructions from external sources and prevent misuse. This prevents users from being exposed to fraud and phishing attempts through things like malicious instructions hidden in emails, documents or websites.
We firmly believe that the only way to build AI is to be responsible from the start and we'll continue to prioritize making safety and responsibility a key element of our model development process as we advance our models and agents.
Gemini 2.0, AI agents and beyond
Today’s releases mark a new chapter for our Gemini model. With the release of Gemini 2.0 Flash, and the series of research prototypes exploring agentic possibilities, we have reached an exciting milestone in the Gemini era. And we’re looking forward to continuing to safely explore all the new possibilities within reach as we build towards AGI.
What was on Kiwis' minds?
Kiwis had their eyes fixed on the world stage in 2024, with the US election dominating trending search queries. Sports captivated the nation. From the UEFA European Football Championship and Cricket T20 World Cup to the All Blacks’ rugby clash against England and the Australian Open, Kiwis proved once again they are sports fanatics.
A curious trend emerged this year: a surge in searches for the humble flat white. Perhaps it was fuelled by a rekindled debate about its origins - was it invented in New Zealand or Australia? Whatever the reason, this iconic Kiwi beverage struck a chord, landing a spot in the Top 10.
The tragic passing of Liam Payne sent shockwaves through the nation, sparking a wave of searches likely driven by One Direction nostalgia or a stark reminder of life's fragility. In a year marked by uncertainty, Kiwis sought escapism: flexing their vocabulary with the New York Times' Connections game or indulging their bargain-hunting instincts on Temu.
"Raygun", inspired by Australian Olympian Rachael Gunn's breakdancing, was a viral sensation in New Zealand, topping the memes chart. The "Demure" trend, with its emphasis on kindness and composure, resonated with Kiwis’ friendly and welcoming spirit. Classic memes like “What’s up brother” and “Knee surgery” also proved popular, showing that Kiwis ultimately appreciate a good laugh above all else.
How-to searches reveal a nation eager to learn
Kiwis' "how-to" searches in 2024 reveal a nation eager to learn, adapt and explore the digital world. "How to watch the Olympics in NZ" emerged as #1, showing a sporting nation keen to catch the action even from afar. Beyond sports, “How to lock Facebook profile" reflects a growing desire for online privacy.
“How to make human in Infinite Craft" and "How to say Happy Matariki in te reo” showcased a country embracing both digital innovation and cultural heritage. And who could forget the "How to mew" trend? This tongue-positioning technique, promising a sculpted jawline, demonstrates the unpredictable nature of online trends.
Finally, no Kiwi year would be complete without the America's Cup, with "How to watch America's Cup in NZ" rounding out the top searches.
This year's Google searches paint a vivid picture of New Zealand in 2024: a nation connected to the world yet proud of its unique identity, embracing both tradition and technology, with a keen interest in everything from sports and current events to quirky online trends.
To bring the year's top trending searches to life visually, we collaborated with the Kākano Youth Arts Collective, a programme that supports vulnerable young artists. Our collaborating artist has brilliantly used birds to show some of the key moments of 2024 – very cool and very Kiwi!
Kiwis will soon have greater protection from online scams as Google rolls out Financial Services Verification in New Zealand, a step forward in the company's decades-long efforts to combat scams and create a safer online experience.
Starting November 7, select advertisers of financial products and services will be required to complete a verification process before they can run ads on Google's platforms. For most advertisers, this will entail obtaining verification that they are authorised by New Zealand authorities, including the Financial Markets Authority and Reserve Bank of New Zealand, to ensure only legitimate financial service providers can advertise on our platforms. We believe this will reduce the risk of Kiwis falling victim to fraudulent financial schemes online.
Scammers are constantly evolving and devising new tactics to evade detection, making this program a defence in ensuring only authorised financial providers can reach users on Google. By verifying select advertisers, we hope to give Kiwis greater peace of mind when interacting with financial services online.
This program builds upon Google's existing multi-layered approach to combating fraud and scams, which includes strict policies against misleading financial information, advanced enforcement using machine learning and human review, and an advertiser verification program launched in 2020. These efforts work together to protect users and maintain a safe advertising ecosystem.
The support of the New Zealand Government is critical to the success of this program, and we appreciate their collaboration.
Google will continue partnering with the New Zealand government and industry leaders in the ongoing fight against online scams. Together, we can create a safer and more secure online experience for all Kiwis.
We are proud of our long standing contribution to New Zealand’s news industry. Our services help connect New Zealanders with quality journalism every day, driving valuable traffic to publishers. Through our local partnerships and investments, we continue to contribute to a sustainable, diverse and innovative news ecosystem in New Zealand, including through Google News Showcase - a licensing program that covers over 95% of New Zealand digital news publishers and results in us paying millions of dollars per year to almost 50 local publications. [1] Our investments in New Zealand news are targeted to help journalists and news publishers evolve in response to the changing way people are looking for and consuming information. Today, people are getting news from sources like short-form video, newsletters, social media, and curated podcasts, and many are avoiding the news entirely. We want to continue making contributions to the news ecosystem to help news publishers navigate this inflection point. We’ve already helped New Zealand partners navigate these changes by supporting work to increase paid subscribers, create technology solutions that are fit for this future and personalise user experiences. The Bill as currently drafted would put these contributions at risk.
Google's Concerns with the Bill
We believe the proposed "link tax" model is fundamentally flawed and would generate unintended consequences and unsustainable models. Here's why:
A Path Forward We’ve been engaging with New Zealand publishers and lawmakers throughout the legislative process and have proposed reasonable and balanced alternatives to the draft Bill. Google is currently the only tech company providing financial support to New Zealand's news industry - as we have been for over two years. Further strengthening New Zealand’s news industry will require additional public and private support from both the New Zealand Government and a broad base of private companies. Looking ahead, we encourage the Government to reconsider the current Bill and engage in constructive dialogue to find alternative solutions that will ensure a sustainable future for New Zealand journalism. We are confident that, together, we can develop a better path forward.
[1] This includes: NZ Herald, 1News, Stuff, Stuff Auckland, RNZ, The Post, Otago Daily Times, Otago Daily Times - Dunedin, The Spinoff, Newsroom, Waikato Times, The Star - Dunedin, The Star - Christchurch, Southland Times, Southland Express, Ashburton Guardian, Crux, Mahurangi Matters, Hibiscus Matters, Nelson App Online, Marlborough App Online, Northern Advocate, Northland Age, Kāhu, Waikato News, Bay of Plenty Times, Hawkes Bay Today, Rotorua Daily Post, Whanganui Chronicle, Stratford Press, Manawatu Guardian, Kapiti News, Horowhenua Chronicle, Te Awamutu Courier, Gisborne Herald, SunLive, Pacific Media Network, Scoop.co.nz, Taranaki Daily News, Manawatū Standard, Nelson Mail, Marlborough Express, The Press, Timaru Herald, Wairarapa Times-Age, Times Online, Wanaka App, Te Ao Māori News
[3] For example, independent analysis by The Media Ecosystem Observatory estimated a loss of Facebook traffic to Canadian news outlets of between 64 and 85% following similar legislation being introduced in Canada - similar impacts in New Zealand would be devastating for struggling smaller publishers.
Kia ora koutou! As country director of Google New Zealand, and a passionate advocate for te reo Māori, I'm thrilled to share our small role in celebrating this year's Te Wiki o te Reo Māori.
If you often use te reo Māori when you're chatting with friends, family or colleagues online, you might find this Chrome extension handy. Google Input Tools makes it easy to add macrons to your writing on a Chromebook, and switch between different languages with just a click.
If you wish to learn te reo Māori in a fun, interactive way, check out the Kupu app. We worked with Spark on this innovative app that uses Google's tech to turn everyday things into Māori lessons. Just take a photo of something, and the app will say what it is in te reo. Kupu has gotten even better with more words, and recently, a new "Stories" feature where you can share photos and new words with others! It's a great way to make language learning social and enjoyable.
We're also proud to have added Māori to Google Translate over ten years ago. It's been a big help for people learning Māori, Māori communities talking to others, and New Zealand translators. Recently, we added several more Pacific Island languages to Google Translate too, like Fijian and Tok Pisin, which will benefit even more people across the islands.
Finally, I couldn't be prouder of the grassroots initiatives blossoming within the Google New Zealand team. Each week, a dedicated colleague shares new Māori words and phrases during our team call segment "Te Reo Tuesday”, enriching our vocabulary and deepening our connection to the Māori culture. Another colleague of mine has, in consultation with our cultural partner AU, created an engaging online training module for Googlers, delving into Aotearoa's history, te Tiriti o Waitangi, and the relevance of Māori values in our workplaces. Whenever we welcome honored guests, the team and I would sing them a Māori song as part of a Māori welcome at the door. These ground-up efforts are invaluable in fostering a deeper appreciation of te ao Māori within our team.
But we know there's still more to do to support and protect this language, which is so important to us in Aotearoa. We’re open to hearing your ideas and thoughts, and excited to see what the future holds for te Reo Māori. I know we can make a real difference in preserving and promoting this beautiful language, and I'm grateful for the chance to be a part of this journey.
He mihi nui ki a koutou katoa! Happy Māori Language Week.
If you are a member of the press, please email our communications team at: press-australia-nz@google.com For all other inquiries, please visit our Help Center.
.jpg)
