How well do Google’s new AI features really work in Gmail?

If it seems like everybody is racing to build AI into their products these days, it’s because they are. 

Google is one of the latest to announce how they’ll be building AI into their product, starting with Google Workspace tools like Docs and Gmail. As part of their rollout plan, they’ll release a first version of their AI features to trusted testers. We got early access and were able to play around to see how well their ai writing assistant tools work. 

Overall, we found the tool to be cleanly designed and intuitive, but they fell short of their potential. The AI writing tools missed the mark by not leveraging data in Gmail and Docs and including certain capabilities that could have transformed it into a truly useful writing assistant. 

While we’re sure they’ll be continuing to update and improve the tool, we’ll share our initial impressions from testing Google’s AI features below.

<a href="#google-ai-features" class="anchor-link">How Google plans to incorporate AI into Google Workspace products</a>

• <a href="#generative-ai" class="anchor-link">What is generative AI?</a>
• <a href="#ai-features-docs-gmail" class="anchor-link">AI features for Docs and Gmail</a>
• <a href="#responsible-approach-ai" class="anchor-link">A responsible approach to AI</a>

<a href="#how-google-ai-gmail" class="anchor-link">How Google’s new AI features work in Gmail</a>

• <a href="#testing-gmail-ai" class="anchor-link">Testing Gmail’s AI writing assistant</a>

<a href="#what-worked-well" class="anchor-link">What worked well</a>
<a href="#what-could-be-improved" class="anchor-link">What could be improved</a>
<a href="#conclusion-google-ai" class="anchor-link">Conclusion: A decent V1 feature with a lot of potential</a>

<div class="anchor-wrapper"><div id="google-ai-features" class="anchor-target"></div></div>

How Google plans to incorporate AI into Google Workspace products

Google’s recent announcement of “A new era for AI and Google Workspace” describes how they’ll responsibly incorporate generative AI into their Gmail and Docs products. 

The announcement is carefully crafted to demonstrate the thought they’re putting into this launch and the weight of responsibility they assume when incorporating AI into an app used by billions of people.

<div class="anchor-wrapper"><div id="generative-ai" class="anchor-target"></div></div>

What is generative AI?

“Generative AI” refers to a subset of artificial intelligence (AI) models and algorithms that can create or generate new content, such as text, images, or videos, that has not been explicitly programmed or designed by humans. 

In contrast, other types of AI models, such as predictive or classification models, are focused on analyzing and making predictions based on existing data. These models typically do not generate new content or outputs.

As you might imagine, generative AI lends itself well to Google’s products like Docs and Gmail, where users spend a lot of time writing content. 

<div class="anchor-wrapper"><div id="ai-features-docs-gmail" class="anchor-target"></div></div>

AI features for Docs and Gmail

Google’s first AI features will be found in Docs and Gmail and will focus on helping users write better content and emails.

Google Docs is an extremely versatile tool used for drafting job descriptions, writing blog posts, taking meeting notes, and more. With Google’s new AI features in Docs, you’ll be able to enter a topic or prompt and generate a complete draft. 

The AI built into Gmail has similar functionality, but will allow you to enter notes or prompts in an email draft, then choose a series of options to turn it into a complete message. 

Based on the type of content you want and even what voice or tone you’re looking for, you can transform your notes into an email with a single tap or click. Once you have a draft, you can use Google’s pre-set options to adjust the output.

The options to personalize the AI-generated content include: 

• Formalize
• Elaborate
• Shorten
• “I’m feeling lucky”

The final “I’m feeling lucky” option is a nod to the way AI can often delight users with something slightly different and seemingly unpredictable. This will write your email in “a new playful voice” to try something out of the ordinary.

<div class="anchor-wrapper"><div id="responsible-approach-ai" class="anchor-target"></div></div>

A responsible approach to AI

Google’s announcement emphasized the importance of “building features with… great care, thoughtful experimentation, and many interactions driven by user feedback.” 

In fact, they have a set of AI principles that are designed to “keep the user in control.” The principles include guidelines like “Be built and tested for safety” and “Be accountable to people” along with “Uphold high standards of scientific excellence.” 

In order to gather user feedback, Google plans to launch the new AI features in March to their trusted tester program in the US. Once they’ve completed testing, they’ll make the AI tools available to more users and in languages beyond English. 

<div class="anchor-wrapper"><div id="how-google-ai-gmail" class="anchor-target"></div></div>

How Google’s new AI features work in Gmail

As part of the trusted tester program, we got early access to Google’s AI features in Gmail and Docs. Overall, the feature is straightforward and very intuitive, but we found that it fell short on a lot of its potential to be a more useful writing assistant.

Below, we’ll share our experience with Google’s AI features and how we think they could be improved.

<div class="anchor-wrapper"><div id="testing-gmail-ai" class="anchor-target"></div></div>

Testing Gmail’s AI writing assistant

We tested the “Help me write” feature in Gmail, which is designed to accept a prompt and draft a complete email.

The new AI icon can be found in the toolbar and allows you to enter a prompt. We started with a simple one: “email andrew about taking a motorcycle ride tomorrow”. 

The draft that ensued was fairly good, taking a few liberties with suggestions, like when and where to meet. 

While the tone seemed a bit stiff, you can use the “elaborate” function to hone in on the voice and tone you want. Of course, you can always manually edit parts of the message to better match your own voice and tone. 

Obviously, we went straight for the “I’m Feeling Lucky” option to see what we’d get. The results were odd at best and unusable at worst.

The first result was a mildly Shakespearean poem that followed no clear rhyming scheme. However, it did include relevant information like an invitation to go for a motorcycle ride and a suggestion of when and where.

The second email draft created by the Gmail AI was completely unusable. It’s attempt at something that might be delightfully witty completely missed the mark by not including any mention of a motorcycle ride, the one piece of information I included in the prompt.

I’m pretty sure I’d be getting a concerned call from Andrew if I actually hit send on this one. 

<div class="anchor-wrapper"><div id="what-worked-well" class="anchor-target"></div></div>

What worked well

Overall, the AI writing tools worked as expected: the experience was simple and intuitive, and the standard output produced an acceptable email draft.

The most impressive part of Gmail’s new AI features is the UI. It’s polished, clean, and looks like a slightly futuristic version of the Gmail UI we’re all familiar with thanks to a new color gradient. 

While the current version doesn’t offer much above and beyond OpenAI’s ChatGPT or the hundreds of other writing assistant apps and extensions, its main competitive advantage is being built directly into Gmail. Pretty soon, we think Google Workspace users will be able to say goodbye to writing assistant extensions and ChatGPT browser windows - at least when drafting emails in Gmail.

<div class="anchor-wrapper"><div id="what-could-be-improved" class="anchor-target"></div></div>

What could be improved

Google had the opportunity to do so much more in creating a “smarter” writing assistant by leveraging some of the data that’s already inside our Gmail accounts. Without doing this, Google is basically offering a ChatGPT equivalent in a more convenient location.

Here are a few of the ways we think Google’s AI features could be improved:

Use sent emails data to write in my own style

None of the drafts that Gmail’s AI writing assistant produced sounded like they came from me.

One concern about shifting to more and more AI-generated content is that we’ll end up with masses of lower quality communication. With thousands of emails that I’ve previously written in my sent folder, I would have liked Gmail to be able to hone in on my writing style to craft a more personalized email.

Use my Google Calendar to suggest meeting times

In the test email, I asked Google’s AI to write an email to Andrew about “taking a motorcycle ride tomorrow”. 

Since Google already has access to my Google Calendar, it could have reviewed any meetings and events I already have scheduled, and suggested a time that I have at least an hour free during the day. 

Instead, it seemed to pick a time at random that didn’t work with my schedule.

Fill out all email fields

In order to differentiate itself from an AI text generator, Gmail’s AI could also fill out the recipients and subject line so my email was truly ready to hit send.

How does it know who to send it to, though? Just take a look at my inbox. Since I’ve already emailed with Andrew in the past about riding, it would be logical that I’m sending him another message about the same topic. 

Similarly, a subject line would have been an easy bit of text to generate to complete my email draft.

Include more known variables

After using the same Gmail account for years, I find it hard to believe Google doesn’t know my name. 

It’s in my Gmail address, my signature, and my Workspace profile. In almost every email in my inbox, the sender addresses me by my name… and yet, it includes a [your name] variable at the end of the email draft. 

Signing off emails with my name seems like a shockingly simple way to personalize the experience and make it feel a bit more magical to the user. On top of that, it should know whether I typically sign off an email with “best,” or “cheers,” or “talk soon.” 

<div class="anchor-wrapper"><div id="conclusion-google-ai" class="anchor-target"></div></div>

Conclusion: A decent V1 feature with a lot of potential

Google's new AI features in Gmail and Docs fall short of their potential. The tools are intuitive and the user interface is polished, but they lack certain capabilities that could have made them truly transformative writing assistants.

While the current AI features are useful, they don't offer significant advantages over existing writing assistant apps. The main benefit is their integration within Gmail, eliminating the need for additional extensions or browser windows.

To improve, Google should leverage Gmail and other Workspace data to suggest meeting times, auto-fill recipients based on previous conversations, and personalize the writing style.

It does seem clear that this is a cautious V1 launch, and we’re looking at their first feature rolling out. Google’s AI shows promise, but requires further development to become an indispensable writing assistant.