Last week, Google announced a new initiative from their Trust & Safety team aimed at raising awareness of common online scams.
It's not horribly interesting news. But it was the top unread blog post I had from the various Google internal blogs I subscribe to, so I wanted to use it for a demonstration on why, at least for now, we should use but not trust AI.
I asked for a summary of the article with a simple prompt. Here's what I input:
summarize this article in two paragraphs with bullet points to outline the specific scams Google is highlighting. https://blog.google/technology/safety-security/a-new-way-were-helping-others-track-frauds-and-scams-online/
A simple request, right? Here's what I got back from three different LLM models.
The Responses
OpenAI's GPT-4o
Google’s Trust & Safety teams are intensifying efforts to combat online scams by launching a regular advisory to raise awareness about emerging fraudulent tactics. This initiative aims to protect the public and the broader digital ecosystem by sharing insights into the latest scam trends.
The advisory highlights several specific scams:
•Realistic public figure impersonation campaigns: Scammers use deepfake technology to create convincing impersonations of public figures.
•Investment scams: Fraudsters lure victims with promises of high returns on investments.
•Tech support scams: Scammers pose as technical support agents to gain access to personal information.
•Romance scams: Individuals are deceived into believing they are in a genuine relationship, leading to financial exploitation.
•Phishing attacks: Fraudulent communications trick individuals into revealing sensitive information.
By disseminating information on these scams, Google aims to equip users with the knowledge to recognize and avoid falling victim to such fraudulent schemes. (Google Blog)
OpenAI's GPT-4
Google’s recent advisory on online scams sheds light on several sophisticated fraud tactics that users should be aware of to protect themselves . Here’s a summary of the key scams highlighted by Google:
• Realistic Public Figure Impersonation: Scammers utilize deepfake technology to create convincing impersonations of well-known figures, tricking people with fraudulent investment opportunities and fake giveaways.
• Crypto Investment Schemes: These scams promise high returns with low risk, often using doctored images and videos of trusted public figures or brands to lure victims.
• App and Landing Page Cloning: Fraudsters create fake web pages or apps that mimic legitimate sites to steal personal information or distribute malware. They also target company login pages to gain unauthorized access to corporate environments.
Google is actively updating its policies and deploying technology to detect and prevent these scams. For instance, they have updated their Misrepresentation Policy to handle impersonations in Google Ads, developed tools like SynthID to identify AI-generated content, and enforced strict guidelines against deceptive apps and web pages . Always remain vigilant for signs like unnatural expressions in videos, strange promotions from public figures, or unusual page layouts that could indicate a scam.
Google's Gemini 1.5 Flash
Sure, here is a summary of the article in two paragraphs with bullet points to outline the specific scams Google is highlighting:
- Google is using AI to identify and combat scams. This includes developing machine learning models to detect phishing emails and websites, as well as identifying suspicious activity in Google products like Gmail and Google Search.
- Google is also working to protect users from financial harm. This includes providing warnings to users about potential scams, as well as making it easier for users to report scams.
What went wrong?
GPT-4o
With the first response, using OpenAI's "Great for most tasks" model, GPT-4o, the summary looks good at a glance. But only at a glance. The two paragraphs are accurate. This initiative is just about awareness and sharing information. But the bullet points are where it falls apart.
The first bullet is accurate. The second is pretty accurate, as the original blog has "Crypro investment schemes" as the 2nd point. The original also mentions "too-good-to-be-true returns". The third point in the original article covers app/landing page cloning for the purposes of phishing attacks. And specifically speaks to such attacks being aimed at business users (e.g. a fake login page for some SaaS tool you might use). There is a mention of tech support scams being a subset of these, but is a broader issue. I'd say the summary of bullet three is misleading.
The fourth bullet is straight up fabricated. The actual article talks about "Landing page cloaking", which is showing Google one page in order to bypass ranking/policy checks, but showing users something different. Generally a fraudulent copy of an e-commerce site or a page that makes it look like you have some terrible computer virus.
The last bullet in the original article is "Exploitation of major events". ChatGPT said "Phishing attacks". Google's summary of this scam is not about stealing personal information, but instead about using tragedies or major events generally to push sales of scammy products. E.g. some "NASA approved eclipse glasses" that have nothing to do with NASA.
If you were to pass this off as a summary of the article, anyone who read or even glanced at the original would immediately know you didn't actually read the piece.
GPT-4
The previous flagship model from ChatGPT actually did a great job summarizing the article. Really the only problem is it failed to summarize the last two scame examples. But the first three were nicely explained and were accurate to the original article.
The real worry here is how much important detail gets missed in other summaries. Two of the five items Google outlined did not get summarized, that's a pretty big gap in information.
Gemini
Oh boy! Let's just break this one down one bullet at a time (there's only two of them, so it's not a huge task).
Google is using AI to identify and combat scams. This includes developing machine learning models to detect phishing emails and websites, as well as identifying suspicious activity in Google products like Gmail and Google Search.
- The article does not mention Google using AI to identify and combat scams. Even the resources linked in the article are all policies for Google Ads and Google Search, which talk about what the policies are, but not how they are enforced or monitored.
- There's nothing about emails. No mention to any phishing-detection systems that GMail might have. Again, this article was focused on Search/Ads (and to a small degree, Play). The word "mail" doesn't even appear in the article.
Google is also working to protect users from financial harm. This includes providing warnings to users about potential scams, as well as making it easier for users to report scams.
Again, nothing about warning/detection systems. Nothing about reporting systems. Policies are linked to, but this article is about awareness, not Google's role in preventing scams.
And not to mention, my request to give a bulleted list of the different scams was ignored. My request to summarize in two paragraphs was ignored. I got two bullet points that were two sentences each.
Why did this happen?
In the earlier days of LLMs (you know, like 18 months ago), this commonly happened due to limitations of context windows. That is, the amount of information an LLM can hold in memory while generating a response. Think of it like delivering a book report in front of your class in school. A large context window would mean you could rememeber the plot/characters/etc of the entire book and summarize all of it accurately. A small context window would mean you could only remember the first few paragraphs or just the chapter titles. But huge chunks of information would be forgotten, leading to a really poor book report.
Context windows are still a liminitation for most LLM models. But generally, they are large enough to answer a single request about a short blog post.
Instead, this is more than likely an issue of temperature. LLM's work in a probabalistic way and use their training data to predict the most likely response to a prompt. Temperature is a setting that determines how much freedom the model has in that response. If an LLM has a very low temperature and you ask for two examples of ice cream flavors, it might say "chocolate and vanilla" every time. If the model has a very high temperature, it might say "chocolate and horseradish-mint."
You want to strike a balance of being mostly accurate but not super bland and predictable. There's no perfect temperature.
But what happens is, when summarizing an article that touches on several distinct topics (even if it's a short article), the model is trying to balance summarizing what it read with introducing those levels of randomness in order to make the output interesting. The longer the response goes, typically the more inaccurate it gets.
Try it Yourself
Take a blog post or publicly/freely available news article. Ask your LLM of choice to summarize that article in one or two paragraphs. Then ask it to take each major point of the article and summarize those in bullets. Or to list 7 key takeaways. Something that leads to a longer and more detailed answer.
Of course, with a short summary it's easier for a response to be vague enough that it is hard to prove inaccurate. But again, the longer the response gets, typically the more hallucinations you'll see.
The key is to understand these limitations so you can avoid looking silly when you reference some work AI did for you.
