May 2024 should go down as an historic month for Google. The company will hope that it is remembered for the successful launch of AI Overviews to hundreds of millions of US users, but other news may overshadow that. Soon after the release of AI Overviews (the new name for AI-generated responses in Google Search) the new feature got some bad press. Then things got worse for Google with news that code from its search engine had been leaked. Some people in the SEO industry think that the leak proves that Google lies to us about how search works and others say it should change the way we do SEO going forward. We don’t think so, yet, but more about that below.
May 5th – Manual actions for Site Reputation Abuse
Announced with the March 2024 Spam Update, Google brought into effect a new spam policy against site reputation abuse on May 5th. The new policy meant that Google could issue manual actions against sites, but there was no dedicated update to ranking systems as expected. Danny Sulivan from Google confirmed that this “will indeed come but is not live yet” via the @searchliason X account.
In Search Engine Roundtable, Barry Schwartz reported that manual actions had been given to “some large reputable sites … including CNN, USA Today, LA Times, Fortune, Daily Mail, Outlook India, TimesUnion, PostandCourier, SFGATE and many more”. As we discussed in our recent article What to do When Your Site Disappears from Google? a manual action means that someone at Google has decided to give a site a penalty, usually meaning that the site disappears completely from Google. In the case of sites like CNN and USA Today, it was a subdomain (like https://coupons.cnn.com/) that has been removed from Google and not the main site.
May 14th – Launch of AI Overviews
As we shared in March, rumors were that Google would release AI-generated responses to all Google Search users on May 14th to coincide with the Google I/O 2024 event. This indeed happened with the news that generative responses previously available as the Search Generative Experience (SGE) would now be known as “AI Overviews”.
AI Overviews rolled out to all US-based users from May 14th to May 17th with Google promising that the feature will roll out over one billion users worldwide by the end of the year. Options will be added to AI Overviews to allow users to simplify or break down responses as well as provide planning capabilities.
As reported by EMarketer, many SEO specialists had warned that the technology was not ready for wide release and it did not take long for users to share strange answers they had received from AI Overviews. Although some screen-shots proved to be fake, advice to add glue to Pizza or eat at least one rock a day were among those answers confirmed by CNET. Many concerns about AI Overviews are also linked to the fact that it does not clearly indicate sources of information, and it will reduce traffic to websites even when the content of these sites is used to generate an answer.
Google reacted to negative reaction with a post AI Overviews: About last week written by Liz Reid, head of Google Search, on The Keyword blog. She stresses that errors are very few and that the first improvements have already been made to AI Overviews systems following the recent feedback. It is thought that AI-generated responses are shown for around 20% of searches. If you are in the USA, you should test to see if this feature is showing on top of search results for your keywords. Rank tracking tools like SEOPress Insights continue to report on organic results – being number one in organic results may generate less traffic if an AI-generated response features first.
There is no specific method to block Google from using content from your website in AI Overviews. You can use the nosnippet directive (SEO > Titles & Metas > Advanced in SEOPress) to stop Google from quoting a page in search results, but this will affect normal search results as well as AI Overviews.
Leaked Google Search API Documents
On May 28th, Rand Fishkin revealed on X and a blog post what he described as “mind-boggling” news that he had been given access to a leaked document containing thousands of internal files containing code from Google Search. Originally quoted as an anonymous source, the person who contacted Rand has now been identified as Erfan Azimi an SEO expert who claims to have accidentally stumbled upon the code on GitHub.
Rand shared the leak with SEO Mic King who then spent some time analyzing over 2 thousand files before publishing his findings on his own blog post Secrets from the Algorithm: Google Search’s Internal Engineering Documentation Has Leaked on May 27th.
There has been a flurry of hyperbole around this announcement including accusations that Google has deliberately lied about how search works. But it is worth noting that the leaked code is not actually Google’s algorithm, but simply the attributes that may be used by its ranking systems. Google officially reacted to the leak in an email to The Verge on May 29th. They quote Davis Thompson of Google as saying, “We would caution against making inaccurate assumptions about Search based on out-of-context, outdated, or incomplete information. We’ve shared extensive information about how Search works and the types of factors that our systems weigh, while also working to protect the integrity of our results from manipulation.”
At SEOPress, nothing that we have seen so far in the leaked documents has surprised us or made us want to change advice we give on doing SEO. WordPress is not mentioned anywhere in the leaked code. The main revelations (Google uses domain authority, click data from search results and data from Chrome) do not seem that surprising in the light of testimony from the recent anti-trust trial in the US. It is very probable that Google has wanted to keep this information secret to discourage manipulation by spammers.
Rand Fiskin, meanwhile, reacted to indifference by some leading search experts by saying, “If you’re not changing anything about how you do SEO following the Google Leak… You’re not paying attention.” He went on to outline in an 11-minute video how marketers and publishers should change the way they do SEO in light of this news. Mic King also wrote a follow-up post for Search Engine Land How SEO moves forward with the Google Content Warehouse API leak where he answers some questions about the importance of the leak, encourages site owners to access and analyze the files for themselves and shares his advice including
- Pay more attention to click metrics
- Content needs to be more focused
- Page titles can be as long as you want
- Use fewer authors on more content
- Focus on link relevance from sites with traffic
Unconfirmed update
While the above news was going on, it is worth noting that despite no official update rolling out, the month of May saw a lot of volatility in search results with a peak on May 31st / June 1st as shared by Barry Schwartz on SEO Roundtable. This sort of activity in SERP tracking software and chatter in webmaster and SEO forums is normally a good indication that something has changed in Google’s search ranking (outside of AI Overviews) but as no official announcement is forthcoming then we call this an “unconfirmed update”. If you have lost traffic in May, you may have been impacted by this unconfirmed update and not AI Overviews.
It was certainly a good month for Google to release an update to organic search results without anyone noticing!
Some of the images from SERP tracking tools shared by Barry Schwartz on June 1st
