Google Introduces New Crawler To Optimize Googlebot’s Performance

Apr 23, 2023 | SEO News Feeds | 0 comments



SEO Content Writing Service

Google has recently introduced a new web crawler called “GoogleOther,” designed to alleviate strain on Googlebot, its primary search index crawler.

Attorney Websites for sale

The addition of this new crawler will ultimately help Google optimize and streamline its crawling operations.

Web crawlers, also known as robots or spiders, automatically discover and scan websites.

Googlebot is responsible for building the index for Google Search.

GoogleOther is a generic web crawler that will be used by various product teams within Google to fetch publicly accessible content from websites.

In a LinkedIn post, Google Search Analyst Gary Illyes shares more details.

Dividing Responsibilities Between Googlebot & GoogleOther

The main purpose of the new GoogleOther crawler is to take over the non-essential tasks currently performed by Googlebot.

Attorney Websites For Sale Criminal Defense Attorney

By doing so, Googlebot can now focus solely on building the search index utilized by Google Search.

Meanwhile, GoogleOther will handle other jobs, such as research and development (R&D) crawls, which are not directly related to search indexing.

Illyes states on LinkedIn:

“We added a new crawler, GoogleOther to our list of crawlers that ultimately will take some strain off of Googlebot. This is a no-op change for you, but it’s interesting nonetheless I reckon.

Attorney Websites For Sale 4ebusiness Media Group

As we optimize how and what Googlebot crawls, one thing we wanted to ensure is that Googlebot’s crawl jobs are only used internally for building the index that’s used by Search. For this we added a new crawler, GoogleOther, that will replace some of Googlebot’s other jobs like R&D crawls to free up some crawl capacity for Googlebot.”

GoogleOther Inherits Googlebot’s Infrastructure

GoogleOther shares the same infrastructure as Googlebot, meaning it possesses the same limitations and features, including host load limitations, robots.txt (albeit with a different user-agent token), HTTP protocol version, and fetch size.

Essentially, GoogleOther is Googlebot operating under a different name.

Implications For SEOs & Site Owners

The introduction of GoogleOther should not significantly impact websites, as it operates using the same infrastructure and limitations as Googlebot.

Nonetheless, it’s a noteworthy development in Google’s ongoing efforts to optimize and streamline its web crawling processes.

If you’re concerned about GoogleOther, you can monitor it in the following ways:

  • Analyze server logs: Regularly review server logs to identify requests made by GoogleOther. This will help you understand how often it crawls your website and which pages it visits.
  • Update robots.txt: Ensure your robots.txt file is updated to include specific rules for GoogleOther if necessary. This will help you control its access and crawling behavior on your website.
  • Monitor crawl stats in Google Search Console: Keep an eye on crawl stats within Google Search Console to observe any changes in crawl frequency, crawl budget, or the number of indexed pages since the introduction of GoogleOther.
  • Track website performance: Regularly monitor your website’s performance metrics, such as load times, bounce rates, and user engagement, to identify any potential correlations with GoogleOther’s crawling activities. This will help you detect if the new crawler is causing any unforeseen issues on your website.

Source: Google

Featured Image: BestForBest/Shutterstock

Source link


Anxiety Stress Management

Live a Life of Contentment eBook We all want to be satisfied, even though we know some people who will never be that way, and others who see satisfaction as a foreign emotion that they can’t hope to ever feel.

Newspaper Ads Canyon Crest CA

Click To See Full Page Ads

Click To See Half Page Ads

Click To See Quarter Page Ads

Click To See Business Card Size Ads

If you have questions before you order, give me a call @ 951-235-3518 or email @ canyoncrestnewspaper@gmail.com Like us on Facebook Here

Attorney Lead Generation Websites For Sale 4ebusiness Media Group
You May Also Like

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Contact Us

Contact Us

Personal Injury Attorney

Websites For Sale Personal Injury Attorneys

Criminal Defense Attorneys

Websites For Sale Criminal Defense Attorney

Bankruptcy Attorneys

Websites For Sale Bankruptcy Attorneys

General Practice Attorneys

Websites For Sale General Practice Attorneys

Family Attorneys

Websites For Sale Family Attorneys

Corporate Attorneys

Websites For Sale Corporate Attorneys

Home Privacy Policy Terms Of Use Anti Spam Policy Contact Us Affiliate Disclosure Amazon Affiliate Disclaimer DMCA Earnings Disclaimer