Google Adds New Crawler Named GoogleOther To Help Googlebot

SEO Content Writing Service

Google has added a new crawler to its list of Google Crawlers and user agents, this one is named GoogleOther. It is described as a “generic crawler that may be used by various product teams for fetching publicly accessible content from sites.”

For example, it may be used for one-off crawls for internal research and development, Google explained. The GoogleOther crawler always obeys robots.txt rules for its user agent token and the global user agent (*), and uses the same IP ranges as Googlebot.

The User agent token is “GoogleOther” and the full user agent string is “GoogleOther.”

Gary Illyes from Google explained on LinkedIn that this new crawler will “replace some of Googlebot’s other jobs like R&D crawls to free up some crawl capacity for Googlebot.”

Gary wrote, “We added a new crawler, GoogleOther to our list of crawlers that ultimately will take some strain off of Googlebot. This is a no-op change for you, but it’s interesting nonetheless I reckon. As we optimize how and what Googlebot crawls, one thing we wanted to ensure is that Googlebot’s crawl jobs are only used internally for building the index that’s used by Search. For this we added a new crawler, GoogleOther, that will replace some of Googlebot’s other jobs like R&D crawls to free up some crawl capacity for Googlebot. The new crawler uses the same infrastructure as Googlebot and so it has the same limitations and features as Googlebot: hostload limitations, robotstxt (though different user agent token), http protocol version, fetch size, you name it. It’s basically Googlebot under a different name.”

There is no comment on if this crawler may or may not be used for Google Bard.

So if you see this new crawler, GoogleOther, this is what it is about.

I think they should have named it “MiniMeGoogleBot.”

Just a clarification from Gary:

> used by Google product teams for internally building the Google index

the other way around: Googlebot is used for index building, GoogleOther for the other tasks historically owned by Googlebot

— Gary 鯨理/경리 Illyes (@methode) April 20, 2023

like, random one-off crawls we used to do for learning something. Wanna learn how many sites *require* an accept language request header? run a crawl job. This used to be with googlebot, now it’s gonna be with GoogleOther.
Please don’t overthink it, it’s really that boring.

— Gary 鯨理/경리 Illyes (@methode) April 20, 2023

Forum discussion at LinkedIn.

Source link

Anxiety Stress Management

Live a Life of Contentment eBook We all want to be satisfied, even though we know some people who will never be that way, and others who see satisfaction as a foreign emotion that they can’t hope to ever feel.

Newspaper Ads Canyon Crest CA

Click To See Full Page Ads

Click To See Half Page Ads

Click To See Quarter Page Ads

Click To See Business Card Size Ads

If you have questions before you order, give me a call @ 951-235-3518 or email @ Like us on Facebook Here

You May Also Like

About the Author: 4ebusinessmediagroup99

Leave a Reply

Your email address will not be published. Required fields are marked *

Home Privacy Policy Terms Of Use Anti Spam Policy Contact Us Affiliate Disclosure Amazon Affiliate Disclaimer DMCA Earnings Disclaimer