News|google

Website owners have been excluding web crawlers using the Robots Exclusion Protocol (REP) on robots.txt files for 25 years. More than 500 million websites use robots.txt files to talk to bots, according to Google’s data. Up until now, there has never been an official Internet standard, no documented specification for (more…)

Google’s rich results testing tool has been updated to include a selector for desktop or mobile so website owners can check their results on both platforms. This tool shows if pages are properly structured for display in Google’s carousels, images, breadcrumbs, events, books, and other types of rich snippets. A (more…)

Google is enabling mobile-first indexing (crawling sites with a mobile user-agent) by default for new domains as of July 1, 2019. These are sites Google defines as “previously unknown to Google Search.” Mobile searches are the primary way that users engage with the search engine for the past few years, (more…)