Merge pull request #375 from ByteInternet/hypercharles-GoogleOther-patch

tdgroot · web-flow · commit 34a9c5dcbc24 · 2025-06-23T14:22:31.000+02:00
Update how-to-fix-performance-issues-caused-by-bots-and-crawlers.md
diff --git a/docs/best-practices/performance/how-to-fix-performance-issues-caused-by-bots-and-crawlers.md b/docs/best-practices/performance/how-to-fix-performance-issues-caused-by-bots-and-crawlers.md
@@ -55,7 +55,7 @@ GET /clothing/pants.html?color=1181_1188_1500 HTTP/1.1 Mozilla/5.0 (compatible;
 It is clear that the “size” and “color” parameters are used for layered navigation, because they define subsets of your catalogue. For normal indexing, you only want /clothing.html and /clothing/pants.html indexed. To block bots in this case, you should add this to `/data/web/nginx/server.layered_navigation:`
 
 ```nginx
-if ($http_user_agent ~* 'http|bot|crawl|spider') {
+if ($http_user_agent ~* 'http|bot|crawl|spider|GoogleOther') {
     set $is_bot 1;
 }
 # Here, you should define the layered navigation urls.

Original file line number	Diff line number	Diff line change
`@@ -55,7 +55,7 @@ GET /clothing/pants.html?color=1181_1188_1500 HTTP/1.1 Mozilla/5.0 (compatible;`
`55`	`55`	It is clear that the “size” and “color” parameters are used for layered navigation, because they define subsets of your catalogue. For normal indexing, you only want /clothing.html and /clothing/pants.html indexed. To block bots in this case, you should add this to `/data/web/nginx/server.layered_navigation:`
`56`	`56`
`57`	`57`	```nginx
`58`		`-if ($http_user_agent ~* 'http\|bot\|crawl\|spider') {`
	`58`	`+if ($http_user_agent ~* 'http\|bot\|crawl\|spider\|GoogleOther') {`
`59`	`59`	`set $is_bot 1;`
`60`	`60`	`}`
`61`	`61`	`# Here, you should define the layered navigation urls.`