Author Topic: BYOAC robots.txt (Read 2088 times)

krick · « **on:** August 19, 2009, 08:28:03 pm »

Saint,

You should add some Disallows to your robots.txt so that the robots don't spend a lot of time indexing invalid content. The reason I noticed this is that a LOT "action=printpage" content indexed in google, which is very frustrating when you want the normal page with navigation (no to mention ads)...

http://www.google.com/search?q=site%3Ahttp%3A%2F%2Fforum.arcadecontrols.com%2F+"action%3Dprintpage"

Here's a good thread on how to build a search engine friendly robots.txt for SMF...

http://www.simplemachines.org/community/index.php?topic=67944.0

saint · « **Reply #1 on:** October 11, 2009, 02:58:17 pm »

OK. Change made. Will take a while for Google to work it through.


Main	Restorations	Software	Audio/Jukebox/MP3	Everything Else	Buy/Sell/Trade
Project Announcements	Monitor/Video	GroovyMAME	Merit/JVL Touchscreen	Meet Up	Retail Vendors
Driving & Racing	Woodworking	Software Support Forums	Consoles	Project Arcade	Reviews
Automated Projects	Artwork	Frontend Support Forums	Pinball	Forum Discussion	Old Boards
Raspberry Pi & Dev Board	controls.dat	Linux	Miscellaneous Arcade	Wiki Discussion	Old Archives
Lightguns	Arcade1Up	Try the site in https mode		Site News


Unread posts \| New Replies \| Recent posts \| Rules \| Chatroom \| Wiki \| File Repository \| RSS \| Submit news