What should my robot.txt file look like?

[SpottySocks]

Hi guys, I have had an alert from Google Adsense saying the google robot has been denied. This is out of the blue as I haven't had a problem before.

Instructions say to remove the following from my robot.txt file:

User-agent: Mediapartners-Google
Disallow: /

This is my current robot.txt file content:

User-agent: *
User-agent: Mediapartners-Google
Disallow:
Disallow:/admincp/
Disallow:/clientscript/
Disallow:/cpstyles/
Disallow:/customavatars/
Disallow:/customprofilepics/
Disallow:/images/
Disallow:/modcp/
Disallow:/ajax.php
Disallow:/attachment.php
Disallow:/calendar.php
Disallow:/cron.php
Disallow:/editpost.php
Disallow:/global.php
Disallow:/image.php
Disallow:/inlinemod.php
Disallow:/joinrequests.php
Disallow:/login.php
Disallow:/member.php
Disallow:/memberlist.php
Disallow:/misc.php
Disallow:/moderator.php
Disallow:/newattachment.php
Disallow:/newreply.php
Disallow:/newthread.php
Disallow:/online.php
Disallow:/poll.php
Disallow:/postings.php
Disallow:/printthread.php
Disallow:/private.php
Disallow:/profile.php
Disallow:/register.php
Disallow:/report.php
Disallow:/reputation.php
Disallow:/search.php
Disallow:/sendmessage.php
Disallow:/showgroups.php
Disallow:/subscription.php
Disallow:/threadrate.php
Disallow:/usercp.php
Disallow:/usernote.php


#Begin Attracta SEO Tools Sitemap. Do not remove
sitemap: http://cdn.attracta.com/sitemap/3406644.xml.gz
#End Attracta SEO Tools Sitemap. Do not remove

#Begin Attracta SEO Tools Sitemap. Do not remove
sitemap: http://cdn.attracta.com/sitemap/3406643.xml.gz
#End Attracta SEO Tools Sitemap. Do not remove


Shall I remove what Google Adsense says to move? How should my robot.txt file look?

I use vBSEO so not sure if I should do something via that in the AdminCP or if I should edit the robot.txt file manually.

Thanks all.

[thetechgenius]

This is what my Robots.txt looks like:



User-agent: thetechgenius-Google
Disallow:

User-agent: *
Disallow: /cache.php

User-agent: *
Disallow: /go/
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /attachment.php
Disallow: /search.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /editpost.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /login.php
Disallow: /subscription.php
Disallow: /private.php
Disallow: /report.php
Disallow: /sendmessage.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /postings.php
Disallow: /sendtofriend.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /showgroups.php
Disallow: /announcement.php
Disallow: /downloads.php
Disallow: /downloads.html
Disallow: /**********/ #<- Renamed AdminCP Directory -- Blanked out for security reasons
Disallow: /*********/ #<- Renamed ModCP Directory -- Blanked out for security reasons

Sitemap: https://thetechgenius.net/dbseositemap.php #<- This is my Sitemap from the DBSEO Addon --



As for the Sitemap, you have to put the URL that generates the Sitemap. If you go to my Sitemap URL, it will Generate and Start downloading the Sitemap. I'm not sure how vBSEO handles Sitemaps.