uCoz Community » For Webmasters » Site Promotion » Indexing Policy & Robots.txt
Indexing Policy & Robots.txt
Sunny
Posts: 9296
Reputation: 456

Message # 1 | 10:41 AM
Website's Indexing Status


All uCoz websites have Indexing status that is displayed at the top of the Control Panel's main page (/panel/?a=cp). The parameter shows whether indexing by search engines is allowed for the website or not (whether the website is in quarantine).
The indexing status can show one of the two options: "indexing is allowed (quarantine is removed)":



Or "indexing is prohibited (the website is in quarantine)":



The status "indexing is prohibited (the website is in quarantine)" is assigned by default to all newly created websites.

Quarantine Removal Policy


A website can become available for indexing either automatically (if a premium plan is purchased) or upon the website owner's request. If the website does not have a premium plan and the user wants the quarantine to be removed, a request should be submitted from the website's Control Panel:



There will be a pop-up window with the info on the quarantine policy:



After the request has been submitted, the website will be checked automatically according to a number criteria: the website's age, presence of a custom domain name, content, verified phone number etc. On the basis of these criteria the system decides whether the quarantine should be removed. We cannot provide a more detailed description of the algorithm.

Note! If the quarantine removal was denied, the next request can be submitted no sooner than in 7 days.


Robots.txt


A website's robots.txt file is located at http://your_website_address/robots.txt. A website with the default robots.txt is indexed in the best possible way – we set up the file in such a way that only pages with content are indexed, and not all existing pages (e.g. login or registration page). Therefore uCoz websites are indexed better and get higher priority in comparison with other sites where all unnecessary pages are indexed.

That's why we strongly recommend not to replace the default robots.txt by your own.


If you still want to replace the file by your own, create a text file using Notepad or any other text editor and name it "robots.txt". Then upload it to the root folder of your website via File Manager or FTP. Note: while website indexing is prohibited, no modification of the robots.txt file is possible.

The default robots.txt looks as follows:
Quote

User-agent: *
Allow: /*?page
Allow: /*?ref=
Allow: /stat/dspixel
Disallow: /*?
Disallow: /stat/
Disallow: /index/1
Disallow: /index/3
Disallow: /register
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /index/sub/
Disallow: /panel/
Disallow: /admin/
Disallow: /informer/
Disallow: /secure/
Disallow: /poll/
Disallow: /search/
Disallow: /abnl/
Disallow: /*_escaped_fragment_=
Disallow: /*-*-*-*-987$
Disallow: /shop/order/
Disallow: /shop/printorder/
Disallow: /shop/checkout/
Disallow: /shop/user/
Disallow: /*0-*-0-17$
Disallow: /*-0-0-

Sitemap: http://forum.ucoz.com/sitemap.xml
Sitemap: http://forum.ucoz.com/sitemap-forum.xml



Robots.txt during the quarantine looks as follows:

Quote

User-agent: *
Disallow: /




Robots.txt FAQ


Informers are not indexed because they display information that ALREADY exists. As a rule this information is already indexed on the corresponding pages.


Question: I have accidentally messed up robots.txt. What should I do?

Answer: Delete it. The default robots.txt file will be added back automatically (the system checks whether a website has it, and if not – adds back the default file).


Question: Is there any use in submitting a website to search engines if the quarantine hasn't been removed yet?

Answer: No, your website won't be indexed while in quarantine.


Question: Will the robots.txt file be replaced automatically after the quarantine has been removed? Or should I update it manually?

Answer: It will be updated automatically.


Question: Is it possible to delete the default robots.txt?

Answer: You can't delete it, it's a system file, but you can add your own file. However, we don't recommend to do this, as was stated above. During the quarantine it is impossible to upload a custom robots.txt.


Question: What should I do to forbid indexing of the following pages?
_http://site.ucoz.com/index/0-4
_http://site.ucoz.com/index/0-5

Answer: Add the following lines to the robots.txt file:
/index/0-4
/index/0-5


Question: I have forbidden indexing of some links by means of robots.txt but they are still displayed. Why is it so?

Answer: By means of robots.txt you can forbid indexing of pages, not links.


Question: I want to make some changes in my robots.txt file. How can I do this?

Answer: Download it to your PC, edit it and then upload it back via File Manager or FTP.

I'm not active on the forum anymore. Please contact other forum staff.
Sunny
Posts: 9296
Reputation: 456

Message # 46 | 11:03 AM
Freakzzstar, I fixed the first post.
I'm not active on the forum anymore. Please contact other forum staff.
Freakzzstar
Posts: 6
Reputation: 0

Message # 47 | 12:22 PM
Sunny,

wat it means ?/

Quote
Informers are not indexed because they output information that ALREADY exists. As a rule this information is already indexed on the corresponding pages.

sorry din understand that can u explain me ?? if u don have any problem

Sunny
Posts: 9296
Reputation: 456

Message # 48 | 1:11 PM
Freakzzstar, it means that the robots.txt is set in such a way that informers are not indexed by search engines. That's because informers duplicate information, e.g. latest forum posts. Is it clearer?
I'm not active on the forum anymore. Please contact other forum staff.
Freakzzstar
Posts: 6
Reputation: 0

Message # 49 | 3:44 PM
Sunny, hmm but who is informers here ??
and why not my site will be indexed before 30 days ?
sorry but can u plzz explain it caz i am new in this feild
Thanks in advanced
Sunny
Posts: 9296
Reputation: 456

Message # 50 | 3:52 PM
Quote (Freakzzstar)
hmm but who is informers here ??

http://forum.ucoz.com/forum/37-457-1

Quote (Freakzzstar)
and why not my site will be indexed before 30 days ?

As a result of fight against people who create doorway pages, and in order not to pollute search engines with empty websites. It is not reasonable to submit a website to search engines immediately after it is created, it must be filled with content first.


I'm not active on the forum anymore. Please contact other forum staff.
Freakzzstar
Posts: 6
Reputation: 0

Message # 51 | 4:23 PM
Sunny, hmm okii nice policy biggrin

one more question but i dont knw how to fill conents in website n who will help me to fill contents in website sad

Sunny
Posts: 9296
Reputation: 456

Message # 52 | 9:31 AM
Freakzzstar, this question is not related to this thread. Create a new one on General Questions.
I'm not active on the forum anymore. Please contact other forum staff.
yet_one
Posts: 3
Reputation: 0

Message # 53 | 11:13 AM
Hi admin

İm web site robots.txt

content

Code

User-agent: *
Disallow: /

http://uzaypandizot.com/robots.txt

Google does not crawl the site because of this problem

Eror photo

http://img835.imageshack.us/img835/2177/hatavp.jpg

Robots.txt error is written in the image

http://img835.imageshack.us/img835/2177/hatavp.jpg

Dear admin Pls help

Post edited by yet_one - Monday, 2010-08-16, 11:17 AM
Sunny
Posts: 9296
Reputation: 456

Message # 54 | 11:24 AM
yet_one, did you read the first post of this thread? Probably your website is less then 30 days old and therefore it is still on the quarantine and is closed for indexation.
I'm not active on the forum anymore. Please contact other forum staff.
yet_one
Posts: 3
Reputation: 0

Message # 55 | 12:26 PM
Oh ok Thank you very quickly answer
seLymmm
Posts: 9
Reputation: 0

Message # 56 | 11:09 AM
www.zoomedia.at.ua

Sorry For my english.
i'm using google translate

(Other translate:
i don't make over robots.txt, search engines doesn't index anyway, i want to use to my robots.txt)

Google web tools, (please look screenshot's)

Zoomedia.at.ua Crawl Page:
http://film.zoomizle.com/siteproblems/zoomediatxt.png
-------------------------------
film.Zoomizle.com Crawl Page:
http://film.zoomizle.com/siteproblems/zoomizle.png
-------------------------------
-------------------------------
Zoomedia.at.ua Fetch As Googlebot:
http://film.zoomizle.com/siteproblems/zoomedia.png
Film.zoomizle.com Fetch As Googlebot:
http://film.zoomizle.com/siteproblems/filmzoom.png

Film.zoomizle.com domain transfered the zoomedia.at.ua

Natashko
Posts: 3366
Reputation: 171

Message # 57 | 11:43 AM
seLymmm, You need to add the website to the Google search engines again, but this time using the new domain name http://film.zoomizle.com/.
seLymmm
Posts: 9
Reputation: 0

Message # 58 | 10:36 AM
Quote (Natashko)
seLymmm, You need to add the website to the Google search engines again, but this time using the new domain name http://film.zoomizle.com/.

Sorry, You Don't See Images? I'm already added search engines wink
Natashko
Posts: 3366
Reputation: 171

Message # 59 | 11:17 AM
seLymmm, have a look at the domain name on this screenshot http://film.zoomizle.com/siteproblems/zoomedia.png
And now look here You website is available for indexing
Attachments: 5510271.png(176.5 Kb)
seLymmm
Posts: 9
Reputation: 0

Message # 60 | 12:07 PM
Quote (Natashko)
seLymmm, have a look at the domain name on this screenshot

You're not knowledgeable, How many pages to see added to: site:film.zoomizle.com
Sample link: http://www.google.com.tr/#hl=tr&biw=1024&bih=580&q=site%3Afilm.zoomizle.com&aq=f&aqi=&aql=&oq=&gs_rfai=&fp=5db38267a0f7a9af
uCoz Community » For Webmasters » Site Promotion » Indexing Policy & Robots.txt
Search: