uCoz Community » For Webmasters » Site Promotion » Indexing Policy & Robots.txt
Indexing Policy & Robots.txt
Sunny
Posts: 9296
Reputation: 456

Message # 1 | 10:41 AM
Website's Indexing Status


All uCoz websites have Indexing status that is displayed at the top of the Control Panel's main page (/panel/?a=cp). The parameter shows whether indexing by search engines is allowed for the website or not (whether the website is in quarantine).
The indexing status can show one of the two options: "indexing is allowed (quarantine is removed)":



Or "indexing is prohibited (the website is in quarantine)":



The status "indexing is prohibited (the website is in quarantine)" is assigned by default to all newly created websites.

Quarantine Removal Policy


A website can become available for indexing either automatically (if a premium plan is purchased) or upon the website owner's request. If the website does not have a premium plan and the user wants the quarantine to be removed, a request should be submitted from the website's Control Panel:



There will be a pop-up window with the info on the quarantine policy:



After the request has been submitted, the website will be checked automatically according to a number criteria: the website's age, presence of a custom domain name, content, verified phone number etc. On the basis of these criteria the system decides whether the quarantine should be removed. We cannot provide a more detailed description of the algorithm.

Note! If the quarantine removal was denied, the next request can be submitted no sooner than in 7 days.


Robots.txt


A website's robots.txt file is located at http://your_website_address/robots.txt. A website with the default robots.txt is indexed in the best possible way – we set up the file in such a way that only pages with content are indexed, and not all existing pages (e.g. login or registration page). Therefore uCoz websites are indexed better and get higher priority in comparison with other sites where all unnecessary pages are indexed.

That's why we strongly recommend not to replace the default robots.txt by your own.


If you still want to replace the file by your own, create a text file using Notepad or any other text editor and name it "robots.txt". Then upload it to the root folder of your website via File Manager or FTP. Note: while website indexing is prohibited, no modification of the robots.txt file is possible.

The default robots.txt looks as follows:
Quote

User-agent: *
Allow: /*?page
Allow: /*?ref=
Allow: /stat/dspixel
Disallow: /*?
Disallow: /stat/
Disallow: /index/1
Disallow: /index/3
Disallow: /register
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /index/sub/
Disallow: /panel/
Disallow: /admin/
Disallow: /informer/
Disallow: /secure/
Disallow: /poll/
Disallow: /search/
Disallow: /abnl/
Disallow: /*_escaped_fragment_=
Disallow: /*-*-*-*-987$
Disallow: /shop/order/
Disallow: /shop/printorder/
Disallow: /shop/checkout/
Disallow: /shop/user/
Disallow: /*0-*-0-17$
Disallow: /*-0-0-

Sitemap: http://forum.ucoz.com/sitemap.xml
Sitemap: http://forum.ucoz.com/sitemap-forum.xml



Robots.txt during the quarantine looks as follows:

Quote

User-agent: *
Disallow: /




Robots.txt FAQ


Informers are not indexed because they display information that ALREADY exists. As a rule this information is already indexed on the corresponding pages.


Question: I have accidentally messed up robots.txt. What should I do?

Answer: Delete it. The default robots.txt file will be added back automatically (the system checks whether a website has it, and if not – adds back the default file).


Question: Is there any use in submitting a website to search engines if the quarantine hasn't been removed yet?

Answer: No, your website won't be indexed while in quarantine.


Question: Will the robots.txt file be replaced automatically after the quarantine has been removed? Or should I update it manually?

Answer: It will be updated automatically.


Question: Is it possible to delete the default robots.txt?

Answer: You can't delete it, it's a system file, but you can add your own file. However, we don't recommend to do this, as was stated above. During the quarantine it is impossible to upload a custom robots.txt.


Question: What should I do to forbid indexing of the following pages?
_http://site.ucoz.com/index/0-4
_http://site.ucoz.com/index/0-5

Answer: Add the following lines to the robots.txt file:
/index/0-4
/index/0-5


Question: I have forbidden indexing of some links by means of robots.txt but they are still displayed. Why is it so?

Answer: By means of robots.txt you can forbid indexing of pages, not links.


Question: I want to make some changes in my robots.txt file. How can I do this?

Answer: Download it to your PC, edit it and then upload it back via File Manager or FTP.

I'm not active on the forum anymore. Please contact other forum staff.
Sunny
Posts: 9296
Reputation: 456

Message # 31 | 1:52 PM
Quote (lotkymaia)
Hello, can someone help me with my problem? i have my site 1.5 years old and still no pictures indexed. My question is: Photo album is robots restricted and i must cahge this in robots.txt document?
Thank you! Have a nice day!

As far as I know there are no restrictions for photos.


I'm not active on the forum anymore. Please contact other forum staff.
adytzu1_vl01
Posts: 251
Reputation: 0

Message # 32 | 7:03 PM
I have a problem
In this address http://styiks.ucoz.net/robots.txt its shows this
Code
User-agent: uBot
Disallow: /a/
Disallow: /stat/
Disallow: /index/1
Disallow: /index/2
Disallow: /index/3
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /panel/
Disallow: /admin/
Disallow: /secure/
Disallow: /informer/
Disallow: /mchat
Disallow: /search

User-agent: *
Disallow: /

and in this http://stx-zone.ro/robots.txt

Code
User-agent: uBot
Disallow: /a/
Disallow: /stat/
Disallow: /index/1
Disallow: /index/2
Disallow: /index/3
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /panel/
Disallow: /admin/
Disallow: /secure/
Disallow: /informer/
Disallow: /mchat
Disallow: /search

User-agent: *
Disallow: /

User-agent: Mediapartners-Google*
Disallow:

whats wrong ? dry

i added this lines

Code
User-agent: Mediapartners-Google*
Disallow:

This site does not support Internet Explorer.
Join me All-Stars.ro
With uCoz since 2009
Post edited by adytzu1_vl01 - Monday, 2010-05-17, 7:04 PM
Sunny
Posts: 9296
Reputation: 456

Message # 33 | 9:48 AM
adytzu1_vl01, it is the same website, right? You added just two last lines to http://stx-zone.ro/robots.txt and didn't edit other lines? Didn't you copy lines from the first robots.txt ?
I'm not active on the forum anymore. Please contact other forum staff.
adytzu1_vl01
Posts: 251
Reputation: 0

Message # 34 | 11:54 AM
yes i copy that from my old robots.txt and added 2 lines
and it's different between addresses

This site does not support Internet Explorer.
Join me All-Stars.ro
With uCoz since 2009
Sunny
Posts: 9296
Reputation: 456

Message # 35 | 1:44 PM
adytzu1_vl01, it will be different because uCoz URL is closed for indexation when there is an attached domain. And according to http://stx-zone.ro/robots.txt your attached domain is now closed for indexation as well. Therefore I suggest that you delete your custom robots, then download the default robots from http://stx-zone.ro/robots.txt and then edit it and upload again.
I'm not active on the forum anymore. Please contact other forum staff.
adytzu1_vl01
Posts: 251
Reputation: 0

Message # 36 | 3:48 PM
ok
This site does not support Internet Explorer.
Join me All-Stars.ro
With uCoz since 2009
kralfmradyo
Posts: 1
Reputation: 0

Message # 37 | 5:11 PM
User-agent: *
Disallow: /a/
Disallow: /stat/
Disallow: /index/1
Disallow: /index/2
Disallow: /index/3
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /main/
Disallow: /admin/
Disallow: /secure/
Disallow: /informer/
Disallow: /mchat
http://xat.com/ferdifon
There is a "quarantine

http://xat.com/ferdifon http://kralfmradyo.com
Sunny
Posts: 9296
Reputation: 456

Message # 38 | 8:23 AM
kralfmradyo, no, this robots.txt shows that there is no quarantine.
I'm not active on the forum anymore. Please contact other forum staff.
Torres
Posts: 5
Reputation: 0

Message # 39 | 2:39 PM
Hey guyz, my site robots.txt file looks like this: http://ewarez.ucoz.com/robots.txt And google won't index my site as restricted by robots.txt what should i do now? Can someone help me to solve this problem? Then i'll much appreciate! cry

Added (2010-06-04, 8:39 Am)
---------------------------------------------
Oh i forgot to say that i attached a domain tk then deleted it cause google won't index my site. Is it a problem during the index??

Sunny
Posts: 9296
Reputation: 456

Message # 40 | 2:53 PM
Torres, what about reading the first post of this thread? Your website is on the quarantine, wait till your website is 30 days old.
I'm not active on the forum anymore. Please contact other forum staff.
Torres
Posts: 5
Reputation: 0

Message # 41 | 2:14 AM
Ok then Google will automatically add my site on their search engine? Or should i have to add my URL after the quarantine? Thank you for helping plz answer to my last question!
Sunny
Posts: 9296
Reputation: 456

Message # 42 | 9:21 AM
Torres, it is better if you submit the URL to search engines yourself. This can help - http://forum.ucoz.com/forum/23-3533-1
I'm not active on the forum anymore. Please contact other forum staff.
Freakzzstar
Posts: 6
Reputation: 0

Message # 43 | 9:36 AM
hi i am newbie here and i want to knw that is my robots.txt file is eligble for posting ads and indexing my site in google this is my site robots.txt file url plzz help me
http://mastizone.do.am/robots.txt
Sunny
Posts: 9296
Reputation: 456

Message # 44 | 10:23 AM
Freakzzstar, at the moment your website is closed for indexation, most probably it is less then 30 days old.
I'm not active on the forum anymore. Please contact other forum staff.
Freakzzstar
Posts: 6
Reputation: 0

Message # 45 | 10:36 AM
Sunny, but in my robots.txt file there
User-agent: *
Disallow: /

in the first post there given that if user agent :ubot
then its closed
plzz tell me in detail if u don mind

uCoz Community » For Webmasters » Site Promotion » Indexing Policy & Robots.txt
Search: