Indexing Policy & Robots.txt
Sunny
Posts: 9296
Reputation: 456

Message # 1 | 10:41 AM
Website's Indexing Status


All uCoz websites have Indexing status that is displayed at the top of the Control Panel's main page (/panel/?a=cp). The parameter shows whether indexing by search engines is allowed for the website or not (whether the website is in quarantine).
The indexing status can show one of the two options: "indexing is allowed (quarantine is removed)":



Or "indexing is prohibited (the website is in quarantine)":



The status "indexing is prohibited (the website is in quarantine)" is assigned by default to all newly created websites.

Quarantine Removal Policy


A website can become available for indexing either automatically (if a premium plan is purchased) or upon the website owner's request. If the website does not have a premium plan and the user wants the quarantine to be removed, a request should be submitted from the website's Control Panel:



There will be a pop-up window with the info on the quarantine policy:



After the request has been submitted, the website will be checked automatically according to a number criteria: the website's age, presence of a custom domain name, content, verified phone number etc. On the basis of these criteria the system decides whether the quarantine should be removed. We cannot provide a more detailed description of the algorithm.

Note! If the quarantine removal was denied, the next request can be submitted no sooner than in 7 days.


Robots.txt


A website's robots.txt file is located at http://your_website_address/robots.txt. A website with the default robots.txt is indexed in the best possible way – we set up the file in such a way that only pages with content are indexed, and not all existing pages (e.g. login or registration page). Therefore uCoz websites are indexed better and get higher priority in comparison with other sites where all unnecessary pages are indexed.

That's why we strongly recommend not to replace the default robots.txt by your own.


If you still want to replace the file by your own, create a text file using Notepad or any other text editor and name it "robots.txt". Then upload it to the root folder of your website via File Manager or FTP. Note: while website indexing is prohibited, no modification of the robots.txt file is possible.

The default robots.txt looks as follows:
Quote

User-agent: *
Allow: /*?page
Allow: /*?ref=
Allow: /stat/dspixel
Disallow: /*?
Disallow: /stat/
Disallow: /index/1
Disallow: /index/3
Disallow: /register
Disallow: /index/5
Disallow: /index/7
Disallow: /index/8
Disallow: /index/9
Disallow: /index/sub/
Disallow: /panel/
Disallow: /admin/
Disallow: /informer/
Disallow: /secure/
Disallow: /poll/
Disallow: /search/
Disallow: /abnl/
Disallow: /*_escaped_fragment_=
Disallow: /*-*-*-*-987$
Disallow: /shop/order/
Disallow: /shop/printorder/
Disallow: /shop/checkout/
Disallow: /shop/user/
Disallow: /*0-*-0-17$
Disallow: /*-0-0-

Sitemap: http://forum.ucoz.com/sitemap.xml
Sitemap: http://forum.ucoz.com/sitemap-forum.xml



Robots.txt during the quarantine looks as follows:

Quote

User-agent: *
Disallow: /




Robots.txt FAQ


Informers are not indexed because they display information that ALREADY exists. As a rule this information is already indexed on the corresponding pages.


Question: I have accidentally messed up robots.txt. What should I do?

Answer: Delete it. The default robots.txt file will be added back automatically (the system checks whether a website has it, and if not – adds back the default file).


Question: Is there any use in submitting a website to search engines if the quarantine hasn't been removed yet?

Answer: No, your website won't be indexed while in quarantine.


Question: Will the robots.txt file be replaced automatically after the quarantine has been removed? Or should I update it manually?

Answer: It will be updated automatically.


Question: Is it possible to delete the default robots.txt?

Answer: You can't delete it, it's a system file, but you can add your own file. However, we don't recommend to do this, as was stated above. During the quarantine it is impossible to upload a custom robots.txt.


Question: What should I do to forbid indexing of the following pages?
_http://site.ucoz.com/index/0-4
_http://site.ucoz.com/index/0-5

Answer: Add the following lines to the robots.txt file:
/index/0-4
/index/0-5


Question: I have forbidden indexing of some links by means of robots.txt but they are still displayed. Why is it so?

Answer: By means of robots.txt you can forbid indexing of pages, not links.


Question: I want to make some changes in my robots.txt file. How can I do this?

Answer: Download it to your PC, edit it and then upload it back via File Manager or FTP.

I'm not active on the forum anymore. Please contact other forum staff.
akash19
Posts: 27
Reputation: -2

Message # 2 | 10:41 AM
how to know that site is 30days old, sothat i an edit robots.txt file.
Sunny
Posts: 9296
Reputation: 456

Message # 3 | 10:41 AM
It is written on the main page of the Control Panel.
I'm not active on the forum anymore. Please contact other forum staff.
Applesdg
Posts: 4
Reputation: 1

Message # 4 | 0:19 AM
Im sorry I read this about 10 times, so im about to get a domain and attach it, and the Robot.txt will go away, and I will be on search engines?
Also just to make sure, your account or control panel will stay with you with the attached domain.
Thanks.

www.EpicFourms.ucoz.com
Sunny
Posts: 9296
Reputation: 456

Message # 5 | 9:01 AM
Applesdg, when you attach a domain your website will stay as it is - it will just have one more url to access
I'm not active on the forum anymore. Please contact other forum staff.
ScottyBoy
Posts: 16
Reputation: 1

Message # 6 | 2:41 AM
I wish I had known about the quarantine before I submitted my site to Google and Yahoo! online2long haha

Do you think they would take me seriously if I submitted my site again after 30 days? wacko


Post edited by ScottyBoy - Friday, 2009-04-10, 2:42 AM
Brazen
Posts: 74
Reputation: 25

Message # 7 | 7:49 AM
Quote (ScottyBoy)
Do you think they would take me seriously if I submitted my site again after 30 days?

Once you submit your website to search engine (Yahoo, Google...) again they will crawl
your website within a week, it will treat your submition the same way it did the first time,
and I don't think server engine would care about seriousness since it's machine and not
a person. So, yes it will crawl your website after it's 30 days old!

Post edited by Brazen - Friday, 2009-04-10, 7:51 AM
babytux
Posts: 2
Reputation: 0

Message # 8 | 12:13 PM
I did upload the robots.txt in the root, but still showing the old file. Google says my robots.txt is in trouble and he can not index my site in search engine. HELP ME!!!
Sunny
Posts: 9296
Reputation: 456

Message # 9 | 2:22 PM
babytux, read attentively the first post.

Quote (Sunny)
There is a "quarantine" for each new website when no modification of the robots.txt file is possible.

I'm not active on the forum anymore. Please contact other forum staff.
akash19
Posts: 27
Reputation: -2

Message # 10 | 2:22 PM
i want to make some changes in ma robots.txt file. what da process of it??
ma site is http://gamejungle.co.cc/ or http://gamejungle.ucoz.com
Armaros
Posts: 1424

Message # 11 | 2:23 PM
Way to change what? You must download it to your PC & upload back via File Manager or FTP-client.
mugenpolo
Posts: 1
Reputation: 0

Message # 12 | 2:23 PM
How can i access the robot.txt of my site so that i can modify it?
Sunny
Posts: 9296
Reputation: 456

Message # 13 | 2:23 PM
The address of your robots.txt file is http://your_site_url/robots.txt but you can't modify it until your site is 30 days old. Even if it is old enough we do not recommend to modify robots.txt because it is adjusted in the best possible way.
I'm not active on the forum anymore. Please contact other forum staff.
l2viruz
Posts: 20
Reputation: 12

Message # 14 | 2:23 PM
What is robot.txt? and HOW CAN I FILE SIMILAR FILES LIKE THESE?
Sunny
Posts: 9296
Reputation: 456

Message # 15 | 2:23 PM
http://en.wikipedia.org/wiki/Robots.txt

Quote (l2viruz)
HOW CAN I FILE SIMILAR FILES LIKE THESE?

What do you mean?


I'm not active on the forum anymore. Please contact other forum staff.
Search: