Author Topic: Several Users asks to show the profile in Search Engines, What to do?  (Read 426 times)

onlineservices

  • Veteran Member
  • *****
  • Posts: 238
  • Karma: +0/-0
Ofcourse community profiles needs privacy. But several of our users provides the feedback to make the profiles visible to search engine only by blocking privacy info like contact, email and phone numbers.

Our users can view profiles of others if they are even guest. But they cant view the personal information of profiles like contact address, phone and email.

But as our robots.txt is completely blocked even for images, its not visible in search engines.

Which disallow command shall be removed for a safe visibility of profiles in search engine. This is my repeated question. As i am afraid of hack attacks, i seek geeks like Mr. Maverick or any other expert's help.


My robots.txt is as follows, Kindly suggest which disallow can be removed for profiles visibility.

Quote
User-Agent: *
Crawl-delay: 10
Allow: /
Disallow: /admin
Disallow: /ajax
Disallow: /events
Disallow: /fckeditor
Disallow: /images
Disallow: /includes
Disallow: /js
Disallow: /media
Disallow: /processors
Disallow: /skins_site
Disallow: /tmp
Disallow: /tools
Disallow: /*.gif$
Disallow: /*.jpg$
Disallow: /*.jpeg$
Disallow: /*.png$

maverick

  • Administrator
  • Veteran Member
  • *****
  • Posts: 3025
  • Karma: +210/-7
    • Maverick Webworks
Re: Several Users asks to show the profile in Search Engines, What to do?
« Reply #1 on: October 18, 2015, 09:33:44 AM »
You clearly don't fully understand the purpose of a robots.txt file, it provides very little if any security.

 It is important to clarify that a robots.txt file is not a way to prevent search engines or bots from crawling your site, it just provides a suggestion which they don't have to obey, only the major bots like Google and Bing obey them.

A robots.txt file does not provide any type of security from spambots, thieves or hackers, adding a robots.txt file is like putting a sticky note on an unlocked door that says ... "Please, Do Not Enter", the bad bots will ignore it and enter anyway.

A robots.txt is by no means mandatory for search engines to crawl and index your site, and if done incorrectly can cause more harm than good. It doesn't tell or force search engines to index pages, it only suggests what not to crawl and index.

For example, Disallowing the crawling of JS or CSS files in your robots.txt file can actually harm how well search engines render and index your content and can result in less than optimal page indexing and rankings. These types of files can easily be accessed and viewed by any knowledgeable webmaster simply by utilizing a browser's 'inspect' tool.

Another example is Disallowing the crawling of directories, folders or pages that aren't accessible to the public, such as the 'admin' folder, if it can't be accessed by the public, it also can't be accessed by the search engines and therefore they won't and can't index it.

Fusion Responsive Template & Free Mods
http://www.maverickwebworks.com
DO NOT PM me asking for personal help. Post your problem or request in the forums so the entire community can contribute and benefit.

onlineservices

  • Veteran Member
  • *****
  • Posts: 238
  • Karma: +0/-0
Re: Several Users asks to show the profile in Search Engines, What to do?
« Reply #2 on: October 20, 2015, 03:18:09 PM »
Yes I understood Mr. Maverick.

But the users are referring the competitors. Means same concept websites show the profile for the fullest except the contact and personal details.

But our website not doing so.... Quality of the Script, Blocking unauthorized use of general users are excellent.

Even after 14 months of script installation, the ranking in search engines are poor....page indexing is fair but not excellent.

The Social Networking score is excellent. but the ranking is very poor...So I think i may need to concentrate in seo for visible pages.

When we test the site in testers like http://nibbler.silktide.com/ with the competitors site we are able to see major difference.

My site needs development in Inbound links, thats the only variation i can fine. But search queries does not work well.. though the competitor sites has no content. Our sites has better profile content..

So some where i may need to give attention in the script. I am not sure where to begin. But checking page by page for meta tags and everything found ok.

Let me analyse to find the factors for better search ranking...

Just started customizing the following codes in .PHPs
Quote
$tplvars['title']='  ';
$tplvars['page_title']=' ';
$tplvars['meta_diz']=' ';
$tplvars['meta_keywords']=' ';



Thanks Again Mr. Maverick

kenli

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
Re: Several Users asks to show the profile in Search Engines, What to do?
« Reply #3 on: December 20, 2017, 04:17:59 AM »
thanks, this a great piece of info :)