[geeklog-devel] Search engine indexing...

[Tom Willett]

Simon,

I know there has been a lot of talk about the problem of urls like Geeklog 
has, but I have never had a problem with any of the search engines spidering 
my geeklog sites and returning results.  I think it is a bogus issue to some 
extent.

Saying that I have noticed one issue with geeklog sites and that is very 
often the links are to pages of the index that no longer point to the 
article.  And the entire article is not indexed only the teaser.  To get 
around this issue I created a simple site index with direct links to all the 
articles on the site.  This works fairly well.  About the same time Vinny 
developed a method to use apache rewrite rules to create a google friendly 
link and we modified the theme to place this link on each article.  You can 
see both these in action on http://gplugs.sf.net.  An article discussing 
them can be found here:

http://gplugs.sourceforge.net/article.php?story=2003011606465372

My searches on google and others have not shown much use of the rewritten 
article links.  I am not sure it is worth the effort.

Some others have modified my siteindex to make it people friendly but my 
testing shows that just confuses the bots.

On Mon, 10 Mar 2003 10:11:06 -0500, Simon Lord wrote
> Great, so if someone got this working, can you please post the  
> solution?  Much appreciated.
> 
> On Monday, March 10, 2003, at 09:44 AM, Tony Bibbs wrote:
> 
> > Simon, this is nothing new.  At last test, the url rewriting stuff I  
> > came
> > up with didn't work on windows because of a known PHP bug.  I know Tom  
> > and
> > a few others took another stab at this and I'm not sure if thiers works
> > under windows or not.
> >
> > --Tony
> >
> >  On Sun, 9 Mar 2003, Simon Lord wrote:
> >
> >> Ok, I found another useful snippet of code that I think MUST be
> >> included into the Geeklog code.
> >>
> >> Here is the first paragraph from the page link below:
> >>
> >> ------------------------------------------------------------
> >> "Anyone who has built template based sites that use query strings to
> >> select content will have at some point hit the problem of indexing (or
> >> lack of it) by search engines. Search engine spiders won't index
> >> dynamic sites, as they are worried about getting stuck in a maze of
> >> twisty URLs, all alike."
> >>
> >> "We want to turn this
> >>
> >> http://www.somesite.co.uk/ 
> >> site.php?section=books&subsection=architecture
> >>
> >> into this
> >>
> >> http://www.somesite.co.uk/site/books/architecture
> >>
> >> which involves masking the file extension for the processing file and
> >> then giving the query string a shave and a haircut."
> >> ------------------------------------------------------------
> >>
> >> This way, all our Geeklog pages get indexed and found by Google.  You
> >> can thank James, my server admin, for pointing me to this page.
> >>
> >> http://www.evolt.org/article/
> >> Search_Engine_Friendly_URLs_with_PHP_and_Apache/17/15049/index.html%22
> >>
> >> Sincerely,
> >> Simon
> >>
> >> _______________________________________________
> >> geeklog-devel mailing list
> >> geeklog-devel at lists.geeklog.net
> >> http://lists.geeklog.net/listinfo/geeklog-devel
> >>
> >
> > -- 
> > Tony Bibbs          "I guess you have to remember that those who don't
> > tony at tonybibbs.com  hunt or fish often see those of us who do as
> >                     harmlessly strange and sort of amusing. When you
> >                     think about it, that might be a fair assessment."
> >                     --Unknown
> >
> >
> > _______________________________________________
> > geeklog-devel mailing list
> > geeklog-devel at lists.geeklog.net
> > http://lists.geeklog.net/listinfo/geeklog-devel
> >
> >
> Sincerely,
> Simon
> 
> _______________________________________________
> geeklog-devel mailing list
> geeklog-devel at lists.geeklog.net
> http://lists.geeklog.net/listinfo/geeklog-devel



--
Tom Willett
tomw at pigstye.net