Search Databases
- Started
- Last post
- 11 Responses
- phirschybar
Ok... so I have this pretty big database and I'm making a site that is primarily dynamic content. Google doesnt care much for pages generated dynamically so whats my best bet for getting this thing properly listed?
Right now, I'm thinking of making a script which writes the entire contents of the database onto one page every night and putting a redirect script at the top of the page to bring users to the sites INDEX page.
Thoughts?
- sparker0
what are you talking about? if the pages that are dynamically generated contain any form of text and/or meta tags, then google will pick them up fine.
the spider that google uses searches the body copy of html/php/etc files. it will pick up on key words and such without a problem.
what format are they being generated in?
- phirschybar0
I'm using php and pulling everything from MySQL. So none of the JUICY content exists in the body of the php files.
So if you claim that this gets indexed by google, what will the LINK be to the page? Should I just make sure that all IDs get passed to the URL? Like "page.php?id=25"?
google will include the variables in the URL?
- sparker0
the data that is pulled from the db is parsed and then interpreted onscreen...it would be the same as if the page were a static html page containing said text.
it just happens that it can change every second.
spiders scan interpreted pages...not source files.
the spider goes out and makes a call to the server to serve up the website (as if a user was visiting it)...it is the same as apps like wget that allow you to "download" entire websites for reading offline.
- phirschybar0
thanks sparker. Definitely feel better about all this.
- Mimio0
You can also modify your Apache's "mod_rewrite" so it won't display those varibles in the URL window. Google still won't have any problems crawling your site and it's more secure.
- sparker0
now, to optimise your site for search engines, you will want to check into things like metadata/metatags and making sure that your site markup is clean and semantic.
:)
- phirschybar0
I'll definitely consider that.
Check this:
I just read on Googles tech notes that occasionally they wont crawl sites with dynamically generated pages because it may cause the site or server to crash. You guys know any tricks or pitfalls to avoid to make sure Google stays on the ball and logs all of the data from your database.
- phirschybar0
you just answered my question... got any links for help / resources?
- sparker0
just look for 'search engine optimisation' online...
zeldmans book has a few references, as well as there being some articles on alistapart.
i doubt googles spider would crash a decent server.
it just depends on how much data there is in the db, and how heavy the generated pages are.
if the pages weigh in at high kb sizes, sure...the transfer might generate a small hicup.
- phirschybar0
Google made it sound like they werent concerned with actually crashing servers but that their BOT has some built-in limits when it comes to indexing dynamic pages.
I did search a bit for answers but when you include the words "search engine" in YOUR search you get a bunch of crap on helping your site get better listings... OH WELL... anyone else have some interesting links on this (or books)?
Thanks all :)
- phirschybar0
This is the answer: