Anyone familiar with Custom Search Engines?

[Saintsaens]

Hi all,

Forgive me if this is in the wrong forum or not really explained clearly, but I had come up with an idea yesterday that might help a bunch of people, but unfortunately I'm trying to learn about it on my own with no one to really guide me and only partially understanding what it takes to do what I want to do. I'm heading into grad school for library information sciences soon, and I guess this is somewhat of a relevant pursuit of ways to make finding information on particular topics easier (which is basically what a librarian does). I'm also trying to learn about some of the new Web 2.0/3.0 features out there on my own, and perhaps trying to apply them to this website (like with the wiki we implemented).

Basically, what I want to do is make a metasearch engine (or a Custom Search Engine, or some other similar tool) that would help me when I want to find information about a particular topic in EQ. I know a LOT of good websites and message boards are out there, but sifting through all that information takes a while. When I have something I want to look up, like say, a quest or a raid I want to find information about, I check all relevant sites (3 or 4 different class boards, Alla's, Raspar's Repository, etc.), and then compile a list of results based upon searching just the sites I'd read regularly. However, when exploring about metasearch engines (which are basically search engines that search more than one index of sites), I found that they may or may not be better than just using a regular search engine to search the same topic, and that Custom Search Engines may do what I wanted instead.

I was looking at the Google Custom Search Engine, but when I tried to create a search on some topics that I knew were discussed on the various boards I had put in the test CSE I had created, the only results I pulled up were from webpages like Alla's or Raspar's. My guess is this is because message boards and forums don't get indexed very well by Google. Does anyone have a way around this, or is this something that each individual board needs to implement on their own (since I would assume that having lots of people using a search engine like this would be a pretty heavy server load)?

[Kakimor]

http://www.metacrawler.com/ one I often use. Best of my knowlege unless a site gives permission search engines cant browse them for specific info that is included in the threads. Other wise guild sites could be and would be raped for strats etc. Usually what you will pull up is front page news etc. unless its an "open source" site like Allaks