RIP Findx. I came across the announcement just now while investigating why searches were not working with them. I just discovered FIndx about a month ago and now it’s gone. This makes me sad.
You should read the announcement above.
Some takeaways:
- The open source Gigablast search engine script is not up to the job, they say. Good to know if you are starting up a search engine project.
- It is very hard to get people to switch away from their preferred search engine.
- Too many major sites block small legitimate search engines by default.
- The web is awash in a sea of spam and it’s very hard to sort that out.
- Crawling the web, building your own search index is very expensive.
- Building a competent crawler is very hard.
