Replying to Kicks Condor.

Thanks for replying.  If the Web is our social network, then the question of finding stuff on the web looms large.  Which is what led me to post about search engines and directories.

I like your observations about recency vs static in directories.  To me the directory of the future needs to be a hybrid of directory and search engine:  It needs to combine both recency and curated static PLUS a crawler.  Reddit is a good starting point for recency if you have a spider crawling to pick up newly posted URLs plus the archives.  Add in Curlie and Wikipedia to the crawl but use sub Reddits for taxonomy.  That’s for general search engine.

Once upon a time: I used Fluid Dynamics Search Engine to create a  sort of crawling Horror Fiction directory.  It used the Meta tags and or user submitted data, plus it crawled the submitted page so it was indexing both meta data plus real on page text for search database.  It worked quite well. I had always wished I had the resources to crawl deeper.

Anyway, I think you are on to something.

The goal is discovery, I think using many means, large and small is best.

Liked this post? Follow this blog to get more. Follow

2 thoughts on “Replying to Kicks Condor

  1. So, with Horror Fiction, how do you decide how to rank things? PageRank clearly
    benefits all of the incumbents. I’m sure recency was important—people want to
    announce their new stories and articles and reviews. I’m sure people want to
    find lists of the most beloved stories. But I’m sure topic was important too:
    demonic possessions, monsters, mannequins. Although there might be spoilers in
    that kind of designation.

    The ability to follow authors has got to be paramount in a community like that.
    So it would make sense for that kind of community to congregate in the Indieweb.
    Authors would post on blogs. (But don’t they usually post on wikis?) And then
    they ping the directory when they have new material.

    Indieweb.xyz doesn’t have a crawler, but it does fetch the page and parse it
    and use its metadata. In a way, it works like a crawler where pages are
    submitted one by one. Even Reddit is a kind of crawler, performed by humans.
    So the crawler question is one of: how much do we need to go out and find
    random stuff unprompted?

    Well, and the crawler would be useful for finding stuff outside the Indieweb.
    Unstructured, mostly undiscovered stuff. But the Indieweb’s insistence on
    structure has the benefit of weeding out spam. (For now.)

    So, yes, I agree, directory + crawler. The next question is: have tags worked
    for discovery? (Good subreddits are just permanent, high-value hashtags.) Or
    can hierarchical directories still find a place?

    Interestingly enough, I think two of the best hierarchical directories out there
    are eBay and Craigslist. And both are self-categorized! I wonder what they do
    to solve miscategorization.

    via kickscondor.com

  2. So, with Horror Fiction, how do you decide how to rank things? PageRank clearly
    benefits all of the incumbents. I’m sure recency was important—people want to
    announce their new stories and articles and reviews. I’m sure people want to
    find lists of the most beloved stories. But I’m sure topic was important too:
    demonic possessions, monsters, mannequins. Although there might be spoilers in
    that kind of designation.

    The ability to follow authors has got to be paramount in a community like that.
    So it would make sense for that kind of community to congregate in the Indieweb.
    Authors would post on blogs. (But don’t they usually post on wikis?) And then
    they ping the directory when they have new material.
    Indieweb.xyz doesn’t have a crawler, but it does fetch the page and parse it
    and use its metadata. In a way, it works like a crawler where pages are
    submitted one by one. Even Reddit is a kind of crawler, performed by humans.
    So the crawler question is one of: how much do we need to go out and find
    random stuff unprompted?
    Well, and the crawler would be useful for finding stuff outside the Indieweb.
    Unstructured, mostly undiscovered stuff. But the Indieweb’s insistence on
    structure has the benefit of weeding out spam. (For now.)
    So, yes, I agree, directory + crawler. The next question is: have tags worked
    for discovery? (Good subreddits are just permanent, high-value hashtags.) Or
    can hierarchical directories still find a place?
    Interestingly enough, I think two of the best hierarchical directories out there
    are eBay and Craigslist. And both are self-categorized! I wonder what they do
    to solve miscategorization.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.