Horror directory: This was really a faux directory. Every category was really a search with the category name as the search keyword. Of course you could do multi keyword searches from the search box. Ranking was mainly done by keyword placement and density: KW in Title carried X weight, KW in description carried Y weight, KW on page carried Z weight, etc. It worked.
Science fiction/Fantasy directory: This was a static directory with categories. Pretty conventional. At first sites were ranked by user voted 1 – 5 star rank next by alphabetical order for those with no votes. At one point I tried an experiment: “SearchHippo” a little search engine had a free API to compute “HipRank” (poking fun at Google’s PageRank). Google’s directory listed sites by order of PageRank. So I had my developer incorporate listing sites in categories by “HipRank”. 🙂
Nobody appreciated the joke. So I reverted back to the old way.
Most of the old directory scripts had a simple meta tag fetcher like you describe. We really could not launch a new niche directory without any sites listed in it. Visitors would come to us trying to find sites and they would never come back if you gave them empty categories. So we would seed the directory with sites, a few for each category. That was a lot of work for one person, but we would use that meta tag fetcher (spider) to help us. I always tried to launch with at least 700 sites listed. That kind of seeding went on throughout the life of the directory. A new TV show would launch and you would add new categories for that show and seed, then add categoires for the actors and seed with fan sites.
Tags: The thing is with a static hierarchical directory you get stuck with the hierachy you built. Tags, even reddit semi permanent tags, are more dynamic and finer grained. By letting a site like Reddit create the categories you help stay on top of what topics are new and fresh. It’s like suggestions coming from the grass roots rather than top-down.
But in a small niche community, with a directory editor who really know their community you could have a hierarchical directory with no problem. Or you could use a combination of search field and a tag cloud. On a niche directory that might be more minimalist.
Categories did serve a purpose: as a prompt to explore deeper:
Star Trek below that TOS, TNG, DS9, ENT, VOY
Below TOS: William Shatner, Nimoy, Fanfic, etc.
Each is a prompt to explore further, deeper, hear other voices. We are too used to Google giving us the a page and then we leave as if that is the entire answer.