Replied to


Replying to: Difficult or Silo?

Heh, well everything not on your own domain is a silo, or at least I can make that argument.  Not all silo’s are bad we’ve all just gotten sloppy talking as if they are.  Silo’s become particularly bad when they become monopolies: ie. Facebook, Twitter, Google.  But what you are trying to do with is exactly the opposite, that is create another independent outlet for both articles to be found and also blogs to be found.  We need more of these not less.

Indieweb et al.: Commendably, you have built using Indieweb tech.  But I’m not sure if the Indieweb, as it sits right now, is the best audience.  I mean, we Indiewebbers all “got ours”, we can talk to each other directly anytime we want, we have: webmentions, chat rooms on, Slack, IRC and maybe other places, we have Twitter, we have our own Indie News self serve aggregator, newsletters and most of what we talk about with each other is Indeweb related.  And there is nothing wrong with any of that, it’s just that, maybe gets overlooked by that audience.  I see you have opened things up so that more diverse non-indieweb equipped blogs can syndicate and I think this is wise.

There are blogs and bloggers out there, basically talking to themselves because nobody can find them.  They don’t know if anybody is reading or appreciates what they post because nobody comments and they have no clue about Indieweb.  They want to reach out but they don’t know how, maybe they aren’t techie, maybe they majored in English at Uni.  How do we reach them?  How do we get them to get off their duffs and nailing manifestos on  I have to confess, there are times when I forgot to syndicate to xyz because my post was not linking or Indieweb related.  I’m getting over that.

Difficulty: I think it is best if we have to do a little work to syndicate to xyz.  I’ve thought about what if we could syndicate via RSS but that would spam xyz out.  It does not need all my drivel, only my better (or longer) posts.  And if you incorporated RSS then the real spammers would take over sooner or later.

There is a generational thing here too.  I’m not talking about the Indieweb, but others on the web.  They have become passive, or at least it seems that way to me.

Keep at it.  You are experimenting with something good.  When all the massive silos like FB and Twitter go down or wall themselves off, people will discover the need for places like  They just don’t know it yet.


Also on:

Replied to

In reply to: Chris Aldrich replying to the Blog Snoop Memo.


Cow Paths:  Yes we have this concept in both New Urbanism and planning municipal bike/hike trails.  It’s a good analogy for what I’m doing: “What’s available off the shelf and free as a stopgap until somebody designs something better?”

Blog Discovery:  I’m sure directories are not the best solution for blog discovery, but like blogrolls they have a place at the table because they are low tech and cheap. Here’s a rough hierarchy:

  • Do In A Pinch: Blog acting as a directory.
  • Minimum: A proper directory script (ie. phplinkdirectory or similar) This allows for blog owner to submit their blog, write a description etc.
  • Better:  A directory script that not only lists blog URL but also lists blogs RSS feed.
  • Better x 2: Directory described above which also generates it’s own RSS feed for each category and subcategories.
  • Better x 3:  Some sort of fusion of the x2 directory above, and Indieweb stuff to some degree.  Maybe a fusion of a standard directory with Kicks Condor’s  This is just brainstorming.
  • Better x 4: Probably an RSS search engine like the old defunct Daypop.  Because this leads the searcher to individual posts about a topic in close to real time.  Such an engine could use Post Kinds as filters for the searcher to refine their search.  There used to be a lot of RSS blog search engines I could find only RSSMicro today.
  • Best: Some sort of hybrid directory/RSS/crawler engine listing only blogs.  The search crawler digs deep into a blog for those posts from 2015 or before that are buried and won’t appear in feeds.  The RSS search engine for the newest posts.

Something like that.

I don’t think we have tapped RSS fully.

OPML: I use Inoreader and I subscribed to your Indieweb list from your Following Page.  Brilliant list and thanks.  I think there is a future for this for deep discovery.

rel=directory: My instincts tell me this is important and they are on the right track.  The directory also is a low tech way to sort by topic.  Because it’s not enough to just have all the Indieweb plugins, but people need to know what the blog’s topics are.

Vouch: Yes. Needed in some form.  I used to move at the periphery of the spam crowd although I was never part of it and I can tell you, right now Indieweb stuff, (webmentions etc.) are in a honeymoon.  The spammers will figure out how to exploit these in ways you never dreamed of.

Thanks to you and to Kicks Condor for taking the time to reply so extensively.  The first step needed for Blog Snoop is to have an extensive index of blogs.  Only then will it be of any use.  And this was a useful discussion because it gave me some ideas that might work.  Again thanks.

Also on:

Replied to

In reply to: The Awesome Directories.


I think I have an answer to your linkrot concern. Not for Awesome but a hypothetical directory – “someday”.

The idea really isn’t mine, I got it thinking about the directory, which is really a new Indie style webring that also has a List of Sites page aka a flat directory page.  When you join you get a webring code to place on your site, plus listing in the directory.  Cool.


On some hypothetical future niche directory it’s that webring code that prevents the linkrot.  As long as the webring/directories robot keeps finding that validation code you stay in the directory, no code and eventually you will be dropped. Not perfect but it automates the process a bit.

It wouldn’t have to be a webring code, it could just be a validation code, I suppose.  I like the idea of indiestyle webring/directory combined.  My worry would be scale on the ring.  Old style webrings had a sweet spot of 50-200 sites.  Over 200 and they become difficult to manage and it was found that bigger destination sites in the ring siphoned off too much ring traffic and gave very little back.

Still adding that directory page was a good evolutionary move for indie-style rings.

Massaging this idea a bit:  one could let ring members self define subject categories when they join very much like you did on (tags sorta) which would help when it gets larger.

This was also posted to

Also on:

Replied to

In reply to: Catalog of Internet Artist Clubs

Kicks!  Great find.  That is one of the best done websites I’ve seen in a long time.  I love the timeline bar graphs.

I’m beginning to think that the failure of early directories was that they were just piles of links with no sense of an editor or curator.

This got me thinking.  There was/is another directory model that I don’t think we have discussed much: the expert guide model.  When I first got on the Web I ran across The Mining Company which later became  A similar one was Suite101.

The Mining Co. was fantastic, but when they ditched the unpopular topics and became they were not as good.  Still About was decent for awhile.

Anyway the model was each topic or subtopic had an expert guide they would write lots of different essays on their topic, link them all together, and at the bottom of each essay was a list of links to resources and other expert pages on the subject.

This was a sort of hybrid: not just a collection of links but not a wiki either. The bottleneck was finding experts and keeping them.  Wiki’s were developed partly in response to this bottleneck.

These sites were excellent starting points when you knew absolutely nothing about a topic but needed to learn.

Also on:

Replied to

In Reply to Kicks Condor Cataloging Horror Fiction.

Track Expired Links: Many directory scripts had a bad link checker but that really didn’t work well.  Basically you had to go through the directory categories and subcategories manually and get rid of dead wood.  Now I was in a “hobby and fan” niche so a lot of sites listed were static sites on Geocities, Tripod or other free host.  (Blogs, on Blogger and, were new and didn’t have Pages yet) Surprisingly the free hosted sites were more stable than sites on their own domains.  Webmasters on free hosts might abandon their site but they left them up and Geocities didn’t delete so even without updates their existing essays and info were still good.  Sites on their own domains: would either go dark or get snapped up by the p*rn industry and go p*rn.  We did have a way for users to report bad links but only a few went to the trouble.  So clearing out dead wood was a time consuming chore and part of what I mean when I talk about difficulties of getting directories to “scale”.

Updates: No way to know really.  On the Horror search engine I had the spider reindex the Index page every few months so changes made there would eventually show up.

Wikis as replacement for Directory: No and sorta.  A directory is about navigating the Web.  The job of a directory is to get rid of the searcher by helping them find what they want as quickly as possible.  You are sending people off to a primary source.  A wiki is about providing information and if it links out, that is a secondary function.  Take Wikipedia as an example: Wikipedia (which I use daily) is a silo.  It’s just not a commercial silo.  Wikipedia has filled the vacuum left by the closure of free hosts like Geocities.  Before Wikipedia the experts were all building pages about their areas of expertise either on free tilde pages from their ISP or university or on GeoCities and the like, now they are editing Wikipedia pages.  Then Google intentionally filtered out free hosted sites, so it fell to niche directories and webrings to help free sites get found, then the plug got pulled and Geocites went dark.  But that is how we found information before Wikipedia.  Wikipedia is a mixed blessing.  It provides a needed service – free, but it also smothers out many small informational sites from being created (unless they can contain more useful detail than Wikipedia.) It’s more efficient, but lacks funkiness and fun.

Blogs can act as curated directories. I have a small experimental one. But they really can’t do it as well as a directory even though they might aid in discovery they are improvisations.

Webrings are like taking a subcategory and adding navigation between like sites in that subcategory.

Crawler:  Having a real crawler that you control is like having the secret decoder ring!  Even if you can’t see all the uses right now, you will quickly find all sorts of new uses and utility.  You can detect bad links, changed content, who they are linking too, etc.

Moderation: yeah I hate to say it but you will eventually need it.  I closed down many of my directories partly because they were obsolete but partly because those old directory scripts did not have spam protection.  They got spammed out by submit bots.  It got to be too much work deleting out hundreds of spam Add URL’s each week from the review cue.  Right now you are flying under the radar and the Indieweb movement is small, but once both your directory and the Indieweb blips up on the screen the spammers will come.

As you have pointed out, directories are still around us: Yelp, yellow pages, Craig’s List, eBay, Amazon etc are all directories.

Directories can exist without a hierarchy.  The now defunct Searchking “search engine” was really just a linear directory.  You submitted each page of a site (no crawler) you wrote a title, description and 10 keywords (Tags?) and your page was instantly findable in the index.  That was okay in the early days of the Web when sites were static and small.

Your experiments are going to be a lot of fun.  The technology has changed from my day and that will work in your favor.  I can’t help with coding, that’s above my pay grade, but if you have any questions keep asking, I be happy to tell you what I can.

Also on:

Replied to

Replying to: Kicks Condor.

Horror directory:  This was really a faux directory.  Every category was really a search with the category name as the search keyword.  Of course you could do multi keyword searches from the search box.  Ranking was mainly done by keyword placement and density: KW in Title carried X weight, KW in description carried Y weight, KW on page carried Z weight, etc.  It worked.

Science fiction/Fantasy directory:  This was a static directory with categories.  Pretty conventional.  At first sites were ranked by user voted 1 – 5 star rank next by alphabetical order for those with no votes.  At one point I tried an experiment:  “SearchHippo” a little search engine had a free API to compute “HipRank” (poking fun at Google’s PageRank).  Google’s directory listed sites by order of PageRank.  So I had my developer incorporate  listing sites in categories by “HipRank”.  🙂

Nobody appreciated the joke.  So I reverted back to the old way.

Most of the old directory scripts had a simple meta tag fetcher like you describe.  We really could not launch a new niche directory without any sites listed in it.  Visitors would come to us trying to find sites and they would never come back if you gave them empty categories. So we would seed the directory with sites, a few for each category.  That was a lot of work for one person, but we would use that meta tag fetcher (spider) to help us.  I always tried to launch with at least 700 sites listed.  That kind of seeding went on throughout the life of the directory.  A new TV show would launch and you would add new categories for that show and seed, then add categoires for the actors and seed with fan sites.

Tags: The thing is with a static hierarchical directory you get stuck with the hierachy you built.  Tags, even reddit semi permanent tags, are more dynamic and finer grained.  By letting a site like Reddit create the categories you help stay on top of what topics are new and fresh.  It’s like suggestions coming from the grass roots rather than top-down.

But in a small niche community, with a directory editor who really know their community you could have a hierarchical directory with no problem.  Or you could use a combination of search field and a tag cloud.  On a niche directory that might be more minimalist.

Categories did serve a purpose: as a prompt to explore deeper:

Star Trek below that TOS, TNG, DS9, ENT, VOY

Below TOS: William Shatner,  Nimoy, Fanfic,  etc.

Each is a prompt to explore further, deeper, hear other voices.  We are too used to Google giving us the a page and then we leave as if that is the entire answer.

Also on:

Replied to

Replying to Kicks Condor.

Thanks for replying.  If the Web is our social network, then the question of finding stuff on the web looms large.  Which is what led me to post about search engines and directories.

I like your observations about recency vs static in directories.  To me the directory of the future needs to be a hybrid of directory and search engine:  It needs to combine both recency and curated static PLUS a crawler.  Reddit is a good starting point for recency if you have a spider crawling to pick up newly posted URLs plus the archives.  Add in Curlie and Wikipedia to the crawl but use sub Reddits for taxonomy.  That’s for general search engine.

Once upon a time: I used Fluid Dynamics Search Engine to create a  sort of crawling Horror Fiction directory.  It used the Meta tags and or user submitted data, plus it crawled the submitted page so it was indexing both meta data plus real on page text for search database.  It worked quite well. I had always wished I had the resources to crawl deeper.

Anyway, I think you are on to something.

The goal is discovery, I think using many means, large and small is best.

Also on:

Replied to

In Reply to: Chris Aldrich – Webrings have come back to the web.  Gah!  This brings back half forgotten memories.  I was a RingMaster on both and Ringsurf.  Plus I actually ran a Science Fiction/Fantasy webring hosting site via a nice free Perl script for a few years.  I never thought in my wildest imaginings that somebody would revive the webring concept, but it fits with the theme of the Indieweb movement.  (BTW: Ringsurf appears to be still around but I have no idea how it works today.)

Also on:

Replied to

In reply to: Intertextrevolution

Thanks for the kind response.

I’ll answer here your webmention reply didn’t show on my site. I’m sure I probably have a setting for that wrong somewhere. 🙂

You are right, WordPress is a handful to get set up. If you add in the Indieweb stuff it becomes even more of a handful. In theory you only have to do this once which is good. You folks are on the right track with the Indieweb plugin – it makes things much easier.  All I knew, after doing a lot of reading on Indieweb stuff, then seeing the Webmentions on other blogs, I knew this was what I wanted.

After writing that post I got to thinking: one approach to documentation is to treat it like cooking recipes and cooking classes. (ie. I’m making a pie crust, the recipe calls for peanut oil which will do this, but can I substitute corn oil and what will that do to the pie crust? etc.)

Another way is to think like a blogger: “3 Best ways to Syndicate Your Indieweb Blog to Twitter,” type articles.

I understand the Indieweb eco-system is not yet complete, templates, feed readers are all still being developed.  Someday, this will all be close to plug and play, but in the meantime we need some big FAQ’s and comparison of real world (read WordPress for now) services.  Unfortunately, my trial and error approach burns up too much time.

Thanks to Chris Aldrich who has been doing a good job of explaining a complex eco system.

Also on: