Does this exist: a standardized way of documenting a file-system structure

At work, I'm in charge of maintaining the organization of a whole lot of varied data on a standard file-system. Part of this is coming up with sensible classification (by similarity, need, read/write access, etc), but the bigger part is actually documenting it: what documents/files/media should go where, what should not be in this directory, "for something slightly different, see ../../other-dir", etc.

At the moment, I've documented this using a plaintext file readme in every directory I want to document. If someone is unsure what's meant to be in any directory, they read that file.

This works alright, but it seems odd that I have this primitive custom solution to a problem that any maintainer of a non-trivial directory structure must experience. Every company I've known of, for example, has some kind of shared file-system where agreed terminology for categorization is important. In my experience, people just have to learn what's what by trial-and-error and experimentation.

So allow me to propose a better solution, and hopefully you can tell me if it exists. Any directory on any filesystem can have a hidden plaintext file named .readme. Its contents are descriptive human language. It uses some markup like Markdown, with little more than bold, italic, and (relative) hyperlinks to other directories. Now a suitably-enabled file browser will check for a file named .readme whenever it displays a directory. If it exists, its contents are parsed and displayed in an unobtrusive pane near the directory-path widget. Any links therein can be clicked, and the user will be taken to the target directory of that link.

I think that the effort of implementing such a standard would pay back many times over in usability gains. We would have, say, plugins for Nautilus, Konqueror, etc.. It could be used to display directory information in the standard file lists served by webservers. And so on.

So, question: does such a thing exist? If not, why not? Do people think it's a worthwhile idea?

As far as I know there is no standard. Here are a few ideas from my experience.

Set it up, never change it

This is were most companies fail. Nothing is worse than an ever changing file-system structure. If it is not possible to keep it constant then a pure file-system is just the wrong container to organize your information. Use a database or a Content Management System.

Use descriptive and consistent directory names

Nobody has the time to read a .filing file or anything else. If your directory names are not self explaining you probably are lost anyway.

Write a documentation for your directory structure

Write a document where you explain the role of every directory. Give lots of examples. Make it available to anyone who has to work with your structure, but don't believe anybody will read it. It should be more like a Bible to you. It's not easy to find an example for such a document, because obviously companies don't publish them. An example from open source software is the Filesystem Hierarchy Standard.

If this sounds a little negative, it is. I've never seen a non-trivial repository based on a file system with more than five users work in the long run in practice. The problem is that whatever categories you'll set up, people will have completely different ideas about them. So to finally answer your questions:

Does such a thing exist?

No, I don't think so.

If not, why not?

In my opinion: For a small static hierarchy with a few users it's overkill. For a large changing hierarchy with many users it won't work because the idea of categories (=directories, folder) doesn't scale.

Do people think it's a worthwhile idea?

Hm, it's an interesting idea. To see if people will use it, someone has to implement it. Instead of a .filing file you could store that information in an alternate data stream (yes, folders can have ADS too). You could use extended attributes on Linux and OSX. The biggest problem would probably be to patch the file browsers.

wasabi is worth a shot. At your project root, the plain-text source will serve as a solid project overview and you can throw the same document at a browser for some fancier results.

Granted, it's not a filesystem-based solution, but until all the filesystems around support something common, wasabi (or your own implementation of it) could well be the best option.

Your idea has some merit, but I am afraid that when you need something like this it really means that you need something more structured. I.e. a CMS, maybe just a lightweight one, but definitely something more than just a textual file.

Especially if you want to restrict writing (or even accessing) specific documents (and therefore their containing folders) to some subset of your userbase.

You don't specify your OS, but there are excellent (and free) products like alfresco that could maybe serve you better than your current setup.

Here is an idea. Write a script that asks the user various questions about the file and/or do some matching in the file content itself, and then recommends the location to put the file or puts it there itself. The script can be as simple or as complex as you want it to be.

The script acts as a file management system, decision support system, and a living documentation of the filesystem hierarchy. The Linux Filesystem Hierarchy Standard is a bit of a myth if you think about it, because there's a very large variance between distros. However most Linux/Unix users don't actually have to learn about the filesystem hierarchy themselves because various software installed in the system exists that manages the hierarchy in a standardized way (package manager, configuration tool, etc). Various application framework also creates scripts to manage its directories, e.g. django has management commands to create new project, or new app module, or squash migration files, etc.

This has the additional advantage that the script can create various symlinks if the file need to be searched for in more than one way, for instance an index based on the file author or creation date or whatever business rule you have. You can simulate tagging with symlinks. A tag is simple an symlink from a tags directory, so tags/mytag/myfile may be a symlink to actual/myfile.

Additionally it will also be possible to change the filesystem hierarchy structure without changing the user interface.

A filesystem is a database. Not a relational database, but hierarchical database. Think about it like managing a database, you don't really want to require people to have to learn the relational structure, but rather you want the application to present to the user as various tasks they need to do (e.g. are you doing a monthly XYZ report, great, then put it in this folder, and create this symlink, and then you'll need to modify another file to record that you've done the report for this month, etc).