(Archived) IDEA: Deeply integrating into the OSX file system

[rsweetland 2]

After frustratedly rearranging my folders on my mac today for the 1000th time, I found myself longing for the power and simplicity of the Evernote filing system.

I think it's time someone disrupted this whole traditional folder-based file system.

There are several file-tagging solutions available for OSX, but how annoying it is to have such a great filing system in Evernote, and then have a separate filing system for my local files. And with Evernote, the lines between "notes" and "files" are now totally blurred.

Dropbox, Box, Google Docs also have this same traditional file-based organization.

I'm wondering if there are other people like me - that are willing to jump ship and go to a completely tag-based file system.

If I there are others like me, what if Evernote published a deeply integrated solution for all of my files. Those files that I tag might then be automatically added to my Evernote world (auto-synchronized, versioned, OCR'd, etc). It would be incredibly valuable.

So in short, my request is this: Evernote, please replace my folder-based file system. I'm tired of it and you guys could do a much better job!

[GrumpyMonkey 4,320]

After frustratedly rearranging my folders on my mac today for the 1000th time, I found myself longing for the power and simplicity of the Evernote filing system.

I think it's time someone disrupted this whole traditional folder-based file system.

There are several file-tagging solutions available for OSX, but how annoying it is to have such a great filing system in Evernote, and then have a separate filing system for my local files. And with Evernote, the lines between "notes" and "files" are now totally blurred.

Dropbox, Box, Google Docs also have this same traditional file-based organization.

I'm wondering if there are other people like me - that are willing to jump ship and go to a completely tag-based file system.

If I there are others like me, what if Evernote published a deeply integrated solution for all of my files. Those files that I tag might then be automatically added to my Evernote world (auto-synchronized, versioned, OCR'd, etc). It would be incredibly valuable.

So in short, my request is this: Evernote, please replace my folder-based file system. I'm tired of it and you guys could do a much better job!

hi. welcome to the forums!

i like your idea, and i wouldn't be surprised if we eventually get to a tagging system, but i doubt evernote will be getting involved in that kind of deep integration anytime soon. more to the point, though, i think evernote is currently set up to store files well, and to let you do some basic notetaking. however, frequently modified attachments are not really its strong suit. in other words, every time you edit a word document, you open it in word, you make changes, and when you save it the file goes back to your note as an attachment, but it has to be uploaded in its entirety each time you sync. so, this eats into your upload allowance. i tend to use dropbox for current projects and evernote for completed ones, so that my workflow looks like this:

1. start project as a note of some sort in evernote

2. finish project in a third party app like word or scrivener

3. save a copy of completed project in original file format and pdf file format (to make it searchable in evernote)

[jefito 5,598]

Alternatives to traditional hierarchical file systems are talked about, but have not made it into the major consumer desktop operating systems. I don't know about anything on the Mac side, but Microsoft has been talking about this for years, mostly in terms of a file system based on the relational database model (I went to a developer's conference in 1993 where they were talking this up). See the Wikipedia article on WinFS here, http://en.wikipedia.org/wiki/WinFS, for an overview.

[JMichaelTX 4,119]

I'm wondering if there are other people like me - that are willing to jump ship and go to a completely tag-based file system.

I sure there are others like you. However, there are lots of other people, including me, that do NOT want a tag only file system.

Both Tags and hierarchical folders have there uses, and could be used together.

So in short, my request is this: Evernote, please replace my folder-based file system. I'm tired of it and you guys could do a much better job!

Frankly, I don't want Evernote to even spend 1 second on this. Evernote has its plate full of just making a stable, consistent product across all platforms, plus many, many features requested by users. Given the stability of Evernote, I would *never* allow Evernote to take over my Mac OS file system.

[rsweetland 2]

Thanks for the input everyone.

@grumpymonkey Thanks for sharing your workflow. I also use Dropbox and Evernote in a similar way.

Somehow Dropbox solves the bandwidth and continuous uploading issues. If Evernote could do this while maintaining the right economics, it seems that augmenting or replacing the traditional file system is a pretty big market. While Dropbox, Google drive, Amazon, etc are huge, Evernote could own the market segment for those of use who are willing to think outside the box - or folder - as the case may be ; )

But, @jmichael, to your point, I have a bunch of other feature requests in advance of this. This post was mainly to surface a hair-brain idea that could be fun and useful some day (think years, not months).

------

EDIT: Well look at that. Here is a super-light version of what I was thinking. Send A File Right To Evernote From Within Finder

[wrmay 2]

This idea also has some appeal to me. I thought about how I might implement this and came up with Evernote Cloud API + Mac Fuse and it appears I wasn't the first to go down that road: http://rpj.github.com/evernotefsapp/

It might be worth checking out, as a starting point if nothing else. It appears to be 3 years old so there's also a good chance it won't work.

Either way, someone else had enough interest in this idea to spend some time on it.

[robheus 0]

The idea is intruiging to replace the traditional way in which data is stored as files, and replace that concept with the idea of a relational file system. Since UNIX we are kind of tied to the philosophy that everything can be presented as a file (a stream of bytes), even devices and processes. To replace that with the concept of a relation (like in a relational dabatase) or something similar (a node in a network, the mathematical concept of a graph), since we would wish to accomodate also unstructured data, digs deep into how almost all operating systems are built up, and how practically all applications intereact with the operating system.

The problem what comes to mind then is that in order to avoid having to rewrite all your applications is that the OS should still simulate the traditional hierarchical file system. I guess that such an extra layer of abstraction would simply be too much of a burden on the performance of the system, as also that you need to redesign the OS kernel from the bottom up to accomodate this concept.

Or, alternatively, when such does not seem practical, you would need to add on top of the traditional file system a layer of abstraction to accomodate tagging. Tag based file systems do exist already.

Related to this idea is also the Internet File System (IFS), which is storing the fileystem in a database. Such systems also already exist (just Google: "Internet File System").

What comes to my mind also relating to this idea that instead of seeing files as a series of bytes, replacing that with the idea of a relation, is that for software development, the sources would not be stored in a text file (as a series of textual characters,containing code, compiler directives and comments), but in their grammatical form (as required by the language standard, compiler and flavour of the language used and compiler flags), which would simplify software modifications and would eliminate many errors during software development. For example changing the name of a variable or function would only require to change it in one place (it would be simply be a label or name of the node of that function or variable) and all related modules which use or refer to that variable or function, would automatically refer to that function or variable under its new name. This idea is actively being researched by Microsoft under the name "Intentional Programming".

If the underlying operating system would be built on the concept of "everything is a relation" (instead of the currently used paradigm that " everything is a file") the relationships between sources, (intermediate) objects and libraries and targets, would be already stored in the database file system, and you would not need to explicitly define those relationships in make files. More generically, every object/relation (what we currently call a "file") would be seen as either a source or target, or in most cases both (since for example, the source would be itself a target of the input or editing process or some other process, like in generated sources or lexical scanners that produce source files, and additionally the final executable (or whatever type of target is produced) is itself a source for the load&execute process of the OS), for which a process exists that creates the target(s) from the source(s), using some process parameters. Both sources, targets and processes (and their parameters) would be implemented in their relational form, stored in the database. You would always have complete information on dependencies between objects, including versioning information of every source and process and library used in building the target.

For all kind of development projects such an alternative operating system based on relationships, instead of files, would seem very handy.

[rsweetland 2]

@robehus you just took this idea to a whole new level : )

You bring up some interesting points in here, and while a few are over my head (or below my level in the stack as the case may be) I do have one philosophical contribution: in order for that to work, I think it needs a real-world analogy. Ie, for users to understand an abstract concept like this, it needs to be related to something they already know. The proverbial "file", is easily understood as a piece of paper where things get stored. An abstract system of relations of things may break people's mental model of things...

But then again, we may be talking about two different ends of the same problem: I am proposing a mental model for an end-user (or higher-level developer). You may be proposing a lower-level technical implementation which could be adapted to any such mental model. (It definitely sounds more flexible!)

Interestingly, tags are already supported in OSX...they're just a bit hidden.

I think Dropbox did an amazing job at integrating a new solution into the traditional file system.

If I had an "Evernote" folder on my system with all my images, notebooks, etc, I'd be quite happy.

[robheus 0]

@robehus you just took this idea to a whole new level : )

You bring up some interesting points in here, and while a few are over my head (or below my level in the stack as the case may be) I do have one philosophical contribution: in order for that to work, I think it needs a real-world analogy. Ie, for users to understand an abstract concept like this, it needs to be related to something they already know. The proverbial "file", is easily understood as a piece of paper where things get stored. An abstract system of relations of things may break people's mental model of things...

But then again, we may be talking about two different ends of the same problem: I am proposing a mental model for an end-user (or higher-level developer). You may be proposing a lower-level technical implementation which could be adapted to any such mental model. (It definitely sounds more flexible!)

Interestingly, tags are already supported in OSX...they're just a bit hidden.

I think Dropbox did an amazing job at integrating a new solution into the traditional file system.

If I had an "Evernote" folder on my system with all my images, notebooks, etc, I'd be quite happy.

Ok, let me elaborate a bit on this concept "relation" if it is not obvious and a bit too abstract. Let me give some examples that explain what "relation" (as in " relational database") is all about and what the difference is with the concept "file".

Take for example an education (or school) system, where there are STUDENTS, COURSES and TEACHERS. Now the relationships can be defined as follows:

TEACHERS teach on one or more COURSES. COURSES have exactly one TEACHER. (nb. this does not always need to be so, some schools have more as one teacher that teaches the same course)

STUDENTS take class in one or more COURSES. COURSES have one or more STUDENTS.

Now from these definitions you can extract the relationship between teachers and students, because for every course a student takes, there is exactly one teacher (in this example). So, students have one or more teachers (one per course), and teachers have one or more students.

Relationships type are categorized as:

1:1 One-to-One

1:N One-to-Many

N:M Many-to-Many

and we can subcategorize on wether the membership on either end of the relation is mandatory or not. In our education system example: a Student may temporarily not take any courses during absence (sickness), and the same might apply for teachers.

What the relational model achieves is that we avoid having to store redundant information. For instance if we would need to store the full information about a course including teacher and student info, we would have to supply the data about student and teacher too. This would lead to redundant information, and possibly also incorrect data (for instance, when we misspell a students name). In the relational model, all you would store about a course are the info directly belonging to the course itself, and for the teacher and students, you would only store the unique ID's of the records of students and teacher. A misspelled name would only have to be corrected in one place.

In the file-system/directory hierarchy we in fact only have one type of relationship between directories and files. A directory contains one or more files and/or (sub)directories. Which in relational tems mean that neither a file or directory can be at the same time member of more then one directory. And for directory, they have always a parent directory except for the root directory.

As a provision for bypassing this hierarchical filesystem, we can have (symbolic or hard) links, to achieve that a file or directory might be visible and accesible from more then one directory.

Further, a file in fact pre-supposes unstructured data. The OS has no more knowledge about the file contents then a series of bytes, and only the directory stores some meta-information (user that created the file, the file permissions like read/write/execute for user,group and world, file creation and last modification dates, an archive bit). Most OS-es "guess" the file contents from the file extention (the last part of the file name after the dot) - which is not always exact because some extentions have more as one file-type associated with it and anyhow the extention part of the filename might not represent the real filetype or could be left out) and/or from the last time the file was opened, and store in some other file which application is normally used for opening the file.

But sometimes (dependend on the file type and content of the file) file data is in fact structured. But this structure is then imposed on the file (when reading / writing) by the application and not by the operating system.

A strict implementation where the OS dictates that anything that is a file must have a structure (fitting the relational model) is not possible because there are too many exceptions of file content that we could not cope with (although there are standards defined for a number of file types, real world data shows that there are too many exceptions, and many software companies create their own closed data formats), so for that reason I think this idea is not practical (or: at least the OS should accomodate/allow unstructured data storage as a mere sequence of bytes, and leave it to the application how to interpret such data), and also it might make the OS too heavy implementing it and lead to performance problems. Yet, in cases where the data we deal with is in fact structured and fits the relational model, maybe this is acceptable to implement at low level.

Most implementations of an RDBMS (relational database management system) still operate at the basis that they use the underlying OS concept of a file for datastorage, yet they partly abstract from the file concept. For example table data is often allocated in a so-called tablespace (which has for every table a datasegment, which consists of extends which in turn consists of a number of database blocks, which are often a multitude of the OS block/sector), and the tablespace itself is allocated in the form of one or more operating system datafiles, formatted in a specific way to be usable by the RDBMS.

The example I gave about source files written in some higher level programming language, would in fact require that we should store the file in it's grammatical structure (the parse tree). This is in fact the objective of the Intentional Programming research project, because it would ease many programming development tasks (the example given was renaming an identifier, which is in many cases quite cumbersome, because the same name might be given to different programming constructs and they refer to different objects - in the context of Intentional Programming these different objects refer to different nodes, and when we want to rename only one of them, there is only one place where we can adjust the name, and all objects that reference the same object are automatically renamed - so it eliminates the problem of false matches when we use traditional programming tools that only look for a textual match, and are not aware of the grammatical context).

As a provision for the lack of relational data stored by the OS, many OS-es (not only OSX, as also Windows 7 and 8 can provide this, although I didn't use it myself) provide the possibility of tagging files (at least for some filetypes), but to my understanding this is not something implemented on the OS level but by some other means.

What this in fact accomplishes is that by tagging you can make the file that you tag a member of more then one 'virtual' directory as indicated/named by the tag. That is, you can then have so-called many-to-many (N:M) relationship between the file and the group or 'virtual' directory (as indicated/named by the tag) to which it belongs. So then you don't need to conform to the OS strict file/directory hiearchy which only allows 1:N (one-to-many) membership relationships between file and directory, and have greater flexibility as provided by using (hard/soft) link.

[JMichaelTX 4,119]

There is a lot that can be done with the current Mac OS:

• Tags
  
• File/Folder Alias
  
  • Allows you to put an "alias" (shortcut or link) to a file in as many other folders as you wish
    
  • Would allow you to, for example, store your latest auto insurance policy under "Auto" folder, and put links to it under "Insurance" and "Financial" folders.
    

  [*]The Mac OS search engine Spotlight allows you to quickly search across all drives on your system very quickly