ACK!!'s Journal: Unix Truths

Everything is a file -- One of the unique things about Unix as an operating system is that regards everything as a file.
Files can be divided into three categories; ordinary or plain files, directories, and special or device files.

Directories in Unix are properly known as directory files. They are a special type of file that holds a list of the other files they contain.

Ordinary or plain files in Unix are not all text files. They may also contain ASCII text, binary data, and program input or output. Executable binaries (programs) are also files, as are commands. When a user enters a command, the associated file is retrieved and executed. This is an important feature and contributes to the flexibility of Unix.

Special files are also known as device files. In Unix all physical devices are accessed via device files; they are what programs use to communicate with hardware. Files hold information on location, type, and access mode for a specific device. There are two types of device files; character and block, as well as two modes of access.

Block device files are used to access block device I/O. Block devices do buffered I/O, meaning that the the data is collected in a buffer until a full block can be transfered.

Character device files are associated with character or raw device access. They are used for unbuffered data transfers to and from a device. Rather than transferring data in blocks the data is transfered character by character. One transfer can consist of multiple characters.

Everything flows from the root -- The UNIX filesystem is heirarchical (resembling a tree structure). The tree is anchored at a place called the root, designated by a slash "/". Every item in the UNIX filesystem tree is a file even a directory. A directory is a file that holds other files. A directory can contain files, or other directories which are merely files to hold other files.

A directory contained within another is called the child of the other. A directory in the filesystem tree may have many children, but it can only have one parent. A file can hold information, but cannot contain other files, or directories.

UNIX supports access control. Every file and directory has associated with it ownership, and access permissions. Furthermore, one is able to specify those to whom the permissions apply. Every UNIX system has a special user, called root or superuser, who has unique and powerful privileges associated with system administration. Root can access all files, regardless of access permissions. Root can read, write, or run any file; search any directory; and add or delete a file in any directory. Root can change a user's password without knowing the original password. Root can halt the system and change ownership of files. Because root privileges are so powerful, they can be destructive, intentionally or not. Root privileges should be used carefully.

Root is the top of the filesystem. The Root user controls that filesystem. Everything in Unix is a file. Root is the top of the hierachy of the files and the user that controls the top of that hierachy of files. Root is the superuser he who controls the top of the hierachy of the system.

Everything flows through the pipe -- The concept of pipes is one of the most important Unix innovation (the other two are probably hierarchical filesystem and regular expressions) that had found its way to all other operating systems. Let me state it again: the pipe is the the most elegant and powerful features of UNIX.

Dennis M. Ritchie once said, "One of the most widely admired contributions of Unix to the culture of operating systems and command languages is the pipe, as used in a pipeline of commands. Of course, the fundamental idea was by no means new; the pipeline is merely a specific form of coroutine.

Even the implementation was not unprecedented, although we didn't know it at the time; the `communication files' of the Dartmouth Time-Sharing System did very nearly what Unix pipes do, though they seem not to have been exploited so fully."

Pipes are elegant implementation of coroutines on OS shell level and as such they allow the output from one program to be fed as input to another program.

Hence the command: cut -f4 vism.txt | paste - vism.txt | cut -f1,2 | sed -e 's/$/.blah.com/'

Which allows a user to take the fourth field of a text file paste it back into the output as the first field and then take the 1st and 2nd field of output and tack a domain at the end in one command.

This is a beautiful and elegant thing.

All things may be matched by a regular expression -- In the forties, Warren McCulloch and Walter Pitts created neuron-level models of how the nervous system operates. The mathematician, Stephen Kleene, later described these models using his mathematical notation called regular sets. Ken Thompson incorporated that system of notation into qed (the grandfather of the UNIX ed) and eventually into grep. Ever since that time, regular expressions have constantly seeped into UNIX and UNIX-like utilities.

A regular expression is a string of characters that can be used to match a set of character strings. For example, to globally search for all occurrences of the word "and" would require a search for "and", "And", "AnD", "AND", etc. Without regular expressions finding all possible occurrences of "and" would require eight separate searches. Using an regular expression the search could be done with one command.

Regular expressions are used when you want to search for specify lines of text containing a particular pattern. Most of the UNIX utilities operate on ASCII files a line at a time. Regular expressions search for patterns on a single line, and not for patterns that start on one line and end on another.

Therefore the following string is not just gibberish but gibberish that allows you to find not just a word or a set of words but match divergent strings in text.

m{ \(
      ( # Start group
      [^()]+ # anything but '(' or ')'
      | # or
        \( [^()]* \)
      )+ # end group
  \)
}x

An Operating System should not be designed to stop you from doing stupid things --
"UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." --Doug Gwyn.

Unix utilities and applications should not try to anticipate all possible needs, and they should never try to keep you from doing something that might be dangerous. Your program may be so useful that it outlives you. Even if it doesn't, it should most certainly outlive your current project whenever possible: Don't try to guess at how people -- even you yourself -- might want to use it in future. This feeds back into the simplicity argument. Trying to prevent undesirable consequences can be a disaster. (For instance, imagine how much less convenient it would be if programs that used $EDITOR tried to enforce the assumption that it will be an interactive program.)

This is the organically grown Operating System -- Unix did not spring whole from the mind and effort of one company or set of individuals.

Unix has instead grown organically in a zen-like fashion over decades of use created by Dennis M Ritchie and Ken Thompson at Bell Labs and then in use by universities absorbing influences from Berkeley Unix and Bill Joy or various corporations such as Sun, HP and DEC.

Or from the zeal of developers like Richard Stallman and Eric S. Raymond and the Open Source movement.

This natural and somewhat haphazard approach is both maddening to the newb and delightful for the explorer leading one to never grow complacent in the idea that they know all that Unix has to offer. Because expectation of knowledge only goes to show you know what is coming next and you don't.

"Those who don't understand UNIX are doomed to reinvent it, poorly." --Henry Spencer