If you have ever used a UNIX-based system, such as Linux or macOS, you may have noticed that some files and directories start with a dot (.) character. For example, .bashrc, .gitignore, .htaccess, or .config. These are called dot files or hidden files, and they are not visible by default when you list the contents of a directory using the ls command. Why are these files hidden and what is their purpose?
It’s not a feature, it’s a bug
The origin of dot files goes back to the early days of UNIX, when the ls command was first written1. The ls command used a simple logic to display the files in a directory: it would loop through all the entries and print them to the standard output, except for the current directory (.) and the parent directory (..) which were special entries that always existed. However, some users started to create files that began with a dot, either by accident or intentionally, and they noticed that these files were not shown by ls. This was because the ls command treated any file that started with a dot as if it was either . or .. and skipped it.
This behavior was not intended by the original authors of ls, but it turned out to be useful for some users who wanted to store configuration files or other data that they did not want to see every time they ran ls. Thus, the convention of using dot files for hidden or system-related files was born and adopted by many programs and users. For example, the .bashrc file contains commands that are executed when a new bash shell is started, the .gitignore file specifies patterns of files that should be ignored by git, and the .config directory contains configuration files for various applications.
A Useful Unexpected Feature
Dot files are not really hidden in the sense that they are inaccessible or encrypted. They are simply ignored by the default behavior of some commands and tools that follow the dot file convention. You can still access them using their full name, or by using options or flags that override the default behavior. For example, you can use ls -a or ls –all to list all files in a directory, including the dot files. You can also use wildcards or glob patterns to match dot files, such as ls .* or ls .[a-z]*.
Dot files are a common and useful feature of UNIX-based systems, but they also have some drawbacks and limitations. For instance, some users may find them confusing or annoying, especially if they are not aware of their existence or purpose, especially on systems like Windows where they are not used. Some programs may create too many dot files or directories in the user’s home directory, cluttering it and making it hard to manage. Some dot files may contain sensitive or personal information that should not be exposed to other users or programs. Some dot files may conflict with each other or with the user’s preferences if they are not properly documented or standardized.
Therefore, if you use or create dot files, you should follow some best practices and guidelines to avoid potential problems and ensure compatibility and security. Here are some of them:
– Use dot files only for hidden or system-related files that are not meant to be seen or modified by regular users. Do not use them for normal files that contain user data or content.
– Document the purpose and format of your dot files, either in comments inside the file itself or in a separate README file or manual page. Explain what your dot file does, how it is used, what options or settings it supports, and how it interacts with other dot files or programs.
– Follow the existing conventions and standards for common dot files, such as .bashrc, .profile, .gitignore, etc. Do not create your own versions of these files with different names or semantics, unless you have a very good reason to do so. Respect the user’s existing dot files and do not overwrite or modify them without their consent.
– Store your dot files in appropriate locations, such as the user’s home directory, a subdirectory under it (such as .config), or a system-wide directory (such as /etc). Do not scatter your dot files all over the file system hierarchy.
– Use descriptive and meaningful names for your dot files, starting with a dot and followed by a word or acronym that identifies your program or project. Avoid using vague or generic names such as .data, .settings, .temp, etc.
– Use consistent and portable syntax and formats for your dot files, preferably plain text or a widely supported markup language such as JSON, XML, YAML, etc. Avoid using binary formats or proprietary formats that require specific tools or libraries to read or write.
– Protect your dot files from unauthorized access or modification by setting appropriate permissions and ownership. Use chmod and chown commands to restrict who can read, write, or execute your dot files. Use encryption or hashing techniques to secure sensitive data in your dot files.
- Pike, Rob (2 August 2012). “A lesson in shortcuts”. Google+ (via Internet Archive). Retrieved 11 May 2023.