You are writing an application, and you plan to distribute it to lots of people.
Some people like black text on a yellow background.
Others prefer white text on a black background.
- People are too stubborn to come to a consensus on the "right" foreground and background colors.
- It's too much hassle to interview each user as to their preferences, and then build each one a customized executable.
- When the program is first run (or if it can't find the preferences file), it creates a "default" preferences file in the appropriate place.
- When the program is run any other time, it reads the preferences file for various start-up settings.
- When the user hits "Edit Preferences" in your application, a structured preference editor shows the current settings and allows the user to change them.
- When the user hits "OK" in the preference editor, it saves any new settings to the preferences file.
- Most users overwhelmingly prefer a structured preference editor over having to edit text configuration files with vi.
(I'm attempting to pull the PreferenceEditor? information out of PowerOfPlainText. This is my first attempt to put something into a Pattern. Feel free to clean up the mess I've made of it. Should I be using ThereforeBut or CompactForm? Or perhaps it's not a pattern at all, and the page should be renamed PreferenceEditor?? -- DavidCary)
(moved from PowerOfPlainText. If I've moved too much, feel free to move it back.)
binary configuration files: pros and cons
Text configuration files suck, because:
- The program needs to handle errors.
- It's possible to screw them up very badly with a TextEditor
- The program has to parse them
Binary configuration files are much better, because:
- The program needs to handle errors (Yes of course)
- It's possible to screw them up very badly using the configuration program (how: it's the job of the config program to prevent this OK, but the (l)user will want to edit his config files in vi regardless of your config program.)
- The program has to parse them (why: they're binary after all? And binary files magically teleport themselves inside your program?) Yes. See teleport below.
- A human can't easily read them without having the config program installed ("that's what the config program is there for'')
- Your other tools -- grep, sed, awk, perl, emacs, etc. -- can't be used to manipulate them. (WhyWouldYouEverWantTo? On the Mac all preference files are managed by config programs, and I for one feel better off because of it)
- You can't easily tell when they're corrupted. (In my experience, corruption is primarily (not exclusively) a problem under Linux and Windows and almost never happens on Macs. I use all three regularly.)
Storing resources as plain text makes them easier to maintain and increases their longevity.
Strongly disagree, based on experience with Macs, where plain text preference files are unheard of, and every program is expected to provide its own structured preference editor. I suspect the real reason plain text resource files are common in Unix is that Unix programmers can get away with forcing the user to do things that the programmer should have done in the first place. That is hardly an ideal we should strive towards. -- BradCox
Every program is expected to provide its own structured preference editor
This is a very bad situation. This means that every program goes out of its way to provide its own buggy, feature-poor and non-standard reimplementation of a structured text editor. Why not have the user edit the file with his favorite text editor? Why not give the user a choice? -- StephanHouben
But it works on the Mac. They aren't text editors, they are preference editors. -- RalphJohnson
Also note that storing your file as plain text does not mean that you don't provide a structured preference editor. E.g. Emacs provides a GUI to edit many of its options.
Indeed, Mac OS X has migrated preference files into XML behind the scenes...
But OS X also
gives the programmer a standard API for editing those XML property lists (either directly or by serializing a tree of CocoaFramework
objects), and gives the administrator/developer standard tools for editing arbitrary property lists (NetInfo?
Manager and Property List Editor).
Another example of a structured preference editor for plain-text configuration files is The Dotfile Generator, by Jesper Pedersen. It's available from http://sunsite.bilkent.edu.tr/pub/linux/dotfile/
and is intended for UNIX systems running Tcl/Tk. It has modules for bash, elm, fvwm, ipfwadm, procmail, and tcsh. There had been an Emacs module, but it seems to be gone now. -- EricJablow
Q: Why would you ever want to use grep, sed, awk, perl, emacs, & etc to manipulate configuration files?
A1: Because AutomationIsOurFriend
A2: Because it is far easier and reliable to make a common change in a lot of places or files with a script than to manually walk through some sort of GUI and make changes one at a time. Ask Microsoft folk about the joys of regedit.
There's a programming elite (I consider myself one of these) for whom AutomationIsOurFriend
. And there's 'the rest of us' for whom this claim is most emphatically false (ask my wife, or my mother). Which group would programming elites be better off building software for? (Hint: ask Bill Gates, or Steve Jobs, or more recently the Gnome/Helix project: all programming elites that really get it.) -- BradCox
Brad, I think yours is an important point. I recently started using Linux as a development platform, and love it because it a natural fit for a developer. (Automation, documentation, developer tools, etc.) But, at the same time, as a Linux newbie, I was frustrated by the LearningCurve
required to take advantage of the power. So, depending on the perspective, Linux may be great, or it may be inferior to other platforms. It's the old trade-off between flexibility versus complexity, and perhaps usefulness for developers versus usefulness for users. Disclaimer: The previous paragraph is full of generalizations, but was just mentioned to agree with the tension Brad mentioned. The PowerOfPlainText
is real, but may not always be appropriate for the desired goal. --
Thanks, but I'm not sure that the tradeoff you speak of is real. I've been using Unix since god made dirt, and am reasonably capable at Windows as well. But my main system is a Macintosh. By that I mean I do all of my editing there via filesharing. I've never felt the slightest need to grep the preferences files.
OK, but all the arguments you give are pro config tools, not anti text file. Again, having your data stored in a text file does not
mean that you can't have a config tool. So why are you insisting that your config files are unreadable to the "naked Vi"? -- StephanHouben
I've nothing against text files (except the feeling we could and should move beyond 7-bit ascii as discussed elsewhere on this page). To restate the objection again clearly, my objection is to the claim that users can/should do (via a text editor) what the programmer should have done by providing a specialized preferences editor inside the program. Specifically, forcing users to edit config files with text editors (as in apache, jserv, tomcat, XFree86, many others) instead of encapsulating the same files within specialized config programs. The Macintosh provided the value of this for 'the rest of us', Windows picked it up from there, and Helix/Gnome recently started moving Linux in this direction.
Of course the fact that Gnome/Helix doesn't include a single non-text configuration file is beside the point? Granted they're almost all either XML or MS .ini style format, but they are all plain ascii none the less
. In fact Gnome goes even further. To the best of my knowledge, not one Gnome application uses a binary format as its data format! (Note that these are Gnome applications not necessarily all applications using the Gnome libraries). XML provides an escape from the fragility of many text formats, and gzip adds the size benefits generally attributed to binary formats (gnumeric's default format is gzip'd XML, this is a common approach). -- AndraeMuys
(refactor as desired)
One other point worth mentioning is the example of XFree86 given by Brad above. In the case of the XF86config file, which is a plain text file. 99% of the time the file generated for you by the configuration utilities works perfectly. However there have been occasions where there were new protocols supported by the server, but not supported by the config utility yet, or screens (especially lcd's) with strange requirements. In these cases either the config tool becomes horrendously complicated, or you go in with a text editor and change the file manually.
Remember it is a significant advantage to all
users if advanced
users can make hand-tweaked modifications to config files when serious problems arise.
So, to summarize: "ThereIsMoreThanOneWayToDoIt
". Think of the standard MVC model for user interfaces: you have a model (the text file) and you have multiple ways to edit it: vim, emacs, configEditor, &Edit->Preferences, etc.
In fact, to generalize the "gzip-it" approach, a full binary file format is adequate as long as you are given 2 methods with which to process it: "exportAsTextFile()" and "importFromTextFile". It's slightly more convenient to have the text files all the time, but any script guru should be able to automate the calls to the export and import methods ("Automation is our Friend"). But if you choose to use a binary format, make sure it's not a PrematureOptimization
. -- DaveWhipp
but all the arguments you give are pro config tools, not anti text file.
Do we need a PreferenceEditorPattern
And binary files magically teleport themselves inside your program?
Many programming languages provide direct support for serializing and deserializing language objects as (platform-neutral) binary files. Examples of languages that can do this include JavaLanguage
and several Smalltalks. If writing out a file can be as simple as obj.serialise("mydatafile"), then inventing your own text file format sounds like a serious YAGNI violation.
(This looks like a BIG security threat. If you know in what programming language was created your program, you can upload arbitrary objects to it)
In C, I can define a huge global structure to hold all the my program's prefs (a collection of ints, floats, enums, etc.), then "teleport" them into my program with a single line:
Unfortunately, the C method is not platform-neutral.
(Incredibly, the C method doesn't look as insecure. Still looks somewhat dangerous, though)
From: "Paul Lutus"
Subject: Re: HELP: struct data members alignment???
Date: 07 Jun 1999 00:00:00 GMT
About this subject, and in seemingly endless threads, we get two kinds of
1. Why can't I just write raw structure data to a file or network socket?
Writing it as text or packaging it in a portable, binary form is a lot of
work, and I probably won't be moving the program to a different platform
2. How come my program on platform B cannot read the raw structure data from
The answer to (1) is (2). The answer to (2) is (1).
OK, I have no objection to including a StoryCard
that states: "User can edit preferences with text editor". We can then decide its priority. Perhaps we can start off with the binary version and add the text conversion later. However, there is a class of AcceptanceTest
s that are much easier to write if you start from the text file. So the priority of the story may be increased for engineering reasons -- DaveWhipp
That sounds like you want to do a UserStory
first because it's easier, i.e. has lower risk. WorstThingsFirst
says to do the GUI first if you aren't sure how to do it, and it is as valuable, or more valuable, than the "text editor" story.
But if you're comparing against a language's build-in binary serialization feature, then the text mode becomes the more difficult (and probably more valuable, too). Then, WorstThingsFirst
says to do the text mode before the binary mode. I'm sure we can debate round this circle a few times without coming to any conclusion about which one should really be first. It depends on the project.
Actually, projects are underway to make the conversion from object graphs to XML documents automatic, so that serialization might be done directly to an XML format. An instance of GenericityFirst
, I guess. Not sure that the result of such a process can yield an understandable
XML file, though.
I believe that the real point is not whether the data is stored in plain text, but the availability of tools to change the data.
In general, I feel any data requires a user friendly tool for editing it. It is simply too easy to fat finger data in a text editor or violate arcane integrity rules. Give the user an easy way to get the correct results. That said, however, we must also recognize that the editor tool will probably fall short of perfection. In those cases, it is valuable to have an alternate tool to view and correct the data. Notepad, vi, emacs, etc., provide wonderful fall back tools. Nothing is quite so frustrating to the user and developer alike than to have to say, I'm sorry you will have to wait for the next software release to fix your problem.
In general, provide both a safe data edit mode and an unrestricted edit mode. Use of a plain text data format will allow built in operating system tools to serve as the unrestricted edit tool.
The blog post
"No matter where you put an advanced setting, somebody will tell you that you are an idiot"
seems to be related.