Frameworks Should Automate Not Hide

On this wiki, somebody said "Frameworks should automate the repetitious parts of app development, not substitute for understanding of underlying technologies" (paraphrased). Others and I felt it was a great statement. In other words, they should not be a replacement for knowledge of the technologies on which the app relies. [I cannot find the original quote so far. Still looking. I found something similar in WizardsAreDangerous.]

The main reason is that most frameworks are a LeakyAbstraction. If you use them beyond the trivial, you will usually either expose the leaks, or create bugs or performance issues tied to abstraction leaks (not to be confused with memory leaks). If you don't know what's underneath the framework or what the framework is actually doing, you will have difficultly working around or fixing the leaks; becoming a victim of them.

In practice, this is hard to avoid. A new-comer may not understand a GUI toolkit or SQL, for example, and the framework helps them get up and going. However, a more experienced mentor perhaps should keep an eye on them if possible.

While I agree with the premise you've borrowed - that frameworks shouldn't substitute for understanding of underlying technologies (which is necessary to understand the fundamental limits of the framework, and understand what can (fundamentally) be done (or be done efficiently)) - I disagree with the conclusion you've made from that premise.

First, Frameworks aren't 'LeakyAbstractions' unless you poke holes through them so you CAN access the underlying technologies components, which I believe is a very large source of bugs (especially those related to premature optimizations) but also performance issues (since the ability to make optimizations is ALWAYS based on the set of assumptions you can make, and you can't make assumptions about what is happening when some idiot is poking holes to bypass the abstractions you've provided). Sure, the poking holes to provide your 'leaky abstraction' allows for better hand-written optimizations, but it is harmful to the overall system.

Poking holes in frameworks is like Haskell allowing assembler code and pointer and registry manipulation arbitrarily because Haskell is too high an abstraction - doing so would break both Haskell and a good chunk of its optimization assumptions (such as referential transparency). Or like a Relational DBMS giving access to 'pointers' to the underlying representations of tables and indexes so you can muck around in them. Even if you simply note that a particular implementation of, say, inner-joins is unusually fast (or slow), depending on that fact (or working around it) is quite far from a brilliant idea. It's a hack, and, as anyone with an axe to grind will tell you, hacking long enough at any sort of framework will invariably break it.

If a particular framework needs some enhancements to efficiently support certain operations, then the proper solution is add a higher-level expression of intent to the framework, not to poke holes that leak the underlying technologies. (Or, in the case where something in the framework is broken: fixing it, not working around it, is the RightThing.)

Once the higher-level expression of intent is available, you can have your framework or system compile-down (from this higher abstraction) to an efficient implementation of that intent - and thus remain as near efficient as is theoretically possible. Having the higher level expression still isn't an excuse for not understanding why the framework has the limits it does, which is of critical importance when determining what sort of things can be usefully added to the framework.

Second, Frameworks should hide details that are to be encapsulated - not only for the reasons above, but also because encapsulation has its own benefits. Encapsulation, fundamentally, allows change - that which you can hide, you can change... and that which you wish to change (or for which you anticipate a need), you MUST hide. 'Porting' a framework from one system to another is one form of 'change'. 'Optimizing' part of a framework is another sort of change. 'Integrating' new technologies as they arise represent yet another sort of change. Anything the Framework 'hides' is something for which it becomes portable - to different CPUs, different machines, different technologies, enhancements to existing technologies, etc. - and, really, allowing meaningful communication between different systems (which requires portability) really is a huge part of most frameworks' utility. Anything the Framework 'hides' is also something it can optimize, since there are no external dependencies upon that which is hidden. I am convinced that 'performance issues related to abstraction leaks' most often come from failure to hide things that aren't already 'perfect' and can be further optimized (which is most things). When you have enough of these, you need to pretty much scrap the whole framework (at least politically) because fixing things will break the code of everyone to whom you've 'exposed' the naughty details of your framework.

FrameworksMustBothHideAndAutomate?. If they don't, they'll be failures as frameworks in the long run. Only compile-time automated optimizers should be allowed to break encapsulation between frameworks, abstraction-layers, and components, and even then: only those parts of the framework that aren't supposed to be runtime-pluggable components. Optimizers get a special pass because they don't introduce source dependencies: when something is changed, it calls only for a simple recompile of all statically linked components that must also inherit the change. As automated optimizations get stronger (and that's a whole field in itself) higher-level and layered abstractions will become more and more efficient, and poking holes in the framework will become more and more obviously a mistake (if the Haskell and SQL examples aren't obvious enough).

Hacks can still be used when you're stuck with a weak framework that you lack the (political, technical, etc.) power to fix. But they should be recognized as hacks, and certainly oughtn't be on the right-hand-side of a 'should' or 'ought'.

Indeed. In a sense, one can consider a relational DBMS to be a "Framework" that hides the nasty business of dealing with files, disk blocks, BTrees and other gnarly structures, buffers, concurrency mechanisms et al. Or, a compiler is a "Framework" that hides having to muscle out machine code. I suspect the topmost (pun intended) conclusion on this page owes to having endured poorly-engineered frameworks -- especially "application frameworks" -- which are all too common. It is not necessarily the case with good frameworks, few though they may be.

Remember that a library is something that provides proper flow of control, while a framework relies on InversionOfControl. Be careful of what you mean, precisely, when you say "framework." SQL and compilers do not use inversion of control, and are therefore not frameworks. GUI frameworks, server frameworks, application frameworks, etc. all depend on inversion of control, and are appropriately named. --SamuelFalvo?

True, which is why I used the phrase "in a sense" and put quotes around "Framework". My point was about the value of hiding, using illustrations in which (I would hope) the original author would instantly recognise the value of hiding, rather than strict terminological accuracy. Perhaps I should have said that good frameworks hide details in a positive manner, as do DBMSes, compilers, etc.

The above objection to FrameworksShouldAutomateNotHide is independent of the InversionOfControl issue and applies regardless. It depends only upon the fact that intent to particular effect must somehow be expressed (usually by a programmer) to the framework as part of working with the framework. You might be delivering specifications regarding when and how and under which conditions you wish to be 'called back' in the traditional "framework" InversionOfControl manner, or you might be delivering specifications for what you wish to happen in the framework immediately in the traditional Imperative manner; it doesn't affect the above objection.

The whole idea of 'InversionOfControl' or "proper flow of control" being some sort of mathematically precise statement is entirely bogus anyway. If you attempt to make it precise, InversionOfControl fails when encountering the 'DataAndCodeAreTheSameThing' issue. Delivering information to be utilized in a 'callback' is fundamentally the same thing as delivering any other sort of data to be utilized in any other sort of call. As such, 'InversionOfControl' is a fuzzy abstraction and describes at best a fuzzy set of systems - those closest to this 'idea' of sending messages in expectation of receiving 'callbacks' 'later' rather than 'sooner'.

Arguing that IoC is illusory is obviously not going to hold water to whoever wrote this page, and positively doesn't hold water with me. My mind is already made up on this issue, after dealing with IoC for some 20 years now. IoC has its place, but as the author of this page clearly states, it must not be a substitute for learnability.

And I'm quite positive you're wrong. Further, you're making it quite clear that you're uninterested in truth: your mind is "already made up on this issue" and 'you hate them, you hate them, you hate them' - I can't really expect any sort of rational approach to this from you. And whether IoC is illusory or not is an issue of complete irrelevance to the above objection regarding 'FrameworksShouldAutomateNotHide' (which happens to apply whether IoC is there or not). So it doesn't need to hold water with whoever wrote this page, and it was rather pointless of you to bring it up in the first place without first identifying a dependency on what you call 'proper flow of control'. And, sure, a framework shouldn't substitute for understanding... even I agree with that.

When you go through school, you learn the basics first, then you learn the shortcuts/higher-level tricks of, well, whatever. When it comes to automotive repair, you start out doing the simple things (oil changes) before working on the complex (engine rebuilds). When learning to use a framework, why should it be any different?

FalseAnalogy. When it comes to using automobiles, you start out doing the simple things (like driving) before working on the complex (Tokyo drift). When learning to use a framework, why should it be any different? And 'automotive repair' is a horrible analogy - you're not 'repairing' the framework, and thus you don't need to know its internals. 'Understanding' of how your car works, though, would be of great importance in figuring out the limits you can push it to when using it.

Moreover, why should any framework go to such lengths to hide its operations? Information hiding and abstractions are good things, but sometimes, sometimes, you really need to know what's going on, and/or how to overcome an inherent problem.

Encapsulation. Not framework specific. Change. Not framework specific. Optimization. Not framework specific. Portability. Certainly not framework specific. - I'll agree: these are "Not framework specific." These issues just happen to apply to a whole variety of computation systems in which one goes about expressing intent to action or effect, including but not limited to libraries, frameworks, AND language design. Logically, these apply to frameworks and thus constitute a proper and correct answer as to "why should any framework go to such lengths to hide its operations". That it happens to be the same as the answer to "why should any computation component go to such lengths to hide its operations" makes it no less a valid answer.

Did you not read the above objection to 'FrameworksShouldAutomateNotHide' - you know? the one with several paragraphs explaining exactly "why should any framework go to such lengths to hide its operations?" Or did you just see a page on what could make frameworks 'good' and start ranting?

I read the whole page. And, apparently, you actively chose not to understand very much of it.

I think we must agree to disagree on this issue, and just let it go.

Perhaps we shall.

Re: since one cannot make any precisely meaningful statements about what separates "framework" from "library"

A library is a collection of lightly-related or unrelated utilities. A framework is a series of utilities meant to be used together. Library utilities stand on their own more or less, while framework utilities would mostly seem out-of-place by themselves. Although, I agree there are things that straddle both definitions.

This should be obvious, but if the most precise statement you can make to distinguish "framework" from "library" is that they sit on either side of a vague, poorly defined line between 'loosely related' and 'meant to be used together',and that sometimes they straddle this line... well... I think you can see where I'm going.

Anyhow, I'd disagree: even individual libraries are components that can, should, and generally do adhere to some degree to the principles of CouplingAndCohesion - i.e. the components (the values, the procedures, the structures) are meant to work together. Language standard libraries aren't very cohesive only because they carry anything and everything the language-designer believes a programmer should have available when staring at a blank page. But most other libraries I've seen are at least moderately cohesive (well... at least those without a bunch of cruft that has built up and never been removed over time).

Frameworks tend to assume somewhat arbitrary conventions and data structures that are used to communicate and coordinate between the items. If you took out a framework item and stuck it in a library without the context of the rest of the framework, then it would like somebody made up a goofy interface for the hell of it.

As far as a precise definition to clearly separate the two concepts, I doubt one exists. Framework-ness is a continuous concept. The more the components depend on each other and assume a certain "culture" for doing things, the more "frameworky" they are.

See also: HelpersInsteadOfWrappers

See also: ProgrammingWithoutRamDiskDichotomy (contains a related discussion e.g. whether transparent persistence should also abstract control of the storage)
CategoryAbstraction, CategoryFramework

View edit of February 28, 2008 or FindPage with title or text search