This is a position statement that frameworks are (in all essential manners) languages. This is intentionally a weaker claim than ApiIsLanguage
. See ApiIsLanguage
RE: "You don't have to learn new syntax or idioms to use an API. APIs aren't language. They're vocabulary."
I've heard this opinion expressed by many different people, but, in general, I disagree.
When the designer of an API has expectations or requirements on how calls to an API are ordered and otherwise synchronized, then the API is not merely a vocabulary; it is a FrameWork
that embodies various protocols and imposes behavioral requirements on the user... sometimes these behavioral requirements are fairly shallow (create comes before delete, don't delete the same thing twice, query before you expect a response, etc.), and sometimes they are very pervasive (special protocols for accessing shared resources to avoid DeadLock
, communications context for security or compression or DeltaIsolation
, contracts with preconditions and postconditions and invariants, temporal and spatial demands to achieve realtime performance or network quality assurance on embedded systems, and so on), but they always exist.
The degree to which an API can be considered a dedicated language is really the degree to which these behavioral concerns are imposed upon the user of the API. Some of these behavioral concerns are derived of EssentialDifficulty
- a necessary evil
in order to achieve properties of the protocol that are not provided directly by the language. Some of the behavioral concerns are derived of AccidentalDifficulty
- an unnecessary evil
that exists wherever the API is dodging around or accommodating limitations of the language, or leaking implementation details.
Examples of EssentialDifficulty
include the need to construct a shared key (e.g. DiffieHellman?
) to participate in secure communications with an unknown remote system, or the need to publish data to a named topic before it can be distributed, or the need to register a service before others can find it. Examples of AccidentalDifficulty
include working around the sequential programming model of a language to add concurrency and other synchronization patterns, taking extra steps to ensure an object won't be accessed in one part of the code before destroying it in another, demands on code composition to support PrevalenceLayer
, requiring the programmers carefully ensure that the call stack leading to a particular API call not be involved in certain mutexes for risk of deadlock, sandboxing of code to eliminate as much ambient authority as possible before running scripts and such provided remotely, BoilerPlateCode
to 'initialize' a library (as opposed to individual uses of a protocol), and so on. AccidentalDifficulty
often requires violating OnceAndOnlyOnce
because you need to implement something 'once' in a manner suitable for ABC and 'once' in an incompatible manner suitable for XYZ and so on. Most complexity we experience today is still AccidentalDifficulty
is very problematic. Protocols, services, and contracts are difficult enough to compose based purely on EssentialDifficulty
. When AccidentalDifficulty
born of various MissingLanguageFeature?
s is added to the mix, well, there is a reason FrameworksConsideredHarmful
is given one very harsh kick to the nads then left crying and whimpering on the floor. So long as you only need the one framework, you'll do well enough, but the framework as a whole is SecondClass
: not abstract, not composable, full of gotchas and idioms that you and any MaintenanceProgrammer
will need to learn and understand, very little obvious from the API itself, and effectively needing its own FooOneOhOneInSevenDaysForDummiesInaNutshellSuperBibleUnleashed
It is in these more extreme cases, where the behavior imposed on the user of the API is pervasive and frustrating, that ApiIsLanguage
is clear and obvious. So what's on the other end of that spectrum? Well, at a guess I would say that the other side is where the API feels like a straightforward extension to the language in which it was written, where each element of the API feels and acts and perhaps even looks
(syntactically) like a simple or primitive extension to the language. SymmetryOfLanguage
isn't (further) violated: the result is just as FirstClass
, abstract, and composable as everything else in the language (which, for some languages, might not be a very impressive claim). Users don't need to know anything about the API that they wouldn't also need to know about other language primitives. Etc.
I suspect that, when dealing with APIs at the FrameWork
and pervasive DesignPatterns
end of the API spectrum, use of TypefulProgramming
, and other MetaProgramming
mechanisms to guide proper expression of the API is highly justifiable. I don't have numbers, but I suspect both original developers and MaintenanceProgrammer
s will have an easier time learning, reading, writing, and verifying the resulting 'language' after such extension efforts. Further, with the TypefulProgramming
approach, or with FirstClass RealMacros
(as supported in Scheme), the IDE and compiler can get in on the action by detecting errors statically, providing syntax highlighting for the extended language, allowing one to drill down into expanded views of macros when debugging, etc. (DrScheme
provides these mechanisms.)
Still, I do understand the folks who resist such efforts, who want to believe "You don't have to learn new syntax or idioms to use an API. APIs aren't language, they're vocabulary"
. It would be nice to believe that, no matter what the problem, it can be solved via an API that tosses a few new vocabulary words at it without any unusual requirements being imposed on the user.
If we ever have a language where 100% of frameworks and protocols and unanticipated DomainValue
s and entirely new DomainModel
s and ad-hoc MetaModel
s with their associated inference engines and MultiValuedLogic
s can all be expressed without any unusual
behavioral impositions or gotchas relative to the rest of the language - and we achieve this without LanguageIdiomClutter
and with acceptable EconomyOfExpression
(such that it isn't "usual" to jump through hoops) - then maybe
people will stop writing APIs that impose behavioral requirements on the user.
But it isn't possible. Even if we captured all the AccidentalDifficulty
imposed by existing language designs for all known problems, there would still be both EssentialDifficulty
of known problems and AccidentalDifficulty
of the unknown problems. The theoretical best any LanguageDesigner
with intentions towards MinimalDesign
can do is eliminate individual classes of difficulty (one at a time, in an effort with rising combinatorial complexity) then provide the language with a clear path for upgrade while supporting forward compatibility of existing programs. Useful ExtensibleProgrammingLanguage
mechanisms include: some form of extensible syntax, ability to annotate code (HotComments
) and modify the PostProcessor?
to operate over them, support for TypefulProgramming
, a standard HomoiconicLanguage
representation of the AST for reflective operation by said PostProcessor?
, low-level DependencyInjection
at the code layer (related to HyperSpace
s and AspectOrientedProgramming
), and requiring a clear statement at the top of each page of code describing the exact version of syntax in use.
Once these ExtensibleProgrammingLanguage
mechanisms are in place, they act as a buffer allowing extensions to the language without upgrade to it, and allowing certain classes of upgrades to the language without breaking existing programs or requiring pervasive changes to existing code (since the same ExtensibleProgrammingLanguage
mechanisms can be used to downgrade a syntax to an earlier version). Further, such systems obviate the need for many classes of language upgrade, especially those related to DomainModelling
. If one wants an InformLanguage
style declaration of FirstClass InteractiveSceneGraph
s for a VirtualWorldInterconnect?
, one writes up the appropriate extensions to the set of communications services, adds some new syntax for describing and composing SceneGraph
s and the available interactions, and provides the defaults and assumptions in order to hide semantic noise. (That said, as with all LanguageDesign
work, it isn't "trivial" to provide composability, flexibility, static verification, avoid boilerplate, etc. - there are huge differences between well designed language extensions and poorly designed ones just as there are huge differences between well designed APIs and poorly designed ones.)
support reduces the need to perform 'actual' language upgrades to features that either must be pervasive to be useful (such as DistributedTransaction
s, subscription to mutable state, automatic distribution of code, ObjectCapabilityModel
, failover redundancy, system-wide GarbageCollection
, or end-to-end ProcessAccounting
policy via ExplicitManagementOfImplicitContext
in a distributed system, etc.) and various forms of expression that cannot be optimized through composition of other primitives (such as supporting 'sets' with their disordered communication rather than just 'lists').
Actually, a greater problem than upgrading a language with new features is upgrading it with LiberatingConstraint
s. I would suggest that any LanguageDesigner
consider those at time of BigDesignUpFront
because there is no way to add them after the language becomes popular (that is a penalty of premature success; related is AlanKayOnMessaging
, compile-time confinement, removal of AmbientAuthority?
, some classes of static TypeSafety
, guarantees of Termination (of the sort sought by TotalFunctionalProgramming
), shunning of mutable state (KillMutableState
) and shunning of names (NominativeAndStructuralTyping
) to support distribution, purity in the lower layers, and much more beside all fall into the whole 'LiberatingConstraint
Well, it seems I've wandered onto subjects that interest me but aren't entirely topical... so I'll stop there.
The point is that ApiIsLanguage
for almost any API that solves a difficult problem. Arguing that one "needs to learn new syntax" may be true but is nonetheless not
(in general) a point against use of an extensible syntax mechanism... because, for APIs, one (in general) "needs to learn the gotchas, idioms, DesignPatterns
, and protocol, that the RealMacros
can guide you in avoiding or utilizing respectively."
Those who argue, "oh, this new language feature eliminates the need for macros or extensible syntax,"
, should bear in mind that the new feature likely just moves
the need for macros a bit further towards the vast language frontier, allowing one to deprecate and transition out some language extensions, and perhaps optimize or improve composition of existing language extensions.