Programming In Wiki

Why shouldn't programming be as easy as wiki?

Easy to navigate, easy to document, easy to share comments, easy to peer review, programming by gradual accretion ... massive re-use through linking to defining WikiWords, can be programmed linearly, chaotically, rapidly.

Why shouldn't an IntegratedDevelopmentEnvironment be like wiki?

Why shouldn't an IDE BE a wiki? (good question. See WikiIde.)

Why is programming harder than sketching algorithms and diagrams in a notebook?

Increasingly the PainOfProgramming comes from managing such rigid development environments. Think of the multiple files we manage, in different languages, in different file systems. The XML configuration files. The SQL scripts to build databases etc.

Now imagine all these different files replaced by WikiPages.

Imagine if we could

• embed all code (Java, Perl, SQL, config data etc) in freeform text pages;
  • could MarkupLanguageNine be used for this? Discussed below.
  • is having multiple languages a GoodThing? Discussed below.
• allow free text annotations,
• allow easy Wiki linking to related pieces of info: from code to config "file", from bug reports to code blocks, code blocks to programmer homepages (with contact details)

At the moment, WE do the hard job of arranging things for the environment. But a make script could easily crawl a wiki picking up various pieces of code, data in order to build source files, config files, etc.

The make script would itself be written on a wiki page for easy reference. It could save the various components in their traditional places before calling the compiler.

---

Are the LiterateProgramming people converging on something parallel to this?

But I want something less structured. Code embedded in wiki rather than comments embedded in code.

LiterateProgramming is supposed to be code embedded in documentation rather than comments embedded in code. Take a look at TexTheProgram. Granted, we're accustomed to BarelyLiterateProgramming (JavaDoc etc.), but that doesn't mean all programming has to be like that.

I wonder if an existing LiterateProgramming tool could be adapted to crawl a wiki. Seems like a good way to handle collaborative projects. There might potentially be a problem with dependencies (look how long it takes pages to get ReFactored here) or in ConcurrentEditing?, but in does seem like the ultimate in Bazaar-style development.

• HyperPerl is an attempt to mix literate programming and wiki.

I love LiterateProgramming. Nonetheless, the biggest problem I have with LiterateProgramming, and the one thing that keeps me from using it beyond toy projects, is the difficulty of keeping the documentation and the code in sync. --SamuelFalvo

---

The first paragraph builds a vision far beyond just mixing code and documentation - it's programming in a collaborative environment: users and programmers changing code fragments or modules whenever needed. The WikiIDE will care for version control and data migration. In my opinion if ProgrammingInWiki becomes real this will change everything we knew about SoftwareDevelopment. CommunityProgrammedWiki could be a first step -- MirkoBlueming and the PfompVision.

---

Sean McGrath? on mixing code and data: http://www.itworld.com/nl/ebiz_ent/03112003/

---

I have taken the first steps down this path for a project which I have all but abandoned (attention deficit :) ). Over on my wiki, I have a utility called WikiExtract. It crawls my wiki and pulls out specially-formatted PHP code. You can look at (but not run) the source code at:

http://www.llamacom.com/~sky/wiki/phpwiki-1.3.3/index.php/WikiExtract (404 - File not found! Try http://web.archive.org/web/20050326181313/http://www.llamacom.com/~sky/wiki/phpwiki-1.3.3/index.php/WikiExtract. -- 20060328)

Note that it's tested only on my wiki, which runs under PhpWiki, and it only writes into my directory. Feel free to grab it and modify, though.

-- PatrickConnors

---

I have my own UnfinishedProject? here (a wiki with code generation): http://www.nooranch.com/synaesmedia/beach/wiki.cgi/CodeGenerationExperiment. (BrokenLink 20070402) -- PhilJones

---

Ned Gulley wrote an interesting article on a closed access wiki-like programming competition, really goes to show the speed benefits of such a system above a standard comparable OSS system; blowing away the (already pretty small) restrictions to editing placed on traditional OSS projects seems to make a huge difference. http://portal.acm.org/ft_gateway.cfm?id=986264&type=pdf&coll=GUIDE&dl=GUIDE&CFID=37969099&CFTOKEN=20840995

You can also find this paper at http://www.starchamber.com/gulley/pubs/tweaking/tweaking.html. Content is merely -> "Describe the new page here"

---

• http://jot.com is a wiki with structured forms and programmable pages. Jot's wiki markup includes search, if, etc.
• IBM's QEDWiki looks in the right direction too : http://files.zend.com/qedwiki/ , http://services.alphaworks.ibm.com/qedwiki/

---

[EditHint: This discussion is something of a gem, and really needs its own page.]

Imagine if we could embed all code (Java, Perl, SQL, config data etc) in freeform text pages [...]

Q: Could MarkupLanguageNine be used for this?

A: Probably, though MarkupLanguageNine possesses a strong ErlangLanguage bias when it comes to values descriptions (in the metadata), and yet lacks strong referential semantics between meta-data components or from the meta-data into the source (creating the barrier discussed below). Something can be provided by the interpreter module, but making it meaningfully common across different interpreter modules would be a chore. Fortunately for ML9 (and, I'm sure, considered at great length in its design), the vast majority of languages don't reserve '@' in column 1 for anything - even those that do leverage the '@' symbol can place it in column 2 without problem. From what I've read, it wouldn't be a bad option as far as options go. That said, we'd need to extend it to support anchors and linking efforts, dependency specifications, etc. and that semantics barrier (especially across Java/Perl/SQL/config data/etc.) is something that needs looking at, as would be providing IDE support that crosses so many different languages in a single wiki (e.g. syntax highlighting, debugging, etc.)

ML9 also lacks, from what I can tell, the ability to nest tags. Maybe I'm missing something? --SamuelFalvo

ML9 provides full Erlang-value support for the meta-data associated with each tag. (@tag {name1=value1,name2=value2,...}). that is, every except the 'data' meta-value, which must be a flat string. This allows for arbitrarily complex data structures via use of tuples and/or lists. The 'data' meta-value is generally loaded from the text between the tags (though it may be loaded explicitly - (@tag {data="a flat string"})) and must be a flat string because it could be a description in any language (including text commentary) and even when describing Erlang code, this text is NOT describing an Erlang value). The author advertises this as being 'completely the opposite of XML' since in XML the attributes are flat while the 'data' section between tags has structure - and I have to agree that, regarding the purpose for which ML9 is intended, this is a good thing.

Multi-Language + Coordinator vs. Single Extensible Language with integrated InternalDomainSpecificLanguages

The scenario above ("imagine if we could embed all code (Java, Perl, SQL, config data etc) in freeform text pages [...]") favors implicitly the use of multiple languages. Ignoring consideration of whether ML9 could accomplish it for a moment (though I don't doubt it could - ML9 was designed for exactly this purpose), the question of whether using multiple independent languages is a GoodThing deserves some extremely critical consideration of its own. Further, the use of a 'coordinator' language, be it ML9 or not, also deserves scrutiny. My own conclusion is that the decision to coordinate multiple independent languages would be a BadThing (for a number of reasons), but that the extensible single-language solution can (if certain provisions are made) be quite elegant yet still provide the vast majority of the power you'd be looking for with multiple languages (and possibly even more). But I'll attempt to present the evidence for you to make your own judgement.

There are several implications in coordinating multiple independent languages.

• First, the set of tools (or plugins) you require for the coordination grows significantly. If we were to have a single 'WikiIde', it would need to support syntax highlighting, compilation, debugging, code-stepping, error reports, etc. for each of the languages. Even if the tools were perfect and possessed no errors, it will be a challenge to coordinate all of them and integrate them usefully with the WikiIde. This challenge grows further if there are multiple versions of languages in use.
• Second, coordinating and, especially, integrating parts written in different languages becomes very difficult. You can't just write an algorithm in one language, reference that algorithm with a WikiWord from another chunk of code, and use it. Instead, the algorithm is written in some independent language (LanguageA) and if you're writing in LanguageB, you must have not only a ForeignFunctionInterface to LanguageA, but the ability to reference the compiled object formed from the chunk of code containing TheAlgorithmInLanguageA in order to access it (which results in non-trivial naming and reference problems - to where does TheAlgorithmInLanguageA get compiled? should we use a WikiWord to reference it? or some semantic UniformResourceIdentifier understood by the underlying OS/filesystem and injected via the coordinator language?) Lazy programmers will violate OnceAndOnlyOnce simply to implement algorithms in each language.
• Third, a coordinator language that also has the task of marking up and formatting text for display must ultimately integrate with the code-descriptions whilst not interfering with the processor of the independent language. This can be provided via some common interpreter-module of the coordinator language, of course, but it is worth considering the direction being taken: just one step beyond providing markup annotations for display of code is starting to provide macro annotations and functions over the code itself. After all, any coordinator language is essentially one massive language preprocessor. One could allow macro-support even in languages that don't support it, using WikiWord macros with parameters as a sort of HyperText MetaProgramming in languages that don't support it naturally. (Even ML9 - @inject {macro="MyWikiWordMacro", param1="embedded-code",param2="more-embedded-code"} - would, with a little switch to the preprocessor module, allow for macros.} However, this lands us one step shy of creating a 'single extensible language', but without access to the semantic context or environment coordination provided by a single language.
• Fourth, consider a goal of writing the various language-tools within the WikiIde. This could be useful for a number of reasons - among them, tools are unlikely to be perfect and ARE likely to evolve over time, so the ability to debug them (or reference them by version) would be useful. In addition, one might wish to reference tools with WikiWords to provide for easier coordination (as opposed to, for example, strings to be identified by a shell). WikiWords could either reference pages that contain code for the tools, or contain code for accessing the tools. E.g. the compiler for Java could be written on a page called 'TheJavaCompiler'. But if the tools themselves are to be written in the WikiIde, we end up with two basic choices: (a) either part of the tool is written in a coordinator language, or (b) the tool is written wholly in independent languages coordinated by the coordinator language (possibly itself - TheJavaCompiler written in Java). The latter results in a bootstrap problem (since every independent language is written in itself or another independent language that must be bootstrapped) - and such a bootstrap problem must be somehow resolved by the coordinator language (or in yet another independent language). The former requires a TuringComplete coordinator language. Either way, one ends up just one step shy of diving into the 'single extensible language' (since one is self-extending via WikiWords referencing languages compiled into the Wiki), except that we went about getting there in the least intuitive and least efficient way possible - instead of automatically gaining the syntax highlighting and debugging and compilation of every sub-language, we must write specialized debuggers and highlighters for every independent language we add.
• Fifth, and in general, there are simply a ton of barriers between languages (including the coordinator language) creating all sorts of AccidentalDifficulty and adding to the PainOfProgramming. There are referential and namespace barriers: one can't simply reference an algorithm written in another language, or even written in the same language on another part of the same WikiIde - not without extending the language and its tools to support WikiWords (which results in a specialized form of the vaunted 'independent' language). There are semantic barriers: one can't expect types, representations of values, objects, etc. to have the same meaning across languages, nor can one expect the coordinator language to have any 'real' idea of what is going on beneath, and a refactoring browser would also require significant per-language plugins. There are communications and debugging barriers; any debugger tool or plugin that must cross a ForeignFunctionInterface into another language must somehow make this transparent to the WikiIde and handoff the debugger to the other specialized debugger plugin, which must support the virtual language-runtime-environment created by the original language (which may itself be running in a virtual language-runtime-environment from yet another 'original' language); also, there is communications in the cross-process sense, but that tends to be easier to handle (since the WikiIde can be aware of both the message source and destination runtime environments). And then there is the lack of cross-language optimization and code-verification; barriers are bad things where automated optimizers and provers are concerned.

Even if we limited everything to one single language plus coordinator, the Wiki coordinator language itself becomes a layered preprocessor for that language with semantics barriers and referential barriers reaching into the core language.

If I were ProgrammingInWiki, what I'd want is a single, common language in which all code AND all commentary and annotations are written. Obviously this imposes some requirements on this language; among them is required support for markup annotations that affect only the display of code and commentary, not the meaning. Another is the need for configuration-builder support - i.e. the linker and set of available services needs to be controlled from within the language. In addition, on a per-configuration basis it would be useful to control all sorts of aspecting (to allow for aspect-oriented programming).

The main benefit is that the WikiIde suddenly gets a single, common processor, a common reference space, common syntax highlighting, the ability to reference any piece of code or commentary from any other piece of code or commentary (especially with anchors), a single debugger, code-stepper, compiler, etc., a single bootstrap, OS independence (no need to modify or intercept calls to the OS to support injected referential URIs back into the Wiki when referencing some pieces of code from others; no need for a panoply of shell tools to support 'makefile' configuration specs, etc.), the ability to reference any algorithm on the WikiIde and know it will work in your piece of code, and the ability to do so without naming a language (as opposed to needing to reference TheAlgorithmInLanguageX, or needing to use complicated references through obtuse ForeignFunctionInterfaces?, or need to perform communications to other processes through pipes or sockets, etc. - all with combined comms, security, type-safety, and optimization costs.)

But this doesn't exclude the possibility for getting your java/erlang/smalltalk/perl/SQL/Prolog/makefile/pseudo-code syntax where you want it. If the core language used for ProgrammingInWiki is an extensible language (extensible syntax and semantics), especially a non-monotonically extensible language (such that parts of the syntax and semantics can be removed from the language from within the language). It wouldn't be bad at all if one simply said (in something similar to ML9) '@JavaLanguage', and had it automatically capture most of the syntax and semantics of JavaLanguage, yet providing the necessary extensions for commentary and markup. The 'JavaLanguage' WikiWord page in the WikiIde would include the basic syntax and semantics transforms and post-processor-wrapper support to get from Java back into the core extensible language, and further provide commentary about the Java language, the reasons behind the syntax and semantics, etc. The syntax highlighting for the code under the '@JavaLanguage' tag could either be performed automatically from the specification of the Java syntax, or could involve a secondary specialized markup-annotations post-processor for Java (or could use the latter if available, and the former if not - the latter allowing for programmers to drop in their own preferences for viewing the WikiIde - e.g. DavidBarbour might have in his special page DavidBarbour:Prefs a line such as 'use WikiView.JavaLanguage.SyntaxHighlighting = AnotherPopularJavaSyntaxHighlighter').

More likely, of course, such a language would discourage the creation of other general purpose languages within it. You don't need JavaLanguage syntax and semantics within the language all that much (though it'd be useful to include when writing a JavaLanguage compiler within the language, since it'd then become a very short process... almost like a MetaCircularEvaluator). Instead, it'd be the mini-languages for various purposes - data descriptions, GUI configurations, communications hookups, etc.

As to how one gets a nice, extensible language? Well, ExtensibleAttributeGrammars atop a TuringComplete core language with nice communications support (e.g. ActorModel based) is a very a viable option, possibly with TypeChecking (be it optional or not), plus need for annotations (for markup, but also useful for any other post-processor of the language - such as optimizers), compile-time executives (to support anything a makefile would need to do), and the ability to modify the post-processor (for aspect-oriented programming, injecting typecheckers, full language redesign based on annotations - e.g. adding unit-tests to a language that was missing it, injecting optimizations, etc.). There is plenty of research in this area regarding LanguageOrientedProgramming. It is likely that AwelonProgrammingLanguage would work in this role (it already has the support for annotating code, including marking it up, built into the language design... along with support for control of the linker, access to services at compile-time, and extension of the language) but there are no promises from the author as to whether that particular language shall ever grow beyond its infancy. Maybe if it does, a WikiIde will be designed for it (as the primary development environment).

And commentary itself needn't be so 'independent' of the code - no binary division of code and commentary. Commentary can just be written in a specialized commentary language (much as you see with the various documentation-readers today, plus support for markup), which creates annotations of the code, which is utilized by the post-processor when it comes time to display said commentary. Post-processors would still be able to investigate the code with which the commentary is associated in order to create interface descriptors and whatnot.

-- DavidBarbour

---

ProgrammingInWiki

• This idea can be actuated, if it is organized and expressed so as to include all requiredDefinitionAndImplementation and if it has reachableValues
• It requires a new way of thinking about how one uses words, by considering hyperWords as having twoFacets, where the name has a descriptiveAndStructuralFacet as well as a proceduralAndActionableFacet
• Incorporates the use of hyperWords as hyperComponents, hyperFunctions, hyperObjects, hyperMethods, hyperContainers and hyperDocumentation (where documentation also includes an ability to add comments, reminders, programmingInformation, developmentSchedules, identificationOfRemainingToDoItems, expressed as hyperWords within or linked from within the SourceCode (making it reachableAndDisplayableFeatureOfTheExecutable
  • indentificationOfIntent
    • for all variables, constants, objects, structs, constructors, includes, etc
    • for visualComponents used in the Gui: guiControls, guiContainers, and relationalDevices and locationalDevices as visualInterfaces
• namingConventions
  • Using hyperWords for:
    • functionalNames
      • functions
      • routines
      • operations
      • processes
      • routings
    • elementIdentifiers
      • controlledElements
      • uncontrolledElements
    • Components
    • Definitions
    • informationSpaces
    • Storage
      • Reference
      • temporarySpaces
      • verifiedSpaces
      • controlledInfoSpace
    • Procedures
      • uncontrolledExternalProcesses
      • controlledVerifiedProcesses
    • addIns
      • hyperInterfaces which facilitate wikiProcProcedures to act as extendedCapabilities

-- DonaldNoyes 20080222.1345.m06

---

See WikiWithProgrammableContent, SoftWiki, XpSystem, SelfProgrammingWiki, HyperPerl, LanguageOrientedProgramming, WikiIde