Challenge Six Versus Fp Discussion

This is an attempt to provide a coded example that FunctionalProgramming (FP) proponents can use to show FP simplifying and/or shortening code outside of the systems software domain and beyond mere foobar examples. Although there are lots of different potential metrics to use, this particulate topic focuses primarily on CodeSize?.

http://www.geocities.com/tablizer/chal06.htm (NoteAboutGeocities)

That link includes:


Summary of arguments (as I view them, you are welcome to present an alternative summary)

FP: FP significantly reduces code size

Top: I would like to see an example

FP: See, it can simplify your Example 6 loops.

Top: I could re-write them to simplify the loops without FP.

FP: But that does not properly close the database handles/pointers on error.

Top {Issue 1}: However, the challenge was to "significantly reduce code-size", based on an FP'er claim. You are moving the goal-posts. "Significantly less code" was never demonstrated for the given example (#6).

Top: {Issue 2}: I could reduce or eliminate the need to leave handles open by practicing SeparateIoFromCalculation such that the code-size difference is negligible.

FP: But SeparateIoFromCalculation has limitations...

Top: (Discussion then spins off to various topics about pros and cons of result sets versus cursors. I agree that result sets have limits in some cases, but they are rare enough I claim to not affect total code size much. The vast majority of loops are not affected by the drawbacks of result sets.)

The HaskellLanguage forces separation of I/O and calculation because it is not good style to put calculation in monads except when you must.

It's not really about SeparateIoFromCalculation, it's about pluggable architecture. It's about "How can you create a function that, logic used deep down in the function calls, can be customized at the site of function call". You can not have this as easily without FP

I would like to explore a specific need. That was the point of using challenge #6. If you have another fairly complete or complete example that illustrates such, I would be happy to take a look. But, I am skeptical of claims of heavy need for such without seeing for myself. Some of your coding styles seem so different from mine that the frequency may not be an issue if my style is assumed. (see PageAnchor: coding_thumb_rules, for example.) -- top

....

FP: FP better isolates variables to specific blocks

Top: I tend to agree, but 1) Problems caused by such "loose" variables are fairly rare and 2) it may be a language-specific thing that can be "fixed" without FP.

....

FP: Example is not a good demonstration because....

(I am fuzzy on exactly why it is being rejected.)

....

Top: My conclusion thus far: Demonstrated FP advantages shown so far are quite minor such that adding complexity to a language/design is not really justified, or breaks roughly even at best. I still see no slam-dunk evidence that justifies the FP bragging I have encountered. If you personally like it, I can live with that, but suggestions that it revolutionizes ALL software are off the mark so far.

-- top

Why does adds complexity to a language more than ?

Well, this is sort of off-topic, but it seems you want an infinitely tweakable language. I might like that if I was the only reader and writer, but other people would just reinvent their language to fit their own head and consistency would go out the window. Every shop would have a self-rolled language more or less. If one could demonstrate significant improvements using such an approach, I might be more open to it.

By the way, I am not for try/catch blocks. SeparateIoFromCalculation reduces the need for that also. Some of those others I might cross out also. Basic list handling can be implemented via string libraries alone, no new syntax, and bigger stuff would often be best in a database.

  list="1,2,3,4,5,6";
  while(x = getNextInList(list)) {
	print(x);
  }
This can be implemented by simply chopping off one item per iteration in the getNextInList function (pop).

I cannot believe you actually show that code. It's, IMHO, ugly... The list cannot contains object of other type (string, file, socket). One will never use that as a replacement for list. Did you think even a bit before showing this example? Or did you just want to put up something?

As for infinitely tweakable language, What's the different between allowing one to reinvent the language which fit their own head, or the problem domains, and disallowing it? What's the different between this two library?

 re = createRegexp();
 g1 = createMatchManyNotRange('0','9');
 re.addGroup(g1);
 g2 = createMatchManyOfRange('0','9');
 re.addGroup(g2);
 re.match("hello 1234");
 if(re.matchGroup(g1)){
	// do 1...
 }
 if(re.matchGroup(g2)){
	// do 2...
 }

and

 regexp-do [
  /([^0-9]*)/ { do 1... },
  /([0-9])*/  { do 2... } ]
I think the point that allowing one to build a language for their library is moot. Because, after all, when you are using a library, you are really using their new language, the API IS the invented language. The only different is that, in language that does not allow FP. the invented language looks ugly, you see kludge that shows how the new language doesn't get treated as first class language construct. It's like if you have to draw a picture, one system only let you write description of picture, while another just let you draw the picture. What's the different?

I am not sure what your example is trying to illustrate. There are many ways to clean up reg-ex syntax/API's without using FP. We are wondering away from challenge #6 here anyhow. If you wish to start a topic such as FpMakesRegexBetter?, be my guest.

I'm not saying there is no way to clean up Regexp API without using FP. I'm saying that there's NO WAY to clean up Regexp API without reinvent custom language (the regexp part). Just like there is no way to Clean up Database query without having invented SQL language. It shows that there is no benefit of disallowing one to reinvent languages for specific domains. Because, after all, API call is also an invented language; the different is just that it's invented on the language that discourage inventing another sublanguage; That's why it's uglier than another.

Move Some to ApiIsLanguage

Languages such as Tcl are pretty good at allowing/making sublanges without explicit FP, I would note.

Yes, by using eval. However, we already talk about how using HOF is more suitable than Eval in DynamicStringsVsFunctional.

I have not seem decent real-world/realistic examples there.


Moved from TopMind. Maybe move it back when it is cooked, but can we barrow a topic for a few months to rant around with? Thanks.

10/2004: Under HowCanSomethingBeSuperGreatWithoutProducingExternalEvidence and DynamicStringsVsFunctional, they kept claiming higher-order-functions or FP can produce significantly less code, but could not prove it using Challenge #6 (see link below).

Lies.

{Damn! My evilness has been exposed. Oh the shame!}

Nor, did they clearly state why their techniques could not shorten #6.

More lies.

They made some vague insult about it having SQL in it, but refused to clarify. All talk and no walk. I am holding you to your claims, guys. No wiggling allowed on my watch. You simply failed to deliver less code for #6, and without a good explanation.

You were shown many times, you just didn't understand it.

I don't have to "understand it" to know whether it is significantly less code or not. A quick eyeball scan will tell that. I made a big deal about the claim because such is something that has an easy-to-see metric rather than just another LaynesLaw battle. This time you can't pull the old "you would see it was better if you were smart like me" escapist trick.

It may later turn into a battle over how to score code size, but you have not presented anything to compare yet. Since you claimed "significant", there should not be much incremental bickering about the code size metrics. I suppose you could also offer fake code to trick me, but the challenge is to produce signif. smaller runnable code, not to fake me out.

Can I rightly claim that you walked away from the challenge?

No, but you can rightly claim ignorance.

[ Indeed, he can. TopMind was shown the use of higher order functions to make new control structures many, many times. He was shown how it could shorten loop-oriented code, how to extract regions of code out and make them into HigherOrderFunctions that handled common tasks, and how to even use common HigherOrderFunction tools to make domain sublanguages.

Top's ultimate response was that, in the face of for loops, case statements and the IteratorPattern, all of these things were rendered pointless. Even when his approach was dutifully extended to show it was several times larger than the counter-examples using HigherOrderFunctions, he still claimed they were not practical or didn't offer a net code savings.

I don't understand this. Please clarify. If you mean the "weather example", I have insufficient info about the requirements to compare. Challenge #6 is better documented and I supplied runnable code. You did not. Runnability is a fact, not opinion. Further, I offered to answer ANY question about #6, but the reciprocal for the weather example was not offered. If you want to use the weather example, then please do it right and finish it.

Quite interesting to notice how all the code samples that proved you wrong, have somehow disappeared from that page.

Apologies, I had the pages confused. Yes, ArrayDeletionExample pretty clearly proves the less code claims.

[It's amazing how carrying an axe with the intent of grinding it can distort your vision.]

Yes. It made me completely not see your Great Short Version of #6. It is totally invisible to me, as if it never existed. FP fanatics are not exactly mainstream either. So what's with this "axe" talk? Glass houses.


Top, you like to point out challenge #6, but whenever we point out challenges, you seem to take delight in claiming that is "not your domain." What I don't understand is why this works for you (in that you can claim TOP is superior, say it is only for certain domains, then act like it's better for all of them) but not for anyone else?

How about this. You're the one with something to prove, TopMind.

No! YOU claimed your techniques result in significantly less code.

Table Oriented Programming is a simple technique, really a footnote in the collective consciousness of programmers. You use it when it makes sense to use. You're claiming it always makes sense.

Did not. I don't claim my favorite techniques are optimized for all domains. Do you?

So why not tackle http://www.frank-buss.de/challenge/index.html which SlashDot recently posted. See if you can shorten it. You're competing with PascalBourguignon?'s version.

You up to it?

Why not have him compete against Dirk Thierbach's 6-line HaskellLanguage version, or Konstantin Metlov's 3-character JayLanguage version?

Sure. Compete against any of the good ones. As long as the feature set is comparable, it's fine.

Challenge #6 existed first. Stop changing the challenges until you find one that breaks TOP or is outside my scope of familiarity. Can't FP improve anything beyond university math-lab toys? If FP cannot reduce 6's code size, then I consider your claim null and void*. Now code or erode!

[*] Unless you come up with a detailed justification for why your FP tricks don't work on that particular example. If FP has limits, lets find and document them. There is no reason to be afraid of such, is there? Besides, a language tuned for lab toys may be completely different than one tuned for real business projects.


<rant> Stop arguing about FP, TOP, I now present you the new language and challenge, my language is called writing CRUD-SCREEN-REPORT-001 language. It's CRUD screen for Employee and department oriented language. It has only one line of code you can do with it. The code is

 run().
This code start up the database server running 24*7, with full logging, and administration for The company name taken from Machine environment variable.

Other flexibility is out of my language's scope.

Now this is my challenge, you TOP mind please do coding that is shorter than this in your language? Or else your claim of TOP is null and void*. And don't challenge my language and paradigm by other algorithm, 'cause that is out of my language's scope.!!! </rant>

Is there a point to this? Hints of DomainPissingMatch.

Sure there is, I'm tired of seeing top, saying "No, that part of the problem is not in my application domain." He tried to say HOF has no benefit over his TOP oriented, but when someone showed how HOF is used to achieve simplicity in some problem top can not do, he would just said that it was not in his domain. He is not trying to learn anything. There may be smug lisp weeny, But even those guy knows about what ever programming practice you are talking about, Top is just not so. Me as a CRUD-SCREEN-REPORT-001 programmer, I would just said Top things are useless because, I do all I want in just one line of code and I don't wanna learn anything new.

May be TopMind is not Top at all.

The "significantly less code" (SLC) claim did not exclude CrudScreen apps. If you want to revise your claim, be my guest. Something about #6 seems to get under your skin such that you change the subject. It is hard for me not to conclude that you are afraid of something in #6. The original SLC claims were written with such certitude. That certitude is quickly draining away, like a fast-talking street racer who is finally pressured to race his car instead of just talk about it.

Why does anybody keep giving this crank airtime? Seriously, just go read his pages. Rant, rant, rant. OO is evil. OO proponents are misguided. FP people are ivory tower theorists. No-one recognizes my genius, because there's a conspiracy by OO tool developers to market their stuff. No, really, read this crap. He goes on and on about requirements, but can't write coherent requirements documentation for this or any of at least five other challenges. Most of the pages don't even describe what the challenge is.

Content outside of the "challenge" page is a separate issue. Somebody claimed higher-order-functions or FP can produce significantly less code on this wiki and I will hold them up to that claim. My rants and their claims are orthogonal issues. One side being an alleged rant-head does not dismiss the other side from providing empirical evidence of their claims. Note that others may be interested to see the FP claims backed with code despite what the "ranthead" says about it. The results live outside the ranthead.

Re: "Most of the pages don't even describe what the challenge is"

Unfortunately, those who initiated the challenge are the ones responsible for providing one. What you are reading is the response, not the initiator. But, in the case of #6, there is a reference implementation to use as the "requirements".

Questions can be asked and answered below.


I don't understand this library size issue that keeps popping up. Will functional need less libraries? Is it easier to make libraries with functional? Given the same or similar libraries, will the functional program be smaller? If there are too many variables (factors) to compare, then how does one know if a functional version will be significantly smaller? The fault in the alternative may be the design or quantity of the libraries, not the paradigm. It almost sounds like the functional side is implying that if one has good enough libraries, one does not need functional to reduce the code size or code complexity. Good libraries seem to be the GoldenHammer factor affecting code size according to this argument and not functional. It is the allegedly great or thorough or domain-specific libraries that made the code example small enough that functional cannot reduce the size by much. It seems the reader is being sold on libraries, not functional.


The problem with what TopMind is asking is that it largely misses the point. He's basically asking us to take one tool from our toolbox and use it to attack a small application without considering other tools. Using AnonymousFunctions and the FunctorPattern? in your code is a very significant win in some cases (when you work with collections), but were I to attack Challenge #6, I would use many more patterns. Certainly, AnonymousFunctions iterating across collections would be one of them, but...

You are allowed to use as many FP features as you want.

I think TopMind is trying to compare Apples and Oranges here, and that's why our examples have fallen on deaf ears. TopMind considers TableOrientedProgramming in its broadest sense, whereas the one point of FP we argued on (acting upon collections) is but one of the tools of the larger FunctionalProgramming picture.

While it should be easy to beat the line count of the given example (you don't need anything fancy to beat Challenge #6 there, the design is... very conservative) I believe it wouldn't prove anything to TopMind. Until he learns FunctionalProgramming and groks it, the advantages of such a system will be lost on him.

As explained in HowCanSomethingBeSuperGreatWithoutProducingExternalEvidence, I reject the idea that something can be significantly better without being able to give specifics. The "if you just knew it as well as I did, you would feel the benefits" is absurd. The fault is your articulation skills in that case, NOT my understanding. Everything objective is demonstratable in smaller examples. I agree that smaller examples sometimes are misleading when applied to larger ones, but at least they can demonstrate patterns to be found in larger stuff. We may later debate about the frequency of those patterns compared to real stuff, but at least the patterns (lower case) themselves are demonstrated so that we can then move to the frequency issues.

There are tons of external evidence to support the merits of FunctionalProgramming paradigm, you just pretend not to see them or refuse to look for them. Don't confuse your ignoramus status with the lack of evidence. See IcfpProgrammingContext? for example, let's see if using your TableOrientedProgramming you can even attempt to solve any of the examples out there, much less to compete with FP hackers out there. If you complain that those challenges are too elaborate for you to try, I have simpler challenges for you like EnumeratingRegularLanguages, or any algorithmic problem. Take the recent one proposed by somebody here, like SicpIterationExercise. Even as the last one is very amenable to TableOrientedProgramming, a Haskell or ML solution will still be much more elegant, but let us see your FoxPro solution first.

I presented a challenge first. But, I will take a look (when it is ready).

What was your challenge? If I'm not mistaken it was no coding challenge, it was a request for examples where FP is the best approach, so I pointed you a bunch of them, and I can point to you tons more, so please tone down your handwaving in the meantime.

I moved the link to the top.

Given the link you posted, this page should be deleted as it has no value? What's the purpose to compare challenge six against FP? None whatsoever as far as I can tell. FP is not designed to substitute SQL, nor is it intended to solve all problems int he world.

Can it help the parts that are non-SQL? If not, I would like to know why. That might tell us something about the nature of FP and where it helps and where it does not help. We may all learn something from such an answer. The original "significantly less code" claims did not have caveats for specific domains.

If an environment (be it FoxPro or DelPhi or VisualBasic or whatever) already offers you precanned components to interact with a SQL database, run reports, etc, then that's the deal. FP was not designed to solve all challenges in the world. It's like discussing that differential geometry is not adequate to solve diophantine equations (in case you wonder the two have nothing in common).

The "precanned" stuff is mostly just library API's. You can assume or make the same API's for your example if you want. If API's are the secret to competing with FP, then why sell FP? The "magic" is in powerful API's, not FP under that logic. I am thus confused about your API complaints. Regarding your diophantine analogy, if FP's benefits are limited to certain domains or techniques, let's get it out in the open. I would probably solve most problems I encounter for any domain with TOP techniques. Performance, lack of tools, or management paradigm mandates are the only thing that would stop me. If TOP is a flop for other domains outside of performance or microsecond timing issues, I would like to know about that also. (AreRdbmsSlow)

So bottom line is: you try to mix unrelated things in the same bag. You challenge six cannot be used for any discussion whatsoever about FP, and all your comments related to FP on the basis of this challenge are to be discarded. If you are honestly curious about what FP is good for, try to learn first, and handwave less.

I want to know why they are allegedly unrelated.

Because nobody targeted FP either in general or in a particular implementation language to address your particular domain of interest (boring business applications).

See DomainPissingMatch and AreBusinessAppsBoring.

Instead FP shines at solving mathematical and algorithmic problems, while producing high quality (low bugs, low maintenance code). FP environments in general don't have precanned libraries and WhatYouSeeIsWhatYouGet IDEs to make them compete with VisualBasic, BorlandDelphi and others in slapping together GUI screens and database queries.

Um, that is a web app. HTML, basic HTTP/CGI-like libraries, and a little JavaScript. There are some GUI screen shots, but they are mostly just to show the ControlTable content. I purposely stuck with a web interface there to avoid a GUI library war. (BeenThereDoneThat and I agree with you that it makes comparing difficult.) BTW, what is an example of an "algorithmic" problem?

[Actually, there're several decent libraries available for FP languages, eg. WASH for HTML/CGI, Fudgets for GUI, and HaskellDb for database access. Unfortunately, most of these are student/hobbyist projects, which means they pay lip service to conveniences like easy installers. I spent 2-3 days trying to get Haskell and WASH installed, and I haven't even tried HaskellDb because of the long list of dependencies. Preliminary investigations are encouraging though; the equivalent of RptList?.asp was half as long (10 lines vs. 19). I expect bigger savings with the main program, as I can utilize HaskellDb's composability of queries and Haskell's PatternMatching features. -- JonathanTang]

Although it could be done in principle (and some disparate efforts proved that high quality components), no big company invested the necessary money in making FP a success story for this domain.

So one should not use FP for biz apps???

So your challenge 6 is completely futile and borderline trolling: nobody pretended FP is the optimal solution for the domain, there's no reason why FP ought to be the optimal solution for that, nor the fact that FP currently isn't the optimal solution for that domain detracts in any way from the value of FP.

Again, the original claimers seemed pretty adamant and intense.

Therefore the very title ChallengeSixVersusFpDiscussion is borderline idiotic. To drive the message home I can make ChallengeSeven?: write a good sonnet about wiki. And then create a page ChallengeSevenVersusChicagoManualOfStyle?, and then let's get it out in the open that ChicagoManualOfStyle does not help you write a good sonnet. And then let's criticize the well-respected writing guideline for the reason that it doesn't cover writing sonnets.

That would be the same thing as you do here.

I am still not getting your analogies. Biz apps are not outside mainstream. In fact, I expect offshoring to hit physics-oriented stuff harder than biz because Newton's laws are the same in Bangalore, but business culture is not.

Not quite. The thing is, physics is already heavily offshored, because Americans never had a monopoly on smart physicists to begin with. The bulk of physics discoveries have come from other countries, and then their discoverers have moved to America to make more money afterwards. As a result, physics is already a pretty egalitarian playing field, where you need to be smart to do it in the first place, but it doesn't matter whether you're a smart American or smart Indian or smart Chinese or smart European.

We can probably expect biz apps to tend towards the physics situation as what you know becomes more important than where you're from. Sucks for the LessAbleProgrammers in the U.S, but it's great for all the computer whizzes in India. And you could argue that this is how it should be. -- JonathanTang

Perhaps the offshoring stuff should be moved to a different topic.


You can prove this by looking in ArrayDeletionExample. We took his code examples, extended them out, and then showed how a FunctionalProgramming approach could reduce code count and increase code clarity. It even allowed us to give arbitrary verbs to collections (he immediately noticed we use database verbs frequently). But, despite the simple comparisons present on that page, TopMind doesn't get it yet. This is most likely because he can't envision and entire program designed around such principles (which is hard to do unless you've tried it), so let's not get too angry at him.

I don't know why you guys keep defending ArrayDeletionExample. I have read it multiple times (and clarified some old points in it), and don't see any magic. I reworked some of the code to be more competitive size-wise, I would note. Nothing is more than a few percent smaller in the end. You have not shown any slam-dunk size reduction. The weather example was admitted to be limited by a Non-disclosure Agreement. Thus, key questions about why it has such odd design constraints cannot be answered; and this makes it useless for detailed comparisons and analysis.

Top, you need to understand what we're talking about. FunctionalProgramming is a separate but related paradigm to TableOrientedProgramming. It's very hard to see the forest when all you're doing is focusing on one tree.

Sometimes I think you FP'ers are fixing the wrong forest. Why do you keep implying the problem is me instead of just show obvious improvements? I don't get some of you guys. You're so evasive yet make such strong claims. Are your claimed benefits objective or subjective? Make up your mind.

You make up your mind, Top. You're currently full of BS clutching at straws (i.e. irrelevant petty examples), when you have been directed what it is you need to look at. Let me repeat: IcfpProgrammingContest, and algorithmic problems like EnumeratingRegularLanguages, or if you whine that those are too complex you can try even something as simple as SicpIterationExercise. Put your code where your mouth is or just shut up for a while, and stop polluting wiki with repetitive ramblings. You cannot whine continuously that nothing has been shown to you, as long as it has been shown and you haven't done your homework.

My challenge was first. I turned on the TV, so why do you get to keep changing channels? Besides, those are university lab toys. I want to see something that helps for biz apps, not university lab projects.

Your challenge was first to be irrelevant as it has been explained to you.

Your explanation was unsatisfactory. They don't depend heavily on libraries. You are flat wrong about that.

The other are real challenges, as a proof of that anybody who knows an if then else can write your "biz apps",

DomainPissingMatch talk and ArgumentFromIntimidation. If they were so simple than FP should knock them out of the ball park during the first swing.

IcfpProgrammingContest challenges are for GrandMasterProgrammers only. Your exclamation "university lab projects" sounds very much like the grapes are sour'. Why don't you show us that you can solve at least the simplest one so that we can appreciate your mastery of the ArtOfComputerProgramming.

The sour grapes are on your side. My challenge was first, Bucko! Further, most of those appear to be contests where the programs compete with each other rather than fitting requirements. In other words, more like making a program to play chess rather than make a chess game. That makes it too difficult to compare. We want to study and compare code here, not output.

Whether biz apps are boring or "easy" is irrelevant. Can FP significantly reduce the coding size (or whatever objective metric you propose) for them or not? Just answer the frippen question with biz code. Stories about where "real programmers" hang out is off topic and smells of social intimidation. I don't challenge FP for making chess playing software, that is just not the issue.


On the size reductions of Top's new code in ArrayDeletionExample, specifically Mandy and MandyTwo?, TopMind totally missed the point of what the code was doing. I think part of the reason he isn't seeing how cool HigherOrderFunctions are is because he's not thinking in both runtime and compile-time terms.

If you look at his MandyTwo? code, what he did was parrot a HigherOrderFunction example with a pair of named functions, offering none of the flexibility. This suggests to me Top doesn't get how you can dynamically alter behavior with HOFs, trivially. It also suggests to me he thinks of applications in a very web-application-oriented sense. You write something, and it's done. People don't really extend it, and if they do they don't touch old code so much as make entirely new regions. This mimics the development of websites.

If he could see how a program uses HOFs to control behavior by managing code like data, placing HOFs at will all over the place, then he might be more interested.

This can be used even in software that traditionally centers around tables and doesn't care much for speed. For example, check out this procedure that you might find in a weblog (in a pythonish, rubyish, phpish pseudolanguage):

  define output_entries( blog_entries, output_type=Outputs::HTML )
	for each entry in blog_entries do
	if( output_type == Outputs::RSS )
	output << RSSEntry( entry )
	elsif( Output.type == Outputs::HTML )
	output << HTMLEntry( entry )
	else
	output << TextEntry?( entry )
And if someone wanted to add new types, they'd have to extend this further. This is exactly the approach that TopMind said he espoused, and you can see on ArrayDeletionExample. This is basically a glorified case statement. It's annoying, hard to extend, and prone to breakage. Let's rewrite it with a HigherOrderFunction in mind.

  define output_entries( blog_entries, output_func )
	for each entry in blog_entries do
	output << output_func( entry )
Now, if we were to add a few HOFs, like the following, we won't see a massive lines-of-code savings right away:

  output_as_html = { |entry| return HTMLEntry( entry ) } # HOF that returns text of HTMLEntry on its argument
  output_as_rss  = { |entry| return RSSEntry( entry ) }  # HOF that returns text of RSSEntry on its argument
  output_as_text = { |entry| return TextEntry?( entry ) }  # HOF that returns text of TextEntry? on its argument
Or we might see it used inline, as is popular in Ruby and Lisp:
  output_entries( blog_entries, { |entry| return HTMLEntry( entry ) } )
We can invoke output entries with whatever we want. We've saved a little code, but not too much. Within 5%, as TopMind says. But, what happens when something changes? For example, it turns out our users are reporting that every so often, bogus entries are coming up in their blogs. Maybe our master database queries are broken somehow? But we can't easily reproduce this problem on our test server. So what we need to do is put a log on the live system and hope we catch the bug in action.

Well, with the original case approach, this would suck. We'd have to go through each entry in the case statement and add the same code over and over again. Cut'n'paste code at its worst, and clearly it's a pain to handle. But if we had the higher-order-function version, it's simple and painless and reliable!

 # Assume that the script parsed the URL above, and set "desired_output" as one of 
 # our output_as_* functors.

output_entries( blog_entries, { |entry| debug << DebugInfo?( entry ) ; return desired_output( entry ) }
See? That's much less code. Using HigherOrderFunctions makes your code more flexible. Imagine if only some of the blog sites were breaking. Since you might have thousands of users, you might want to be able to dynamically enable this logging to save on disk space and I/O bandwidth until they report the problem. Again, this would be easy with higher order functions, but much harder with the case-like statement. You'd have to add if-statements into every if statement, massively increasing the size of the code.

Flexible code leads to long term savings. Even though it wasn't immediately obvious, the design paid off when we had to modify our code. This kind of savings often comes when you least expect it. It's why when I wrote my Weather app (I'm the guy who talked about it), I used them. This is not to say they're a GoldenHammer. It's undeniable that they don't save much code in the short term, and that they are often more complex (especially in languages that don't support them well, like C++). But even if the code savings is minimal, the ease with which someone familiar with the pattern can extend the program shouldn't be taken lightly.

You might argue that "Eval" could mimic this behavior, and you'd be partially right. Certainly, eval does something very similar to what HigherOrderFunctions can do. But you're using Eval at the wrong time. Eval is a very slow and powerful tool. Really, it's meant only for use when you have no other option. HigherOrderFunctions have strengths ideal for this, and they can encapsulate eval calls at a later date, if you feel you need them. Claiming that you'd forego HOFs here for Eval is like saying you can't be bothered to take the parking break off your car, it's so much more convenient to just drive with it on, who cares if it ruins your gas mileage and break, right? Cars are getting better all the time.

I hope you can see were we've been going with this discussion now, Top.

Reply

First of all, why are you assuming that each output "type" is mutually exclusive? (Be very careful using the word "type" around me.) Is it not possible that multiple output "types" may be needed per request? You talk about making stuff be change-friendly, and to be change-friendly we cannot (overly) hard-wire the assumption of mutual-exclusiveness into the design.

A CollectionOrientedProgramming viewpoint would be to have the output formats (code snippets?) in one table/collection, and the input in another, and then join them as needed. Going from a one-to-many relationship to a many-to-many relationship does not require a change to the original output format table, only the introduction of an intermediate many-to-many table.

But, let's stick to a case-statement comparison right now.

Second, you claimed, "We'd have to go through each entry in the case statement and add the same code over and over again." I don't see why this would be the case. Let's assume a statement like this:

  function outputMessage(msgRecord, outputFormats) {
	result = "";
	initialization_stuff...;
	if (listContains(outputFormats,'html')) {
	result .= do_html_stuff();
	} else if (listContains(outputFormats,'rss')) {
	result .= do_rss_stuff();
	} else if (listContains(outputFormats,'text')) {
	result .= do_text_stuff();
	}
	post_processing_stuff...;
	return(result);
  }
We can put the tracing at the beginning (near "initialization_stuff") or at the end (near "post_processing_stuff"). I see no reason to put it under each of the output format blocks unless it is somehow specific to that format, in which case you have the same issue to contend with. Either it is sharable code or specific. Specific goes inside the decision blocks, general goes outside.

I also assumed orthogonality here (an easy change from a case list) by using a potentially multi-value list and also just appended the results to a string. I don't know what would be done in practice because I don't have the full requirements. If they need to be separated into different files or records, that does not change the general strategy here.

	general_pre	// common to all
	block A  // stuff specific to A
	pre_A
	process_A
	post_A
	block B  // stuff specific to B
	pre_B
	process_B
	post_B
	block C // stuff specific to C
	etc...
	end blocks
	general_post
If we follow SeparateIoFromCalculation, then we can intercept the results (or pre-sults) outside of the format-specific blocks.

As far as your weather example is concerned, like I already said, your claimed limitations are very suspicious. (See "perfect storm" comment under ArrayDeletionExample.)

Further, perhaps this discussion goes under ArrayDeletionExample instead of here. However, that topic is getting too long.

-- top

SwitchStatementsSmell contains somewhat-related discussions about case statements.


Top: If you factor out all the "template filling in" stuff, how many lines are left actually doing real work? It seems to me that this problem may be difficult to reduce because it's essence is trivial once all of the template code is ignored. HelloWorld usually has a "smallest correct value" in any given language that no amount of cleverness can reduce. If challenge six gets most of its bulk from the HTML and SQL code that is being used essentially as a template, the rest may not be reducible because it doesn't do much. In short, in your attempt to come up with a non-trivial example, maybe you just made a long trivial example.

I do not know what you mean. You may find the example boring, but that is irrelevant. This is not an entertainment contest. What are you calling "template code", and how does that differ from non-template code? I always consider template code as something that you copy-and-paste and then fill it for specific circumstances. This is not that (although it could be used that way if one does not want to extend it directly, but so can anything).

I mean code to fill in the blanks in SQL and HTML statements. To take a trivial example:
  print "Hello " + name + "."
Replace "Hello " and "." with three page long fairly non-redundant strings, and you will have a six page program that just can't be shortened (ignoring exotic things like data compression).

Maybe that is because SQL and to a lesser extent HTML are doing most of the "work". I don't see that as a bad thing. It is a problem-solving approach: use high-level tools to do the work for you. If it is not something that FP can simplify, then we have learned something and the "signif less code" claimers should put caveats into their claims.

I am looking for techniques to simplify my domain, not university homework assignments that find the shortest path to all combinations of pi, etc. I chose that example because it is a bit closer to my domain than most of those offered. (But still not a perfect example. A better example would probably have more seemingly arbitrary business rules tossed in.) If you have a biz-oriented example you wish to present, that would be great also.

Since neither HTML nor SQL are particular functional, that doesn't strike me as much of a concession. I'm not a big FP guy, but I'm happy to take a crack at it. How's this for a caveat: "Functional techniques are of limited use when you are forced to use a language that does not support them, such as HTML or SQL."

Are you suggesting you can create signif shorter code for the same thing if you did not use SQL? Be my guest to try. (I don't think SQL is the ideal relationallanguage, but it is generally compact for what it does.)

Not exactly. Although I think it may be possible, I suspect that challenge six is just too trivial to matter. Let me try to translate the idea into "structured programming" terms. Imagine that someone is trying to claim that subroutines can reduce code size. Imagine that someone else says "Prove that subroutines can shorten code by showing how it can make the following significantly shorter":
  for(i = 0; i < 100; i++)
	printf("Hello, world!\n");
  for(i = 0; i < 100; i++)
	printf("Wheeee!\n");
The program just doesn't do enough for subroutines to be of any use in shortening it. Subroutines, recursion, data structures, garbage collection... they just won't help. However, they are all capable of vastly reducing code size in more complicated programs. Does that make sense?

No. I would like to see an example from a "complex" business application. The trick is to not let something get too complex by dividing it into individual tasks. The large-scale communication is via the database, not so much algorithms. More about this in ProceduralMethodologies. And, I do heavily use subroutines in #6. Anyhow, as far as simplifying the above:

  repeatPrint("Hello world", 100);
  repeatPrint("Wheeee!", 99);
But let me get this strait. You are agreeing that FP won't help much with challenge #6 code-size-wise. Correct? Any other PF fans disagree? -- top
I disagree. And I'm working on proving you wrong. Unfortunately, you didn't even see fit to include an SQL schema with challenge 6. Urgh. -- JonathanTang

The schema is shown as screen-shots near the bottom of the page, and there is a link to the data dictionary page which shows example data. The "reports" table only shows the column headings, but the types are easily guessed. The "reportID" and "rptSequence" columns are integer, the rest are character/text. -- top

Yeah, my complaint is more that I have to take data_dictionary.txt and manually convert it from comma-delimited to SQL, which is a huge pain. I tried mdbtools on the Access DB, but mdbtools's makefile barfed on my machine for lack of cpp. Oh well. Just expect it to take a bit longer. Also, your link to Jim's Ruby version is broken, and I'd very much like to see it, as I can read Ruby easier than ASP/VBScript. -- JonathanTang

I indeed wish there were an open-source MS-Access-like tool for quicky DB RAD stuff. Jim's server croaked and he never got around to recreating it IIRC. But he never implemented most of it anyhow, only created a rough framework.


Apparent reasons given for not producing an FP equivalent (you are welcome to increment the vote tags):


The example is boring, unchallenging

I don't see how that is relevant. To some going to the moon is boring. All those space-ship parts to keep track of? Why bother.

and badly specified. It takes quite an effort even to understand what it is that is required,

I think if you study the screen-shots and some of the code, it should be fairly obvious. If you have an NT box, run it and play with it a bit. I will answer specific questions.

and nothing valuable seems to entice the reader to solve such a grunt work exercise.

You seem to be looking for MentalMasturbation instead of solving semi-realistic problems. If it's grunt work, then use your brilliance to automate it. That is how a real developer should approach "grunt work".

Plus the "solution" as presented suffers from too bloody obvious SqlInjectionAttack? which means that if the originator didn't put good enough effort to produce some credible code from where others can learn something, why should anybody else bother?

I don't know what you mean. I cannot help it if you are an SQL-hater.

 Select ... where criteria=''; Delete * from users; ....

These defects of the challenge have nothing to do with functional vs OO vs visual basic or whatever, but it's just not enough of a challenge. Opposing it to FP as if such a challenge can prove anything one way or the other about the merits or demerits of FunctionalProgramming is completely misguided.

The only valid criticism is unclear documentation. I will answer specific questions if you first bother to study what is available.


[On the contrary, I'm finding just installing the software to be enough of a challenge. 3 days and counting just to get GHC, PostgresQl??, WASH, c2hs, and Dbconnect installed. Ugh.

Any idea why ld is giving me "cannot find -lpq" when I try to link? libpq.so.3 is in /usr/lib and I've got a -L/usr/lib option specified. -- JonathanTang]

Like I told you, unnecessary wasted effort. What do you hope to learn in the end? That Haskell is entirely inadequate to tackle database driven web apps. I could have told you long time ago. How much time do you think SimonPeytonJones spends on integration with ODBC issues? You just hope that a bunch of developers out there programming a couple of afternoons now and then are going to offer you a software stack comparable with that which is the results of many man years dedicated to the comparable .NET or Java solutions? The only sane solutions to such difficulties is to move over. Life's just too short to be wasted on lost causes.

It is wishful thinking to think you'll get lucky with Haskell. After you'll fix the libraries, you'll discover that WASH will not hold to any kind of stress test, or that you'll have problems adding SSL, and so on, so forth which would make the idea to use such a stack for any kind of production software completely ludicrous.

Relax. It is a toy example. It just has to run for a few local tests. He's not planning on taking on eBay. I would think that there is something more nimble than PostGre? though. -- top

On the contrary, I expect to find that Haskell is quite good at CrudScreens. I'm looking for at least a factor-of-2 improvement in line count; that'd be about 200 lines max. I was talking to DougMerritt a couple of days ago and he also didn't see any reason why it wouldn't work. He also mentioned that a friend of his had read ArrayDeletionExample, tried a couple of the techniques in a custom biz app for a Fortune-1000 company, and found them quite helpful.

SimonPeytonJones doesn't spend his time writing ODBC drivers, but that doesn't mean other people don't. And the Haskell community is small enough and elitist enough that they often turn out very nice designs. I'm quite impressed by WASH and HaskellDb, for example, even if I'm using Dbconnect instead of HaskellDb in the end.

I'm using PostGres? because it's the only supported DB for Dbconnect. My options, living in the open-source Linux world, are PostGres? and MySql. Dbconnect (which comes with WASH) doesn't work with MySql, so I'd be forced to use HaskellDb or some other database library. HaskellDb has its own dependency stack (for example, it requires HSQL) and I really didn't want to go through dependency hell again.

Also, my own language design efforts have recently shifted towards the web/database apps world (as that's where I think existing solutions suck the most), so this is good practice with various database webapp solutions. I'm already familiar with MySql; add PostGres? and I've got both major OpenSource databases covered. And WASH is state-of-the-art as far as web frameworks go. I want experience with both before I try and write something better. -- JonathanTang


Top... just one question. Did you have a hand in designing Challenge #6 here?

Yes, why? The general concept as a comparison was brought up by John Urberg IIRC.

That explains it. The article raises some objections to OO that I've heard you raise, and I was curious if I was right in reading your hand into it.

It's pretty typical stuff for top. Demand vaguely, then rant and rave when people fail to deliver according to some exacting specification that's super-secret. When presented with answers, he selectively filters out anything he doesn't want to hear.

Damn, you make me sound so evil. What specifically have I "filtered" out?

He hasn't even managed to write a coherent specification for this "Challenge Six".

All flames aside, I do agree that Challenge #6 is underspecified as it stands. Every time I consider trying to do it, I have to scroll up and down repeated trying to match requirements, terminology and implied data from screenshots. A clear, concise summary would be better.

If you have to scroll back and forth to the same areas, then open up two browser sessions. Or, print them out on paper and spread them on the coffee table. I often do that when studying poorly-documented systems. I will try to provide some more details {since added below}.

Or even better, maybe we should have a short programming contest. Challenge #6 is boring, but that doesn't mean we couldn't make a slightly more fun project and then everyone could submit and judge results.

I don't know why you guys find it boring. CrudScreen frameworks are generally interesting in my opinion. They are not trivial because of all the potential options and variations they can have. It is tough to make a truly generic one that handles all possible business requirements and it still be strait-forward. But it makes for an interesting challenge to search for the best mix of such competing factors. (This example is not meant to cover all possible combinations and features by any stretch. It is pretty much an illustrative experiment.) -- top


PageAnchor: more_details

More Details

Let's take a look at this screen-shot:

It is using report number 2 (selected from the simple RptList?.asp report listing). The listing (output) at the top is based on a query created in part using the "Reports" table. If you look at the "selectClause" column in the "Reports" table screen-shot (see original page), you will see an asterisk. This means the SQL Select clause selected every column in the table. In this case the data table has 6 columns. The database table is determined by "fromClause" column in the Reports table.

The top part shows the results and the bottom part shows the user's criteria because the criteria is repeated (echoed) to the result page so that the user can modify their already-entered criteria if they want. This is why both are on the same page.

I'll come back to the results later. For now let's focus on the bottom part, the criteria input form. This is generated using the "reportCriteria" DataDictionary, illustrated in part below:

These criteria fields come from the rows that have "reportRef" equal to 2. See the second column. (The column name is partly truncated to make room, but all the columns are shown in full in the schema on the original page.) You can see that the rows with reportRef = 2 match closely with the above screen-shot. This is a very typical DataDictionary. "Field-dictionary" would probably be a more fitting name in this case because these do not necessarily correspond one-to-one with actual database columns.

The only real oddity in this set is the price range (lowest price to highest price). Our data dictionary does not have a native "range" type, so we used two fields that refer to the same database column (listPrice). When somebody puts a value in a criteria field, by default for numbers it generates an SQL sub-string such as:

  and X = 999
This assumes the database column is 'X' and the user entered '999'. It basically just uses the WhereAndAnd technique. The "fmtType" column determines whether it has quotes around it or not. And, for strings the default is a "LIKE" operator.

But for ranges we don't want an equal comparison. Thus, the "Comparer" column has ">=" for the first range part, and "<=" for the second. If the user types in "5" and "20" for the range, both "listPrice" rows will generate two strings:

  1. " and listPrice >= 5"
  2. " and listPrice <= 20"
These are then internally concatenated together as part of the final SQL statement. (A fancier version could perhaps use an SQL "BETWEEN" clause instead.) Here is the snippet of code that performs most of this process:

	'---Comparer
	if isblank(rs("comparer")) then
	useComparer = " = "
	if fmtType = "T" then
	  useComparer = " LIKE "
	end if
	else
	useComparer = space(1) & rs("comparer") & space(1)
	end if
It is basically just constructing "AND" sub-clauses for the final SQL, as described in QueryByExample and WhereAndAnd.

The two other data dictionary columns that play a significant role in the range approach is the "keepWithPrior" column and the "sequence" column. If "keepWithPrior" is set to "true", then the input field stays on the same line. This is just a formatting nicety issue. What is "prior" is defined by the values in the "sequence" column.

The "in stock" option is format type "Y", which produces a pull-down list of "Yes", "No", and "(either)" as the choices. One can also use the "theList" column to enter a comma-delimited list of options. This particular example does not use that feature. A fancier version perhaps could separate the list description and the list value. (Note that I called the column "theList" instead of "List" to avoid overlapping with reserved words. I don't know if it would be an issue in this language or DB, but I generally avoid column names that risk such overlaps.) -- top

This is still completely unhelpful. You're explaining "how" and not "what". A good spec states the business requirements, and only the business requirements. It should have no code (unless one of the business requirements is that managers can write code for it), and it should lay out everything that a user can do with the system. -- JonathanTang

{That is not true. Only a little bit talks about code. I am mostly describing the UI and how it relates to the control tables (which I suppose is a kind of UI for app admin people). You don't have to use control tables if you don't want. You can use EssExpressions if you prefer. I won't complain if it is anything declarative (non-executable) and fairly easy to edit. Personally I would rather use a table editor for entering such info, but if you dig EssExpressions for whatever reason, go for it. If you want it in requirements format, then try this: "Must provide a way for an experienced operator or administrator to add and edit report configuration information without having to know programming." -- top}

Top, it is true. The document is awfully light on requirements and awfully heavy on how something should be done. We're well aware we don't have to use control tables. What we're confused about is the exact specs of this boring system. Tell us what we need to store, what we need to be able to query upon, the limits of those queries, the specifics of the preview function, and all the details in between. You keep telling us the how like it matters or gives us insight. Like I said before, I had to keep referring to the pictures just to figure out what data would be operated on. That's not acceptable as a requirement professionally or personally.

The "how" is especially useless when you're asking people to do the "what" in a better way. Could this be a case of GoldenHammerTintedGlasses? :)

Ask questions. I added a section below for that with an example. And please stop mentioning that you find it a boring problem. Unless you can tie boredom to being less practical as an example, there is no reason to keep bringing it up. The people who sign our paychecks are not necessarily in business to entertain us. If it bores you that much, just admit FP defeat and move on. If you can find an "exciting" somewhat practical and typical biz example, you are welcome to present it as an alternative. -- top

The problem is that you don't sign any paycheck, and as long as you offer it as a challenge, you have to assume the responsibility to make it interesting. Otherwise you have no reason to complain that people criticize it as boring and waste of time.

  Boring != Waste_of_Time
Or to be more precise:

  Equiv(Boring, Waste_of_Time) != True
You have failed to find a better example for biz apps, so you are stuck with mine. One thing I have learned from the biz world is that you better have a ready alternative before criticizing the existing state.

Let me break down the math for you, Top:
 (Boring * No Money) ^ Trying to educate someone who rarely listens + Student likes to twist examples so he can ignore them 
	= Waste of Time
We're doing this for you, not because we have some deep-seated desire to implement mind-numbingly boring CrudScreen apps. This kind of work is why we have software interns. Me, I'd like to write something that, you know... does something, besides tell a database what to keep.

Do it because you want the geek bragging rights of using FP to kick a skeptic's ass in the biz domain.

Sounds like sour grapes from somebody who is stumped to deliver their claim. As far as "rarely listens", I don't want words. They only lead to LaynesLaw battles. I want to see the claims in code, not moving lips. By the way, Phd's in India are only $2 an hour on the world market. Who needs interns? -- top

Ahem, I'm a software intern, and the work I did this year is much more interesting than this. This reminds me of the stuff I did freshman year (I didn't go back to that company, BTW), or what I the stuff I still do for free for friends. Except, I'm not a freshman, and we're not that friendly. ;-) BTW, PhD's - whether in India or the U.S. - are a lot more than $2 an hour. There're plenty of good programmers in India. But the good ones get paid as much as the good ones in the U.S. -- JonathanTang

I already proved my claim. See the above equation to see my feelings. We showed you examples, you turned them down. I'm not an FP golden hammer addict. I have lots of tools in my toolbox, including most of yours. And no, Top. There are no PhDs? from respected universities making $2/hr for software in India (barring freak examples). This kind of business app has no challenges. It's just like stamping out license plates. It's good for people who can't handle more than mindless repetition. I might do this kind of work to get a product out the door (because I'm being paid, and someone has to do it) or because I'm the FNG of the team, but not because I'd want to. It's like taking pride in making cookies in an EZ-bake oven. Sure it's great, I guess, if you're someone who finds that kind of thing challenging.

{You are confusing personal entertainment with use. I too would rather be doing AI research, but there is no money in that except for the lucky few. If CrudScreen stuff were as repetitious as you say it is, then it would be easy to automate and you could put hundreds of thousands of programmers out of business. As explained in DomainPissingMatch, biz apps have plenty of complexity.}

This application has no use. While some people may make an honest living doing such work (using tools which help automate it and expedite development, of course), they receive a paycheck for the work. We will not.


I am not sure what you (realistically) want then. I just want to see an FP size demonstration with code that is applicable to my domain. That is all the hell I am asking for.

Here's what I don't get, Top. If you really want to see it, then why not try it yourself? It's not a matter of time, because if you had just started coding you'd have spent a hell of a lot less time than arguing with us for hours on end. Pick a language that has something similar to HigherOrderFunctions and go to it (either build your own, or download and understand some of the example code that exists). You can pick all kinds of different languages for this. At the very worst, you don't like it but you understand it much better. At best, you like it and you have another tool to use. Both options are wins for a software developer.

There is plenty of other more promising stuff to study. I don't see enough initial value in FP to pursue that path right now. I just wanted some realistic evidence from those who claimed "significantly less code". I thought that one would be easier to demonstrate since it is more easy to measure than the vaguer claims such as "cleaner code", etc. -- top

No. What you thought was that you could get one of us to do the groundwork for you. Top, I don't care what you study, or what other "more promising things" you study. You do what you like. But stop saying, "You can't prove it." We don't have to prove anything to you, Top.

There are others who also think FP fans are mostly full of hot air. (Implying that there is a wider audience for proofs.)

When we asked you to prove that TableOrientedProgramming was superior to ObjectOrientedProgramming (as you claim), you didn't respond with proof. You responded with talk. Now you demand code from us.

I don't claim it to be objectively better (unless perhaps consistency can be measured). Just not objectively worse. You made an objectively-measurable claim and I am simply holding you to it. Are you afraid of the RealityPolice?? Here's my proof, and the way I decide what to study next. I see something as I read around online that is not just interesting, but was used to make an amazing product. I study how that product was made. There are tons of bang-up awesome things written in FunctionalProgrammingLanguages and in ObjectOrientedProgrammingLanguages. These is absolutely nothing interesting out there written by you.

When you get to define "interesting". Top, maybe we should take a poll on the wiki. But even worse, there is nothing out there by you at all! For all we know, you've never written more than tiny demo apps. Get someone else to do your homework, Top. I can tell you that I don't have time for your kind of BS without somehow eroding my position and claims. This is not college, Top. Eventually you stop getting examples from people and start getting scenarios and patterns. It's up to you to make the examples that will satisfy you.

I don't understand Top's attitude either...

It's not my or anyone else on C2's responsibility to educate you, Top. I gain nothing from showing you FP. Heck, if you really cared enough to learn it, I could end up gaining a competitor, which would be a net loss for me.

I learned Lisp because a couple folks (PaulGraham, PeterNorvig) made a couple tens of millions of dollars with it.

Bill Gates made much much more off of assembler (DOS) and Visual Basic (written in C). Do you really want to measure by money made? Hell, COBOL would probably win, followed C.

That seemed like a good reason to me, though perhaps I have different values than others. I learned Haskell because PaulGraham mentioned this whole BlubParadox thing, and I realized that his and other SmugLispWeenies' comments about Haskell made Lisp seem very Blubbish. It also helped that GuySteele, who literally wrote the book on Lisp, is quite a fan of Haskell.

I'm interested in this challenge because I think actually programming a real application in Haskell will make me a better programmer. I'm interested in putting the results up because I think some lurker (probably not TopMind, I've basically given up on him) might find them interesting. I would never have learned about advanced programming languages had others not put what they know up on the Internet, free for the taking. I figure here I can give back in the same way.

But I've been spending the bulk of my time wrestling with PostGres? and libreadline and the fact that Mandrake can't put together a decent RPM to save their life, which is not educational for me at all. And I doubt anyone here cares what a Haskell webapp looks like. So if you really think that you have better things to learn, and no anonymous lurker speaks up, I'll go do more productive things instead of responding to this page. -- JonathanTang


And like I said, we're not doing this because we stand to gain anything. We're doing it because we're trying to explain something very simple to you and you're stubbornly saying, "No no no."

Nope, it is "Show me, show me, show me". That is not the same as "no". I am from the MentalStateOfMissouri. Why are you painting me as evil or stubborn just because I won't take your word for it???

I won't make any DomainPissingMatch claims, and I have not done so to-date. I leave that to the Haskell and Lisp gurus. Sometimes, a functional approach to programming can save huge amounts of code (or give other favorable results, like ease-of-extension or maintainability), like in my weather example. Likewise, sometimes it does not. It is the same with OOP, COP, and LOP.

I can't see the real code and/or requirements for the weather example. I provided runnable code and offered to answer any questions. You have not reciprocated. I think you are possibly exaggerating with the weather example. I don't want to have to take anybody's word for it. I want to see the code land on the figurative Whitehouse lawn.

If you're really going to maintain that this is not true, then this discussion is clearly over, because it's literally an argument between someone who refuses to use anything but a hammer (And your excuse about your job could be, "But all I have to do is hit nails.") and someone with a toolbelt saying, "Look, when you work with a screw, a hammer really sucks!"

I suggest we try to form an application idea and hold a little code-off. It might even be fun. We could implement something like blogging software and see who would come out on top. As long as we will actually use in real life, and keep the scope small, we might actually get some real valuable information (and some working pieces of software, besides).

Like maybe implementing TopsQueryLanguage :-)

I think it's time to concede. Most functional languages cannot implement CRUD screens with less code than FoxPro, a tool designed for writing CRUD screens. Top, go take a victory lap. Then go write a thermal flow simulator in FoxPro.

It sounds like a thinly disguised ParadigmPissingMatch insult, but I will take it as a victory. Actually, FoxPro was/is not very good at CrudScreen GUI's. I like it for its nimble table handling and CollectionOrientedProgramming features, not for it's GUI's. And, maybe FoxPro does indeed suck at thermal flow simulators. No tool is best for everything. I am curious, though: Do engineers write flow simulators from scratch? It seems like a problem that is standard enough that existing tools or libraries would already exist for it. -- top


Q & A


(Moved from ObjectiveEvidenceAgainstTopDiscussion)

It doesn't matter if SQL is implemented by Higher Order Function; The SQL IS language WITH limited form of Higher Order Function. Your SQL query statement is so compact and comprehensive because: SQL is actually like this (#' denotes function that can be passed to another function):

 filter-for-interested-column( #'select-clause, remove-if-not(#'where-clause, join-table("table1", "table2",...)))

you would not have this SQL power if SQL didn't have HOF in its language. If SQL is like C/C++ you will be doing this in SQL:
 //hypothesis Non-HOF SQL syntax
 join_result = join("table1", "table2",...)
 filtered_result = []
 for( record in join_result){ //can't have where-clause because I cannot pass a function into another function
	if(is_valid_by_where_clause(record)){
	filtered_result << record;
	}
 }
 result = []
 for( record in filtered_result){ //can't have select-clause because I cannot pass a function into another function
	result << 
 }
You enjoy so much of this expressive power in Data Retrieval Domain Because of HOF. Yet you said HOF is not much useful in your domain, YOU ARE USING IT YOURSELF ALL THE TIME AND YOU CAN'T LIVE WITHOUT IT. Now, In challenge #6 you raise, NO FP language is going to reduce the query code to be smaller than SQL; Because FP with HOF and SQL solutions WILL BE THE SAME, It comes down to as much the same size. But NO language without HOF, is going to be at the same size with FP or SQL. That's why we said HOF is so useful, after all you don't do just select/insert/update/delete. We don't other operation too!! See after you have query all the data out to series of row, with your procedural language, you have to come back to for loop again. But No, FP'ers don't have to do that. They can still enjoy then same expressiveness as in SQL with whatever the operation they are going to use!! They can pass criteria, filtering to everything, not only SQL engine!!! That's where the code of FP language will start to shrink from Your Procedural parts. But, there are so small things to do in that challenge after the data is extracted, people didn't bother to accept the challenge because it wouldn't be anything left to challenge about (You use HOF feature to reduce code size already, and you are challenging FP to use HOF to reduce the code size again; how is that possible?). (Don't bring eval into here, that's another topic).

Why don't you want the same level of expressiveness as in your query? You are happy that you have SQL which itself is HOF enabled language, yet you are all against HOF? What is that?

This may be a definition issue about what HigherOrderFunctions really are. Your definition seems to be loose enough that the "eval()" techniques I mention may also be considered HOF. Your conjecture is an interesting approach, but I still would like to see an actual example of it reducing code in a biz app. I selected challenge #6 because it is far more representative than the other examples given (not perfect rep in absolute terms, though). But, then you suggest that something about it makes it not very reducible. Let's call it Factor R. What we need to find out is whether this Factor R is because it is closer to a real biz app, or something else altogether. I have found an example of something that is not very reducible by your techniques it seems, so it makes an interesting exploration tool for FP relevancy, would you agree? It could be that SQL already absorbs the primary benefits. SQL and Eval together seem to cover so much of the territory that FP would otherwise cover, that there is not a lot left over for "direct" FP to help with. -- top

SQL and eval together *are* FP. Especially when you're storing operations in tables.

Again, I think that is a stretch of the definition of FP. Somebody else called expressions in tables OOP. I see it as DataAndCodeAreTheSameThing. My ideal is exposing the run-time engine (AdvantagesOfExposingRunTimeEngine) so that it does not matter, each is just a view of the same thing. I suppose what one calls it depends upon their favored EverythingIsa view. Eval does not really care what is passed to it, it just evaluates it. It could be a function, expression, etc. Whatever they are called, I find them useful. The real issue is whether going to a more "native" or full-blown FP will make a significant difference.

I find it hard to show the benefit of HOF to you because you don't even yet know the different between HOF and Eval. Eval can do everything HOF can,

But it's not necessary to use Eval. You can wrap your whole Program in one eval() too, or you can use Eval instead of direct assignment statement also, but that would be stupid. This is the same, you can mostly use Eval inplace where HOF is used, But Eval has it's own quirk that it should not be used as much as HOF. That's why we prefer HOF over Eval.

You don't use it, fine. We don't know what kind of applications you write either. Obviously for PutTheDamnDataOnTheDamnScreen? kinds of scenarios one doesn't need much advanced techniques. However, you're trolling when you generalize from such a limited experience and such a limited education.

It is not me who generalized. You are guys are the ones implying that examples from domain A are applicable to domain B.

Running a quick statistic over my current Java project, out of 1028 classes some 588 classes were inner classes, thus some form of closures (Java's more verbose corresponding concept to HOF). Of course, this project is peculiar because it is multithreaded and depends on all kinds of events happening between components distributed over the network in unpredictable (non-deterministic) fashion so the use of closures was dictated by a framework needed to keep the concurrency in check. But other Java projects that you can examine (open source) have a healthy doze of closures as well - and you cannot complain that java projects are as peculiar as FP examples coming mostly from university. Thus people do use closures and HOFs, and for you to pretend that you know better that constructed strings can substitute those, well, that is pure unadulterated trolling. It's again a form of ShiftingTheBurdenOfProof. -- CostinCozianu

Even if it was true that they were common (we only have anecdotal evidence here), that does not necessarily imply they are better.

The main reason why dynamically constructed strings should not be used as a replacement for functional values (HOF, closures, inner classes in Java, etc.) is because when you construct a string and eval'ed later the programmer has a much harder time proving (convincing himself to a degree of certainty) the correctness of the execution of constructed string. For example, one would have to account that any values pasted into that string from current variables don't interfere with the syntax of the language, and that's just one of the many crappy details that become a burden when one goes the eval route for all but the simplest of examples. Remember programmers live with a ProofObligation.

This sounds like it follows a typical debate between static typing and dynamic typing. No need to repeat that stuff here. As a convincing proof that you don't know what you're talking about, the very code you presented for ChallengeSixVersusFpDiscussion is not correct for all values provided via user input (you construct SQL strings to be submitted to the database). What you should have used there (had you known any better) was prepared statements with bound variables, which is a mechanism somewhat similar to a closure. And this closes the "strings versus closures" discussion.

One has to check the SQL-targeted values either way. Unless you reinvent and mirror SQL locally, the computer is going to just treat it as a string even in the tightest of languages. (I talked about SQL and security earlier above.)

Buy better books. Oracle and Sql Server, the two biggest vendors, and others I'm sure have supported prepared statements for a while, doing dynamic sql without using prepared statements is ignorant and unsafe, and amateurish at best. Prepared statements have nothing to do with java and counting google links while pulling statistics out of your ass isn't sufficient research to rebut something you're ignorant of. You're lack of knowledge on all this is astounding, as usual, amazing someone pays you.

[EditHint: move sql-injection stuff to SqlInjection or SqlStringsAndSecurity]


1. Go see [Marker 00001] in DynamicStringsVsFunctional That explain why HOF is more suitable in most case than Eval. And 2. I don't know why such expert in Biz app and DB like you don't even know about prepared statement. That's funny.

I have worked in about dozen companies as both an employee and a contractor, big and small, and not a single one used prepared statements, at least not in volume. A good many used stored procedures, which are very popular. Why I haven't encountered them, I don't know.

Out of a hundred technologies, to pick a sample size, if one works for a dozen companies and each one is used in 50% of the companies, then by typical Piosson (sp?) distribution, a handful will probably slip through. But it is immaterial to this topic. (By the way, I have not seen how they allegedly allow dynamic clause creation. Anybody have a link? Thanks)

Further, a Wiki Double Standard has been exposed. Somebody suggests using nested loops instead of joins, which is considered poor form in the industry, and none of you say a single word about it. Yet you find that I don't know about an obscure database feature outside of Java, and suddenly I am stamped the dumbest guy on the planet. You have been caught double handed. Your bias drips with the vital fluid of twisted truths and social manipulation.

[I agree with TopMind that the attack seemed unwarranted. I hadn't heard of prepared statements by that name at all. They're mentioned only very briefly in Date & Darwen's GuideToTheSqlStandard?, third edition, as "preparable statements". I'm willing to believe that's a failing on my part, but the vitriol over that detail doesn't seem warranted. Perhaps they go under different names in vendor's implementations. -- DanMuller, DeleteWhenCooked]

TOP's particular ignorance of this basic mechanism is not the issue. If TOP was just a mere ignoramus he'd be excused and spared the public embarrassment. However the difference is that Top's ignorant and proud of it. His self-infatuation in this case and other cases is beyond belief.

I told him of the problem a few weeks ago and told him it was trivial to fix the code against "SQL injection attack". A simple google for it would have led him to prepared statements, he instead handwaved about it. Above I told him once more. After he was told he was wrong and that he should have used "prepared statements with bound parameters" any self-respecting contributor would have checked first what that was. There are countless articles on how prepared statements save yourself the embarrassment of SQL injection problems. He instead posted four more irrelevant replies without even thinking about it, just hoping that by mere luck he may find something that can stick. All his replies above cannot be justified for a contributor who wants to participate honestly in a discussion.

This trolling attitude is uniquely branded to TopMind. I don't know of any other contributor to Wiki that after given a clear indication that he's wrong and why he's wrong he'd contribute more irrelevant replies. The sole purpose of such ThreadMesses is solely for TopMind to titillate his own ego.

Prepared statements are a well-known mechanism. In ODBC the API name is SQLPrepare, in ADO if I'm not mistaken is Prepare, and tons of examples are everywhere one cares to read about how to interact with SQL databases. Almost all vendor specific client APIs had a prepare call for longer than I can remember. frankly, I cannot even remember when the Prepare call was first introduced in a client API. For the duration of my software engineering career it was always there in one form or another. If you think you have a solid knowledge of things database related but you are not familiar with this concept all I can tell is that you have a serious problem with the selection of books or documentation material for your study, but that's a side issue.

How is this particular example relevant? Because prepared statement solve a problem that also happens with eval: the interference between values picked from environment (especially string values) and the language syntax that the evaled string has to conform to. Because of these, to everybody but to TopMind, it is perfectly clear that thinking about the correctness of the evaluation of a dynamically composed string, is much harder than thinking about the correctness of a closure. He repeated by the way of handwaving that such things are not important and when I pointed out to him that such an interference bytes his ass in the very example he posted in a domain that is supposed to be his area of expertise, he ended up generating this ThreadMess rather than taking the opportunity to learn.

Because above everything else, his positions need defended at no matter what cost. Honestly or dishonestly, informed or ignorant, these criteria are totally outside Top's concern.

Sorry, but I still don't see the connection to this topic. Eval was not an issue in any of the SQL protection comparisons. It is a toy example and intended for intranet use by a handful of managers who would have read-only access to most of the tables anyhow. And, there are multiple approaches to protecting the SQL, each of which I feel is not relevant to this topic. Further, I am not sure standard ASP (a now dead language) supported them and maybe didn't support the clause-addable version, which I suspect does not even exist. -- top

Another mindless reply (5th). Why don't you go and read up a little bit, including what was written above and when you are able to do more than to argue from ignorance come back then.

Sorry, but I missed the direct connection to the topic. You seem preoccupied with compile-time checking and generally anti-dynamic. We don't need to repeat that debate here. If you want to use PS's in your version of #6, be my guest. That is why it does not matter. -- top

EditHint: move the prepared statement stuff to SqlPreparedStatements

Here, sample in javascript, with some of your vbscript as the body, to add a little realism. You're proved wrong top, now extrapolate a little and see how this will help you.

 //with each row, a higher order function that would eliminate 6 repeated loops within your sample
 //and guarantee never having a bug caused by forgetting to movenext, or close.
 //this is a function you desperately need
 function withEachRow(sqlString, aFunction){
	var rs = stdConn.execute(sqlString);
	try {
	while(!rs.eof){
	  aFunction(rs);
	  rs.movenext();
	}
	} finally {
	rs.close(); // ohhh, it's better than your code, it'll close the connection, even under an error condition, yours won't.
	}
 }

For one, we might not want to close it after each QueryAndLoop but leave it open for several loops to gain a little efficiency. Second, good web systems allow global default handlers to be defined for database errors that can close the connection, and even better the connection should automatically close after each HTTP event (submit). There is no reason to keep it open, error or not. Third, sometimes we want different handling for zero or multiple results. For example, a custom "sorry, not found" message for zero result rows, and sometimes there should be one and only one result, such as a count query. Yours is not generic enough to handle all that and making it generic enough will just bloat it up with creeping featuritis, especially in a larger app. Finally, a different contractor would come in and have to learn your custom query looper and all its little features, cranking up the learning curve for shop-specific conventions. I agree that it may shorten some loops by about 1.5 LOC (depending on language), but may result in only a few percent total. The claim was "significant reduction". 5% is not even close to what I consider "significant" to mean. -- top

So this one trick reduce 5% of the code. not enough to claim significant reduction. But who say we can only use one trick per app? Bring in another loop HOF construct, reduce another 5%. Bring in some MAP, INJECT. reduce another 10%. and all that it reduced also increase stability in your code; every part that use "withEachRow" will never leave its connection open. hand typing may forget one. reduce 15% of code size and increase stability of code by 15-30% is not something you can do easily. and that's significant because it BOTH reduce code size and increase stability of application.

For the record, I don't see why it would make "withEachRow" bloated just to add <close_on_exit?> flag and <if_empty_rersult_do> function callback, both default to current behavior; it will only add three more lines to above code. And you are being funny here about "sometimes query only return one result". Duh, the above code is utility for use in LOOPING over ROWS of result set; if there is only one row then DON'T USE IT. It's funny you are complaining like "'for loop' is not generic enough because in some case we will only want to execute only one time"; Don't use the for loop there, man. For query that return only one value like <count>, you already know that it will never return more than one value, so someone would be so stupid to not understand that above code is not for such usage. And The above code already always close the connection, error or not.

 //so you can keep writing your nested conditional code, within the safety of a nice HOF to protect you
 //and eliminate all that rampant duplication in your sample code for this challenge.
 withEachRow("Select * from Users", function(rs){
	fldValue = trim(rs("fldValue") & "")
	fmtType = ucase(trim(rs("fmtType") & ""))
	if fmtType="N" and len(fldValue) > 0 then
	if not isNumeric(fldValue) then
	 appendErr "Invalid Number: " & fldValue
	end if
	end if
	if fmtType="D" and len(fldValue) > 0 then
	if not isDate(fldValue) then
	 appendErr "Invalid Date: " & fldValue
	end if
	end if
	if rs("Required") and len(fldValue)=0 then
	appendErr "Field is Required: '" & rs("fldTitle") & "'"
	end if
 });
 //hey look, used it again, wow, isn't it flexible, it even works with your code.
 withEachRow("Select bla bla", function(rs){
	sql2 = "UPDATE userFields SET fldValue='" & trim(request("fld_" & rs("itemID"))) & "' "
	sql2 = sql2 & " WHERE userID=" & userID & " AND rptItemID=" & rs("itemID")
	stdConn.execute(sql2)
 });

//I'm betting you could also stand to use this one a lot. function withEachColumn(aRow, aFunction){ for(var index = 0;index < aRow.Fields.Count -1; index++) aFunction(aRow[index]); }
I rarely use integer-indexed loops. To me they generally indicate a yellow alert.

then say..

 withEachColumn(aRow, function(aCol){
	hout("<th>" & aCol.Name & "</th>");
 });
I usually loop through the data dictionary, not map arrays. Plus, maps by definition do not define order, which generally creates problems. Looping thru maps is also a yellow alert.

there's all the external evidence you need, if you don't see it, then you're fucking blind, because your program is rampant with duplication that these two simple HOF would eliminate entirely.

Below I argue that some of the bloat is due to the design of Microsoft's API rather than an inherent fault of non-FP itself. One may be able to build wrapper libraries, but I didn't bother in the example.

Moved discussion to ResultSetSizeIssues.


PageAnchor: code_management

In your Non-FP version, you increase complexity of a program by introduce a variable "rs" which is visible for whole method, but is only valid for uses inside the loop. In FP version the uses of "rs" is scoped to inside the block. There can be no mistake of referring to this result set again after it's used. This may be small change, but it's C/C++/Java or whatever languages with external iterator's good practice to scope the iterator to only where it's used. For example, it's good practice to code.

 for(int i = 0; i < length; i++){
	....
 }
than

 int i = 0;
 for(i = 0; i < length; i++){
	....
 }
So FP approach reduce only small code size but also reduce code complexity. How is that not significant. As I state FP is not just significant because it can only reduce code size. FP is significant because it both reduce code size and increase code comprehensibility/abstraction.

I agree that is a nice feature, but I find it hard to call "significant". Again, you seem to be offering solutions to problems that I generally don't encounter in practice. I won't say I have never encountered problems from "leaky" loop scopes, but that probably would not make my top 10 list. -- top

It is probably possible to have a dynamic language where the scope is only in the given block. Thus, any variable created in a While loop will only be visible to that loop. But it may still clobber existing variables created before. But it is possible to have a language in which inner declarations don't clobber outer ones if explicitly declared.

For the record FP version can also be made to read the whole Result set to cache first.


I am not against dynamic typing or agile language like ruby, I love it. Yet SQL made by concatenating string together has the same bad thing; You get the dynamic behavior that you can not reason about.

For this SQL :
 Query("SELECT * FROM employee where name = ?", name_param)	#Prepared statement
 Query("SELECT * FROM employee where name = '" + name_param + "'") #Concatenated String
In case of Prepared statement the query will work no matter what value are binded to the '?' place. And you can always be sure that it's operation is to do a query on employee based on name. In case of concatenated string, it will work ONLY IF the name_param does not contains any ' or comment character for SQL, any statement can even be added by escaping the string and put ";" to separate a statement. You can not be sure what execution path it is for this SQL unless the name_param is know. This is not kind of Dynamicness one will ever want. Or could you give an example of where that is desirable?

Clause addable is always possible with prepared statement. PreparedStatement is not something that has to created at compile time. You can create them on the fly. For example, this prepared statement can be created at runtime:
 SELECT * FROM employ WHERE name = ? AND phone = ? address = ? AND ... # add as many "xxx = ?" programmatically.
And you can after that bind each '?' to the value. And there is no chance that the query result will change just because some of the data has '#' (comment character) in them.

This is the same objection I have for eval. Suppose your eval statement is to do assignment to variable. So you create this:

 def eval_append(var_name, var_value)	#<var_name> is a string, <var_value> is from DB and never know.
	eval var_name + " = '" + var_value + "' + 'foo'"	#result in, for example, eval "x = 'bar' + 'foo'"
 end
Now this would work fine for

 eval_assign("x", "baz")
But it will not work this string

 eval_assign("x", "boom' ; 'x")	# assume ";" is statement separation. now we have: eval "x = 'boom' ; 'x' + 'foo'"
I can never think of the reason you will want this. you want <var_name> to always be string you can assign to something no matter what it's value is. (Or do you like a language that "'x = y' works as long as y does not equals 0, 57, and 12357"?) You can not really be sure, with eval and string building, that the execution path of the data can be determine based on only the code you wrote.

"Code and data are the same thing" is fine for me; to the extent that I can be sure that what I want them to do to be data doesn't suddenly choose to behave itself as code without my decision.

There is a different degree of dynamicness one feels comfortable with. For me, I want dynamicness of code that is reasonable by looking at the code. I will never want some undetermined execution path to come just because some weird data enter in to the systems.

Secondly, Eval is hard to play nice with itself. In HOF, one of the best convenient is that a function can take another function as parameter and return function as a result of execution. Can Eval take another a string that is the result of another eval to use in its code, or return a string for another function to use in eval statement? Sure it can but, with the problem that code for eval is just string, you will have to double/triple escape strings depending on where you want it to really behave as string and where you want it to behave as code; which is confusing process to get right at least.

As far as prepared statements, I don't care either way. I am tired of hearing about them. Use them if you want and if it affects the code size count in the end, I will accept an adjustment to compensate.

PS does not reduce your code size, it is just safer to use than string building. Just like HOF and Eval.

As far as Eval going bad, It has not been a problem for me. I focus on where actual problems occur in practice. If I needed to use Eval a lot per app, it might start to become an issue, but I don't. It works just fine for the few places that extra indirection is needed. And, I have not figured out how to store HOF references in a database. But, I can put function names in a database. (It is not an issue for #6, I would note.) But generally I end up putting expressions in the DB for most Eval uses that I recall, not function names alone.

So now you lost the claim that putting function dispatch in DB is across language; you can't eval TCL code in Python.

I'm not sure what you mean. I do it out of editing convenience and OnceAndOnlyOnce. If you put the same in code then you have to carry the primary key in the code also to match up with the table, which contains the attributes. It then becomes a one-to-one relationship, which is something to generally avoid. For example, to add or subtract something from the table requires you to remember to do the same to the code-based list to keep them in-sync.

For example, if you have a menu system where the menu descriptions and nesting are stored in a table(s), then we probably want a way to associate the menu items with behavior. If we cannot put function names or expressions in the table, then somewhere in the code the primary key probably needs to be duplicated:

  // duplication example
  switch on menuID
	case 234: functionX(...)
	case 432: functionY(...)
	case 532: functionZ(...)
	otherwise: error(...)
  end switch
If it is in the menu table instead, we don't have to duplicate the key and remember to keep them in sync. Thus, it is better OnceAndOnlyOnce.

It's already stated that we can put the function name in DB and use reflection to get the function to call. Don't you remember? You said "If we cannot put function names or expressions in the table...". We can put function name in the table and use reflection to get the function to call, no duplication example above will occur. When I said about you

Reflection is a different thing than HOF and closures. You keep adding features to solve problems. You need three different features to do what Eval does. Plus, what if we change it from putting function names in the DB to expressions in the DB? Now you need four features to do what Eval does:

 switch(f){
 ..case 'foo': foo();
 ..case 'bar': bar();
 ..case 'baz': baz();
 }
compare to
 (funcall (symbol-function f))
Or

  eval(funcName)
And it scales to parameters, expressions, etc.

I didn't go out of my way at all to make an Eval problem example. It's a common mistake in Eval. You want to passed both data and code to another function; this is possible in both HOF and Eval. But in either approach, you want it to behave as what you want when you want it. If you are intending to use it as data now, it must only be data; if you want to use it as code now, it must behave as code. Eval doesn't give you that. See from above? I intended to use <name_param> as data. But just because that data contains some character, it can't be used!

Are we talking about security or programmer convenience? Assuming database security is done right, the hacker should not be able to munge function names or expressions stored in tables anymore than they could munge the source code.

Grrr.. I'm neither talking about security or programmer's convenience. Security hole is only one of many results of using eval this way. Using Eval, you cannot be sure that which part of the string will be data and which part will be code. The user of the system may not be hacker. But they just want to type " ' ;" as part of the data! Why can't they do that? And if talking about security. With eval/string concatenated SQL. They don't need special permission to bomb the system. They don't have to munge with the expression in DB; they just type the malicious expression directly in the User interface you provided them!

You seem to be mixing up different issues. The Prepared Statement issue is generally unrelated to Eval. The code that Eval executes does not necessarily have to come from the user. Let's evaluate specific UseCases of Eval. -- top

The code that eval execute doesn't have to come from user. But the data WHICH IS PART OF THE CODE that is passed to eval in someway has to come from user. Sure the following code does not come from user:

 data = ....		  // get data from somewhere
 code "print '" + data + "';"	// print the data as string (see those two 's?).
 eval code ;
But did you notice that data is actually PART OF THE CODE? If data is, Say, " '; print = 'some more". You will end up with a side effect that only happen because the data contains a single-quote which escape the string literal you intended. But looking at the code above, Can this flawed can be easily found by just looking at the code, I don't think so. Will simple test case discover this? (most people will just pass in normal data, no "'", as test data).

I don't see how the above is significantly more likely or less testable than any other dynamic-language bug. You keep talking about how risky eval is, but again it has not been a source of big problems in my experience. I don't need it that often, and when I do need it, if one exercises a little care, it does its job. You keep insisting that brand X is going to turn my laundry green, and with more than a decade of use, my laundry hasn't turned green. I can think of only one app where the user created the expressions to be evaled, and it was a local-use app with two power-users, not something all over the company. Most of the rest were either internally generated, or came from a ControlTable.

Ok, is there any syntax error in this code?

 eval "print('" + data +"');"
I bet you cannot tell me whether there is syntax error or not until you know what the data is, correct? Syntax error because of data? Good?

I don't understand what you are trying to get at. I restate, in practice, Eval has not been the boogey-man you make it out to be. Quotes and stuff may tend to trip you in particular up, but that does not mean that they trip everybody else. Different people are tripped up by different things. -- top


Note that the #6 example does not use Eval. Perhaps the Eval discussion should be moved.

Move it to DynamicStringVsFunctional? if you'd like.

That is also TooBigToEdit. When the dust settles, we need to clean these discussions.


There seems to be a reoccurring pattern to some of our differences in opinion: I practice more SeparateIoFromCalculation and don't mix things that don't have to be mixed. For example, loops are simpler under such separation because preparing the loop and performing the loop become separated issues. I tend to loop on prepared temporary result sets, not direct active behavioral interfaces (such as files or pipes). Similarly, error handling seems simpler because high-risk activities are separated from low-risk activities such that recovering from high-risk activities is then much smaller in scope. It tends to be part of my data-oriented-interface view. Communication between "parts" of algorithm is via data structures (preferably tables in many cases) instead of via behavior and behavioral interfaces. Your examples and demonstrations over-use behavioral techniques in my opinion. You try to do too much at once. You nest processing that does not need to be nested (at least not in my domain). It is not that FP does not help in general; it is that it does not help much when doing things my way. Now, whether my way is "bad" is another topic. But it's similar to some of the reasons my views conflict with OOP also, which tends to be behavioralistic.

PageAnchor: coding_thumb_rules

Here is a summary of practices that would reduce the frequency of need for many of the things you FP's try to sell:

Now, I will grant there are occasional times where one cannot follow these principles and FP may indeed simplify those particular spots. However, I question whether it is wise to double our tool set just to improve a very small percent of the total code. One should not violate KISS unless there is a compelling need to gum up the works.

-- top

"Now, whether my way is "bad" is another topic."

No, I think it is part of the same topic. If I'm understanding it right, it sounds incredibly inefficient for many applications. What do you do when you encounter a performance problem? Do you then deviate from the techniques that you advocate, and if so, what techniques do you use then? -- DanMuller

Machines are cheaper than humans (at least in the US). And, in my domain the bottleneck is usually the network, not the CPU. Also, the RDBMS can do a lot of auto-optimization for you if you let it. Given these conditions, I will happily let the machine do the grunt work so that I don't have to. If I encounter a specific performance problem, then generally a little rework is all that is needed. Most performance problems I witness are due to bad schemas and/or bad queries, and not from principles such as SeparateIoFromCalculation. -- top

I'll willingly agree with all those points, and work from similar assumptions myself. But the key words are "usually", "can", "generally", and "most". There are exceptions that I find to be infrequent but far from non-existent, such as large batch processes that require a sophistication of processing which the query engine can't handle for me. So what do you do in those cases? Just tell the user that they have to live with the long processing times incurred by all the data copying? Or is this when you resort to cursors? (Note that I'm not trying to enter this debate in any way, I'm just curious.) -- DanM

I never said never. {Oops, I caught myself in a recursive lie here.} If you really need client-side cursors, use them. The FP wrapping techniques suggested seem to only materially matter if there are a lot of cursor loops. It appears that one is sufficient for that case. That one loop may indeed be 3 more lines than an FP version. But using multiple paradigms to tends to bring diminishing returns and increases the expense of staff. It might be great if you are the receiver of the paycheck, but businesses do weigh those diminishing returns which are cancelled out by staffing issues. The numbers are against it.

 Benefits of each additional paradigm (illustrative only, not based on actual studies)

1 | *********** 2 | *************** 3 | ***************** 4 | ****************** 5 | *******************

Costs (hiring costs, staffing costs, staff transition delays, training, debugger cost, etc.)

1 | *********** 2 | ************** 3 | ***************** 4 | ******************** 5 | ***********************
Businesses want PlugCompatibleInterchangeableEngineers. They will only abandon this goal if you can show a compelling benefit. Going from 3100 lines of code to 3097 lines of code is not even near "compelling". Related: HackerLanguage. -- top

Well, much of the discussion on wiki is about how to improve software engineering, not how to kowtow to bad practices propagated by inertia in management and vendor products. Anyway, thanks for the answer, but no thanks to all the extra stuff (including invented graphs) that came with it. -- DanM

Business managers sometimes piss me off also, but your assumption that most business practices are inherently wrong is misguided in my opinion and in this FP case I agree with their general rejection of FP. A good case for it has not been made in the domain. Its features seem better suited for systems software and communications tools. I will agree with this though: if staffing/training/education and language-complexity are not factors being weighed, then I could possibly side with you. I am just trying to put myself in the shoes of those who decide on languages and methodologies from a capitalism perspective. Whether that is the "right" perspective is perhaps another entire philosophical topic in itself. But, hopefully we have identified and narrowed down the areas where we differ. -- top


Just another invented Chart

 Benefits of each additional paradigm (illustrative only, not based on actual studies)

1 | ******* 2 | ********** 3 | ************* 4 | **************** 5 | ************************

Costs (hiring costs, staffing costs, staff transition delays, training, debugger cost, etc.)

1 | ************ 2 | ************** 3 | ***************** 4 | **************** 5 | ***************
Now we all should be using Multiparadigm language shouldn't we?

NOTE:
Also note there is the cost of those who make big messes for the sake of job security. The more paradigms you can mix up, the more weapons you have for job-security. I have seen cases where boredom led to MentalMasturbation, and multiple paradigms just gives a dude more dicks.

You seem to have a distrustful view of your employee, don't you think?

In the current work-world, there is not a lot of incentive to write readable code. I know one guy who claims one shouldn't comment code because once he saw his comments confuse the optimizer hint system, and the boss buys that excuse. -- top

It is also possible that someone will use DBMS just to kill boredom. It is also possible that someone will use Eval just for mental masturbation. It is also possible that someone will program in TOP just for mental masturbation.

Some indeed do. But it is a matter of quantity. Code where 2 paradigms are abused is better than code where 5 are abused.

[Ehh... No, if it is a mess, it is a mess, even if it's a single paradigm program. Quantity wise, 10 line of code will be better than 20 lines of code. And 2 lines of code will be better than 10 lines of code. Yet we know what PerlGolf is And we know how Verbose XML is. And if many choices is bad, then you shouldn't only count paradigm, but count library out there. How many thousands of free libraries are there for Java/C#/Perl? Any programmers can download a library and abuse it in his program. Why should we allow that? Abusing uses of Library is no better than abusing uses of paradigm. Why don't we have a language where one cannot create a library? What's the difference? Disregarding mental masturbation, who force him to use a library where he doesn't need? Who force him to use a paradigm where he doesn't need?]

And it is certain that, in the current work-world, there is not a lot of incentive to know enough to program TOP effectively (Only topmind seem so). I once saw a guy trying to simulate multi-dispatch using Database Table. Does that also mean we should discourage TOP too?''

What is wrong with using tables for multi-dispatch? I am not saying it is always the way to go, but I would like to know specifically why it didn't allegedly work. I've worked in shops where tables where heavily used for such and they were productive and reliable.

Because the language already provide the function?

No, you have to build the dispatch matrix from scratch. And, tables are better OnceAndOnlyOnce because an OO version repeats the "cell header" over and over again (method declaration).

Because you are doing things unconventional to your codeveloper?

So the status-quo is a self-fullfilling prophecy? ExBase shops often used expressions in tables. It was not "unconventional" to heavy ExBase people.

Because your dispatch is based on unsafe eval?

Not this anti-scripting debate again. Besides, one can use case statements instead if you really want static "comfort".

Also, there are shops that use multiparadigm and works just like your shop that table works. So what decides that multiparadigm should be discourage while TOP should not?

Tables and procedural complement each other well. Thus, you have only 2 paradigms which generally don't fight over territory.

Higher order functions are used to do something like this
	SchemeLanguage Code 	
	(map-query "select * from foo where bar = baz"
			(lambda (x) (printf "~a~%" (* x x))))
And this does what? what is "~a~%"? The SQL does not change, so why run it in a loop?

It's not in a loop silly the first argument is the query and the second is the body of the loop. its lisp format strings

What use is it over more conventional techniques? Stuff like "~a~%" is not very self-documenting, so I'd knock points off for that.

(Who has been doing screwy grammar editing of late? Please be a bit more careful.)
How does adding one keyword Lambda, and removing restrictions on function passing have on complexity vs having to add a keyword for every control flow abstraction. Lets say i have a table and i want to map over the tuples therein imperitive code
	 for(i = query("select (quux.foo,spam.bar,eggs.baz) from quux,spam,eggs where quux.foo = ? and spam.bar = eggs.baz","abc");
		!done(i);
		next(i))
	 {
		printf("%i,%i,%i",i.get[0]+1,i.get[1]+22,i.get[2]*3);
	 } 
Functional code
	 foreach(query("select (quux.foo,spam.bar,eggs.baz) from quux,spam,eggs where quux.foo = ? and spam.bar = eggs.baz","abc")) (foo,bar,baz) => {
		printf("%i,%i,%i",foo+1,bar+22,baz*3)
	 }

What is being demonstrated here?


For what it's worth, I was upset by the on-all-sides-argumentative and fruitless conversation on this page, and decided to do something about it. http://github.com/ods94065/challenge-six is my tilt at the windmill. The solution uses a grab-bag of Lisp tricks, including some FP, to shave off somewhere between 9-30% of source code size depending on how you measure it. Not bad for a simple demo app IMO. -- Owen

Looks good to me. Just a minor nitpick though. C++ has made some changes recently (i.e. C++11) because they've "seen the light", as it were, on the value of expressions. You can now do the following.

	std::vector<std::string> x()
	{
	    return {"foo", "bar", "baz"};
	}

Continued at ChallengeSixLispVersionDiscussion...
CategoryExample, CategoryBusinessDomain

EditText of this page (last edited June 16, 2013) or FindPage with title or text search