Nuts And Bolts

I've read around the site and one thing becomes clear to me.... (I cleaned this up a little to be easier to read.)

SoftwareEngineering is not considered mature because we don't have the equivalent of standardized: So I am Proposing a new topic line: What is our toolbox?

Ground Rules: One thing at least a couple of us agree on is this seems to be a major part of the toolbox. I do add the following note: I do understand that artifacts are language dependent, but we do not specify which language the artifacts belong to. -- WyattMatthews

I think there's a certain danger in over-using mechanical engineering as a metaphor for software engineering. SoftwareEngineering isn't mechanical engineering, it isn't creative writing, it isn't math. It's a thing unto itself, SoftwareEngineering is, and I don't think it deserves to be deprecated by unnecessary analogy. That said, aren't our "nuts and bolts" algorithms, data structures, and software patterns? -- DaveVoorhis

I would hope so. It is my intent to clear that up not only in my mind, but everyone else's as well.

Is this a duplicate of SoftwareEngineeringBodyOfKnowledge effort ? That one is all but failed for practical purposes, but still has a lot more than this page. I don't think wiki is a good place to start such enterprises. -- CostinCozianu

Costin, it is not intentionally a duplication. I will re-read both pages and make adjustments if needed. -- WM

This page seems to be an attempt to set some lower threshold on what software engineers "engineer". The assumption is that for any given category of engineer there is some scale of matter that separates raw material from finished product. The title suggest an engineer who designs structures using nuts and bolts, but would never design a new nut or bolt. The implication is that every possible type of nut and bolt is available to that category of engineer.

One problem with this metaphor is that engineers do create new kinds of nuts and bolts when existing nuts and bolts don't fit a new use or new environment. They may not necessarily be in the same category as the engineer who uses the new nuts and bolts, but novel situations demand novel materials.

If we look around at current software technology, it seems that we haven't created every possible type of any building block. We haven't even created the theory that would identify the possible parameters for such blocks or establish their scale.

The first issue that will need to be addressed is: What are our nuts and bolts as Engineers? (contrast with, what are the mediums we use as Artists?) Do they reduce to the Pattern system we are working with, or are they just implementations of the DTSTTCPW?

Secondly, how should we define these meta descriptions of these tools? Again, are they just Patterns that we all already employ?
The problem is that almost every EverythingIsa item can potentially be made into a standard building block. There are just too many ways to achieve the same results, leading us back to HolyWars again. It is a problem of narrowing down the choices, not lack of.
Then in an effort to avoid the HolyWarSyndrome, let's conceptually discuss how to narrow down the choices. Is there a way? Is it applying the DTSTTCPW and OAOO to everything even a pattern?

I took a stab at responding to this, and then realized I don't quite understand what is being asked. Clarification, please? -- DaveVoorhis

It has been pointed out that there are too many candidates for basic building blocks, starting with things such as EverythingIsa items. The question is: Feel free to add more potential questions.

Sorry, you'll have to forgive my density here. I still find the above too amorphous to work with. I find it difficult - and maybe this is my own lack of imagination or understanding, I admit - to interpret anything above this paragraph except in terms of algorithms, data structures, and patterns. While it may be possible to further subdivide algorithms and data structures into more primitive and/or abstract notions of reference, transformation, state and so forth, I am not sure this is useful at the level of software engineering except in terms of understanding a given algorithm, data structure, and/or pattern. There are also constructs that are nearly universal, such as identifiers, but I am not sure their near ubiquity is sufficient - only absolute ubiquity will do. Therefore, algorithms, data structures, and patterns are the nuts and bolts of our profession. They are universal and fundamental, as they are applicable to and required by every language and software system, whether implemented explicitly by the programmer or available implicitly in the language or software system. Language features or constructs, programming paradigms, development approaches, and various pre-implemented algorithms and data structures - the attributes of the aforementioned EverythingIsa items - are largely not the nuts and bolts of software engineering. At best, they may be - if the analogy can be stretched this far - ways in which we arrange the nuts and bolts, ways to understand arrangements of nuts and bolts, or ways to assemble nuts and bolts, but because they are (by definition) not universal except from the point of view of a given EverythingIsa item proponent, they cannot be considered basic building blocks. They are structures built from basic building blocks, or tools used to arrange basic building blocks. Only algorithms, data structures, and patterns are inarguably universal. Without these, there is no software to engineer. However, that is not generally true of items on the EverythingIsa list. -- DaveVoorhis

Perhaps that's all we need, but I worry that it is over-simplification. I am looking for what is really at the heart of things after all. I guess what I am asking is: "What do we consider core to our discipline?" and "What do we expect to be already present when we sit down to design or engineer a software project?" Just algorithms, data structures, and patterns?

Again, I know I am being a little dense or repetitive, but: Is that all there is? We now have listed the following:

Is there nothing else? No standard implementations of the principles in relation to the various artifacts?

No. Are the Principles and the Artifacts not enough? A developer who understands the Principles and can select an appropriate Principle for the task at hand via a rational, reasoned process - i.e., can appropriately apply experience, knowledge, and expertise - and can effectively use the Artifacts to produce acceptably reliable software on time and within budget, is a good Software Engineer.

What would "standard implementations" of the Principles in relation to the various Artifacts be? A library of "reference standard" source code? Due to the diversity of problem spaces, options, and individual requirements, I doubt it would be possible to come up with even a single agreed reference implementation of, for example, a disk-based B+TREE in ANSI Standard C, let alone come up with a reference collection of all Principles in relation to an agreed upon (good luck!) set of Artifacts. That would be a bit like coming up with a "standard implementation" of an electric motor. The closest we have might (emphasis on "might") be TheArtOfComputerProgramming. -- DaveVoorhis

I'll give you that currently there is no way of coming up with a single agreed upon reference at this time due to the number of potential problem spaces.

What I think we will need to continue this discussion is a SoftwareEngineer (let's call him 'SE') who has a specialty in a particular ProblemSpace (let's call it 'X').

Suppose SE asks others who specialize in X, "Why have we not been able to yet come up with an agreed upon reference that factors out as many tailorable requirements as possible?"

My gut reaction is that there is no single real-world problem space small enough to support a realistic complete enumeration of tailorable requirements, while still being a significant enough problem space to be worth bothering. -- DaveVoorhis

It was not my intent to forward any agenda. I started it hoping to clear up my own confusions about what the readers of this wiki, and perhaps myself, see as necessary tools for the job of a SoftwareEngineer.

I also ask, should SE have a standard reference for the problem space of 'X'?

It strikes me that attempting to define such a standard reference is a little like trying to define a reference piston for a universal internal combustion engine. (How could there be a universal internal combustion engine??? Ah hah! See?) We certainly recognise pistons are pretty much ubiquitous in internal combustion engines (rotary engines aside), and we are aware of the general conditions and constraints that determine their design, but the specific real-world variability in those conditions and constraints means we can never create a meaningful general reference piston. We could, however, create a specific reference piston for a specific model of engine. Similarly, we can create (for example) a reference implementation in Java of a given Java Community Process's Java Specification Request, but that applies only to a particular and very small subset of the overall software engineering space. Defining a useful reference implementation of, say, a general purpose CrudScreen or a database table or a lexer or an accounts payable system or an A* algorithm or some other common software device is unlikely, but if someone produces a specification for (say) a particular A* algorithm implementation in a given language, or even one to be accessed via some language-neutral protocol, then it is possible to produce a reference implementation of that specified A* algorithm. But that's not what you mean, is it?

Looking at it from another point of view, let's say you are an SE with a speciality in X. Can you create a reference implementation for even a subset of 'X' that other SEs in X will agree with, in sufficient quantity, to generally regard your reference implementation as standard? -- DaveVoorhis

I acknowledge that what I am looking for may be the pattern of the piston rather than the ideal implementation of a piston. Perhaps X's standard reference is a list of standard patterns needed to resolve the OnsiteCustomer's RequirementsAndGoals presented to SE.

I think I've got it now. Using the piston example, we can certainly agree on a piston pattern: It will be an approximately cylindrical blah blah blah wrist pin blah blah oscillatory motion blah blah sealing blah forces blah blah... Likewise, for (say) developing accounts payable software there should be a collection of standard patterns that are specific to accounts payable, but are (for the most part) standardized across all accounts payable development. Is that what you're suggesting? In other words, each problem domain has, or should have, a set of domain-specific patterns that constitute standard approaches? -- DaveVoorhis

This does appear to be part of what I needed.

All right, I think we've determined that for X, SE needs a standard set of Domain-Specific Patterns. But this is only part of our requirements. Does X demand its own Algorithms and Data Structures, or are these implicit attributes of correctly created standard patterns?

Some X have X-specific algorithms, or at least algorithms that appear more frequently in X than elsewhere (for example, the A* pathfinding algorithm shows up in games more than elsewhere), but the creation of new algorithms and data structures is going to be, in general, relatively rare compared to creating domain-specific patterns. Algorithms and data structures - BTREEs, queues, lists, binary trees, etc., are more far more general and universal than domain-specific patterns, and so are far more likely to simply be used than new data structures invented. -- DaveVoorhis

Add to our mix a SoftwareResearcher, SR. It is his job is to create or identify the Patterns in X and submit them to the standardization process.

SE reviews SR's standardized patterns and goes to apply them. Here's the next question: Does SR's research generate the Algorithms and/or the Data Structures outlined above in the Principles and Artifacts list?

There's no reason why an SR wouldn't create new algorithms and data structures if appropriate. In many problem domains, I suspect this will be infrequent. I can't think of any reason why Accounts Payable development would demand any new, previously unknown algorithms, for example. Cutting-edge game design, on the other hand, may well produce new algorithms and data structures. -- DaveVoorhis

I think I can summarize our conversation to this point as follows:

Is this really intended to be a summary of the discussion so far, or is it a draft of a formal development process roughly derived from the discussion? It seems to be the latter. Furthermore "XP Processes" seem to have worked their way into the picture, and while I am a proponent of XP, I'm not sure it's sufficiently foundational to be considered a Principle. The same goes for IDE, and why have Algorithms and Data Structures left Principles and joined Development Environment? Are we still in the territory of NutsAndBolts software engineering foundations, or have we moved somewhere else? -- DaveVoorhis

Dave, you are right, I think I drifted from the intended summary to a process. I rearranged things because it seemed where the discussion was heading. Perhaps a better summary would have been:

And Ask, do the following belong in the list: -- WyattMatthews

Next question: (As a preemptive strike, Yes I know that LAX is Los Angeles' airport designation.)

Presume that SE is using a specific language (call it L).

Presume that SE is using a set of artifacts (Call it A).

"Best" and "should" are only meaningful in the context of metrics, otherwise they are purely subjective and therefore questionable. -- DaveVoorhis

(Man, I think I need to Refactor).

Yes, I guess, SoftwareEngineering is not considered mature until we see the equivalent of standardized nuts listed at the top of this page ;-)

[there are so many standardized components and tools] hmmm, many surely, but standardized? I'd consider RFCs to be standards and languages are also standardized, IEEE does its job as expected. Let me annotate the table:

But maybe you mean de-facto standards as opposed to de-jure. If a large company is behind it (MS, IBM, Sun), whatever it is, it will quickly be "standard".

Most of what you list that were not on previous lists are specifically Representational or Execution artifacts. Perhaps we can summarize your list as follows: What I am looking for is to place the categories for Language Specific Principles, Methodologies, IDE Artifacts, User Artifacts, and Other off of SE's shoulders unless they apply to X.

If you do not like my summary, please feel free to comment.

Again, If SE's specialty is some ProblemDomain 'X', he should not be responsible for having to develop Non-X equipment. Does the bolt-maker also press out the bolt-maker's parts or only the bolt? If X is a VM, then yes, SE would be responsible for a JVM, but if X is accounting, then why should SE be responsible for the design and construction of the VM, Interpreter, and/or Compiler? That's too much knowledge/work for any one person.

I would think that would be the Pattern, Algorithm, Data Structure - not including their language based implementations. At the individual level, we need to decouple all the problem domains from the main body of the definition of who SE is. SE needs a common system of development notation (the three mentioned above) and then possibly in their specialization, X, more specific data. Do I really need to include in the discipline's definition all X, Some X, or No X? I think the discipline should include very little direct data on X and concentrate on what is common to all who practice. X is not common to everyone. Take a look at medicine. Does every doctor know the general principles and all the specifics?

It is not my intention to reduce SoftwareEngineering to Pattern, Algorithm and Data Structure; but to explore these as tools of a SoftwareEngineer. I pushed the methodologies/practices off because I don't want to get bogged down into the politics of them.

I ask that someone else summarize this if I don't do so by the sixth. I am trying to objectively find a way of doing so that does not get into a politicized discussion. It may turn out to be that there is no easy way. I know that I am not trying to find out the body of knowledge itself, but beyond that, I am not certain.

See also: IsComputerScience

View edit of December 4, 2005 or FindPage with title or text search