Please attack/defend/discuss the following proposition (yes, a ThreadMode
page about threading):
- Threads are just too difficult for normal, real-world programmers. They introduce too many opportunities for developers to introduce obscure bugs which are hard to reproduce, hard to find, and hard to fix. It requires an expert programmer to work with them safely. Therefore, "normal" developers (those not developing servers or application frameworks) should be discouraged or prohibited from using threads.
What you need are some SynchronizationStrategies
. What you don't need is JavaLanguage
? Please explain.)
Java 1.5 has added additional synchronization primitives and additional JavaCollectionClasses? which implement various BlockingQueue? strategies. -- RobertField
Here's an attempt at summarizing this page. Please don't add discussion here, put it later and add to this summary only if there's something genuinely missing.
- In favour of threads:
- Threads (and concurrency) can be indispensable for some applications.
- Kernel threads can be scheduled on multiple CPUs of a multiprocessor system (however, introducing multithreading for this purpose may well be PrematureOptimization unless you can show your app needs it).
- Good threaded code is wonderful. Why? Sequential code is often better because it is easier to understand and modify. A: It depends on the situation. For some problems, there is a natural concurrent solution, and the purely sequential version is awkward.
- Some sample scenarios would be appreciated.
- Some tasks may be truly independent; having independent simultaneous flows of control is useful.
- But: Separate processes may be a better solution.
- On some OS's (ie Windows) that is much more expensive than separate threads (on Unix derivatives, separate processes are much cheaper).
- Points against threads:
- It's very, very easy to get threads wrong:
- have subtle, hard-to-find bugs (DeadLock, RaceConditions)
- SharedStateConcurrency is hard
- Almost impossible to debug or otherwise prove correct.
- get worse performance than expected by an order of magnitude
- Not all threading is done for performance reasons
- Using threads for performance reasons is an AntiPattern.
- At least when done without significant profiling which identifies serialization as a bottleneck.
- Good threaded code seems especially difficult for inexperienced programmers
- Using the database to manage concurrency of state is often easier and safer and more consistent across languages.
- Some languages and OSs seem to go out of their way to make it hard to get threads right, even when you do know what you're doing.
By telling that threads are harmful, you're also telling that multicore CPU are harmful.
However, they have been built though, and they certainly improve the performance of a PC or server a lot. So, the old serialised times are gone, and that's exactly the reason why we MUST program with threads nowadays. Otherwise you use only 1 CPU and your performance won't be improved at all.
Anyhow, deadlocks are no problem in Java if you use (guarded) synchronized blocks and encapsulate the synchronisation commands into each other (like "synchronise(a), then synchronize(b), then do what you need and finish it!). Anyway, hash tables and arrays don't need synchronisation for changements, only for addition or removal of elements, so we don't have performance problems here, still using arrays isn't really great for it (since you need to block the hole array for a search, blocking the access of elements for other threads!). At best you could minimize the synchronisation efforts in Java by using hash tables. Do you know what? Hash tables and threads should be learned in every beginners class, because altogether they are nothing complicated at all! They are like a team, like man or woman needing each other. The same as good hardware needs good software to be useful, and good software need good programmers using threads. Cheers, Mark Weiler
The problem isn't that the idea of threads is bad (though a lot of programmers have gotten the wrong idea of threads from the available implementations)...The problem is that the implementations provided by most of the existing OSes are inefficient, and the primitives provided by those OSes to manage the interactions between threads (a) are many and varied, (b) are complicated, (c) are asynchronous, and (d) make you explicitly do both communication and synchronization in all the right places. I have no doubt that there is a better way. :) See ThreadsAreComputationalTasks and SendReceiveReply.
-- ShamelessPromoterOfSynchronousSystems? (er... WylieGarvin)
I have seen this premise begin to take root several places: the policies of some development shops, the design of EnterpriseJavaBeans
specifically prohibits the use of threading within a bean, and some web application servers (like WebLogic
) discourage it as well. They say "We handle the threading... you just write your code single-threaded or multi-threadable (tell us which), and our framework will schedule it all. You must never start a thread of your own, nor perform wait() or sleep() calls which affect thread scheduling."
Attacking the premise:
Threading is a powerful tool, which allows us to do things which were extremely difficult without it (you might even say impossible). I once built an EnterpriseJavaBean
with a synchronous interface (you called the function doTheWork(), and it returned an Answer object). Behind the scenes, it needed to call an asynchronous module (it needed to call startTheWork(), which returned a Key, then later call getTheAnswer(key) which returned an Answer).
To do this, you need threads. Well, you could maintain a list of active Keys and run a tight loop which called getTheAnswer(key) with a timeout repeatedly, interspersed with calls to startTheWork() for new requests coming in. But essentially what you're doing is implementing your own threading... which is one of the FiveSignsAprojectIsSpiralingOutOfControl?
. Besides, because of details of java's IO model, calling getTheAnswer(key) blocked until an Answer was ready.
So what I did was to violate the specifications for an EnterpriseJavaBean
. The specifications explicitly say that you can't launch a new thread or execute any code which affects thread synchronization. I did it anyhow, and fortunately it worked. But can't we have something better? I admit that writing robust threaded programs can be quite difficult -- that even experts often err -- but to take the tool away seems too extreme a reaction. -- MichaelChermside
Defending the premise:
Have you ever worked with a beginning developer - one just out of school - on a project involving threading? If so, I rest my case.
That's the best time to get a beginning developer into threading - while they're fresh, and before they learn a bunch of prejudices and bad habits that they'll have to unlearn later!
Have you ever worked with a beginning developer using a linked list? A similar set of problems arises. This is more a trait of new developers than the technologies they use.
Yeah, I wrote the thread library and taught all the experienced programmers how not to deadlock the system. What's your point? Old people are dumb? I cautiously disagree.
Are we on the same wavelength here? I am saying that new developers tend to have problems with things that experienced developers do not, and thus stating that threads are bad because new developers cannot use them correctly is poor logic, because new developers also make mistakes using other constructs that we know are good and useful.
Are the limitations on threads in EnterpriseJavaBeans
not also there to make EJB implementations easier for the application server vendors?
Yes, but that's just too bad. Lots of things about writing an EJB container are hard. You have to support lots of complicated transaction handling. You have to handle remote invocation. The intent of the whole EJB thing is to set standards that container vendors must meet so that the beans will be portable across servers. There is no reason to impose unnecessary limits on the bean authors. No one suggested it would be easy to write the containers, and this is something that the container writers CAN handle.
Threads are necessary in complex systems for many reasons. If you haven't used them, young or old, doesn't matter, it's difficult. The problem is having threads exposed to developers. Java having suspend/resume available is almost criminal. Place a message passing agent model on top of threads and deadlock becomes very rare and the usability of the system becomes high. -- AnonymousFool?
Java deprecated the suspend and resume methods a long time ago.
Like any other tool, threading is a good thing to have around in case
you meet a task it's suited to. It's a way to sacrifice a bit of efficiency for clarity.
I'd say ThreadOveruse?
is a bad thing, though - many processes running in the same address space can easily abuse it. Threads should be chosen judiciously, not thrown into a program willy-nilly.
Someone who programs in ErlangLanguage might disagree. Threads are cheap; why not have plenty? Just write your programs in an almost-functional style and you hardly ever need any locking. -- StephanHouben
That is almost correct. In Erlang, threads are not only cheap, they are very simple and safe because they are CommunicatingSequentialProcesses
, and are completely isolated from each other. So much so that we use them not only when we need concurrency for performance reasons, but simply as a way of modeling the domain. Threads (confusingly called processes in Erlang) are Erlang's modeling equivalent to objects. What is incorrect in your statement is that the functional style is what avoids the need for locking. In fact, Erlang concurrency and asynchronous messaging is the part of the language that is not functional. Only the sequential parts of the code are in a FunctionalProgramming
paradigm. The thing that makes locking unnecessary is that Erlang processes (i.e. threads) are isolated, and cannot share data -- DominicWilliams
That's called LockFreeSynchronization. It is a cousin of AsynchronousProgramming. -- GuillermoSchwarz
No, it's MessagePassingConcurrency
is something entirely different.
The criticism may be specific to C/++ threads, in that they share an address space.
So do Erlang threads. I believe the commonly accepted definition of "thread" (as opposed to "process") includes the notion that address space is shared. -- sh
All right - now I'm confused. At erlang.org, the FAQ, the tutorial on concurrent programming, and the 'Programming Rules and Conventions' depict 'lightweight processes' which seem to converse through message passing; no obvious mention is made of shared memory or threads. My second comment, above, assumes these processes are what you mean by 'threads,' but the definitions you assert contradict this. What are Erlang threads? Are they the same as Erlang processes? Do they duplicate the C/++ hazards of memory corruption and of one segfaulting thread taking down an entire process?
Well, in Erlang the difference between threads and processes is (intentionally) fuzzy, since Erlang doesn't have assignment. This means that there is no way for the programmer to find out if the address space is shared or not. The system can thus figure out itself whether it should use threads or processes. And of course, Erlang is a safe language, so there aren't any segfaults. -- sh
No, the above statement is incorrect. Erlang processes are extremely lightweight threads managed by the Erlang runtime (virtual machine). A single Erlang runtime is called a node, and is seen by the host operating system as a single process. In that sense, all Erlang process share an address space. However, from a programming perspective, an Erlang process is completely isolated: it can only communicate with other processes by sending and receiving asynchronous messages in which data can only be passed by value. -- DominicWilliams
At this time, given the current lack of maturity of our tools, techniques and libraries, threads are difficult and dangerous to use.
But I think we, as the computing industry, need to find a way to make threads >not< harmful.
To better serve the needs of our users, I think our applications should work more like browsers:
Display the information you have now, and allow the user to interact with it.
Don't lock up the user interface waiting for every last bit of data and graphics to trickle down through the network interface;
offer the maximum possible functionality at each moment of the loading process.
Unfortunately, current tools and libraries don't seem to offer the abstractions necessary to make this practical.
[I explore this idea in more detail in MultiThreadedGuiWouldBeGreat
Threads are only harmful if they actual share data/resources. If you reduce the amount of data you need to share between your threads, perhaps by making a copy for each thread, you can go a long way towards making thread safe programs. Once you've done that you can isolate the bits of your program that do need to share data between threads and make sure they're correctly synchronized.
Try hard to avoid statics and singletons.
Threads wreck performance. They context switch, which ruins the cache and the pipelining. On the very same multiple processor machines they were supposedly made to take advantage of, they cause cache coherency chatter beyond control in many cases. You wait for locks. so what you're saying is that, for programs where threading reduces complexity, serialization (removing threads) is a PrematureOptimization?
You lock too much. You don't lock enough. The program crashes. You try to debug the program. Debugging a threaded program has a lot in common with medieval torture methods, IMHO.
I work in HPC (high performance computing), with very large vector supercomputing facilities, and I will tell you the ugly truth: only Wu-Tang and Shaolin monks can do thread in all confidence, if they have the Tao in them. Isn't Shaolin actually Ch'an (Zen)?
added multithreading to Quake III: Arena (Q3A), a recent OpenGL first-person shooter game. He's widely recognized as one of the best programmer in this industry, without equal at squeezing features and speed out of his graphics engine. When he added a second thread to Q3A to take advantage of his dual processor machine, performance dropped by 50%
!!! This is from memory, but the number is around that. Wasn't it supposed to double, or at least increase in some manner? Yes, of course. He worked at it a lot, and while he managed to get some kind of improvement out of this (not
close to twice as fast), the things to remember was:
- avoid threads if possible
- if you have to have threads, then have only one per CPU
- avoid threads if possible
- share as little data as possible between threads
- are you sure a separate process with shared memory or other IPC wouldn't do?
Don't make the mistake of assuming that Carmack is super human. He can mess up anything just like anyone else. In fact, if you dig in the various archives of the internet, you can easily find communication from him in a time when he knew very little. IMHO, what's super human about Carmack is he drives himself at a pace that only a handful of programmers in the world could match. Anyhow, threading in a game is a poor example, as there's little independent work to be done. Threading has been successfully used to implement background loading and such, but as far as threading the main simulation or the graphics core, it simply isn't worth it. This doesn't generalize to all possible applications however; it doesn't somehow mean that your httpd should be single thread!. -jpw
- Do you mean AvoidThreadsForOptimizations?
Not sharing data between threads is very hard to achieve, because you "share" data without really knowing, because two unrelated things are in the same memory page, which cause a cache flush for coherency, and messages to go from one CPU to the others. Maybe a specialized allocator may help there.
Here is a link to more information about multithreading: http://www3.sympatico.ca/pphaneuf/thread.html
But all is not lost! Non-preemptive threads (also known as "fibers" in the Win32 API, see FibersExplanation?
) can give you the same "feel". "Fibers" sound a lot like CoRoutines - same thing?
Not at all - Win32 Fibers are just threads scheduled in user space, that don't get preempted by kernel code to give running time to other fibers in the same process. They're meant only for porting apps from Unix systems that have similar threading models.
comment that as an industry, we needed to make threads not harmful, as to better serve the needs of our users. What does threads have to do with the needs of our users? On a single CPU, threads only give the illusion
of things happening at the same time, an illusion that good olde event-driven programming can do just as well (as always, you have to KnowWhatYoureDoing?
). Good event-driven programming is hard? Well, debug a complex multithreaded program and come back to tell us about it, okay?
Now, many libraries don't make it easy (i.e., they often don't have asynchronous interfaces). A famous example is the gethostbyname() call. As much of a Unix fan that I am, I find the Winsock asynchronous interface to the resolver very nice. But they have an "enforced" event loop to maintain (the Win32 message pump), so it's easier to implement everywhere. It would be quite feasible to make a resolver object for libXt for example, where you can hook up sockets and timers to the event loop (it has probably already been done).
So write an async version of gethostbyname(). I'm not being facetious, that's what I did last time this came up some years back. I've also fiddled with the X11 socket, but of course that's inherently non-portable (not all X implementations use sockets; shared memory is also common).
I would reply to GlenStampoultzis
with a question of my own: why use threads at all if you isolate the parts of your program properly? Processes with message passing could do just as well, no?
(If you had a software application where you needed to upload FTP files while you were still working in the application..you'd want to utilize a thread so that your FTP uploading wouldn't rob all your CPU power and halt the application for a few seconds/minutes. Of course, you could
fire up a separate FTP process but
- The process will be slower to fire up than a thread within the app. In a speed critical application you may not want to load a completely separate process, because maybe that process will take 2 seconds to load and initialize, instead of say 0.2 seconds for a thread to load immediately within the app. (Maybe in Unix it's different?)
- You couldn't as easily access a separate process with your code...
- i.e. set parent of a dialog window that popped up from a thread, kill the thread when the parent application closes (you could kill a process, but it would require a lot more code,and your objects/events in the separate process wouldn't be accessible easily, without complex message coding).
- Utilizing event driven programming is tough between processes. Events can't really easily be accessed/controlled across processes, but really easy if within the same process.
- Say you wanted to immediately upload files to the web. You want (CTRL-S) similar response and speed. Uploading 40 files while still working within the application... Then say you decided to upload more files while those 40 were uploading. With messages and separate processes, that gets into complex cross-process message coding, and each new process might annoyingly take 2 seconds to load versus 0.5 for a thread. (p.s. is unix different? thread actually take a longer time to load than a process?).
- Events again: If the user killed the application, it'd be easy to kill all the threads and deinitialize/free up things within the same process. But across a separate process, that's going to be some complex message coding.
- The objects/references/strings/data etc. that you are trying to manage are already within your process and at your fingertips, easily accessible with your programming language. Sending messages usually involves moving away from your programming language and resorting to (in win32) sendmessage syntax, writing procedures functions to decode the messages, etc. (see also below.. unless a wrapper event system was written for cross process?)
- Processes can waste a lot of memory, especially if the thread is just performing a simple, temporary, but processor intense task. In win-32 each process will have to load up several of the same duplicate modules in the memory. For example if all processes rely on kernel32.dll and several other DLLs (even simple small applications rely on several DLL's, it's going to have to be loaded with each new process. With a thread, a process wouldn't have to fire up all these modules (maybe different in unix?).
Or take another example
[JeffGrigg expanded his comments on "a MultiThreadedGuiWouldBeGreat." Non-preemptive threads (Win32 "fibers"), along with good GUI tool support, would meet the needs described there.]
- Your web server has a program on it which hogs a lot of CPU power. You don't want 50 web visitors at once to fire up a new process because after 50 processes are open, your computer runs out of memory and the server does not respond. If you utilized threads, maybe you could have squeezed 100 threads versus 50 processes? I don't know the numbers... just playing random guessing games. So maybe hardware costs would go down significantly on a large web server.
- FTP download within a web browser, example: more control and easier to manage code through objects. If the FTP downloader is within the same process as the browser: say a user kills the browser with other processes open, how do you access and manage the process easily, and shut it down and control the deinitialization within the other process, safe file saving, etc. within the outer process FTP program? With a thread, it's like an object and you can access it and manage it easily without writing complex message code. Maybe we just need to write wrappers for sending messages across processes? Create events across processes instead of hooks and etc.? Maybe someone just hasn't thought about controlling processes like objects? )
Don't mistake me for someone who uses threads everywhere. Best to avoid them if possible. Try writing a scalable servlet that avoids threads however. Message passing to separate processes can do the job. It has the advantage of forcing you to tightly define the communication channels you're using. Managing processes and message passing involves a little more work to also. I could also see problems if the amount of data shared between processes needs to be large. Perhaps we need a new threading model that can give us the benefits of both worlds. -- GlenStampoultzis
solution is to spawn off a separate process that communicates with other processes only through the database. Databases already take care of most semaphore-like and/or multi-user transaction issues. It is like client-server where you spin off another temporary client-let. (I can't say this works for embedded systems or systems in need of predictable timing, however, due to the unpredictability of garbage collection schemes.)
One way to use pre-emptive threads to advantage and without pain is a MultiCaster
. Another way is OrganicThreads
. -- PeterMerel
As a bit of an aside, is there any particular good reason a VirtualMachine
should croak on a couple hundred threads, or would it be something that generally just isn't done? I happen to like the determinableness/testability of code written using blocks and safe threading objects which invoke them for me: but apparently I got a bit trigger happy or something... I seem to fall off a cliff of some variety at around 500 threads or so, but I can't seem to track any particular resources spiking other than cpu time... ah, is it the context switches getting me? Or perhaps some too-smart implementation deep within the thread code with O(n^2) complexity or something... Either way, too much blogging, not enough testing/experimenting. (And sleep, as the case may be.) -- cwillu
What is the failure mode? Does the VM crash, slow down or lock up? My experience is that context switches can be expensive.
500 threads seems a bit small, but benchmarks comparing ErlangLanguage
with other languages have shown that a lot of systems die given a few thousand threads. But (here comes that SmugErlangWeenie
), Erlang does fine with tens of thousands of threads (OK, processes), and maybe more. -- BillTrost
In real-time systems context switches are not expensive. Only in heavy OSs are they expensive.
After using threads with both C (scary) and Java (still problematic) the only real way I could see myself coding threads is by having completely separate contexts and by message passing. For instance, the model used for threading with the ToolCommandLanguage
. You can always only have one thread per interpreter. Messages can be read when the interpreter enters its own event loop. Effectively they are kind of light-weight processes with message passing abilities. Thus there is very little chance to get stuck with the usual thread nonsense. Even dynamically loadable extensions tend to work out of the box, assuming they attach any shared information to the interpreter instead of globals (which they should be doing anyway). -- Setok
This page confuses me, since I use threads all the time and don't know any alternatives. How do you write a server that handles concurrent requests?
See the apache 1 source code for unix like operating systems. They use processes.
How do you keep the UI active while doing something that takes a few seconds? How can you time out a remote function call if it doesn't respond after a specified duration? Creating a separate process and dealing with interprocess communication is much more of a kludge than writing multi-threaded code, at least in Java.
Traditionally you use asynchronous logic. Apps decompose themselves into state machines that are driven by async requests and responses that drive the state machine forward. As you usually have only have one CPU there's really no doing things at the same time anyway. You can have as many simultaneous activities has you can have overlapping state machines. An app must take no more time than the greatest desired latency. This is a form of cooperative multitasking. Though in a low-latency priority based system you always have cooperative multi-tasking. You can have code that shouldn't take a long time or shouldn't take a lock for a long time.
I've done that. I called it "writing my own thread management". It still uses threads, they just have to cooperate.
An approach like that would be plausible if threads didn't exist. But in a language like Java where threads are built-in, why not use them?
"As you usually have only have one CPU there's really no doing things at the same time anyway."
Wow. That's dated. Even phones have multiple cores now.
I don't see why you couldn't remove the word "thread" in the original proposition and replace it with "C++". The argument is the same, or at least, very similar.
Those in favour will argue that threads/C++ make coding for mere mortals near impossible.
Those against will claim that people need to know what they are using in order to do effectively.
At the end of the day, I would argue it's about picking the right tool for the job and the team, and most of all, about the software being GoodEnough
I have found that people don't understand even very simple threading code, largely because they can't keep it all in their heads at once. The idea that in just one instruction a different thread can be executing doesn't register. This is the problem behind the entire java synchronization approach. The proposition that you can replace "thread" with "C++" is false, though because you can write C++ code that is understandable, and though C++ is difficult, it is easier than threading.
Threads wreck performance.
Scheduling latency is very OS specific. It doesn't have to be large. If it is it is the OSs fault. Yet still we must consider this latency for the target system when creating an application. Lots of locks is an application issue. Java makes lots of locks all too easy though. Then there's the length of locks. People probably see low performance from threads because their locking and app architecture aren't designed towards parallelism.
As always with performance issues, until there's evidence of performance problems, DontOptimizeYet?
. Even once there is, consider simpler architectural fixes before moving to an entirely single-threaded solution. You should use threads if it makes your code simpler, and optimize when your code is too slow.
I somewhat agree. It's part of engineering to validate your assumptions and requirements. If your requirements aren't ambitious for your platform then don't worry about it. But, if you think your requirements could not be met and the penalty for that is high enough, then as an engineer you must do some up front work. If you must handle Z requests a second and you know a context switch averages X usecs and your architecture requires Y number of context switches per request then and that turns out not to meet your requirements then you need to do something about it. It's not premature to know what you are doing.
Actually, one of the problems I often see with using threads is that people don't abstract the objects. They end up scattering POSIX thread calls across their application code. They then can't use idioms like stack-objects to get mutex locks, and hence spend loads of time trying to track down obscure interference issues. Bit of a whinge in general really... lack of ability to close the gap between the language used and the application domain.
The only real grief I've ever had with exception handling is that the recent versions of GnuCpp
don't like exceptions and threading. The thread which THROW's isn't necessarily the one that CATCHes... *sigh* -- KatieLucas
"having completely separate contexts and by message passing" - yes!
"the gap between the language used and the application domain" - you put your finger on a problem that I felt was a root cause of a lot of the problems we had with application development over the years. Some time ago I sort of accidentally stumbled on the concepts underlying FlowBasedProgramming
(FBP), which has completely separate contexts and message passing over BoundedBuffer
s. It also seems to result in components that have a good match with the kind of words we use in business applications, e.g. merge, update, summarize, report, etc. These functions are hard to build as reusable components in a single-threading environment. For a very simple example, see http://www.jpaulmorrison.com/cgi-bin/wiki.pl?TelegramProblem
I'm also perplexed by the mention above of "thousands of threads" - some extremely complex banking applications that we built using FBP software did not exceed about 60 threads/components. These apps were written about 30 years ago, and have consistently performed well (and accurately!) since that time.
If you have 5 apps talking to a 100 nodes and you have a thread per concurrent activity that's 500 threads. It can get a lot larger obviously. There's no reason not to have a lot of threads if the threading system is well built.
There was a recent article in DrDobbs?
called No Free Lunch. This article points out that Moore's law has allowed us to write software now, and assume that it will continue to run faster on newer hardware for some time to come (that's the free lunch), but Moore's law has recently ceased to apply when measured in purely ClockSpeed?
terms. If CPU speeds today were following the trends they were following a few years back, we should have 10GHz processors by now, and notice that we don't. Also notice how big the cooling systems have become and now much the power requirements have risen just to support the current 2.x GHz processors.
Future improvements in CPU performance will have to come by adding more cores to the die more so than by increasing the clock speed of each core, and that means that for a single program to take advantage of performance improvements in newer processors, it will have to be able to seamlessly scale up to utilize ever increasing numbers of CPU cores. Whether that means writing multi-threaded apps, or something else, I don't know, but multi-threading is the most obvious approach. I guess another way would be the FiniteStateMachine
approach, like StatiCee
uses. Any others?
First class processes would be another.
CategoryEvil CategoryConcurrency CategoryConsideredHarmful