Different groups had completely different results. Some groups consisted entirely of experienced people (some of whom were grad students) and did very well. Some had experienced people who fought like cats and dogs. Others had little domain experience and floundered. Others had little domain experience but had a good time. Some people thought moderators were great, others thought they were distracting. This makes it hard to see how to do better next time.
Some things are clear; people thought we wasted too much time on process, wish things were better defined, thought there wasn't enough time, and need more space and quiet for working.
Some of the things people want are impossible. We can't impose a single design method on everybody, though it might be possible to sort people by the methods they prefer to use. We can't both provide a setting for learning from people with different backgrounds and them make sure that everybody has compatible backgrounds. Even if we go to one day, DesignFest will still be high-pressured and you won't be able to come up with a polished design.
Other things seem reasonable, but experienced people tell me that they don't work. For example, I've been told that it doesn't really help to provide OOA in advance, because groups usually just redo it. Maybe they are wrong, but several people have told me that.
Many people thought that the best part of DesignFest was seeing how other people thought, and that if people prepared in advance then you saw their solution, but you didn't see their thought processes. Some of the people who felt strongest about not preparing were from groups where someone came in with a ready-made solution. It is clear that it is disruptive to have someone who has prepared and someone who hasn't. I figured that people wouldn't be able to prepare very much in the two days before DesignFest, and I only expected people to read the problem and be ready to start work on it. A few people surprised me.
It seems like half the people thought that it would be good to prepare in advance. Perhaps next year we can have two categories, "prepared" and "spontaneous". They would result in entirely different events. It is clear that different people have different goals for DesignFest. Is the goal to learn a lot during the experience, or to come up with a great design? It is clear that these are different goals, and it is not clear that we can achieve both of them with the same event. They are both fine goals, and I was originally hoping to do both, but I am starting to wonder if we have to choose one.
I figured that people who preferred to have DesignFest during the conference days (Tuesday-Thursday) would also prefer half-days, but there didn't seem to be a correlation. I find it hard to believe that people would want to miss a full day of the conference, but perhaps that is true. We will definitely try to have full-day sessions next year, but we might have half-day sessions on a conference day, too.
Several people sent me further comments. Feel free to send comments to me by e-mail. Also, feel free to send comments to the folks who made up the problems. They are:
Doug Schmidt: schmidt@cs.wustl.edu Doug Lea: dl@altair.cs.oswego.edu Desmond D'Souza: Desmond_F._DSouza@iconcomp.com LarryBest: Larry_Best@mail.amsinc.com Ken Auer: kauer@ksccary.com