Monday, October 27, 2008

On System Design

In an essay entitled "On System Design", Jim Waldo, a distinguished engineer with Sun Microsystems, expresses his ideas and opinions on system design, including factors that hinder the effort necessary to do system design.

Some of the techniques that aid in the process of good system design are Six Sigma, or UML. The latter helps in the development of good design, but it is you (as an individual or a team) that has to come up with the design. UML helps you in expressing that design better. Many approaches have been found to work, but there is no general approach that works on all projects. Depending on the project, you have to decide which approach better suites your needs. The only exception to this is given by Brooks, namely that good designs have one thing in common, that is they are produced by good designers. Jim Waldo explained this in terms of probability/statistics: those who have been able to produce a good design in the past are far more likely to produce a good design in the future. Furthermore, designers have to be bale to deal with ambiguity while creating the design, and to know that the question that cannot be answered right away, will be eventually answered at the appropriate time. The author mentions that instead of learning a process of design, it is better to learn how to best design. An indicative to the best way to learn how to design includes two factors:

1. Trying, failing, and then trying again.
2. Being under the guidance of someone who is an expert at the task.

It si the way of thinking that matters most in system design, rather than the subject that can be thought in a particular course. Interesting enough, the author mentions that System Design cannot be learned in one course, but should rather be though in a much larger process, similar to an apprenticeship. The relationship needed between a master and its apprentice is found at the graduate level, where each student has an adviser with whom he is involved in academic research. One other place is at work (in industry) where you get hands-on insight on system design. I would argue that at work, you are more constrained in terms of time; hence, there is less opportunity in learning system design.

Diversity in the sense of academic backgrounds (physics, philosophy, anthropology, computer science, music, etc), and how such a diversity creates different views on a given problem, is described as being beneficial. I would particularize this to software development in the sense of not trying to think of a solution for a problem in terms of a particular programming language, but instead design the solution independent of the software platform and only afterward think of what language can help me express this design the best (similar to the MDA concepts).

The problem with doing design up front is the fact that no immediate code is produced. The pressure to produce something that can be seen, or the measure of productivity by means of the number of lines of code written, still exists. What is needed is "an act of faith by management". He continues by saying that "if you find a manager who is actually willing to give you time to do the design task, stick with him or her. He or she is a treasure much rarer than gold."

The paper also mentions intellectual property (as a note on why system design is not shared among peers anymore), and how any patent has the same value as any other patent. For me this is unbelievable; it is equivalent to saying that ideas that are behind the pattern are the same, namely have the same value. The quality of an idea is not taken into consideration at all by the patent system. Furthermore, in case of a patent litigation, the jury is made up of ordinary (non-technical) people, not technical peers. Hence, "it is very hard to learn about good system design unless you can see and study other system designs, both good and bad. The intellectual property atmosphere in industry has limited the number of designs that are actually talked about, and has convinced many system designers that they should not even look at the designs that are available."

Ways of solving the lack of system design from academia and industry are given in forms of agile methodologies and open source projects, because they enhance the discussion of a system's design in several ways: the communication between two (or a small group) of programmers can help in clarifying and better understanding the design of a system; furthermore, communicating the design to other members of an open source project will help clarify the design of a system. It is interesting to note that the author provides a different view to both agile methodologies and open source projects, namely them being a reaction to the impossibility of doing system design in academia and in the industry. Overall, the paper makes for an inspiring reading. You find yourself nodding and agreeing with the statements made while reading the essay.

No comments: