-By Arun Batchu
Courage: Sitting in the audience exactly one week after the September 11th attacks, I realized how lucky we were to have Linda Rising fly out here in these tumultuous times and talk about patterns and impart some of her knowledge to improve the lives of the small yet dedicated group of object enthusiasts. In spite of the uncertainty of the terrorist situation and the fear still lingering in the air; in spite of the unheard-of air traffic delays, Linda’s love for patterns prevailed over her fear and she made it to the twin cities – one of only 9 in a huge Boeing 727 aircraft. Looking at her slender figure yet resilient energy, she was the embodiment of freedom, progress and indomitable spirit – Lady Liberty herself. She was so energetic – the events seemed to have only strengthened her and given her infinite energy in spite of the grueling itinerary and the morning lecture at West Group. Her husband is correct – she can really talk-lucky for us indeed!
Colors for Feedback: Before she started her talk, Linda explained the meaning of the tri-color index card set that was provided to everybody. Every member of the audience was supposed to show one of these stoplight colors: red for disagreement, green for agreement, yellow for confusion. This little mechanism worked very well to elicit audience responses for various questions.
Best Practices: Linda’s talk was about best practices that included agile processes, patterns and project retrospectives. She started by talking about lightweight processes. She refers us to Martin Fowler’s paper where he summarizes different lightweight or agile processes. This paper can be found at http://www.martinfowler.com/articles/newMethodology.html. Linda mentions that all the agile processes have some common themes. These themes include having small teams, doing incremental development, time-boxed scheduling and having adaptive processes. Complex projects like writing software for Boeing 777 always involve hundreds of software developers. Linda questions whether those huge numbers of resources is really required. Linda believes that if we learned how to build small yet effective teams then we may not need so many developers. She mentions eXtreme Programming (XP) as one of the better-known agile process. But no one in the audience seemed to be using Scrum as an alternate lightweight process. Linda attributes this lack of awareness to the eXtreme Programming camp garnering all the attention with their vocal support and prolific writing in the form of a series of books. Linda jokes about the amount of literature that is being written about a lightweight process. If it is so light, she jokingly asks, then why do we need so many books? The audience, at this remark, burst out in laughter. She had a point. Linda, however, agrees with Kent Beck who says that humans in general do not do a good job of explaining things, especially things that do work. It seems it is easy to point out why things fail but very difficult to nail down reasons of success. That is why books, attempts to explain things that seem to work well, need to be written. Being a prolific writer herself, Linda realizes the need very well. Talking about all the different versions of agile processes that exist today, she likens the scenario to those golden days of yore when a fierce debate of “what is an object anyway?” raged among object gurus. We are at a similar junction today where what exactly is an agile process is in flux and debatable.
Scrumming: The reason Scrum is not so widely practiced as XP is due to general lack of literature about it. Linda went on to explain how she and her team discovered Scrum and went on to adopt it. When the 1996 Telecom Act got passed, it opened up the telecom industry and caused major upheaval after 60+ years of dormancy. People described the industry as chaotic. Things were changing very fast . Heavyweight models like CMM did not seem to work. AG communication systems (AGCS), a subsidiary of Lucent Technologies (where Linda worked at the time) started doing retrospectives on successful projects. Linda and her team discovered that they started seeing some common themes of success – patterns of success. They also analyzed failures and detected that the patterns that they detected in successful projects were absent. At this time they stumbled upon some literature about Scrum. They realized that Scrum closely matched the successful processes that they had identified. They adopted it – their process then had a name and AGCS started encouraging people to adopt best practices in Scrum. Scrumming became a common word. The word Scrum is adopted from the game of Rugby. Rugby involves a team having to work very closely with each other. Patterns are the way teams work. Patterns are the way teams address problems. Patterns are the way of doing a job better. Scrum teams comprise 10 or less people. Fred Brooks the famous author of “Mythical Manmonth” mentions that teams of 10 are optimal. Most XP teams are about 10. There seems that experiential evidence indicates that 10 is a good number for team strength. People, just as in the game of rugby, participate in sprints. The result of every sprint that lasts 1-4 weeks is a useful increment. Each sprint is a time-boxed. Linda mentions that in our software development world, it is all too common to see customers who do not know the requirements upfront. Linda explains this situation by pointing out that the markets these days change very rapidly and the business requirements change along with them. She mentioned an AGCS customer who owned up about not knowing what exactly was required but whatever it was, it was needed by June that year! Linda’s prompt response of ‘We can do that ‘ drew a round of applause from the now enthralled audience. Users in general have fuzzy requirements and it is ok. We can all learn together. Linda prefers this evolution of a system to a system based on rigid but incomplete and outdated requirements that are specified upfront.
Sprints in Scrum are the important periods when actual work gets done. Utmost care is taken to avoid unwanted diversion. There can be no interruptions from marketing or any other non-development agent. In a sprint the entire team gives everything that they have got and hence it is very important that the finish line (scope) never moves.
Frequent, short scrum meetings of around 15-30 minutes are held. This is to catch problems very early so as to not impact the increment delivery date. A ScrumMaster facilitates these meetings. Each attendee is expected to answer three questions : “What have you done?”, “Are there any problems?” and “What will you do between now and the next meeting?” Linda drove home the point by posing the question “How does a project get six months late?” and Fred Brooks’ famous answer: “one day at a time”.
At the end of a Sprint, there is a status meeting with all stakeholders. The health of the project is evaluated. Projects can be even cancelled at this time. New estimates and responsibilities are made. No amount of data, Linda says, can allow guaranteed estimates. However, if we have the same small team, the estimates could be fairly accurate. On-time delivery of increments develops relationships with customers. Trust builds and knowledge grows. One can find more information on Scrum at http://www.controlchaos.com and http://www.jeffsutherland.org/scrum/index.html .
Patterns: After Scrum, Linda talked about patterns. Patterns became very well known in the object oriented (OO) world in 1994 when the now famous ‘Gang of Four’ (GOF) book by Richard Helms, John Vlissides, Ralph Johnson and Erich Gamma was written and released. The best-selling book that documented 23 fundamental design patterns proved a catalyst to the software development community. Now there are many more books and over a thousand patterns. Realizing the need to prevent pattern duplication and provide a taxonomy, Linda wrote “The Pattern Almanac” in 2000.
Linda considers patterns as good, reusable, abstract ideas. She uses Christopher Alexander’s definition to describe it: “Each pattern describes a problem that occurs over and over again in our environment and then describes the core of the solution to that problem in such a way that you can use the solution a million times over without ever doing in the same way twice”. Christopher Alexander is the building architect who first documented patterns to solve problems in building architecture.
Linda does not think that code-libraries work. Code libraries are too implementation specific. First of all, there is the problem of finding a module that would satisfy our requirement. If such a module is indeed found, it may need modification. This may lead to an increase in what Linda calls entropy (a measure of disorder) of the system. Linda finds it more important that we reuse the idea. We need idea libraries and design libraries. That is what patterns do: capture good, reusable ideas.
Many good ideas and things have a quality that is very difficult to describe. We objectively and unambiguously realize the value and the quality in them, but are unable to name it. Christopher Alexander recognized the existence of such a quality and called it a Quality Without A Name (QWAN). Some ideas have such a QWAN. But things do not have to have this QWAN to be useful. Does the Gang of Four (GOF) book have that quality without a name? Linda does not care for the answer to this question because the book has 23 very good ideas that solve people’s problems and improves their lives.
Having laid the foundation of patterns and their usefulness, Linda went on to provide a few more definitions of patterns. Patterns capture best practices – they describe what works in a given context and generate an efficient language of communication. She warns us though that a pattern is not a silver bullet. Patterns do not solve the software crisis. To solve the so-called software crisis, we need paradigm shifts. Reminds me of Prof. Dave West’s lecture in July – we need paraphors (Paradigm Metaphors)! Patterns exhilarate.
Patterns, apart from providing solutions to known problems, generate a new language called a pattern language in a given domain. Nate Kirby describes this language as one that provides a very dense and efficient means of conveying information among those who know it. Jim Coplien thinks patterns help fill a crucial need in the software industry where human communication is a bottleneck.
Patterns help engineers, who are mired in details every day, to think and communicate in abstractions – design and architecture and not be bogged down by implementation details. Teams can understand their interfacing subsystems not only as black boxes but also as a system of patterns. Patterns provide a means for knowledge transfer from the experienced to the novices.
Patterns in software can be grouped into several categories: Architectural patterns, Design patterns, System Test patterns, Customer Interaction patterns, Organization and process patterns. Jim Coplien studied hyper-productive teams in order to extract patterns that could be documented. He found that important patterns were missing from unsuccessful teams.
At AGCS, patterns became a success story and resulted in a book called “The Patterns Handbook”. Patterns help promote a knowledge culture where it is important to capture, document, improve and share knowledge.
Restrospectives: At the end of every project, retrospective sessions can help discover patterns that work and those that don’t. Meetings with all the stakeholders at the end of an increment provide valuable checkups and a means of improving the process. Linda talks about process minstrels who are responsible for documenting and sharing knowledge across teams. These storytellers interact with teams, identify problems and suggest stories, anecdotes, and patterns to improve efficiency. Such process owners are very important. They conduct checkups and postmortems, document patterns and share results. They get involved with outside sources and bring in fresh ideas.
Norm Kerth is an acknowledged retrospectives guru. He has a book called Project Retrospectives where talks about how to do postmortems. A few questions that are important to be posed are: “What went well with the project”, “What should be done differently”,”What did you learn?”, “What surprised you?”, “What puzzles you?”. At AGCS, the results of the postmortems were posted on an internal web site and a summary document identified trends and ideas for patterns.
Talking about common sense, Linda laments that common sense is so uncommon! Common sense is difficult to explain and easy to forget. Patterns provide names to obvious things that worked; a body of documentation that makes it easier to remember. Patterns allow us to become a little better, a little smarter, one step at a time. Re-quoting F P Jones’s remark “Experience is a marvelous thing that helps you recognize a mistake when you make it again”, Linda reminds us that life is too short to learn from our own experience, we need to learn from other people and patterns and their language provide a wonderful medium to do that. Gary Klein in his book “Sources of Power” mentions that experts solve problems using patterns.
In Conclusion: Linda concluded her inspiring talk with a reference to a 16th Century Venetian painting in the National Museum of London that depicts a man with three heads. The present, mature head looks straight at us deciding what actions to take. On one side is the young head looking towards the future. On the other side is the wise, old head looking into the past. The message is that in order to take wise actions in the present we need to learn from the past in order to not damage our future. Patterns help us do just that.