Topic:

Martin Fowler led a discussion on PayingForBetterValue at the Agile Universe 2002 conference. It was by far the largest Open Space gathering there, generating the most interest and involvement. Since then, Clyde has been pondering and seeking an answer to the question that was posed:

"How do we make it accepted that able people who are expensive end up being better value than cheap, less able developers?"

Problem in Context:

We have observed a need to more effectively evaluate the capabilities of software developers. Current certification systems are widely seen not to work. Many of the issues are identified in the words of PayingForBetterValue participants, as recorded by Mr. Fowler:

Can we find a way of charging for value?

People are prepared to pay more for consultants, … But the room doesn't feel they are better quality.

DSDM introduced certification of facilitators. This made a big difference. The CPF is well respected and facilitators can charge more when they have it. Certification is based on demonstrating their skill - have to facilitate a session in front of judges.

Certification is often criticized within the software industry. If we could come up with a good certification system it could be used so people can charge more if they have the certification.

Currently personal recommendation is a key for assessing a developer's skill. But it's an opening to people suggesting untalented friends and is difficult to scale.

Certify people by pair programming with experts.

Does anyone ask for programmer ratings? Is there any business in selling a programmer rating capability to HR departments to improve their hiring and retention processes?

Difference in assessing technical skill (done by programmer peers) and delivery skill (done by customers)

(Quoted from http://monet.objectmentor.com/cgi-bin/openspace/wiki.py?PayingForBetterValue.)

Familiar Evaluation Method Solutions:

• Certification exams
• DSDM's CPF certification
• Formal Licensing
• Master/Apprentice guild system
• Portfolios
• Performance Reviews
• Authorship
• Ad-hoc, such as trusted expert referral

Another Possible Solution: Developer Capability Maps

Prediction of future performance is the goal of most evaluation systems. There is little confidence in certification, despite its frequent use. We need a simple predictor so that customers, project managers and team leads can confidently and successfully select the right person.

One such system is in use in other industries, we will call it ‘Capability Maps.’

What would a ‘developer capability map’ provide to be an ideal evaluation system?

1. It would provide us with criteria to make selection easy at a glance.
2. It would differentiate between similar applicants.
3. It would identify the feedback needed to verify that our selection was correct.

What would this map look like?

Lots of high impact short-stories

Not a certification or ‘achievements’ list

What a Developer Capability Map is…

We will introduce Capability Maps in the workshop so that they can be evaluated alongside the other alternatives to certification. For a rough draft of an example capability map, see http://www.latenighthacking.com/wiki/wiki.pl?JavaCapabilityMap.

Would you agree or disagree with the items listed below of what it takes to become a good OO developer? How could this self-assessment be validated to provide a more convincing demonstration of your skills than test-based certification? Want to learn how to use such as assessment to show your greater capabilities and justify earning more money?

NOVICE:

• Not familiar with OO terminology and concepts
• May be familiar with non-OO programming language(s)

• Has basic understanding with terminology of classes, inheritance, instances and methods
• Knows the syntax of at least one OO language
• Comfortable with browsing classes and methods in their OO language's IDE
• Can edit class and method definitions, with unpredictable results
• Can use debuggers and inspectors to observe in-the-moment runtime information
• Uses cut and paste technique to reuse code
• Can follow an explanation of most UML diagrams

• Conversant in terminology of classes, inheritance, instances and methods
• Can effectively and successfully edit class and method definitions, but unevenly
• Feels comfortable with the most frequently used classes in their OO language
• Has basic understanding of the idea of patterns, may know the commonly used patterns in their language or local environment
• Can use debuggers and inspectors to step through and observe runtime behavior
• Reuses existing code if convenient to do so
• Understands the most common UML diagrams, can create some UML diagrams

• Thorough understanding of OO terminology and the elements of their OO language
• Can use debuggers and inspectors with breakpoints to control and change runtime behavior
• Conversational understanding of the most common patterns
• Uses language idioms, best practices, and local standards
• Basic awareness of industry trends in patterns, methodologies, etc.
• Conversational understanding of the most common patterns, including some outside their language and local environment
• Awareness of impedance mismatch issues between their OO language and non-OO persistence or other non-OO I/O sources
• Familiar with their language and local environment's frameworks and architectural layering such as MVC, persistence, etc
• Looks for existing code to reuse and refactors it if convenient to do so
• Can create the most common UML diagrams, understands all UML diagrams

• Designs and builds frameworks and architectures such as MVC, persistence, etc
• Designs and builds solutions to impedance mismatch issues
• Regularly uses debuggers and inspectors to control and change runtime behavior
• Regularly refactors
• Uses patterns when appropriate, knows which ones don't fit
• Always looks for existing code to reuse, refactors it if necessary, eliminates other duplicate code
• Effectively understands and creates all UML diagrams

• Discovers and documents new patterns

[OTUG home]