Background

We've been grappling with this question for a while. Here is a very old quote from Ward's wiki (author unknown, possibly Ron or Kent): "One of the XP principles is that you want to get as many people making good decisions as possible. Therefore, it's not extreme to create a review board to decide for you if you are extreme. It is extreme to let you decide if your project is extreme. We'll tell you what we think that means. If you decide your project is extreme, you can become part of the community refining the same definitions you just used to decide if you were part of the community. Damned circular, but it's the simplest thing that could possibly work."

"The community" is much larger now than it was then. That is good news. The collective knowledge of that community is also much richer now, which is good news too; it does mean that "the simplest thing that could possibly work" has changed since then. The objective is to keep it simple.

A first attempt (from the Agile Forums) at a definition of a simple solution fitting the present situation: "The Agile Alliance comprises a professional network of software development practitioners, who certify each other in the areas of excellence collectively known as Agile."

There is much more to it than that, but it's the starting point. Some relevant observations:

• the Agile community has a network structure
• some largish "nodes" in this network are user groups
• the Agile Alliance is the node with most members and largest resources

Basics

The basic process, stealing from Advogato and LinkedIn, is this: people sign up to a Web site, after which they can certify people they know. They do this by entering a Statement of Certification.

A Statement of Certification is a strong, solemn, formal statement. Example: "I, Laurent Bossavit, a member of the agile community, certify Rachel Davies as having outstanding skills in Retrospective Facilitation, based on the following evidence: I attended a Retrospective which she facilitated at a conference and took away key learnings from the experience."

It's important that it should include all parts: someone who states that he/she is a member of the community, the name of someone he/she wants to certify, an area of competence, an assessment of the person's skill in that area : serviceable, highly valuable, outstanding; and mention of the substantial evidence on which this judgement is based. The idea is that if you knowingly issue a Certification on false grounds, you are liable to lose standing in the community.

That's the basic user story. Then we do things with the data.

Supporting processes

There are many, many secondary processes arising from that. Sorting them out and finding the most valuable ones (valuable from the perspective of a variety of constituencies) is the meat of the Networks of Excellence project.

Trust metrics

By itself the list of Certifications appearing on a person's profile page has value. If they have many Certifications, from people whose identity can be verified, that's a strong indication that they have good standing in the community. There might be sinister things going on - collusion, etc. - but that's unlikely to happen unless most of the community's determinations of competence are of some value.

We can increase that value by applying various algorithms of the trust metric variety. Determining which algorithms and with what properties is a large subproject. They should be a) simple and b) transparent to the community. We assume that if you're good at something (say TDD) then we should trust you if you say (based on grounded evidence) that another person is good at TDD.

This lets us divide one person's Certifications into two categories: Reliable Statements of Certification, and the rest. The list of reliable Certifications, plural, are that person's "certification", if we must use that term in the singular. (Or we could group them, see below.)

One thing that greatly affects the "weight" of a Certification might be the length of time that the Issuer has observed the behaviours. If you've trained a person for two days, that counts for something. You've interacted with them. If they worked on your team for six months, that's much more reliable. Then again, this could be computed along with Summaries instead (see below).

Training courses

The idea is that you get Certified for anything you did that someone else in the community thinks is a sign you're good. So trainers, like anyone else, can Certify trainees. Individually.

Example of a Certiication that might arise from a CSM course: "I, S. C. Rumtrainer, a member of the Agile community, certify that Laurent Bossavit has at least a basic understanding of Scrum, based on the fact that he took a two-day Scrum course I gave, and after the course asked an insightful question on why it was taboo to extend or shorten the iteration length by one day, but not taboo to give the whole team a day off." (If it sounds like a lot of work to make sure you have twenty individual statements to make, one for each attendee in a class, that's a feature.)

On-site assessments

It's a network, not a family tree. New communities will spring up where none were before. Some "trust metric" algorithm flow trust from a fixed set of seeds. How do the outlying communities enter the network and get Reliable Certifications ?

The Agile Alliance could fund project assessments. The assessors need to be people with good standing. They visit a project at the start, learn about the people working there, the process they use. They come back a while later and based on the results they see, write Certifications for some or all of the people on the team.

One possible idea is that this would be free of charge to the projects being assessed, provided they were "central" enough to the local community. (Of course, if the same team wants to retain the assessor for additional consulting, they'd be welcome to do so; it would not be a condition.) This would be a good use of AA resources.

A single assessment on a sincere Agile team, one with many good people, who would therefore be likely to work with other members of the community in the future, could "flow" trust to many people at once.

Tracking projects and outcomes

As a result of the above, or perhaps as a separate process, we build up a database of Agile projects, and perhaps of how they eventually fare. We have actual data to correlate "number of people with a given number and strength of Certification statements in various Agile areas" with "project outcome".

This is useful for Agile advocacy (successful projects staffed with "certified" people), policing the boundaries of Agile (failed projects staffed with people having weak "certifications"), critical refinement of Agile practice (failed projects staffed with "certified" people), and constructive refinement (projects staffed with people having weak "certifications" that nevertheless turn out well: did they do anything new or different ?).

Individual assessments

We can have something resembling "traditional" certification as well. You have skills you would like to see recognized. You arrange with someone known to have good standing, and who is available to perform that service, to have them come into your project and assess you personally; or to give you an "exam"; or whatever gives them confidence to issue a Certification for you. They can charge for that service, or not. What seems key here is that it's a free market, not a central body which controls who, what and for how much.

Extensions

There are many "wild" ideas that arise from discussions of the basic process. Some of them might turn out, on examination, to be in fact central to the success of the whole scheme. A partial list:

Multiple webs

Rather than derive a single "trust metric", you could derive several based on different sets of "roots". Say, one starting with the Founding Fathers of XP, one with the Originators or Scrum, one with the Leaders of Lean, and so on, as well as a more inclusive one called All of Agile. (For a more extreme idea, we could extend this to allow users to choose any set of roots they like.)

Similarly we might want to associate with different "schools" different sets of Summaries (see below) and rules for computing them.

Summaries

The basic process ends up with users (prospective employers, practitioners looking for teams to join) looking at someone's Web page and seeing a list of things they did that reliably qualify them to take part in an Agile project. For the most qualified, the list will be quite long, as it could include many small things (attending conferences, training sessions) as well as some bigger things (taking part in a project, etc.)

A useful thing to do is to summarize long lists of Certifications into a shorter assessment. Suppose we define a typical profile for "Member of an XP team", which (say) include skill in TDD, pairing, and estimating. If you have several reliable Certifications of your skills in all these areas in at least the Apprentice level, the system computes a summary, in effect saying that if this were a "traditional" certifying body, you might as well be a "certified XP team member, Apprentice".

To compute summaries it might be interesting to define "units" establishing some equivalence between different categories of Certifications (those for two-day exposures to some knowledge, those for six-month experience on a project, those for training other people in the practices, etc.), summing these units to yield a "level".

Job board and "reverse certification"

The site where all this happens would be an ideal place to post job offers. An interesting idea is that current employees might well issue Certifications to their managers, customers or Product Owners. You might choose where to go work based on the Certifications of your prospective employer, as well as the reverse.

Journeyman system

Actually, the sections in "Basics" assume that the process of "certification" does not interfere with existing patterns of joining and leaving teams, it just records facts. The "reverse certification" idea is one thing that comes out of questioning that assumption, but why not go further ? You might want to plan your career moves on how well they will advance your skills and your standing together. You might want to join, as an "apprentice" or "journeyman" interested in Agile, a project where a person known to have high level Certifications is working.

Issues

There are many potential problems with these schemes, or objections that can be legitimately raised. These will be listed below based on comments to this page...

Commercial aspects of certification

One feature of the above scheme is that spending money is optional. Nor is there anyone deciding what courses you should attend in order to get your Certifications, what skills count as Agile or not, etc. You decide where to invest your hard earned cash (or your employer's scarce resources) in the course of your professional development; "certification" occurs as a natural by-product of that.

The usual approach to certification can be considered broken in this regard. There is a limited and predetermined set of "products" that you need to purchase in order to be recognized as having certain skills (with the scarcity driving up the price). Even if you already have the skills and therefore get no value from paying that money. That is in line with Signaling Theory, but not a very appealing characteristic.

On the other hand, a central certifying body can make it easier to control those who provide certification services for money, keeping them "honest". If someone pays you to assess your skills and give you a Certification, if it turns out their skills aren't that good, it could be hard to keep the money and still "fail" them, even though that's what you should do. (Note that a central body could easily have the same problem.) The possibility of issuing Certifications of varying strengths could mitigate that issue.

Community vs cliques

How to implement a certification scheme that a) is faithful to some kernel of "identity" of the agile community, b) doesn't create more division between that community and other communities than already exists and c) doesn't drive a wedge between the sub-groups within the community itself ?

The Scrum Alliance "Certified Coach" program states that one of the criteria for certification is "Scrum Community involvement". You can see how this would be both a good way to maintain community, but also a risk of splintering the larger Agile community. Would there be an Industrial XP community as well trying to maintain itself distinct from the XP community, and so on ?

We "agilistas" are already perceived, outside our own community, as being excessively prone to self-validation. How would a certification scheme (the above or any other) affect that perception ?

Algorithm choice

There has been some discussion on whether the Advogato trust metric might be broken. There's the patent issue, as noted by Brian Marick. Neither seems to be a problem, to the extent that the manner of extracting reliability assessments from the Certification data is an implementation detail. But it is a hairy detail, and one that could take a lot of effort to get right. The idea is to think of all sorts of possibilities for "gaming" whatever scheme is implemented (trying not to rely on security through obscurity), and gain confidence (if possible through proof) that the scheme is not vulnerable to them, or that we have contingency plans if any known vulnerability should be exploited.

Are we even asking the right question ?

Perhaps the most important issue of all is keeping a clear picture of why this scheme is at all desirable. As is being discussed in Goals of Certification, there are different reasons why we could be seeking to establish such a thing. They may even be different from the reasons why "the market" is, apparently, "demanding" some kind of Agile certification.

For an interesting take, see Reg Braithwaite's ideas on "certification as a minimal barrier to entry" (a more proper term would be "license"), and why he would support that kind of certification.

The "networks of excellence" scheme is not, per se, about certification. It is a lot more about leveraging the fact that Agile grew of a networked community. It is about "raising the bar" for excellence: because the more senior members of our community will have well-documented reasons for being regarded as senior, the more junior will strive to emulate these work experiences and learning efforts. At the same time it is also about respecting the values that make the Agile community vibrant and fluid, able to integrate new ideas and respond to feedback: the Agile community, entire, should be given the job of "certifying", or no one should.

Implementation

This could be implemented as an Agile Alliance program. If successful, it might - like the conference program - probably end up being a large part of what people take the AA to be about.