OSGeo Stay out of training - the pros and cons of certification

Tyler is doing an excellent job of broaching difficult topics.

The long and short of it is that the foundation is sensibly looking at revenue streams to support its activities. There are lots of great ideas for what the foundation can do; but accomplishing (or even setting goals) requires resources.

If you are interested in the following rants a more diverse opinion is available on discuss@osgeo.org.

Rant Zero: Income

The foundation does need a source of income; it may not be obvious but we do depend on the foundation for a wide range of things from legal advice, to a friendly voice on the phone (that would be Tyler) to the nuts and bolts of keeping the servers warm and the lights on when people visit osgeo.org.

Rant One: Projects

Clarification: Projects are not a source of volunteers (sorry - not even for really good ideas).

Even if an idea is good, it will need to have some volunteers (or budget) attached for it to be successful. The projects are as I understand it already maxed out ... making software.

Indeed I can think of a lot of ways in which projects could be better supported that have not come through ... yet.

• push to bring more volunteers into the development teams of each project.
• fund maintenance activities (documentation, technical debt, quality assurance, external security review)

The osgeo discussion list also shows individual projects are also seeking ways to raise funds. With a classic tradeoff between an OSGeo backed 'donate button' (difficult to explain where the money goes) vs the 'bounty model' used to fund the development of specific features (difficult to explain how to include maintenance and documentation let alone the foundation).

Rant Two: Training

Training is one of the few "business models" that almost all the projects share to support the various development teams. I cannot really see a way for the foundation to offer training without conflicting with existing providers.

The best I can come up with is taking away the "provider list" and putting it behind a pay wall. The foundation could send out the latest copy of the list on request; in exchange for a referral fee of some sort.

The danger is the foundation appearing to recommend a provider or training course that ends up being terrible. I cannot think of a cheap way for the foundation to supply any kind of quality assurance of providers, making this a risky approach to raise funds.

• Quality could be improved by asking the PSC to review providers (in exchange for some of the referral fee). Good way for teams to raise maintenance funds.
• The foundation could define criteria for evaluation: GeoServer for example organises its "commercial support page" based on participation and provider experience.

Rant three: Certification

Here I am going to cheat and recycle and edit some of my emails:

With respect to Paolo Cavallini commenting about the foundation supporting our work, not competing with it:

While I concur (I don't want to see the foundation set itself up in competition ) there may yet still be a useful roll to play.

What I cannot figure out is how the foundation could expect to make any money from this angle ... any figuring of costs I go through makes it look like a massive effort.

As for the useful role: If OSGeo was able to supply a certification test, provide independent marking, and issue the resulting certification it may actually complement existing training offerings by the "professionals and enterprises" and make training easier to sell. This would both validate the training offered; and act as a competitive advantage - right now given a choice between two training courses people will often choose the option that gives them a chance at sitting a certification at the end (especially if they have a limited budget and don't really care what it is they are learning).

A couple of things are clear to me about this discussion:

• a) I *hate* certifications; I feel they prey on the disadvantaged of our industry right when they are weakest (this goes for both job hunters and those going through a hiring process)
• b) certifications are really required in different markets around the world (especially when industry has lost confidence in the meaning of a university degree).

With the above in mind I feel that certifications will happen; and given a choice I would rather it happen at the foundation level (rather than getting people certified in different product stacks).

So while I have some mechanics in mind (certification to include the open source process; not only use; demonstrate ability; aim for a 50% pass rate for the certification to mean something; offer "bulk" discount to groups wishing to use tests at at the end of a training course; or groups wishing to use test as part of a hiring process).

What I cannot figure out is where the profit is; or how to pay for people's involvement. While groups offering training could collaborate (and possible act in a double blind capability to mark results); it would probably require some paid hours to get projects to look at the tests and make sure they mean something at the end of the day.

Pricing the tests would probably be within market norms; and I would expect a much cheaper retry cost (possibly just covering marking time) if we manage to make the marking process brutal enough to be useful to potential employers.

One thing we have a chance to do well here is stress the soft "open source" skills that a potential employee must have in order to be sucessful. Rather than only mechanical questions about configuration and use. Examples: link to 3 questions you have answered on the user list; two issues you have reported etc (which can be marked for completeness etc...).

Finally you have the annoyance for companies that are already established in this space of having the possibility of competing with new groups that have picked up their certifications and appear better "on paper". I cannot honestly have much sympathy here, competition is as competition does, best advice would be to help define the certification (and allow that to be placed on a resume).

To clarify how certification makes the case for QGIS training courses stronger; and does not conflict with existing training offerings:

I don't think anybody is interested in the foundation competing with existing training courses. (Indeed training is one of the few places where any cost recovery on the udig project occurs).

That said if you don't want OSGeo competing in training - how would you like to pay for the foundation? I am not sure if your organisation sponsors OSGeo? I don't think my employer does (preferring to volunteer marketing effort); and I don't personally sponsor the foundation (preferring volunteer effort myself).

So this is the nice part about certification:

• it would make your training courses stronger (ie more attractive to customers)
• it makes training an easier thing to sell (take training as one step towards getting ready for certification)
• it would make QGIS more attractive (as a technology in which certification was available)
• it provides the foundation with a revenue stream that does not compete with any of the member organisations (Indeed certification is a "service" that very few organisations could offer credibly?)

From the QGIS standpoint the benefit for you really is focused on those first couple of points; certifications would be an additional activity the foundation could perform that would make your training courses more valuable.

My own thoughts on this (using your QGIS project as an example for how certification supports training):

1. Testing criteria
   • organisations offering QGIS training are asked to supply criteria to use for the certification process (If your organisation wants to be involved this is where you would take part)
   • the foundation pays for someone to write the test material for a specific QGIS release (perhaps you? perhaps another vendor?)
   • the test is passed around to those supplying QGIS certification criteria for review; production of an answer key etc...
2. Next time you do a training course offer your customer the option of either:
   • a) taking the certification tests at a later date (you can pass on the foundation contact details; and get a 30% cut in thanks for the referral)
   • b) arranging for a "bulk purchase" where you can offer your customer a discount for doing it then and their (perhaps give the customer a 20% discount to make it more attractive). You would need to play with the numbers to make this attractive (so customers don't just ordering the test for their top people).
3. Each month the foundation hires one of the organisation that defined the testing criteria to mark the tests
   • a) a month is chosen to have enough tests together in one spot to make effective use of time
   • b) the organisation hired should follow a set rotation to be "fair"
   • c) the organisation hired should probably not be responsible for the training of any of the students being marked in order to keep this as independent as possible
4. Marking should be brutal
   • a) the idea is to force a spread so that potential employers can actually respect the certification
   • b) cover open source activities (bug submission, contribution to documentation, participation on the user list). If it is any kind of advanced certification this goes into building the application from source code, applying a patch and building locally (can submit a screen snap of the result), links to accepted submissions etc...
   • c) How brutal? How about if they get everything right they end up with 90%; the last 10% is there to allow markers to recognise "outstanding"
   • d) if you really want to soften the blow you can provide different levels of certification out of the same test (confusion may not be worth it; easier to fail people and ask them to try again)
5. Updates to certification should be cheaper and repeatable
   • a) as each release comes out the certification criteria should be updated
   • b) a cheaper rate for "repeat customers" should be available - to encourage this both as a revenue stream - and as a certification process that employers can trust to be update to date. Why hire someone certified in QGIS 1.6 when QGIS 3 has been released?
   • c) the cheaper rate should also be available to those repeating the same test (partly to soften the blow due to the expected failure rate)
   • d) Updates are going to have to occur often to reduce cheating; we have a slight advantage in that we are testing real skills (the test can ask for maps produced with QGIS) and real interaction (the test can ask for links to nabble showing participation).

The other scenario for using the certification tests is:

Next time you hire someone

• a) Buy a "bulk purchase" of tests
• b) Ask applicants to take the test; and submit review (this is nice for them because it is on your dime; and nice for you as you get an objective evaluation)
• c) The foundation arranges for someone to mark this pronto as part of the service; probably only returning details on the top five candidates
• d) The foundation could change more to access test results in detail

The final touchy subject is "discounts":

• a) arranging some kind of discount for graduate students (perhaps if their professor helps with the marking it could be arranged at the school level).
• b) I hate asking graduate students for money; graduate student money is better spent on beer :(
• c) arrange some kind of discount for "osgeo volunteers" perhaps with an email from a recognised osgeo committee chair (project steering committee, education committee or something). Because I don't mind asking graduate students for volunteer time ...
• d) Being tough enough not to offer discounts to usual suspects (project developers, osgeo sponsors, people we really like ...). The more discounts that are around the lower the perceived value of the certification; we should try and get people to pay full price once; and then pay to retake the certification (either because they failed or because a new version of QGIS came out).

I am being very strict about not using the word volunteer in the above activities; your company / organisation should be paying for your involvement. And the OSGeo foundation should be hiring your company / organisation to set the certification tests, perform marking etc... This is very much pay to play.