JavaBlackBelt is a fast growing community whose members take tests to assess their skills in various Java-related topics. The tests themselves are created also by the site’s members. A small number of Ruby enthusiasts is currently working to add a Ruby exam to this Java-oriented site. I interviewed Mr. John Rizzo, one of JavaBlackBelt’s founders, and asked him what JavaBlackBelt has to offer Java and Ruby developers.

Danny: Why do we need JavaBlackBelt if we already have Sun’s Java certificates?

John: We felt that, in the real world, with real teams and real projects, we have no way to assess, in an objective way, Java developers. Java requires so many things to learn. Sun is responsible for many Java apis, like JMS, Java2D, Swing, JDBC, servlet, EJB and so on. But their exam covers only a very small part of that. For example, the Sun Certified Java Programmer (SCJP) is a coverage of a small part of Java SE; JDBC, simple database access, is not even in it. And there are so many other tools and apis that are covered by other groups: Jakarta, Eclipse, jBoss. Their core business is developing their framework, not doing certifications. So there is nothing for this. If you see what a Java developer has to learn the first six months, or the first one or two years, to be productive in a project… All these technologies, and certification covers a very small part, maybe less than 10%. This is the first reason.

The other reason is that when you have three years of experience as a Java developer, and you’d like to do the first Sun certification (SCJP), you still need to study for a few days before you can take the test. Is that normal? You are an experienced developer, and you need to study, before Sun recognizes your skills. That’s why the exams on JavaBlackBelt are made keeping in mind that the candidates have all the resources available. Books, Eclipse, Google, whatever. They can search the web, search the Java apis, they can try things out in Eclipse. They’re only limited by time. They don’t have to remember things by heart. But the exams are hard, even with that. It depends on the exams, we also have basic exams, to help the beginners and to encourage the beginner to learn. Like Ruby Basic for example. And then we have more advanced exams; not just one big certification with a lot of questions.

What does JavaBlackBelt have to offer to a Rubyist? Can Rubyists contribute to JavaBlackBelt in any way?

In fact, we did not plan to have Ruby exams on JavaBlackBelt. Dmitry Buzdin, the moderator on the Ruby exam, and Nathan Smith, they proposed to have one. And I said, why not, let’s just try, and see what happens. So we welcomed this Ruby Basic exam, and we have a very good acceptance and feedback, from the Java community being already on JavaBlackBelt. We have only the Ruby Basic exam so far; the exam’s objectives are composed of 34 questions, and the database contains 127 questions now.

The easiest way to contribute is to take the beta version of the Ruby Basic exam, there any user can edit any question to improve it. They can vote on questions, to say “okay, it should be in the real exam”, or “no, it should go to the freezer, because it’s a bad question”. And they can comment to suggest improvements. Also, everybody is welcome to propose another exam, probably a more advanced one.

If JavaBlackBelt accepts anyone’s questions, how do you ensure the quality of those questions?

That’s the magic behind JavaBlackBelt, we had to invent a platform where you have thousands of users contributing to thousands of questions. When we announced JavaBlackBelt being ready, we got so many new questions, we could not moderate them. You have to start a dialog with the question author to improve the question. And then for so many questions, even with 10 or 20 moderators it’s not possible. So we changed the model. We added a versioning system for questions, a voting system, a comment system. So the users taking the beta exams, and the real exams also, can directly discuss with the author, and directly improve every question.

We have different zones. If you author a new question, it goes to the incubator zone, and when you take a beta exam, you get questions from the incubator. Then people improve these questions, vote on them. Then some questions go to the exam zone, which are proposed for the real exams. For example, the Ruby Basic exam has not been released yet, it’s still in the beta state. This means you can only take the beta test, because there are not enough questions in the real exam zone for every category. But we are not far from it, a little bit more traffic and we should get there.

At the moment, Ruby (okay, and SQL) is the only language besides Java that is represented on JavaBlackBelt. Do you have plans to offer tests for any other languages?

We could even go further than IT, and have for example, BankBlackBelt, with financial questions. But Java and the IT world are already so huge that we try to stay focused. We try to concentrate on Java, the Java world. So if somebody from a community wants to lead an effort for another technology, we will help and encourage it, but we are not going to lead it in the short or mid term. For example, PHPBlackBelt, we are okay with that, but we need a few PHP enthusiasts, to start making objectives and adding questions, and spreading the word also.

Do you foresee a separate RubyBlackBelt.com in the near future?

It depends on the traffic. If we get more and more interest from the Ruby community and we get more and more exams about Ruby, then why not. We are open to that.

Which are the most popular tests, and how popular is the Ruby test?

The Ruby test is not so popular. To give you an idea, in the last 30 days people contributed for 16 points on this exam, which is the equivalent of 5 new questions in one month. On average 10 people per week take this exam. And most people fail, which probably means it’s not basic enough and we should make it easier. Ruby guys should help us to make it easier. The most active exam, in comparison, is Java SE Core intermediate level. That has 1200 points over the last 30 days, and holds currently close to 2000 questions in the database, for 25 asked during the exams. So people contribute a lot to this exam. It depends on the moment and the exam, but we don’t get a lot of traffic from Ruby people.

The tests are being taken via Internet. Doesn’t that mean that people can look up the answers to the questions, for example in the api docs? Or even that people can have their test done by someone else?

It’s okay, it’s just the way we ‘cheat’ in the real world. When we have a problem, we look in the api, we go to the forums, we try something in Eclipse or another environment. It’s the real world way to fix problems, and we wanted JavaBlackBelt to be real world oriented. We say before starting the exam, to prepare Google and javadoc, and be sure to use it during the exam. But they have limited time and it doesn’t mean so many people pass the exams.

The second problem is another problem: is the real guy taking the exam or someone else? That’s not a real problem in fact. For one or two exams you can ask someone else to take the exam for you. But to have a black belt for example, you will need to go for 30 successful exams. The brown belt is 20. Would you find anybody who commits so much time, to do the exams for you?

We also work with a delay. When you fail an exam, you have to wait 15 days before you can retry. And we ask people to contribute before taking exams; they need contribution points to take exams. We don’t want them to open another account and view all the exams and questions by failing the real exams, and then succeed with their real account.

Do you plan on providing “official” certificates, competing with Sun’s Java certificates?

We will surely do something special for JavaBlackBelt. I think we are going to send a real black belt with JavaBlackBelt’s logo, I don’t know yet exactly. We will give Java SE certifications with a certain level, which goes from beginner/apprentice, to master. Four levels, according on how many exams you have passed in JavaBlackBelt. We could imagine to have a ‘Ruby certification’ track, if we have, let’s say, 4 Ruby exams.

The actual black belt is unattainable at the moment, because there aren’t enough exams yet to acquire enough points to deserve a black belt–the standard has been set very high.

Do you know if JavaBlackBelt is being used by companies, for example to test the knowledge of their employees? Is that a market you’re actively pursuing?

The first step was to build a community and to verify that it was a good idea. And to get feedback, which is massively done, we are very impressed ourselves with the success, in fact. We developed last year (2006) the corporate edition, which consists of private portals. So your-company.javablackbelt.com, where you have your own extranet with only your employees. They can take exams without contributing, and their name will not be displayed on the Congratulations box on the home page, for the headhunters. As an employer, you would like to invest in the skills of your employees. You don’t want it to be used by your employees to find another job, or to be contacted by headhunters.

So, yes, JavaBlackBelt is already used by companies. They are very happy because it’s a way for them to recognize the real Java competence, the real developer skills, in an objective way. Developers have the recognition of what they already know and they’re also encouraged to learn more. So it’s a very attractive model to improve self-study within the company. Not just studying and learning from a book, but going a little bit further, with clear objectives, about what to know and what to study, to pass an exam. I think it’s more efficient than sending people three days a year to a course.

There’s a clear message in here: the Ruby Basic exam needs more input from Ruby developers. The exam needs more questions, and the questions must be reviewed. That way we could obtain a useful instrument to test ourselves and our potential new colleagues. Personally, I think JavaBlackBelt is a more developer-friendly way of skills assessment than traditional certification systems; and certainly it’s much more fun at that.

I want to thank John Rizzo for doing the interview, and wish him all the best for JavaBlackBelt!