From OpenBRR

Contents 1 BRR Objectivity 1.1 Who does the rating? 1.2 Where do we get standards to adopt? 1.3 Where does the data come from? 1.4 Will the data be open? 1.5 How are results published? 1.6 Why does BRR still have some subjective dimensions? 2 BRR Technical 2.1 Is it a popularity contest? 2.2 Why only rate open source software? 2.3 One of your metrics is not good enough, so why is it included? 2.4 Are we comparing mature products with immature projects? 2.5 Do you look at source code quality? 2.6 Are there other works? 2.7 Why a new model ? 3 BRR's Purpose 3.1 How can open source adopters benefit from BRR? 3.2 How can developers benefit from BRR? 4 OpenBRR related 4.1 Who initiated the work? 4.2 How can I/my company get involved? 4.3 What level of involvement is expected? 4.4 Can I use BRR in a bundle with a package under different license terms?

BRR Objectivity

Who does the rating?

• We do not give anyone any exclusive right to use the model, or to rate a certain software package. This includes our sponsors and business partners.
• We do not restrict anyone from being able to rate any software. This means that a committer of a software package can rate his/her own software. Although this may present a conflict of interest, all assessment results are open, so if people publish an unfair assessment, the community will be able to refute the assessment.

Where do we get standards to adopt?

• BRR functionality assessment requires a list of features that are demanded by a certain standard. If such standard for the software category is available from a third party organization such as ISO, W3C, www.linuxbase.org, or cmsmatrix, then we will adopt that. If such standard does not exists, we will form a task force to develop one.

Where does the data come from?

• Sourceforge.net, the project website, the project source code repository, and mailing list archives are great places to start getting data.

Will the data be open?

• Most if not all data about most open source projects are available publicly, often from redundant sources.
• Although the data are public, it is not always easy to find them. OpenBRR plans to be provide a service to gather and deliver this data. We will always cite our sources to ensure transparency.

How are results published?

• We are in the process of building an rating publication interface in our website. Before this tool is available, you may publish your results by sending them to getinvolved@openbrr.org. We will publish them in our website.

Why does BRR still have some subjective dimensions?

• We try to be as objective as possible, but it is not possible to eliminate subjectivity in any assessment process.
• Allowing some subjectivity is in some way a strength of BRR. An assessment process is only correct if it is done according to the users' requirements. Because different people in different situations have different needs, it makes sense for BRR to produce different ratings for these different needs.

BRR Technical

Is it a popularity contest?

• Popularity is a good sign of adoption. However popularity alone does not tell much about project maturity, or business readiness. Realizing this dichotomy, BRR includes a popularity dimension, but it does not put an overwhelming weight on it.

Why only rate open source software?

• At the moment BRR is only used for assessing open source projects. The BRR framework can be used to asses both open source and proprietary software. However, OSS and proprietary software have different criteria to measure their strengths and weaknesses. Furthermore, the data availability for the two groups is not the same. For example, bug data for proprietary software may not be available.
• At some point, it's possible that BRR will be extended for assessing proprietary software. However this extension is not yet planned.

One of your metrics is not good enough, so why is it included?

• BRR is an ongoing project. We may not have everything right at the moment, but it will always get better. If you feel that some metric is not suitable, please bring it up as a discussion item. You may want to check if the same issue has been resolved in the past.
• Metrics are used in context. Some metrics may actually make sense in light of the assessment category.

Are we comparing mature products with immature projects?

Yes and No.

• BRR should be able to classify two projects with different level of maturity accordingly. It is only fair to call a sheep a sheep and a goat a goat.
• However, BRR realizes that less mature projects are attractive for R&D or other non critical purposes. BRR allows different usage scenarios to affect the assessment process to fit the specific need. Therefore an open source project may have higher or lower BRR scores if assessed for different usage scenarios. This notion of usage scenarios is captured in our Function Orientation concept.

Do you look at source code quality?

• Currently BRR does not directly look at source code of the project. We try to measure software quality with other methods, including like bug density.
• We don't feel that the resources (both knowledge and tools) are commonly available for people to do this. As we progress, BRR may at some point look directly at the source code.

Are there other works?

• Yes, there are at least two models to measure the maturity of OSS. Both are called the OSMM (Open Source Maturity Model). One is sponsored by CapGemini (CapGemini's OSMM), available at www.seriouslyopen.org, and the other is created by Bernard Golden, the CEO of Navica, published in his book "Succeeding with Open Source."

Why a new model ?

• Previous models address software maturity from a high level perspective, making the assessment process more subjective.
• The BRR takes testing and code coverage into account.
• We want the framework to be totally open so:
  • The community (and others) can participate in the design of BRR
  • The community (and others) can perform their own evaluations
  • The community (and others) can publish their results
  • The results can be read by others openly and re-assessed if necessary.

BRR's Purpose

How can open source adopters benefit from BRR?

• BRR provides a yardstick for comparing different open source software, and to help adopters make educated choices.

How can developers benefit from BRR?

• BRR helps them understand what the Industry expect from them
• BRR helps projects to identify areas where they might need to improve.
• BRR hopes to motivate open source committers to write better, stable and more secure software, and thus gain a higher rating.

OpenBRR related

Who initiated the work?

• The BRR project was started as a joint initiative by the Carnegie Mellon University West Center for Open Source Investigation (COSI), Intel Corporation, and SpikeSource, and soon joined by O'Reilly CodeZoo.

How can I/my company get involved?

• Send us an email at getinvolved@openbrr.org. Tell us how you would like to contribute to BRR and how BRR can benefit you or your company. Tell us what you are doing and how we can probably work together. We will be in contact.
• Just jump in! Get involved in the community by contributing in our forum or through this wiki.

What level of involvement is expected?

• This is an open initiative much like any other open source projects. You can contribute as much as or as little as you want.

Can I use BRR in a bundle with a package under different license terms?

• Please contact getinvolved@openbrr.org -- we would like to make BRR available to as many people as possible, and we will work with you on permissions to use BRR or any of our content.