Sun, 24 Apr 2005

During my talk on Saturday (which I’ve now added to tieguy.org/talks/ under ‘why everyone needs a bugmaster’ (I really need to combine the three bugzilla talks on that page into one big bugzilla best-practices manual)), I mentioned that the kernel adopting debbugs would make the baby jesus cry. This evening, I tried to use the query interface, and again, the baby jesus did in fact cry. I think, though, it made me finally grok (possibly) what the bugzilla query interface needs to be- clearly, being able to generate crazy reports (see next paragraph) is probably not an ‘often by many’ task, so in some ways debbugs (despite the lack of power) may be closer to what most of us really need there than bugzilla is.

So, anyway, I was playing around this evening with bugzilla stats, and looking at them across several major public bugzillas. Some numbers from the past year:

Project Bugs Opened Total Bugs Resolved %age comments
Debian ~60500 unknown Can’t query for the right data, got the total count by guessing bug #s and checking dates.
Mozilla 49947 29876 59.8
Gentoo 41289 34014 82.3 I’ll slag them less after this.
GNOME 40578 31648 78.0
Red Hat 25927 17335 66.8 Resolutions are funky and underdocumented, so the ‘resolved’ # may be funky.
KDE 24223 14663 60.5
Ubuntu 10094 7741 76.7 Covers only 10 months, does not include Malone.
Mandrake 6033 3954 65.5
Kernel 1948 944 48.4

All stats are for all components, so comparisons are not necessarily apples and oranges, even between distros. NEEDINFO was not counted as a resolution for these purposes in those bugzillas where it exists. Did not count ‘enhancements’, and I’m currently paying by the byte for download, so maybe I should try again later, or if someone wants to recalc with enhancement requests excluded, that would be awesome.

Some lame observations:

  • Big numbers. Just these selected projects are topping 250K reports a year.
  • At LCA, Robert’s interesting talk on source code management for distros had a small chart giving a ratio of packagers to packages for Debian and Ubuntu. It would be interesting to see a similar chart for bugs:hackers who fix bugs for the distros above.
  • Resolution rates are pretty good; several are above 75% and all (except kernel) are at or above ~60%. (Both these numbers would be improved if we excluded feature requests, I expect.)
  • GNOME stacks up well overall, but probably also gets numbers inflated by stack trace duplicates (duplicates make up about 1/3 of GNOME resolved bugs.
  • No real trends I see in this very cursory analysis- I had expected to see larger projects fare worse, but that doesn’t appear to necessarily be the case.
  • Would love to see someone not paying by the byte do more drilldown on the types of resolutions seen- what percentage of resolutions are notabug/wfm/invalid? what percentage are dups? Does GNOME actually get more dups than others, as I surmised above? Does Gentoo get a similar boost from INVALID stack traces? I’ve collected some of that data but I’ll be very busy for the next 4-5 days, between UDU, flight home, and beating my head against the jetlag wall, so anyone else who wants to take a crack, that would be cool.