Priorities in Bugzilla [Re: Helping get 2.2 out the door]

Let me prequalify this reply (aimed at the entire abi-QA crowd) by
stating that:
1) As explained below, priorities are not our first priority, and if
you're looking for ways to spend less time but still help more,
priorities are what to let go (in which case, some or most of the
details of this long email might be negligible to you personally).
2) I should probably polish this explanation a bit and add it to the bug
policy faq or one of the other qa info pages; it's a common question
(what do we do about priorities?), and a frequently misused dimension
(IMEO, and for abisource).

--- Ryan Pavlik <abiryan@ryand.net> wrote:
>
> Are we using Priorities, now, too (p1-5) in addition to severities?
> They sort of boggle my mind.

You can as an optional superlative if you want, but in general P3-P1 can
be viewed as indifferentiable, and Priorities aren't our top priority.
P4 and P5, which are sometimes used to indicate bugs of relatively
little importance, don't have to match severity, and this is set
strictly on a per-case basis, because a number of variables (ie,
comments on the bug from developers) can affect priority without being
directly dependent on severity. For example, a label typo, which may
need correction by 2.2 and which may take less than thirty seconds to
fix, but which is nothing to fret about and should be at the bottom of
everyone's list, might be P5, but targeted at 2.2.

I'm trying to choose the dimensions by which to triage which will give
us the best results (most number of matches with a purely theoretical
truth regarding importance of any given bug) for the most bugs, to focus
very few developers in as short an amount of time as is reasonable,
requiring as few QA resources (time and people) as possible. I decided
that based on our current position with Priorities (they are either not
used or misused, most bugs having the former), and in order to churn the
largest number of bugs possible with only a handful of QAers (IOW, the
least amount of time spent per bug on triage without seriously impacting
results), Priorities could be decidedly negligible (IOW, to save time by
disregarding a dimension, thus churning more bugs, this would be that
dimension).

So if you've decided not to mess with priorities, you can ignore the
rest of this.

More specifically how to use P-states when you do:

Fundamental theorem of priorities: Because Priority-states are
time-independent constants (iow, not associated with versions as targets
are), they should be viewed within the frame of a target. For example,
a P5 bug with a target of 2.2 is of lowest priority within the set of
all bugs _targeted at 2.2_. If you use priorities, please (re)define
them in this manner.

As an effect of this definition, priorities may change over time when
nothing else does. For example, a P4 bug may be bumped to P3 in the
future as more is learned, or as more releases are made.

As an effect of this definition, priorities may change in apparent
contradiction to other dimensions (like severity and target). This
means, for example, a P3 normal severity bug with a target of 2.2 may be
postponed to target 2.4 (skipping .x likely due to heavy code changes),
while it's priority is bumped to P2. The target is based on factors
such as the potential delay of 2.2, while the priority is with respect
respect to the set of all bugs targeted at 2.4.

Another example which might be viewed as less obvious: as we approach
2.2 (with regard both to time and number/value of bugs fixed), a P{2/3}
2.2 targeted bug might be postponed only to 2.2.x, but with its priority
bumped to P{1/2}, because we really want to have it fixed in the 2.2
stable release series, but we just can't keep holding up 2.2.0 much
longer.

The following is approximate, subject to change, and intended more as a
guide and teaching aid than strict specification:

<Edit: I had an ascii table here that was rather hard to read, so I'm
gonna do it up in html and make a page about priorities for you>

So let me go back over common reasons for priority promotion/demotion,
nonexhaustively. Time the fix takes, testing the fix requires, severity
relative to other bugs in its target (the target taking into account the
severity state of the bug, severity state being how bad the bug is),
amount of other 'things' (code, features, areas of the program) affected
by the bug AND by its fix, etc.

Simplistically put, priorities are about a lot of thought regarding
time, and what happens during periods of time (good things happening due
to testing, bad things due to the bugs, good and bad due to what the
coders are working on, etc), and what happens before and after specific
points in time (releases and fixes being points in time (what happens
after a fix for a metafeature includes the testing of its subfeatures,
for example)).

-If you're bored or curious, or want another drawn out example-
A hypothetical example of an unusual (we hope) but possible reason to
promote one bug's priority over another. The reason is directly
relative to QA, a bug's testing impedence. Let's say abi has two
crashers (that we may or may not know about, that may or may not be
fixed). The first, abi crashes whenever one opens Format->Styles (it's
just a hypothetical). The second, abi crashes with dataloss whenever
one opens Format->Styles->Import. They're both highly severe to anyone
who uses styles, they're both crashers, they both are targeted at 2.2
and have high priority; they must be fixed by 2.2. The latter is
actually worse due to dataloss. But the former should receive top
priority because of time. Until the former is fixed, noone could even
test the latter such that we'd even know it existed. If the former was
the last bug we fixed before 2.2.0, we'd have let an easy dataloss slip
into production. It has to be fixed in time for QA to test things
pertaining to styles during beta. Because Styles is a metafeature over
Styles-> {Create/Mod, Import, Stylist}, QA theoretically can't
completely test those three features until all bugs in Styles-overall
are gone. So back to the example, the less severe bug receives higher
priority because its effect over time makes it more important, such that
it be fixed sooner.
Recap: This example was one case of Time (see above) affecting Priority,
by way of QA/testing which is time-dependent.
The more affected area of the program, the higher the priority, even
when the severity of two bugs is the same (or opposite).
-/-

I hope this has been an adequate, and not too confusing response to your
question(s). I'll try to put up a pretty page some time with a more
polished explanation of priorities. As always, if you have a case
that's stumping you, or still don't understand something, please ask.
Things are complicated here only so that we can make it simple for the
coders, who collectively have far less time than needed to fix all the
bugs, to spend their time most valuably.

Best regards
-MG
Received on Thu Jun 10 19:31:55 2004