Post by jamesdoyle on May 19, 2022 13:51:42 GMT
Having spent a few weeks revising the Good Week/Bad Week Index (GWBWI) based on suggestions and responses from here (not the ones that said 'Go away, and take your rubbish with you', obviously), and tidying up the spreadsheet to make input easier, it's now back (retroactively to last Thursday).
Below is an outline explanation of what the GWBWI is supposed to be, and how it's calculated. I'm still open to suggestions for improvements, and if anything comes along, I will amend this header post as required so it reflects the most current iteration of the Index.
The Good Week/Bad Week Index (GWBWI)
The Aim:
To create a mathematically calculated index to assess each major party’s performance in the week’s council by-elections; taking out subjective views, party spin, and expectations management; and relying only on information that is consistently available for all by elections.
Nearly every week, there are a handful of by-elections to vacant council seats. Some seats are held by the defending party, some are lost to a challenger. It should be easy to determine, across the weekly results, who has done well and who badly, but when the observers and prospective interpreters of results have an interest in portraying their own side positively and the opposition negatively, it can be a bit murky to sift the various interpretations to decide where the truth lies.
By basing an algorithm on a standard set of quantitative data available for all by-elections, each result will yield a numerical value for each party’s performance which can be aggregated to assess the weekly performance.
Base Data
Four known pieces of information are used to provide a base figure to determine how ‘important’ the by-election is to each party. The first item is common to all parties:
Council Majority – How close to a majority is the largest party
The remaining three items are calculated separately for each party:
Party Size – How close is the party to a council majority, or to having 0 councillors on the authority
Vote Share – What is the party’s historic vote share
Past Elections – What history does the party have in the ward
Note that only normally scheduled elections are included in the base data: by-elections are strange beasts, so any previous by-elections in the ward are disregarded.
Council Majority
The Council Majority Index (CMI) applies equally to all parties contesting the by-election: how close is the largest party to having or losing a majority?
Gaining and losing a majority are of equal importance, so what we are concerned with is determining how ‘far’ the largest party is from the notional point at which a majority is achieved. For example, if a council has 37 members, the minimum majority is 19 seats (a majority of 1); a party with 18 seats is 1 away from gaining a majority, a party with 19 seats is 1 away from losing it. The closer to the minimum majority, the more important it is.
Party Size
For the purposes of this work, a party can be defined as being ‘large’ or ‘small’ in terms of its presence on the council in which the by-election is taking place. So there is no connection to whether a party is a major or minor presence nationally. Two values are calculated for each party which has a candidate in the by-election based on its number of seats prior to the election.
The first, which we will call the Large Party Index (LPI), is based on the assumption that the value of a seat increases as a party gets close to the minimum majority, from either direction; it is calculated similarly to the CMI, above, but substituting the party’s number of seats for LPI (of course, for the largest party the CMI and the LPI will be the same).
The second, which we will call the Small Party Index (SPI), is based on the assumption that the smaller a party is, the more the value of each seat increases in terms of its power on the council: e.g. going from 1 seat to 2 is an increase of 100%, whereas going from 4 seats to 5 is an increase of 25%.
The circumstances in which the SPI is used instead of the LPI are explained below.
Vote Share
Based on the most recent election in the ward, each party’s vote share (VS) is their vote as a percentage of the total votes cast. If the most recent election was for more than one seat, for parties with more than one candidate the average of the candidates’ votes is used. This is then adjusted for volatility using a formula based on the range in votes from the top to the bottom candidate. [Note: It has been suggested that this needs more explanation. I completely agree it’s opaque, but that’s because I don’t think anyone has come up with anything like this before, and I want to keep it to myself.]
Past Elections
This index is based on the principle that the ‘safeness’ of a ward affects how important the seat is. In this case, ‘safeness’ is considered to be how the ward represented has varied between parties over the past four (or fewer if the ward has not been in existence for four) election cycles (see below): the more seats won by a party, the safer it is for that party. Each party will have a different Safe Ward Index (SWI) for a seat. We use a weighted sample of last four cycles: cycles are weighted 12:6:3:1, from most to least recent.
Note on Election cycles: local elections take place on a four year cycle. Some councils elect all their seats in one year of the cycle – London boroughs, shire county councils, and some others. Generally there will be multiple members elected in each ward. Others elect a third of their members each year for three years out of the four. These are generally shire district councils, where the fourth year is the county council year. Generally one member will be elected in each year. A very small number of shire districts elect half their members in one year, and the other half two years later.
Importance Index
Once we have these four values: CMI, LPI/SPI, VS and SWI, we take an average of the four to calculate the Importance Index for each party, but only if the party is standing a candidate in the by election – otherwise the Importance Index is 0.
Note that two versions of the Importance Index are calculated, using the LPI and SPI respectively.
After the By-Election
Two scores are calculated, for which we use the above factors, the result (who won), and the change in vote share for each party.
By-Election Win
The basic principle is that the score for a by-election result will always be positive for the party that won the by-election, and negative for all other participating parties. Two inputs are used: the Safe Ward Index and the Importance Index.
For ‘small’ parties (where the Small Party Index is greater than the Large Party Index), the SPI version of the Importance Index is used for seats the party is defending, or that they gain in the by-election, otherwise the LPI version is used. This represents the fact that for a small party, holding or losing a defended seat is important, as is gaining one, but losing in a by-election for a seat defended by another party is not.
For the winner, the calculation uses the inverse of the SWI: the safer the ward, the less value there is gained by in winning it. For losing parties, the calculation uses the (negative) SWI: the safer the ward was considered to be, the more value is lost by losing it.
By-Election Vote Share Change
For each party, the vote share in the by-election is calculated as normally, and the historical VS is subtracted to determine the vote change (VC).
As parties may only stand sporadically in some seats, such a party may get an skewed result for VC because of missing a previous cycle. To mitigate for this, a weighting value is calculated, using the same weighting (12:6:3:1) as for the SWI above, applied to the cycles in which the party stood one or more candidates.
Final GWBWI score
The final score for each party is the total of these two scores: By Election Win, and By-Election Vote Share Change.
Some Questions and Comments
Q, Is it right to treat all four elements of the Importance Index equally?
A. Currently all are treated equally. It is possible that one or more should be given more weight. If so, which one(s)?
Q. Why a total, not an average?
A. Imagine that in a given week, party A loses one seat to Party B, getting a score of minus X. And in another week, party A loses five seats to party B, each with a score of minus X. The average would be the same (minus X), but the total of the second week would be minus 5X. It should be obvious that losing five by-elections points to a much worse performance than losing one.
Q. What’s the range of scores?
A. At the moment, it’s not easy to say. Once the Index has been running for a while, a range can be determined empirically.
Q. What’s good and what’s bad?
A. See the previous answer.
Q. Why not use other information as factors in the calculation?
A. It has been suggested that the reason for the by-election, the candidates standing, and the quality of the campaigns by the various parties, and other factors, could all be used as inputs to the calculation. There are a number of difficulties with such factors:
1. Any information may well be partial, in both senses: for example, we may know how one campaign is going, but not others; and the sources for information may be inclined to give a better view of information for one party than for another.
2. A factor might be important in one by-election and not in another: for example, having a local candidate might be important in a rural ward, and not in am urban one. Without further research, it is difficult to estimate how this might vary, and indeed how important it is overall.
3. The subjective and non-numerical nature of such factors: it is difficult to reduce something such as the by-election cause to a numerical value without subjectivity getting in the way. Again, with a great deal of research, this might be possible.
Below is an outline explanation of what the GWBWI is supposed to be, and how it's calculated. I'm still open to suggestions for improvements, and if anything comes along, I will amend this header post as required so it reflects the most current iteration of the Index.
The Good Week/Bad Week Index (GWBWI)
The Aim:
To create a mathematically calculated index to assess each major party’s performance in the week’s council by-elections; taking out subjective views, party spin, and expectations management; and relying only on information that is consistently available for all by elections.
Nearly every week, there are a handful of by-elections to vacant council seats. Some seats are held by the defending party, some are lost to a challenger. It should be easy to determine, across the weekly results, who has done well and who badly, but when the observers and prospective interpreters of results have an interest in portraying their own side positively and the opposition negatively, it can be a bit murky to sift the various interpretations to decide where the truth lies.
By basing an algorithm on a standard set of quantitative data available for all by-elections, each result will yield a numerical value for each party’s performance which can be aggregated to assess the weekly performance.
Base Data
Four known pieces of information are used to provide a base figure to determine how ‘important’ the by-election is to each party. The first item is common to all parties:
Council Majority – How close to a majority is the largest party
The remaining three items are calculated separately for each party:
Party Size – How close is the party to a council majority, or to having 0 councillors on the authority
Vote Share – What is the party’s historic vote share
Past Elections – What history does the party have in the ward
Note that only normally scheduled elections are included in the base data: by-elections are strange beasts, so any previous by-elections in the ward are disregarded.
Council Majority
The Council Majority Index (CMI) applies equally to all parties contesting the by-election: how close is the largest party to having or losing a majority?
Gaining and losing a majority are of equal importance, so what we are concerned with is determining how ‘far’ the largest party is from the notional point at which a majority is achieved. For example, if a council has 37 members, the minimum majority is 19 seats (a majority of 1); a party with 18 seats is 1 away from gaining a majority, a party with 19 seats is 1 away from losing it. The closer to the minimum majority, the more important it is.
Party Size
For the purposes of this work, a party can be defined as being ‘large’ or ‘small’ in terms of its presence on the council in which the by-election is taking place. So there is no connection to whether a party is a major or minor presence nationally. Two values are calculated for each party which has a candidate in the by-election based on its number of seats prior to the election.
The first, which we will call the Large Party Index (LPI), is based on the assumption that the value of a seat increases as a party gets close to the minimum majority, from either direction; it is calculated similarly to the CMI, above, but substituting the party’s number of seats for LPI (of course, for the largest party the CMI and the LPI will be the same).
The second, which we will call the Small Party Index (SPI), is based on the assumption that the smaller a party is, the more the value of each seat increases in terms of its power on the council: e.g. going from 1 seat to 2 is an increase of 100%, whereas going from 4 seats to 5 is an increase of 25%.
The circumstances in which the SPI is used instead of the LPI are explained below.
Vote Share
Based on the most recent election in the ward, each party’s vote share (VS) is their vote as a percentage of the total votes cast. If the most recent election was for more than one seat, for parties with more than one candidate the average of the candidates’ votes is used. This is then adjusted for volatility using a formula based on the range in votes from the top to the bottom candidate. [Note: It has been suggested that this needs more explanation. I completely agree it’s opaque, but that’s because I don’t think anyone has come up with anything like this before, and I want to keep it to myself.]
Past Elections
This index is based on the principle that the ‘safeness’ of a ward affects how important the seat is. In this case, ‘safeness’ is considered to be how the ward represented has varied between parties over the past four (or fewer if the ward has not been in existence for four) election cycles (see below): the more seats won by a party, the safer it is for that party. Each party will have a different Safe Ward Index (SWI) for a seat. We use a weighted sample of last four cycles: cycles are weighted 12:6:3:1, from most to least recent.
Note on Election cycles: local elections take place on a four year cycle. Some councils elect all their seats in one year of the cycle – London boroughs, shire county councils, and some others. Generally there will be multiple members elected in each ward. Others elect a third of their members each year for three years out of the four. These are generally shire district councils, where the fourth year is the county council year. Generally one member will be elected in each year. A very small number of shire districts elect half their members in one year, and the other half two years later.
Importance Index
Once we have these four values: CMI, LPI/SPI, VS and SWI, we take an average of the four to calculate the Importance Index for each party, but only if the party is standing a candidate in the by election – otherwise the Importance Index is 0.
Note that two versions of the Importance Index are calculated, using the LPI and SPI respectively.
After the By-Election
Two scores are calculated, for which we use the above factors, the result (who won), and the change in vote share for each party.
By-Election Win
The basic principle is that the score for a by-election result will always be positive for the party that won the by-election, and negative for all other participating parties. Two inputs are used: the Safe Ward Index and the Importance Index.
For ‘small’ parties (where the Small Party Index is greater than the Large Party Index), the SPI version of the Importance Index is used for seats the party is defending, or that they gain in the by-election, otherwise the LPI version is used. This represents the fact that for a small party, holding or losing a defended seat is important, as is gaining one, but losing in a by-election for a seat defended by another party is not.
For the winner, the calculation uses the inverse of the SWI: the safer the ward, the less value there is gained by in winning it. For losing parties, the calculation uses the (negative) SWI: the safer the ward was considered to be, the more value is lost by losing it.
By-Election Vote Share Change
For each party, the vote share in the by-election is calculated as normally, and the historical VS is subtracted to determine the vote change (VC).
As parties may only stand sporadically in some seats, such a party may get an skewed result for VC because of missing a previous cycle. To mitigate for this, a weighting value is calculated, using the same weighting (12:6:3:1) as for the SWI above, applied to the cycles in which the party stood one or more candidates.
Final GWBWI score
The final score for each party is the total of these two scores: By Election Win, and By-Election Vote Share Change.
Some Questions and Comments
Q, Is it right to treat all four elements of the Importance Index equally?
A. Currently all are treated equally. It is possible that one or more should be given more weight. If so, which one(s)?
Q. Why a total, not an average?
A. Imagine that in a given week, party A loses one seat to Party B, getting a score of minus X. And in another week, party A loses five seats to party B, each with a score of minus X. The average would be the same (minus X), but the total of the second week would be minus 5X. It should be obvious that losing five by-elections points to a much worse performance than losing one.
Q. What’s the range of scores?
A. At the moment, it’s not easy to say. Once the Index has been running for a while, a range can be determined empirically.
Q. What’s good and what’s bad?
A. See the previous answer.
Q. Why not use other information as factors in the calculation?
A. It has been suggested that the reason for the by-election, the candidates standing, and the quality of the campaigns by the various parties, and other factors, could all be used as inputs to the calculation. There are a number of difficulties with such factors:
1. Any information may well be partial, in both senses: for example, we may know how one campaign is going, but not others; and the sources for information may be inclined to give a better view of information for one party than for another.
2. A factor might be important in one by-election and not in another: for example, having a local candidate might be important in a rural ward, and not in am urban one. Without further research, it is difficult to estimate how this might vary, and indeed how important it is overall.
3. The subjective and non-numerical nature of such factors: it is difficult to reduce something such as the by-election cause to a numerical value without subjectivity getting in the way. Again, with a great deal of research, this might be possible.
