June 2, 2015

Why DLS trumps VJD

Steven Stern
The man behind the revision to the Duckworth-Lewis method explains why the upgraded system works better than its closest competitor


Let me start by saying I am the first to admit no mathematical system to reset targets in interrupted limited-overs matches will ever be 100% perfect. To that end, I am always interested in listening to sensible critiques, open to the possibilities of improving the DLS (the Duckworth-Lewis-Stern method, the revised Duckworth-Lewis system) structure, and a strong proponent of full debate regarding the strengths and weaknesses of DLS - and any competing methodology.

All of that being said, however, I must stress comparisons exclusively based on examination of a selected set of examples that are accompanied solely by discussion of what intuitively seems the correct behaviour for targets are not particularly compelling. First, in complex situations, human intuition is not particularly reliable. Second, and perhaps more important, the "intuitively correct" behaviour for a target-setting methodology is never likely to be universally agreed upon.

Nevertheless, it is still instructive to look at various scenarios in as full a context as possible and, in particular, to examine mathematical targets set to actual observed match data. To do so, I will here focus on the four general scenarios presented in S Rajesh's recent article on ESPNcricinfo.

Scenario 1: 50-over first innings, 20-over second innings
In the case where the team batting first has a full 50-over innings and the team batting second has their innings shortened to 20 overs before it starts, S Rajesh gives a table of DLS and VJD (V Jayadevan method) targets for various first-innings totals. That table is reproduced below, augmented with some further rows.

Table 1: 20-over target for shortened 50-over innings
Team 1 score DLS target Diff VJD target Diff
 150  93  0  91  0
 175  109  16  105  14
 200  124  15  118  13
 225  139  15  130  12
 250  154  15  142  12
 275  158  4  153  11
 300  163  5  163  10
 325  168  5  173  10
 350  174  6  182  9
 375  181  7  191  9
 400  189  8  199  8

The question of the seemingly "uneven" increase in targets in this table can be somewhat misleading without both broader context and comparison to actual match data.

First, it is important to note that the basic principle of both DLS and its predecessor, the D/L, was that for matches with a first-innings score above average, there was a steady shift in scoring patterns, which is why we see a break in the pattern for the DLS targets for first-innings scores over 250. This shift in DLS target structure is based on observable patterns in actual match data. To see this, let's look at the relationship between scores after 30 overs and final totals. Figure 1 shows the relationship between scores at 30 overs and final total, broken down by the number of wickets down at the 30-over mark for all uninterrupted ODIs played between July 1, 2010 and June 30, 2014 (the time window used in setting the parameters of DLS). In addition, each panel in Figure 1 shows a line relating the final innings scores to the associated DLS par score at 30 overs.

Figure 1: Scores at 30 overs v final scores

As can be seen, the DLS par scores are in very close agreement with observed match data (with the possible exception of the cases for six and seven wickets down).

Furthermore, we can relate this data back to the values of Table 1 to investigate the consistency of DLS and VJD targets within any given match. (I note that the form of "consistency" of primary importance deals not with changes across games with different first-innings totals, as are investigated in VJD's critiques, but instead that associated with changes to which overs are lost within games with the same first- innings score.) In particular, Table 2 shows not only the DLS and VJD targets for a 20-over second innings, but also the par score at 30 overs with no wickets down and finally the sum of the 20-over targets and 30-over no-wickets par scores.

Table 2: 20-Over Target and 30-Over Par Score for shortened 50-over innings
Team 1 score 20-over target DLS 20-over target VJD 30-over Par Score DLS (0 wkts) 30-over Par Score VJD (0 wkts) Sum DLSSum VJD
150 93 91 58 60 151 151
 175  109  105  67  72  176  177
 200  124  118  77  84  201  202
 225  139  130  87  97  226  227
 250  154  142  97  111  251  253
 275  158  153  118  125  276  277
 300  163  163  138  140  301  303
 325  168  173  158  155  326  328
 350  174  182  177  171  351  353
 375  181  191  195  187  376  378
 400  189  199  212  204  401  403

Clearly, if "internal consistency" is to be maintained, the sum of the 20-over target and the 30-over par score with no wickets down (which represents the expected score without the loss of a wicket for the first 30 overs, precisely the portion of the innings lost when the team batting second is given a 20-over chase) must be equal to the original target had the match been uninterrupted. As can be seen, this fundamental requirement is satisfied by DLS but not by VJD. Indeed, this lack of internal consistency within a match is a direct consequence of the fundamental mathematical structure of the VJD methodology, and it is even more pronounced in other circumstances.

Scenario 2: 30-over first innings with four down, 20-over second innings
In the second scenario investigated in S Rajesh's article, the team batting first reaches 30 overs with four wickets down when their innings is terminated and the team batting second is given 20 overs to chase. Again, we reproduce the table of DLS and VJD targets, augmented by additional rows as well as the associated 50-over DLS projected score (that is, the 50-over final score which would correspond the 30-over four wickets down DLS par score and 20-over DLS target).

Table 3: Over Targets for First Innings Terminated at 30 Overs with 4 Wickets Down
Team 1 score (30 ov) DLS target Diff DLS Projection VJD target Diff
 90 for 4  103  0  165.6  100  0
 115 for 4  131  28  211.6  122  22
 140 for 4  155  24  254.4  141  19
 165 for 4  159  4  283.5  157  16
 190 for 4  165  6  313.7  172  15
 215 for 4  173  8  345  185  13
 240 for 4  182  11  377.5  197  12

Using the given DLS projections, we see that the "break" in the pattern of changes to the target occurs across the point corresponding to the average 50-over score of 250. In addition, examining the four-wickets-down panel of Figure 1 shows that the projected final totals for the various first-innings scores are well in line with observed match data. Further, as the DLS 20-over targets here are in line with the values given in Tables 1 and 2, we again see that DLS scores are in agreement with observed match data. Finally, we note that Table 3 does not present a column for a "VJD projection". The reason for this is such a construct is not uniquely defined for VJD. Specifically, for a first-innings total of either 165 or 166, the 20-over VJD target is 100, the same as for a first innings of 90 for 4 terminated after 30 overs. As such, 165.5 could be construed as the VJD projection in this case. On the other hand, the par score after 30 overs with four wickets down for a first-innings total of either 165 or 166 is only 89. To achieve a 30-over four-wickets-down par score of 90, the first-innings total must be either 167 or 168, leading to a projected score of 167.5, two runs higher than that derived from the associated 20-over target. This ambiguity persists across the scenarios in all the rows of Table 3 and is again a result of the "internal inconsistency" of the VJD methodology noted in previous sections.

Scenario 3: 20-over first innings, 10-over second innings
The final scenarios in S Rajesh's article deal with T20 innings. The first of them sees an uninterrupted first innings followed by a second innings shortened to 10 overs at the start. Figure 2 is the exact counterpart to Figure 1, but for 20-over matches.

Figure 2: Scores at 10 overs v final scores

Further, Table 4 gives the DLS and VJD targets, as well as the "mirror image" par score at after 10 overs with no wickets down.

Table 4: 10-Over Target and 10-Over Par Score for shortened 20-over innings
Team 1 score 10-over target DLS 10-over target VJD 10-over Par Score DLS (0 wkts) 10-over Par Score VJD (0 wkts) Sum DLSSum VJD
80 48 51 33 29 81 80
 100  60  62  41  38  101  100
 120  72  73  49  48  121  121
 140  83  83  58  58  141  141
 160  94  93  67  68  161  161
 180  104  103  77  79  181  182
 200  114  112  87  91  201  203
 220  124  121  97  101  221  222
 240  133  129  108  112  241  241
(Note: The VJD method uses different calculations for T20s and ODIs.)

Again we can see that the VJD method does not maintain internal consistency, since the par scores at 10 overs with no wickets down and the targets for a second innings shortened to 10 overs at the start do not sum to the original target. As noted, this structural inconsistency of the VJD method is endemic to its mathematical structure. Moreover, while the cases shown in the preceding examples are relatively mild, the effect can be notably more dramatic as the following example shows:

Consider a 50-over match in which the team batting first scores 80 for 0 in 20 overs and then has their innings terminated - the VJD target here for a chase of 20 overs is 154 (the corresponding DLS target is 161). By contrast, the VJD target had the chase been shortened to 10 overs at the outset (assuming 10 overs constitutes a valid match, as it does in many domestic leagues) is 89 (the corresponding DLS target would be 94). So, internal consistency would require the VJD par score after 10 overs with no wickets down in the 20-over chase to be 154 - 89 = 65 (and similarly, the required DLS par score would be 161 - 94 = 67). Examining the par score table from the VJD method we see that for the 20-over chase, the par score after 10 overs with no wickets down is instead 56, a nine-run difference from the requirement of 65 for consistency (and in stark contrast to the DLS par score which is indeed the 67 required for internal consistency).

So, while the VJD method is an interesting competitor to DLS, its structural inconsistency within a match makes it untenable for application. In addition, while some of the target values produced by DLS may give the appearance of being "intuitively" inconsistent or inaccurate, close inspection reveals that they follow observed data extremely closely, and the apparent inconsistencies are cleared up when the target values are given their proper full context. So, as noted initially, when making comparisons and critique of rain rules, it is critical to rely on data-based analysis instead of simply picking a collection of "key" examples and attempting to evaluate targets using so-called "intuition".

Australian academic Steven Stern was behind the revision of the Duckworth-Lewis system into the Duckworth-Lewis-Stern system