The night-watchman story - Part II

In the first part I looked at a methodology for determining night-watchman situations and looked at individual performances. In this concluding part, I have done a team analysis and come to a conclusion whether the night-watchman experiment is a success or not. I have also looked at readers' comments.

Note: David Barry has sent me an invaluable file containing the day-end batsmen data for all the tests. I myself had gone to my own Text archive and found that this information is available for most matches. However parsing those files would have required lot more effort than parsing the CSV file David has sent.

I will complete the work and come out with a revised article incorporating the actual player data. I will also do a comparison of the actual results and my own derived results. However this will be after some time after looking at a few other interesting ideas already in the pipeline.

I owe David one for this.

There was a suggestion to use the day-end player data which is available in text scorecards. While accepting that this is available in most scorecards, I have to express the inability to do so at the current stage because I have to download quite a few scorecards, do a text-based data mining to extract this data, do some complex parsing work and link this data to the player and fall-of-wicket data already available. This is certainly possible. However, this will take too much time and resources and it is not possible to do this at this instance. Possibly at a later date.

The question of setting up 25.0 as the cut-off batting average has already been raised in two forms. One, questioning the arbitrary nature of this cut-off value, and two, the applicability of lowering this for the pre-WW1 Tests in which the batting averages were consistently lower.

It is impossible for me to work out an algorithm to do a more objective determination of this cut-off value. Whatever I do is likely to be questioned and be subjected to unacceptable variations. I have done this summary of batting averages, over 1879 tests, to substantiate the 25.0 cut-off.

Total Bat Runs: 1719071
Innings:          66117
Not Outs:          8622
Bat Average:      29.90

This is the overall average. This is about 10% below the mid point of the highest average (68.8, excluding Bradman's freakish figure) and lowest average (0.0). As such a figure which is 25% below the mid point figure seems to be the ideal cut-off point for determining night-watchmen. This works out to 25.8, which is just above the current cut-off. Hence the cut-off of 25.0 is retained.

However there is a justification to have this cut-off at a lower figure of 20.0 for all Tests played before 1914. This has excluded some night-watchman instances. The number has now decreased from 563 to 552 since 11 instances which were earlier determined to be night-watchman instances are now outside the

scope.

There was a suggestion that a situation where a No.7 batsman such as Vettori is substituted by a no.11 batsman such as Martin should be considered as a night-watchman instance. In this particular case, the average differential, 26+ against 2+, makes it a correct and valid suggestion. However, this cannot be generalised. If Kumble's place is taken by Sreesanth, the situation is murky. Kumble has an average of 18.25 and Sreesanth has an average of 15.50. By no stretch of imagination can we deem this to be a night-watchman situation. For that matter these numbers could even be reversed. Hence, while readily acknowledging the validity of the readers' suggestion, I have to, with reluctance, stick to my decision that only nos. 3-6 will be considered as night-watchman positions.

Then we come to the requirement that we have to consider the batting average of the batsman being replaced. This is something I am very loath to do because of the many inherent weaknesses. Until now I have determined a night-watchman situation solely by the measures of the specific batsman, what was his career-to-date batting average, what was his BPA and which position did he bat in. When I am not even sure who would have been the next batsman, such a move is fraught with problems.

Country summary (1879 tests)

Cty   NWI  Tests  Tests/NWI    # 3-6 inns  Inns/NWI
Aus:   96    696     7.25         4688       48.8
Bng:    5     53    10.60          417       83.4
Eng:  124    873     7.04         5880       47.4
Ind:   56    418     7.46         2797       49.9
Nzl:   50    342     6.84         2403       48.0
Pak:   62    335     5.40         2220       35.8
Saf:   55    332     6.03         2318       42.1
Slk:   22    177     8.04         1181       53.7
Win:   69    448     6.49         3023       43.8
Zim:   13     83     6.49          607       46.7
Icc:    0      1     0.00            8        0
552   3758     6.80 (3.40)

Pakistan has used night-watchmen most often and Bangladesh the least. Sri Lanka have been quite reluctant to use the night-watchman option but South Africa haven't been averse to doing so. However, Bangladesh figures may not be accurate since only three of their batsmen have averages higher than 25.0 and a few night-watchman innings would have been lost. I did not want to lower the cut-off for them only to 20.0, which might have been the correct thing to do. It wasn't worth the effort since it might only add couple more instances.

The last column is a measure of night-watchman occurence based on the number of qualifying innings (nos. 3-6) for the concerned country. Here again Pakistan leads with one instance every 36 innings, followed by South Africa, once every 44 innings. Bangladesh, possibly for reasons already discussed, emplys this once every 83 innings. Just for information, Bangladesh have played 106 Test innings. Out of these, they have lost fewer than four wickets only three times - once the innings didn't start, once they lost three wickets and once they lost just two.

Overall the night-watchman instance occurs once in about three-and-a-half Tests.

Conclusion

Now for the difficlut task of determining whether the night-watchman experiment has been a success or not.

There was a very good suggestion to consider factors other than the night-watchman innings itself, such as how the innings progressed, how much the next batsman scored et al to determine whether a night-watchman stint was a success. I am not very comfortable with the idea of linking the actual performance of the night-watchman to what happened in the game itself. If Gillespie came in as night-watchman and lasted 100 balls, it was an uqualified success. Whether Michael Clarke, who Gillespie replaced, scored 0 or 100 the next day doesn't really matter. Whether Australia won or lost beacuse of this decision again does not matter. We are only looking at whether the night-watchman did his job or not. If he scored 1 in 50 balls he had succeeded. If he scored 9 in 15 balls, got out and the next batsman had to bat the same day, his stint was a failure.

Assuming that no captain would be dumb enough to send a night-watchman an hour before close of play, we are looking at a possible maximum of around 8-10 overs to be played during the evening. We must also assume that the night-watchman should last for some time the next day. A valid conclusion is that if a night-watchman bats for 30 balls, he has more than done his job, since he has probably been in the middle for around 45 minutes.

The balls faced will either be the actual number (available in most of the matches) or the one derived from the team scoring rate, as explained in Part 1. While accepting that there could be very good scores by night-watchman of 0s, 1s, 2s ..., there is no way to actually cull out this data. The only concession I will make is that any night-watchman who scores 15 or more has done his job. Outside edges and wild swings (unlikely) could get him around 10 runs but not 15. It is very likely that he has faced a fair number of deliveries, possibly 30+, to score 15 or more. This criteria can now be applied irrespective of the method of arriving at the "balls played" information. 221 out of the 552 night-watchmen innings fall under this either-or criteria (at least 30 balls faced or 15 runs scored).

Out of the total population of 552, it can be deduced that 221 have succeeded in their task, making the success rate of the night-watchmen exercise around 40%. This figure is certainly much more than what I expected. The success stories are very significant, as were the cases with Gillespie, Hoggard, Tudor and Larwood. One great factor in these night-watchmen decisions is that they are sent in with the expectation that he might fail more often than not, especially if his name does not start with 'G'. If they succeed, that is a bonus, and if not, other than the loss of one late-order wicket, no serious damage has occurred. Hence a success rate of 40% seems beyond all expectation.

We have to conclude that, over time, the night-watchman experiment has been a great success. Having said that, there is a lot to be said for top-order batsmen taking up the responsibility of batting in difficult conditions, a task for which they are eminently trained, both in skill and temparament.

I must acknowledge the contributions of Dr.Ashwin Mahesh, my co-founder at Thirdslip.Com who, long time back, mooted the idea of using the difference between the BPA and actual batting position to identify a night-watchman situation.

Click here to see the complete list of night-watchman instances

Click here to see the list of successful night-watchman instances

Click here to see the list of unsuccessful night-watchman instances