June 27, 2008

Trivia - batting

The night-watchman story - Part II

Anantha Narayanan
Matthew Hoggard closes his eyes and hopes for the best as he hooks Glenn McGrath for four, Australia v England, 4th Test, Melbourne, December 26, 2006
 © Getty Images

In the first part I looked at a methodology for determining night-watchman situations and looked at individual performances. In this concluding part, I have done a team analysis and come to a conclusion whether the night-watchman experiment is a success or not. I have also looked at readers' comments.

Note: David Barry has sent me an invaluable file containing the day-end batsmen data for all the tests. I myself had gone to my own Text archive and found that this information is available for most matches. However parsing those files would have required lot more effort than parsing the CSV file David has sent.

I will complete the work and come out with a revised article incorporating the actual player data. I will also do a comparison of the actual results and my own derived results. However this will be after some time after looking at a few other interesting ideas already in the pipeline.

I owe David one for this.

There was a suggestion to use the day-end player data which is available in text scorecards. While accepting that this is available in most scorecards, I have to express the inability to do so at the current stage because I have to download quite a few scorecards, do a text-based data mining to extract this data, do some complex parsing work and link this data to the player and fall-of-wicket data already available. This is certainly possible. However, this will take too much time and resources and it is not possible to do this at this instance. Possibly at a later date.

The question of setting up 25.0 as the cut-off batting average has already been raised in two forms. One, questioning the arbitrary nature of this cut-off value, and two, the applicability of lowering this for the pre-WW1 Tests in which the batting averages were consistently lower.

It is impossible for me to work out an algorithm to do a more objective determination of this cut-off value. Whatever I do is likely to be questioned and be subjected to unacceptable variations. I have done this summary of batting averages, over 1879 tests, to substantiate the 25.0 cut-off.

Total Bat Runs: 1719071
Innings:          66117
Not Outs:          8622
Bat Average:      29.90

This is the overall average. This is about 10% below the mid point of the highest average (68.8, excluding Bradman's freakish figure) and lowest average (0.0). As such a figure which is 25% below the mid point figure seems to be the ideal cut-off point for determining night-watchmen. This works out to 25.8, which is just above the current cut-off. Hence the cut-off of 25.0 is retained.

However there is a justification to have this cut-off at a lower figure of 20.0 for all Tests played before 1914. This has excluded some night-watchman instances. The number has now decreased from 563 to 552 since 11 instances which were earlier determined to be night-watchman instances are now outside the


There was a suggestion that a situation where a No.7 batsman such as Vettori is substituted by a no.11 batsman such as Martin should be considered as a night-watchman instance. In this particular case, the average differential, 26+ against 2+, makes it a correct and valid suggestion. However, this cannot be generalised. If Kumble's place is taken by Sreesanth, the situation is murky. Kumble has an average of 18.25 and Sreesanth has an average of 15.50. By no stretch of imagination can we deem this to be a night-watchman situation. For that matter these numbers could even be reversed. Hence, while readily acknowledging the validity of the readers' suggestion, I have to, with reluctance, stick to my decision that only nos. 3-6 will be considered as night-watchman positions.

Then we come to the requirement that we have to consider the batting average of the batsman being replaced. This is something I am very loath to do because of the many inherent weaknesses. Until now I have determined a night-watchman situation solely by the measures of the specific batsman, what was his career-to-date batting average, what was his BPA and which position did he bat in. When I am not even sure who would have been the next batsman, such a move is fraught with problems.

Country summary (1879 tests)

Cty   NWI  Tests  Tests/NWI    # 3-6 inns  Inns/NWI

Aus: 96 696 7.25 4688 48.8 Bng: 5 53 10.60 417 83.4 Eng: 124 873 7.04 5880 47.4 Ind: 56 418 7.46 2797 49.9 Nzl: 50 342 6.84 2403 48.0 Pak: 62 335 5.40 2220 35.8 Saf: 55 332 6.03 2318 42.1 Slk: 22 177 8.04 1181 53.7 Win: 69 448 6.49 3023 43.8 Zim: 13 83 6.49 607 46.7 Icc: 0 1 0.00 8 0 552 3758 6.80 (3.40)

Pakistan has used night-watchmen most often and Bangladesh the least. Sri Lanka have been quite reluctant to use the night-watchman option but South Africa haven't been averse to doing so. However, Bangladesh figures may not be accurate since only three of their batsmen have averages higher than 25.0 and a few night-watchman innings would have been lost. I did not want to lower the cut-off for them only to 20.0, which might have been the correct thing to do. It wasn't worth the effort since it might only add couple more instances.

The last column is a measure of night-watchman occurence based on the number of qualifying innings (nos. 3-6) for the concerned country. Here again Pakistan leads with one instance every 36 innings, followed by South Africa, once every 44 innings. Bangladesh, possibly for reasons already discussed, emplys this once every 83 innings. Just for information, Bangladesh have played 106 Test innings. Out of these, they have lost fewer than four wickets only three times - once the innings didn't start, once they lost three wickets and once they lost just two.

Overall the night-watchman instance occurs once in about three-and-a-half Tests.


Now for the difficlut task of determining whether the night-watchman experiment has been a success or not.

There was a very good suggestion to consider factors other than the night-watchman innings itself, such as how the innings progressed, how much the next batsman scored et al to determine whether a night-watchman stint was a success. I am not very comfortable with the idea of linking the actual performance of the night-watchman to what happened in the game itself. If Gillespie came in as night-watchman and lasted 100 balls, it was an uqualified success. Whether Michael Clarke, who Gillespie replaced, scored 0 or 100 the next day doesn't really matter. Whether Australia won or lost beacuse of this decision again does not matter. We are only looking at whether the night-watchman did his job or not. If he scored 1 in 50 balls he had succeeded. If he scored 9 in 15 balls, got out and the next batsman had to bat the same day, his stint was a failure.

Assuming that no captain would be dumb enough to send a night-watchman an hour before close of play, we are looking at a possible maximum of around 8-10 overs to be played during the evening. We must also assume that the night-watchman should last for some time the next day. A valid conclusion is that if a night-watchman bats for 30 balls, he has more than done his job, since he has probably been in the middle for around 45 minutes.

The balls faced will either be the actual number (available in most of the matches) or the one derived from the team scoring rate, as explained in Part 1. While accepting that there could be very good scores by night-watchman of 0s, 1s, 2s ..., there is no way to actually cull out this data. The only concession I will make is that any night-watchman who scores 15 or more has done his job. Outside edges and wild swings (unlikely) could get him around 10 runs but not 15. It is very likely that he has faced a fair number of deliveries, possibly 30+, to score 15 or more. This criteria can now be applied irrespective of the method of arriving at the "balls played" information. 221 out of the 552 night-watchmen innings fall under this either-or criteria (at least 30 balls faced or 15 runs scored).

Out of the total population of 552, it can be deduced that 221 have succeeded in their task, making the success rate of the night-watchmen exercise around 40%. This figure is certainly much more than what I expected. The success stories are very significant, as were the cases with Gillespie, Hoggard, Tudor and Larwood. One great factor in these night-watchmen decisions is that they are sent in with the expectation that he might fail more often than not, especially if his name does not start with 'G'. If they succeed, that is a bonus, and if not, other than the loss of one late-order wicket, no serious damage has occurred. Hence a success rate of 40% seems beyond all expectation.

We have to conclude that, over time, the night-watchman experiment has been a great success. Having said that, there is a lot to be said for top-order batsmen taking up the responsibility of batting in difficult conditions, a task for which they are eminently trained, both in skill and temparament.

I must acknowledge the contributions of Dr.Ashwin Mahesh, my co-founder at Thirdslip.Com who, long time back, mooted the idea of using the difference between the BPA and actual batting position to identify a night-watchman situation.

Click here to see the complete list of night-watchman instances

Click here to see the list of successful night-watchman instances

Click here to see the list of unsuccessful night-watchman instances

Anantha Narayanan has written for ESPNcricinfo and CastrolCricket and worked with a number of companies on their cricket performance ratings-related systems

RSS Feeds: Anantha Narayanan

Keywords: Trivia

© ESPN Sports Media Ltd.

Posted by Rod Stark on (July 10, 2008, 23:51 GMT)

This is fascinating! Great job!

I would think that the best definition of a nightwatchman might be someone batting within the top six when his normal average batting position was within positions 8-11. Not sure if this is retrievable from the data or not.

And the definition of "success" should be based on whether more wickets fell before close of play. Whether or not the nightwatchman goes on to score runs the next morning is not really relevant to the question since the purpose is to protect wickets until the close of play.

Just my ideas...


[[The available data, at this stage, allows success only to be determined by runs scored (consequently balls faced). After I incorporate David Barry's inputs, maybe I can do a different job.]]

Posted by James on (July 7, 2008, 21:28 GMT)

While reviewing the so-called "complete list of nightwatchman instances", surely the inclusion of Chris Harris (match #1597) is a case of statistics overriding common sense.

Harris was selected as a specialist batsman in that game. He didn't bowl and batted at number 4 in both innings. He came in to bat in the 7th over of the match, in the very first session.

How could someone possibly qualify as a nightwatchman in such circumstances?

And to write it off as an 'exception' doesn't work either, because the same batsman appears further down the same list in similar circumstances! [[Chris Harris qualifies on all counts for a NWM. Batting average of 20.25, his rounded BPA is 7 and he batted in position 4. The fact that he played as a batsman and dis not bowl are irrelevant. I am afraid once I set the parameters and get my short list, if I go through the list and take away instances based on my own subjective reasoning, I will be doing a great disservice to the analysis work done. It is obvious taht there will be people who have walked under and some who missed out. I would be happy if I got over 90% correct. Your point is well-made and I accept your conclusions as valid.]]

Posted by James Morrison on (July 7, 2008, 13:28 GMT)

As a mathematical exercise, the author has probably done a very good job. Unfortunately, the role of a nightwatchman requires analysing factors that, due to reasons stated by the author, are too cumbersome and time-consuming. Two key factors are the time of day and, more importantly, the situation in which the nightwatchman comes in to bat, and "exceptions" to the mathematical "rules", which would make up a large proportion of nightwatchman cases.

Without considering these factors, the exercise becomes impossible. As it stands, it is about as accurate as an algorithm to calculate which bowler had the nicest action or whose surname was the easiest to pronounce. [[The bottom line is that I have attempted to some analysis based on the data available. What are the criteria. A batsman with average below 25.00 (a few exceptions), who has a career BPA of 7.0 and above, who has batted in a position more than 3 off his normal position. Is there anything wrong in any of these assumptions. Maybe when I have got the day-end batsmen data in (a complex and time-consuming exercise) the methodology could be different.]]

Posted by Arnie on (July 7, 2008, 13:15 GMT)

Your articles are fascinating, including this one - well-researched, logically and clearly presented, and fascinating to read. But perhaps best of all, you are not afraid to reach conclusions and express your opinion. Thank you for your hard work!!

Posted by Michael from Australia on (July 7, 2008, 10:08 GMT)

Not a comment regarding Nightwatchmen as such but thought it might be of interest to some regarding the understated ability of Gillespie as a batsman. Gillespie actually narrowly led the South Australian batting averages this year in the Domestic 4-day competition ( at around 39 with an aggregate of about 300 ). Not a good year for South Australia as you might well imagine! Additionally he was an opening batsman during his youth which explains his forward defensive which shames many a top-flight batsman. [[Now we know the secret. I have always felt that "Dizzie" was a very under-rated cricketer.Who is the best night-watchman now. Probably Lee or Kumble or Mills.]]]]

Posted by tonyp on (July 3, 2008, 18:43 GMT)

Outstanding analysis, and a surprising result. The only remaining issue to be explored is the "robustness" of the conclusion. Is this particularly sensitive to any of the selection criteria.

There are also other reasons for elevating batsmen, to cover for injuries or to compensate for batsmen who were off the field a long time during the previous innings. But the success ratio would still seem valid in these instances.

[[My personal take is that irrespective of the seemingly successful story and the conclusion drawn, the Steve Waugh way is the best. The batsmen should take up the responsibility.]]

Posted by Andrew on (July 2, 2008, 5:56 GMT)

The comment on Bangladesh cut-off has made me suggest to you that, maybe you should have a variable cut-off for each team. After all Australia is more likely to have an average Batting average of say, above 40 while New Zealand, below 30. So a variable cut-off might bring to light more night watchmen or take away some non-nwm cases. [[The idea is quite appealing. However the work to be done is quite staggering. However this is certainly an idea I could use for other analysis. Thanks, Andrew.]]

Posted by D.V.C. on (June 29, 2008, 3:31 GMT)

It is worth pointing out that the current playing condition makes using a night-watchman (to protect a batsman from poor light) a better option than in the past. Now if a wicket falls in what would be the last over that's stumps. However, previously the over had to be completed.

So, whereas before the night-watchman had to get to the end of the last over to ensure the recognised batsman didn't have to face up, now they just have to make it to time. [[Very good point. Even if there are 5 minutes on the clock, a bit of gardening, chatting with the other batsman, adjusting the sight screen and asking for some water should keep the play down to one over or less.]]

Posted by Geoff Bethell on (June 28, 2008, 9:19 GMT)

I'm not too sure about all this. I offer the following comments: 1. The promotion of a nightwatchman means that the recognised batsman has less protection lower down the order and this may be counterproductive later on in the innings. 2. If we use a nightwatchman to avoid exposing a recognised batsman to two starts then why not a lunchwatchman and an afternoonteawatchman as well? 3. Too often I see the nightwatchman being protected by the batsman at the other end. Surely the nightwatchman's job is to take as much of the strike as possible to protect his partner. 4. I think I can see a role for a nightwatchman in conditions of poor light - or maybe when a new ball has just been taken. Otherwise I think the normal batting order should be adhered to.

Posted by Khurram on (June 28, 2008, 8:06 GMT)

Great work! This will definitely go down as an authentic reference on this topic for ages! Hats off to the team! [[ The "team of one" acknowledges your kind words and expresses its thanks.]]

Comments have now been closed for this article


Anantha Narayanan
Anantha spent the first half of his four-decade working career with corporates like IBM, Shaw Wallace, NCR, Sime Darby and the Spinneys group in IT-related positions. In the second half, he has worked on cricket simulation, ratings, data mining, analysis and writing, amongst other things. He was the creator of the Wisden 100 lists, released in 2001. He has written for ESPNcricinfo and CastrolCricket, and worked extensively with Maruti Motors, Idea Cellular and Castrol on their performance ratings-related systems. He is an armchair connoisseur of most sports. His other passion is tennis, and he thinks Roger Federer is the greatest sportsman to have walked on earth.

All articles by this writer