In the latest extract from his autobiography, Behind the Shades, Fletcher also says he believed Hair was unfairly punished for his role in the incident.
"Amid everything that happened on the day Pakistan refused to continue playing, I especially recall attending a meeting with Andrew Strauss, David Morgan, Inzamam-ul-Haq, Zaheer Abbas [Pakistan's manager] and Shaharyar Khan [PCB chairman] Interestingly, Bob Woolmer, the Pakistan coach, was not there," Fletcher wrote in his book, extracts of which were published in the Daily Mail on Tuesday.
"At this meeting Hair completely lost his temper. Inzamam, the Pakistan captain, asked him why his side were being accused. 'You know what was going on out there,' said Hair sternly and got up and left."
Fletcher felt Hair's suspension by the ICC had set a wrong precedent. "[Billy] Doctrove backed him to the hilt but it was Hair who copped all the flak. That was unfair and doubtless it will mean that, from now on, umpires will not want to make the big decisions. That is not good for the game."
Fletcher also revealed how he was not allowed to see the ball on the fourth morning. "I went to the umpires' room and it occurred to me that I should have a look at the ball. Pakistan had only bowled 18 overs at us the previous evening and I was interested to see what sort of state it was in. Normally you would only look at a ball after 50 or 60 overs.
"However, TV umpire Peter Hartley and reserve umpire Trevor Jesty said I couldn't see the ball until Hair and Doctrove had arrived."
He was not allowed to see the ball even on Hair's arrival. "I went out to begin the warm-ups. Hair then approached me. 'I'm not going to show you the ball,' he said bluntly, 'but we've got a handle on it and are monitoring the situation.' 'No problem, we are more concerned about the start of play,' I replied."
The 2006 Oval Test was abandoned after Pakistan refused to take the field after tea on the fourth day, feeling they had wrongly been accused of ball-tampering by Hair. This was the first Test forfeiture in the game's history.