"The tour is only a matter of hours old... but the wry thought occurs to me that reputations will almost certainly be destroyed in the next few months."

So begins Geoff Boycott in his diary of the 1981 England tour of the West Indies, In the Fast Lane. Captained by a young Ian Botham, the Caribbean campaign quickly faltered. England followed on and lost the first Test by an innings, and the Guyanese government cancelled the second game when they revoked Robin Jackman's visa due to his connections with apartheid South Africa. When the series resumed Michael Holding bowled what many hail as The Greatest Ever Over at a hapless Boycott. Yet the Holding barrage was overshadowed by the tragic death of assistant tour manager Ken Barrington, who collapsed with a fatal heart attack that same evening at the team hotel.

Injuries, personal tragedy, politics and personal animosity. And here we are in Australia at the end of 2013. England 3-0 and the Ashes lost, with Stuart Broad hobbling and Jonathan Trott flown home, Graeme Swann and James Anderson wandering the trenches with shell shock, and Captain Cook fumbling through his orders. The cliché reverberating on the airwaves is: "the wheels have come off". Not only the wheels. The bonnet is flapping in the wind and smoke is pouring from the engine and the kids are fighting in the back seat and dad has lost the map.

In all fairness, organising a family holiday to the coast for a couple of days is demanding enough, let alone a three-month series on the other side of the planet, dodging 95mph missiles and well-aimed barracking from the stands and the opposition.

Perhaps England should return to their first-ever tour destination, North America: "On the Evening of September 6, 1859, Twelve Cricketers of England met at the George Hotel, Liverpool, to be in readiness to embark for Quebec the following morning." Penning the original tour exposé, Fred Lillywhite begins with a detailed account of the team's voyage across the Atlantic aboard the steamship Nova Scotian. Although there was plenty of drinking with many "a jolly evening" spent at sea, the ship soon "began to roll fearfully" and the majority of the squad began "casting up their accounts". In late 19th century parlance this translates to throwing up over the side. Only John Wisden "was a thorough sailor, enjoying both his meals and his pipe" and well enough to quip that the bumpy waves simply required a heavy roller. The plucky tourists survived hurricanes, icebergs, and even won a few pounds off the crew, playing shuffle board. They then won a five-game cricket series, pitting their XI and against teams of XXII, and sailed home with a sizeable £90 each for their efforts.

This inaugural cricketing jolly soon inspired other jaunts, and in 1868 a team of Australian Aborigines became the first non-English team to play abroad, embarking on an arduous 47-match programme across England that included spear- and boomerang-throwing performances for the throngs of spectators; notably the aborigines were beaten in a cricket ball-throwing contest by an athletic allrounder named WG Grace. Despite the Times describing the Australian Aborigines as "a travesty upon cricketing at Lord's" the tour was a success, with crowds flocking to see the "Arrival of the Black Cricketers" as ran the headline in the Sporting Life. Although finances were healthy for the tourists, their physical conditions suffered. Aborigine "King Cole" died and two other players returned to Australia ill.

At least they got to go home, unlike England bowler E Pooley on the New Zealand leg of England's 1876-77 tour of the Antipodes. Pooley was charged with assault after beating up a spectator who owed him a wager, which he'd actually conned him into accepting, and wicketkeeper H Jupp suffered from "a bout of insanity". On a cross-country journey between games the team stagecoach floundered in a swollen river and the squad waded ashore only to be stranded for two days in a roadman's hut. With no change of clothes they stood naked together before a fire to dry out. On return to Melbourne the bedraggled England XI were hammered in the first ever Test match by the Australians, before regrouping with several upcountry fixtures and avenging their defeat in the second Test. The Aussie punters weren't happy, accusing England of throwing the first game to promote gate receipts for the return match. Perhaps they had cause for complaint, as each of the tourists sailed home with a handsome purse of £300. Except the jailed Pooley, who was rotting away in a Kiwi cell as the party cruised back to Blighty.

Further tours to England preceding the Golden Age of Cricket were hardly plain sailing. On the Canadians' 1880 visit their captain, T Jordan, was arrested on suspicion of actually being T Dale, a military deserter. The tour lost impetus after the skipper was jailed, and the team went home bankrupt halfway through their fixture list.

Although future appearances from India and West Indies would fill stadiums across the country, their initial forays onto English soil were failures. "The public and press took little interest" in the West Indies in 1900, writes Peter Wynne Thomas in The Complete History of Cricket Tours - At Home and Abroad. About the 1911 Indians he reports that "the fielding was haphazard" and that their run of defeats kept the stands empty.

Being away from home is a test in itself. The 2013-14 tourists Down Under are about to celebrate Christmas, and the urn is firmly in Michael Clarke's grip. After the 1981 failure of England in the West Indies, Boycott concluded: "How you lose matters, and on that score I thought we did not do ourselves justice." The tourists have two games to go, two chances to board that plane back to Heathrow with their heads held up, not high, but at least up.

Nicholas Hogg is a co-founder of the Authors Cricket Club. His first novel, Show Me the Sky, was nominated for the IMPAC literary award