While Farbrace, England's former assistant coach, regards Buttler as a "brilliant professional" and a "selfless cricketer" he accepts he "needs to score runs" in Sri Lanka or "step aside" from the Test side.
Buttler has made just one Test century in his 41 Tests (73 innings) and, since the start of the Ashes, has made one half-century in 10 Tests and averaged 21.31. He hasn't reached 30 in his most recent eight Test innings.
"I am worried about Jos," Farbrace said. "I think Jos should be England's keeper and batter but he knows he can't keep going long periods of time without making contributions.
"What should a player of his ability be averaging after 40 games? It should be 40-plus. It is a lot lower than he would want it to be.
"I would definitely keep him for the Sri Lanka series but he needs to score runs. If not, then he has to step aside."
Buttler does look set to start the Sri Lanka tour as England's first-choice keeper. But the recall of Ben Foakes, player of the series when England won in Sri Lanka in late 2018, is sure to increase the pressure on Buttler's position.
And Farbrace feels Foakes has proved his ability and shouldn't be judged too harshly on a disappointing 2019 Championship season (in which he averaged 26.14) as it may well have been provoked by his being dropped by England. Farbrace retains belief in Jonny Bairstow's ability, too, though he accepts he "did not score enough runs."
"Foakes is England's best keeper bar none," Farbrace said. "He is a quality act with the bat. He showed he is up to it in Sri Lanka.
"Being dropped in the West Indies had a massive effect on him during the English summer. It kicked the legs from underneath him.
"Jonny Bairstow's keeping improved enormously, too, but he got bowled too often and did not score enough runs. To get the best out of Jonny, he bats at No. 7 and keeps wicket. A lot of us think he can nail down No. 5, but maybe No. 7 suits him."
While Farbrace, now director of sport at Warwickshire, does not have the influence around the England camp he once did, his views remain intriguing. In his role with England, he was a steadfast supporter of Buttler. If even he is losing confidence, it suggests Buttler is running out of chances.
Still, Farbrace remains a fan of Buttler and feels he has the talent and technique to make it in Test cricket. Addressing a roomful of young cricketers on the PCA's recent rookie camp, Farbrace told them how the previous team management had urged Buttler to simply relax and enjoy the game. Then, Farbrace believes, he requires only greater game awareness to enjoy more consistent success.
"I was watching the IPL on television a couple of years ago," Farbrace said. "Jos' shoulder were hunched. So I phoned him and asked 'are you enjoying it?' He said 'not really.'
"So I said, 'what would your mum - who coached him when he was a boy - have said?' And he said 'try to enjoy it.' I said, 'well, there you are then.' He went on to make 50s in his next five games.
"We always told him not to worry about the level of expectation. Just go in and play the game and remember why you play: because you enjoy it.
"Jos is a very selfless cricketer. Everything about him as a character and person, you can see why everybody wants him in the team. He has that great knack of saying the right thing at the right time, he is a brilliant professional, he has got a lot of experience and is a team-orientated person like Ben Stokes, Joe Root and Moeen Ali. You want those people in your team.
"There are times when he has gone in and played in a way that worked for the team. He got one of England's best hundreds in recent times against India where he had more dot balls in a hundred than anyone else for ten years [his leave percentage of 24 was actually the highest of England's most recent 30 Test centuries]. It was a rearguard action at Trent Bridge.
"I think it is being able to understand where the game is at and what he needs to do. That is what the best players can do. They work out the situation and play accordingly. That is what Root, Cook and Stokes can do.
"I don't think it is a technique issue. He knows he has the defence. It is a case of trusting that defence and being clear on the situation of the game and playing accordingly. He has to work out what the game needs from him at the given moment. It is what Stokes has done brilliantly.
"Over the last 18 months he has worked out how he needs to play for the current situation. The more he does it, the more confidence he has. All Jos is lacking is consistency of being able to bat according to the situation."
England fly to Sri Lanka on Monday and begin their tour with a three-day match against a Board President's XI starting on Saturday.
George Dobell is a senior correspondent at ESPNcricinfo