Richardson, 34, has signed a two-year contract with Worcestershire and Middlesex have not offered Silverwood, also 34, a new deal. The club offered Richardson a combined coaching-playing role but he has decided to prolong his playing career.
Richardson played 43 first-class matches since joining from Warwickshire in 2005 and Silverwood made 35 appearances after his move from Yorkshire in 2006.
"Informing a cricketer and man of the quality of Chris that his services are no longer required by the club is tough," said Angus Fraser, the Middlesex managing director. "Chris has been a model professional during his four years with Middlesex and remains a highly respected figure. Throughout his time at the club Chris set a wonderful example to the younger cricketers on our staff.
Silverwood said that he was confident of continuing his career at a new county. "Naturally, I am sad to be moving on from Middlesex but I am keen to continue playing because I still feel I have plenty to offer the game. I wish Middlesex every success in the future."