Jerry Yang asked Rice to use her forthcoming trip to China to pressure the government into releasing Shi Tao and Wang Xiaoning.
The two journalists were jailed after a Yahoo subsidiary in China handed over user information at the demand of Chinese authorities.
The case brought scorn from human rights groups worldwide and led to the families of the two men filing a lawsuit against Yahoo in August 2007 for its part in the incident. The case was settled three months later.
Yang claimed in his letter to Rice that Yahoo "deeply regrets" the incident.
"Yahoo was founded with the purpose of promoting access to information and free expression, and the fact that individuals are in prison because they expressed their ideas online runs counter to Yahoo's core values," wrote Yang.
Michael Samway, vice president and deputy general counsel at Yahoo, said in a blog posting that Yahoo has worked with Chinese authorities and human rights groups to get the men released.
However, he added that the word of the US government would hold much more sway.
"We recognise the practical limits of corporate diplomacy, especially next to the powerful advocacy tools available to governments," said Samway.
"We believe that governments have the most leverage to influence the decision-making of other governments."
'We are making good progress on 10nm,' claims Intel
Engineer calculates that Chengdu's plan to replace streetlights with artificial moonlight would cost $100bn
Research could also apply to other 'space weather' events involving hot, fast-moving plasma
Dark matter holds the Universe together - and gravitational waves could help identify it