Google explains why ICS has no built-in visual voicemail app
In theory, this seems like a good idea, because visual voicemail is not yet a standard across all carriers, but our experience with Android has shown that anything that is taken out of Google's hands and put into the hands of carriers tends to cause delays. Of course, if you don't trust your carrier, there is always the Google Voice option (assuming you live in the USA) which will be able to handle your visual voicemail needs. And, there are a number of 3rd party apps which can handle the task.
We'll have to see how this shakes out, but we don't think the news is as bad as it sounds, especially since there is the access to voicemail in your call log. We'll definitely start testing out 3rd party visual voicemail apps and give you a breakdown of the best options though.