Andy Rubin was actually quite forceful when speaking to AllThingsD, going as far as to say:
Rubin said that he ultimately thought that customers "don’t have to go in the store and feel it anymore" when purchasing a device. Given the comments earlier from Jamie Rosenberg confirming Google's support of selling devices through the Play Store, these comments from Rubin don't seem that out of place.
Of course, saying that Google doesn't have plans for retail stores doesn't discount the possibility of more pop-up stores and stores-within-stores for the holiday season. We'll have to see on that.