This article may contain personal views and opinion from the author.
Before we take a look at what people see as a problem, let's just take a look at the points of Google's policy that continually get overlooked. First, Google expressly states in its Terms of Service:
That's pretty plain English stating that what is yours is yours, no troubles there.
So, if you take those two things together, here's what you get (and this applies to Google Drive and most other Google products): you own your data, and no one gets to see your data unless you say so, someone higher in your organization says so (domain admin), or a judge says so (legal purposes). The external processing seems like an outlier, but really that's to cover hooked in services. For example, Google has to be able to share your data with Aviary if you have the app connected to your Google Drive.
Out of context
While people tend to ignore, or downplay the importance of the above sections of Google's terms and policy, there is one piece of the Terms of Service that is always quoted. The part of the TOS that is always taken out of context is this:
The first bunch of verbs ("host, store, reproduce, modify, create derivative works") all cover the process of making your content useful. If Google can't host or store your content, it disappears, which is of no help to you. If Google can't reproduce your content, it would be impossible to have the syncing across devices that makes Google Drive, Music, or Photos useful. Modifying and derivative works covers anything from auto-correct in a document to translations or fixing the color of a photograph.
The Bogeyman vs actual harm
Google has already seen the negative side of making a mistake with user data after the debacle of Google Buzz, and that was just a mistake, not an active misuse of content. Google's entire business plan relies on having a good relationship with users. If people don't trust Google, Google gets no data, and can't sell you ads. Google could certainly take advantage of that trust, but the repercussions would hurt every product that Google has.
So, here's a crazy idea: let's not pounce on Google (or any other company for that matter) for a potential problem, if that company has given no reason to ever be mistrusted. We get huge amounts of value from these products: they allow us to be more social, more productive, learn easier, and find entertainment any time we want. Sure, any of these companies can treat us badly, or misuse our data, but there's no reason to do that. As we've explained before, there is a big difference between "personal data" and "personally identifiable data", and the key to the personal data trade is for companies to use the former, while protecting the latter. So far, Google has done a very good job of that, so maybe we should wait and see if Google actually does something wrong, rather than attacking just because we can imagine the possible Bogeyman.