One issue that users have been talking about is that the Nexus doesn't do so well with facial recognition in low-light situations. The trouble appears to stem from the fact that the display doesn't crank up the brightness enough to compensate and light up your face, and that means the camera find the details it needs for Trusted Faces to work. However, a Google Groups thread is saying that the issue has been fixed in the Android code and an update is on the way.
Given that Android 5.0.1 is the only update we know about (although there are likely others in the works), it seems reasonable to assume that this fix would be a part of this update. We'll have to wait until the Nexus 6 gets the update to confirm, but that seems imminent.