This information was discovered on the code.google.com website and includes emails from two different Google employees, both revealing that they are using the Nexus 4 with the "LRW52G" build of Android. For comparison purposes, Android L on the Nexus 5 is build "LRW66E."
Why wouldn't Google want to make the Nexus 4 support Android L? After all, the entry-level Android One handsets will run Android L and the Nexus 4 is higher spec'd with 2GB of RAM and a quad-core 1.5GHz Snapdragon S4 Pro under the hood. It might not be the Nexus 5 with its quad-core 2.3GHz Snapdragon 800 and 2GB of RAM under the hood, but it should have more than enough zip to run Android L.