In general terms, it means that you will be able to seamlessly transition from one device to another and continue what you were doing. Of course, you'll have to be using Apple apps, but Handoff will bring "continuity" to the experience. Specifically, this means that you'll be able to start an email on any Apple device and pick up where you left off on any other Apple device. The feature will work with Apple Mail, messaging through iMessage, tab syncing through Safari, and it will even allow you to make and receive calls on your Mac, while the call is run through your iPhone as a relay. Handoff will also make it very easy to connect your Mac to an automatic hotspot created on your iPhone. Craig Federighi demoed the calling feature by calling Dr. Dre on his Mac, but the conversation was kind of awkward.
You'll need to be running OS X Yosemite on your Mac (and theoretically iOS 8 on your iPhone) for this to work, both platforms are due out this fall. The Handoff service works through AirDrop, meaning it starts with Bluetooth to create a Wi-Fi peer-to-peer connection. We've seen a lot of these features in various ways from Google with Chrome tab syncing and Hangouts, so it's hard to say how Apple's Handoff is going to differentiate.