I've been using VirtualBox for some months now. It's pretty slick and doesn't get in my way too much. Just for the heck of it I've been experimenting with a few other similar systems: OpenVZ, vserver, KVM, and Xen. OpenVZ and KVM are the only two I've tinkered with long enough to at least have a clue what's going on.
I was trying KVM at the suggestion of someone in #debian who seemed to like it a lot better than VirtualBox. After playing with KVM I'm not sure where this opinion came from. Just getting KVM and libvirtd to the point where I could actually run a VM took over a half hour. The virt-manager GUI looked nice but was buggy in some spots, and the lack of guest OS drivers meant that many things were a bit sluggish. On the plus side, networking was much easier to setup -- but only sometimes. (If you add a network card *after* creating a VM you cannot choose which mode to run it in and are forced to use usermode NAT.) Another cool feature was emulation of a tablet pointing device in addition to a regular mouse device. This means that KVM doesn't need any guest drivers for mouse integration, because it set the pointer position using the tablet.
At the end of the day, VirtualBox still wins for desktop virtualization. I'm still keeping KVM on the radar though.