There's an obvious problem to putting the keychain attachment on the drive itself rather than the cap: your keys remain attached to the drive. This'd be a problem if the port is in a tight place, like the back of the computer, or if you need your keys for something else while you need the drive plugged in.
I think there's an obvious solution, too: put a secure fixture on the drive that will detach via some manipulation from your keychain. Something spring-loaded and small, like a carabiner. If you had a small carabiner permanently attached to the drive that could quickly pop off your keychain (but otherwise stay reliably attached), it'd solve the problem perfectly.
All Comments
I think there's an obvious solution, too: put a secure fixture on the drive that will detach via some manipulation from your keychain. Something spring-loaded and small, like a carabiner. If you had a small carabiner permanently attached to the drive that could quickly pop off your keychain (but otherwise stay reliably attached), it'd solve the problem perfectly.