So the easy or hard to drive a pair of phones is, depends on both parameters, the sensitivity and the impedance. The worst case would be a pair of phones of very low sensitivity and also a very low impedance. Why? because they'll be asking to the source more watts to sound equally loud as a more sensitive pair, and an important part of that power will be asked in the form of current intensity, which is something that most portable players, headphone outs in receivers and players, etc. aren't designed to deliver. This is the case of cans like AKG 701 or Denon D5000.
If your cans are low impedance but are very sensitive (the case of Grados and most IEMs) then despite their asking more current from the source, they still manage to sound very loud because they need very little power to do so.
Most people tend to think that low impedance equals to louder sound, but this is plainly wrong. It all depends on the sensitivity and how much power the cans need to give a high SPL. Also take into account that not all manufacturers offer their sensitivity values and not all them do in dB/mW but do in dB/mV. It's not much of a problem, you just need to convert the mV in mW knowing the phones impedance