I think it will certainly cause a performance hit if the encryption is CPU bound and not IO bound.
If it is CPU bound, then yes, of course. While it may have been the case with MartjeB's Atom D510 (although to be sure we'd have to see figures of read throughput without encryption on the same system), I still don't see any evidence that the Michael's pgbench test on encrypted LVM was CPU-bound.
As you mentioned yourself, even without AES-NI support in the CPU an i5 should be able to encrypt at ~100 MiB/s, that's much more than 10% of X25-M's read speed, so it doesn't explain an almost tenfold drop in pgbench performance. And I'm sure my dbench tests weren't CPU-bound (my i7 740LM does have AES-NI), and still I see massive performance drop in a write-intensive test on an encrypted volume.
One point I'm ready to concede is that in addition to the corner case I mentioned in my previous post, I see at least two types of modern systems where encrypted LVM performance may be a problem: low-spec netbook/smartbook with a fast SSD, and a cheap NAS with ARM or Atom CPU driving a RAID array of fast SATA disks.