The Loudness Wars

Assuming my grasp on logic isn't completely broken, it seems to be that as technology progresses through the years, it should improve. Cars go faster, buildings get stronger, computers get smaller, things like that. Why, then, have CDs started to sound WORSE? Why is it that if I compare ABBA's "One Of Us" on a 1983 CD to the same song on a 2005 remaster, it has less dynamics and feels more muddy? Why does each drum hit stay at the same volume, but make the music behind it go away, instead of just being louder than it is? 

The problem here isn't the technology; how could it be, if the CDs we have now are the same as the ones we had in 1983? The problem, instead, is with the technique. In today's world of iPods (and Walkmen before them), music is fighting a battle with the noise surrounding it, all while being pumped out of (usually) awful headphones. There's no room for dynamics; whatever is loudest is all there is. It doesn't matter if you have a punchy drum beat, deep bass, and all that; coming out of a cell phone, you get some drum noises and mostly garbage. The solution provided by today's record engineers is to compress everything. Make the loud parts quieter and the quiet parts louder, often not by changing the levels on the original recording but taking what you have now and making it so loud that anything above the vocals just disappears. 

Wonderful. Now your music sounds great as a ringtone, or coming out of some earbuds in a crowded subway. Unfortunately, it sounds like garbage to anyone who tries to listen in any detail. It's not rich, crazy audiophiles that notice this; pretty much anyone with a decent pair of headphones or even car speakers will realize that the dynamics are gone. Things aren't louder than eachother. The sound just gets crushed together until you can barely tell what's going on; often you get a static pile of guitar noises, and the bass guitar is just gone. If you're listening to metal, you're in big trouble; if the bass drum was mixed up high, and then the mastering engineer compressed it, there will be parts of the song (sometimes the whole song if the drummer wants to show off his rapidfire playing) where you can't hear anything.
 
This is a bizarre incident of technology stumbling over itself, becoming weaker to become smaller, and bringing the big technology down with it. On top of that, it's an affront against talented mixing engineers, talented musicians (especially drummers, bassists, and rhythm guitarists), and my hearing.
 
A quick YouTube video describing the process can be found here:
http://www.youtube.com/watch?v=3Gmex_4hreQ