MacKay’s paper talks about gaining bits as in bits on a hard drive
I don’t think MacKay’s paper even has a coherent concept of information at all. As far as I can tell, in MacKay’s model, if I give you a completely randomized 100 Mb hard drive, then I’ve just given you 50 Mb of useful information, because half of the bits are correct (we just don’t know which ones.) This is not a useful model.
MacKay’s paper talks about gaining bits as in bits on a hard drive
I don’t think MacKay’s paper even has a coherent concept of information at all. As far as I can tell, in MacKay’s model, if I give you a completely randomized 100 Mb hard drive, then I’ve just given you 50 Mb of useful information, because half of the bits are correct (we just don’t know which ones.) This is not a useful model.