When I first saw “save all weights to on chip hardware”, I thought it would be super expensive, but actually saving like 5 times the GPU’s memory to a seperate flash chip would only cost $20 (80GB*5 at 5 cents per gigabyte for flash storage). It can be way cheaper bc it’s low bandwidth and slow.
When I first saw “save all weights to on chip hardware”, I thought it would be super expensive, but actually saving like 5 times the GPU’s memory to a seperate flash chip would only cost $20 (80GB*5 at 5 cents per gigabyte for flash storage). It can be way cheaper bc it’s low bandwidth and slow.