I know memory is fairly cheap but e.g. there are millions of new videos on youtube everyday, each probably few hundred MBs to few GBs. It all has to take enormous amount of space. Not to mention backups.

  • green_light_stop@kbin.social
    link
    fedilink
    arrow-up
    0
    ·
    1 year ago

    There are also techniques where data centers do offline storage by writing out to a high volume storage medium (I heard Blu-ray as an example, especially because it’s cheap) and storing it in racks. All automated of course. This let’s them store huge quantities of infrequently accessed data (most of it) in a more efficient way. Not everything has to be online and ready to go, as long as it’s capable of being made available on demand.

    Edit: Clarifying that tape medium is typically used for the longest term storage with the caveat that read is slow, so that used for the stuff that is least likely to be accessed. For things that are accessed infrequently but still need to be available relatively frequently you can have a “caching layer” which is what I was referring to with the discs. It’s a tradeoff between speed of access and information density. Here’s an article from 2015 where Facebook/Meta is discussing their design: https://engineering.fb.com/2015/05/04/core-data/under-the-hood-facebook-s-cold-storage-system/