Exactly! If it were just out putting exact data they wouldn't care about making new works and just pivot as the world's greatest source of compression.

Though there is some work researchers have done to heavily modify these models to over fit to do exactly this.