They become semi-side quests once you reach a certain point in the main quest line. Technically, I think you only have to collect just one to three of them to finish the story. I'm not 100% sure though, I too have been too distracted by non-mainline quest thingies.