The problem is the chipsets, which include the radio. They have their own proprietary code, including some built in firmware. Along with things like roaming, negotiating frequencies, requesting MMS downloads and other niggly details, you have stuff like handling sim cards, emergency services modes, and public alerts. All of which I’ve heard are lightly documented and a pain to work with… It’s a lot of compatibility layers built up over the years

You can get a Linux phone today, the consensus just seems to be it’s not ready as a primary phone