It's a solved problem too. You just run the prompt and the result of the generation through a second pass of a fine tuned model checking for jailbreaking or rule breaking content generation.

But that increases cost per query by 2-3x.

Huh, so basically it's like every time my mom said "think before you speak". You know, just run that line in your head once before you actually say it, to avoid saying something dumb/offensive.