Nobody is going to SIM swap you unless you have a shitload of crypto and let everyone know about it. It’s not an easy attack, so it would have to be targeted. Pretty easy to not be a target (not having millions of dollars of crypto on a wallet helps).