I noticed that it doesn't seem possible to remove the primary phone number while two-step verification is enabled. Unfortunately, that weakens the security of two-step verification, even when using an authenticator app.
This Wired article provides a great overview of the weakness of SMS-based two-step verification and its vulnerability to "sim swap" type attacks:
https://www.wired.com/2016/06/hey-stop-using-texts-two-factor-authentication/
I'd love to see the ability to remove phone numbers as an override for authenticator-based two-step verification. I'd also love to see Ev