If you're making a mobile only application. Then you can do away with passwords and keep the phone number as the only way to authenticate the user. Facebook acquired whatsapp
does this and has always been using this method. Only one session of the user stays at any given time. If the user changes mobile phones but has the same phone number, then on re-activation the account on the new phone, whatsapp automatically deactivates the old client application on the old phone.
Password authentication is one that spoils the user experience. But it's a good way of authenticating so it's lived on for so long. Twitter recently launched digits a so-called password-replacement
developer tool. It follows a similar approach Whatsapp uses to authenticate.
You can use it.
Phone number authentication is rather a new concept, but it's catching up and is proven to work.
I mentioned mobile only application
because I'm not sure about how this concept will work on multiple clients and multiple sessions.