Best Practice: Both should be used.
As a social scientist studying people from the United States and their relationships between technology, I can say that in my experience both has always been better.
You wrote:
This seems redundant if the app can already detect whether the user is new or existing.
If you are using non-standard ways to detect if a "user" is logged in, you should provide the option to not use this feature or build it into logging in and logging out.
Example: People on shared computers can be very sensitive about "auto-logging-in", so to keep it simple, allow them to sign up or login, and logout without any side affects. Offer options for longer sessions if you must.
If you want to redesign the login/signup experience based on the logic you've placed here, you need to keep in mind that it's not just about what is "logical" but what is culturally afforded within a groups understanding of the way something works. If my assumption that many users are quite experienced and expectant that a login page has a sign up link and a sign up page with a login page, then your design should alleviate what users understand to be problematic about the affordance. I've observed "users" who already have an account use this affordance as a login strategy
- Enter site
- Then click 'Sign Up'
- Then click 'Have an account? Log in'
- Login
The study showed that some "users" employ this strategy based on the reliability of another ubiquitous feature of websites and apps. We can say it's incorrect but we can also gain an understanding of how certain groups of "users" have learned the way that this part of the web works.
If you want to focus on your "user" base then you should run a study.
FYI:
I don't assume that 3 options will overwhelm the user, assuming the "user" can read moderately and uses technology quite often, may not feel overwhelmed by reading 3 call to actions that point them in various directions. It makes it sound like "users" are hypersensitive neurotics, when quite typically they are faced with much more complicated options like what's for lunch.
In short: