The only answer is to do some user testing (great cadidate for A/B testing).
Just having icons can be confusing, people might mistake it as links to the product/company social media pages (commonly found on most website headers/footers).
You don't want users to be guessing, and so 'Login with ...' is certainly more useful and obvious.
But it also depends WHERE you are showing the links (context). For example, it may be easier to understand if the icons are on a login form (or a dedcated login page). Example -
User testing is a good way to validate your designs and concerns, but sometimes it saves time to go with common/usual design practices and focus your energy on more important UX issues.