For instance, say a site is for merchants and consumers; people should either sign up as a merchant or a consumer – they require two different forms located on different pages. From the home page, should I have two different buttons, each to the appropriate signup page? Or, in an effort to reduce clutter and because more consumers than merchants will sign up, do I make a large "Sign Up" button front and center on the home page for consumers, then just have a smaller link in the top menu that says "Merchant sign up"?
I would have one prominent signup button on the homepage. As there are more consumers than merchants, display the consumers form, but at the top of the form create a link for merchants, something like:
Are you a merchant? Sign up here.
Which would then take you to the merchant signup.
Or for something a little more elegant, if a merchant clicks the link: fade out the consumer form and fade in the merchant form.
I would recommend going with a prominent sign up button but on opening the form, allow the user to choose the form he needs by use of a radio button.
It would also be good if dynamically a message can be shown which informs the user which form he has selected just in case he has made a mistake
Should it be possible, try to provide one type of account with additional rights for the merchants. This will reduce users confusion I think.
If this is not what you can do, I would go the second way you proposed: bigger signup button for regular users and a link for merchants. How much you can or should reduce the link depends on the merchants' awareness of the system. If they are educated to use it, even a link in footer will do. Otherwise you can try to make the link look like it directs to a completely separate section, e.g. You can put it in the top menu to the very right, add different background and name it "merchants section".
It would depend on the frequency of each signup type. A lot of websites (and companies by extension) do not have that many merchants or partners to warrant programming separate signup screens, with the merchants or partners being entered manually and an unobtrusive to main navigation login link for them exists. Another way to look at it is that (typically) the merchant website is entirely different to that of consumers, tying up the two from the outset can put you on the wrong track which you fight all the way.