JohnGB is technically correct that in terms of control, the size of the mouse pointer is not as important as for touch. Here are some more points that may be helpful.
Control
The size of the pointer makes no difference to the pointer's control over one pixel of your screen.
Visibility of the Pointer
The larger the pointer, the easier it is to find. This matters most when users are accessing your site on high resolution/large monitors where small cursors are more likely to get lost. It also matters if you are dealing with a noisy/highly colorful background. Here, both the size and the color of the pointer are relevant attributes.
Interference with the background
If the pointer is too big, it will obscure a larger area of the document to the bottom-right of the 'point'. The importance of this depends on your application, but of course in general terms try to avoid putting important information immediately (i.e. within a few pixels) to the bottom-right of a clickable button.
Meeting users' expectations
One could argue that most users are used to a certain range of sizes and colors with respect to mouse pointers (think of the default small white/black Windows and Mac pointers). Unless your site is supposed to be deliberately quirky, bias yourself towards using basic control interfaces that your users are going to be familiar with. You don't want to try to appear original at the expense of annoying some of your users who feel they're being distracted by their own cursor.