It's quite an age old question in UX/Universal Design. But, with time I feel the importance diminishing. Some reasons maybe true globalization of products/brands/english language/etc. I know companies still have to rebrand their products based on the country the are launching in, but those cases are getting fewer. Once again, just a personal observation.
On a related note, it is a good idea to test for color blindness. You can use something like http://colorfilter.wickline.org/ to test your website. Here is a list of different types of color blindnesses http://www.colour-blindness.com/colour-blindness-tests/.
If your user base is going to be huge, it is a good idea to make sure that you atleast test the site to find out the problem areas, even if you do not use it in the design. You can use the information to see if you need to add some contextual cues in specific regions.