We are naturally drawn to movement, so shaking any button will cause people to notice it more. That however is not necessarily a good thing. It isn't someone noticing it in a good way. You are very likely to annoy or people who are getting their attention drawn to something that they may not want to be looking at. Think of those annoying banner adds that would do things like this.
There are situations where it is useful to draw attention, but those are when you are telling users something useful. The MacOS dock is a good example - if an icon shakes, I know that something has just finished and requires my attention. It one of them constantly shook because author of the software wanted me to use it more, I would simply not use it.
So in short, having a shaking button will draw more attention and might increase conversion rates in the short term, but in the long term you are going to annoy people. You need to decide which is more important to you.
My personal opinion is that you shouldn't, and if I come across a site or app that does it, I will close the page or remove the app.
It depends on the goal of the button. As a "Buy now!"-button on a commercial website, it proved very negative in experiments (A/B-tested, audience ~4 million in total). But I can imagine that a late-stage "Questions? Chat with us live!"-button appearing in your screen and giving a shake might prove much more positive. Of course, I'm assuming you would take into account the costs of having people respond to chat requests and compare it to the increase (or decrease!) of sales, and how much sales are valued even at a higher cost.
Personally, I loathe them. A website throwing a non-crucial element at me that takes up a prominent spot and shakes will swiftly be removed by a browser plugin such as adblock. At that point I will never see that button ever again. But I'm far from the average internet user.
I'd advise against it, but I'd love to see explanations as to why your experiment proved positive. Makes me wonder if I should retry my own experiment.
Shaking is going to draw attention to the icon, but at the cost of attention to other elements on the page, and possibly decreased consumer confidence.
You're right to think that animation will grab a user's attention - human peripheral vision is very sensitive to motion, much more so than to colour - so an animated button will get more looks than a non-animated one. But if visibility isn't the bottleneck on conversion, then all you're going to do is stop users from being able to read the rest of the page.
Worse, if the users have to read the rest of the page before they convert (i.e. to be sufficiently convinced), your distracting button will actually impede conversion. And given how suspicious users are of the web, anything that's perceived as 'spamminess' could be a double liability if you're trying to convince users you're trustworthy enough to give money.
There are much better ways to make an element visible, such as keeping it in the reading line and reducing the weight of surrounding elements. Try those out first before resorting to 'shaky' buttons.