I've published my android app where in every screen I've put a back button in the actionBar. Now considering that every android phone has a back hardware button, should I remove this button or is it okay if it stays as-is even though I guess it will be hardly used?
If you added this back button, to make the app work like an iOS app you should indeed remove these.
Android does suggest adding an arrow (home) action to the action bar on the left hand corner, this however should direct to the parent activity, does not have to be the same functionality as the hardware back button.
And yes, it's available on all devices since forever ;).
maybe usefull, the droid docs and the example I mentioned is on top. http://developer.android.com/design/patterns/actionbar.html
If you have more questions about the android UI, these docs handle most things you would wish to know, give it a read!