The Google Material design guidelines on cards is a good read, recommended.
You didn't specify whether your cards are of equal size or not but based on not "mess up the grid" they probably are of equal size.
Grid of different sized cards is a possibility but you should be extra careful not to overload them with actions.
From Google Material design:
The primary action in a card is typically the card itself.
Supplemental actions can vary from card to card in a collection, depending on the content type and expected outcome; for example, playing a movie versus opening a book. Within cards in a collection, position actions consistently.
Supplemental actions within the card are explicitly called out using icons, text, and UI controls, typically placed at the bottom of the card.
Limit supplemental actions to two actions, in addition to an overflow menu.
Although cards can support multiple actions, UI controls, and an overflow menu, use restraint and remember that cards are entry points to more complex and detailed information.
So consistency and restraint is the key here as Michael Lai said:
if you are not careful you'll find that it becomes a dumping ground for information and functionality
So your grid could look something like this:
download bmml source – Wireframes created with Balsamiq Mockups
These cards are really similar with media on each and supplemental actions at the bottom. But at the same time kind of inconsistent with different amount of supplemental actions.