It's good to keep buttons in the same place.
Here is a use case that could be pretty common. Let's say a user wants to just quickly glance at the additional information for the meeting. It's easier for the user to put the mouse in one location and click to expand (and retract), as opposed to clicking, expanding, moving the mouse to the new button location, then retracting. It's less mouse movement on the part of the user.
Along with the icon change and dropdown motion, this should be enough visual feedback for the user that their next click action in the same location will retract the dropdown they just expanded.
Here is an example of what I am talking about. This is how the dropdown would start:
As you can see, the arrow is point down to indicate to the user that there is more content. Then once it's clicked, this is how it would look to the user.
The button would stay in the same location, but the arrow would change directions, indicating to the user that if they click in the same location, the dropdown will collapse.
In this particular example, they had the button change colors as well. For your particular case, I don't know that you need the arrow color to change, but it couldn't hurt to do some user testing to find out what your users prefer.