I would split the issue in 2 parts:
- Remove single line button
- Add new line to the list button
In the original design the "minus" button was outside the input field, but then you moved it inside. I can see that it could be more aesthetically pleasing, but users may accidentally hit it while trying to click inside the input field to edit/select the text. Then, while the user is editing a line, the right side of the input field would be quite crowded and confusing, since it would say "28 characters left: Remove".
On the other side, a circle button with a minus inside of it could not be that clear (the minus icon has many other meanings): probably a trash can icon would be more clear, since it is the standard icon for the "delete" action. A red button with the trash icon and the "Remove" label would leave no doubt to the user.
While the "remove" button has to be quite close to the input field, since it relates directly with the item, the "Add branch" button should not be bound with any particular item. The "Add branch" relates with the whole list and not with a specific entry of the list.
So it makes completely sense that it isn't paired with a button (the minus) that has the opposite meaning and that relates with another thing.
An alternative implementation of your button would be adding an empty line to the list, with a green action button "+ Add Branch" on the left, so users, in order to add a new branch, would only need to select the empty input field, enter the branch name and click on "+ Add Branch" or press Enter.