This answer comes in two parts - activity & terminology.
I think the difference here is mostly a matter of intent. While the actions are similar on the surface, the reasons to take those actions differ, so the fine details of those actions also differ.
Typically, if you are shadowing somebody, the implication is that you are trying to learn how to do what they do so that you can do it yourself in the future. The intent is to gain more skills by observing those who already have them.
If you are performing a contextual inquiry, you are trying to learn how they do what they do. The intent is to better understand their goals, needs & actions in context.
The actions are similar - but the intent differs. As a result, the nuance of what is recorded or learned in each situation will likely differ too - even though the mechanics of the process is similar in each case.
Shadowing is easier to say (and to spell) and generally a more accessible term to those outside the field. It's less specifically accurate for the reasons stated above, but is exoteric (meaning it is understood broadly, outside of its specific field). A layperson will know what shadowing is, and it's a close enough word for their needs.
Contextual Inquiry is more specifically accurate within our field for the reasons already stated, but is esoteric (meaning it is only understood by a narrow group). If you're communicating outside of the narrow group (UXers) then the exoteric word will create less friction, but in the narrow group the esoteric word will convey the more specific meaning. A layperson probably won't know what a contextual inquiry is, but a UXer will know the phrase...
It should be noted that shadowing also has esoteric meanings in other fields (HR, training, security, tradecraft, etc...) which means that while it's understandable to laypeople, it may give wrong ideas to those who know it from those other fields.
In most cases, there won't be any confusion caused by the difference, but if things start to get very specific, then the difference may become more relevant.