Your main issue here isn't really "how to visualize that the frozen pane is frozen".
That part is easy. Add some gray background and some bold border, and you're halfway there :-)
Make it look different and title'ish.
Your real question is: "How do you visualize that there is more data above the top row?"
And question doesn't only apply to Excel's frozen top row. That question is relevant in other situations as well.
(Have you ever followed some link and after a while you realize that you're in the same web-page as before, just in another #Name section. Try this: http://en.wikipedia.org/wiki/Microsoft_Excel#File_formats. How do you know that you're in the middle of the page?)
The best way to visualize that there is something above, is to show a little bit of that content. So that the user doesn't believe that the topmost thing is the beginning of the content.
Unfortunately, Excel only scrolls by whole rows, so it's impossible to show half the content of the row above (or am I wrong?). The only (really lame) workaround I can think of is to merge two cells vertically. That way you might show half the content of the row above - leaving no doubt to the user.
The other way you can visualize that there is more above, is to use the Gestalt law on closure and give the user the feeling of an un-closed area above the visible area.
(Sorry about the bad visual appeal. Please put a little bit more work into the look.) :-)
I would also add that this depends a lot on the content you are showing, and the task the user is performing. If the user is looking at a list of numerically ordered items or alphabetically sorted names, the he will know that he will find more names above - especially if he is looking for it. If he is not looking for it, does he need to know that it's more content above?
Where is customer No#3 and where is Arthur???