Users will have no sense of context
What if they got on that page from a different perspective? Possibly…
… they got there through Google
… they open the browser with an old session restored
… they open the page in a new tab in the background
In these cases they might not know or remember what website this page belongs to.
Yes it’s easy to scroll up or to check the address bar, but will people expect that to be necessary? If the context is lost, relevancy can also be gone, and it’s just as tempting to close the page as it is easy to scroll up.
You risk losing visitors
@SamsonTennela has a good point that this is also bad for the branding of the site and I share the idea of the smaller header. A small title and/or logo, a few navigation options and even breadcrumbs are better than giving no context at all.
Technically not ideal (possibly)