Ouch. At first, I was convinced of one option. But thinking more carefully the answer, in my opinion, is Depends, and all 3.
1. Depends on the input field
a. As user types
There are different types of inputs, and some inputs need different validation.
A password could be validated as the user types. If your site does not accept for example simple passwords like "123" that could be validated on the fly. You see this in "password straight" evaluators.
Choosing a username could be also validated on the fly as user types.
Typing a phone number with non-numeric characters, like letters, slashes or spaces could display a warning when detecting one.
b. When user leaves the field
This should mark some possible typing errors, for example, a not valid email, a name with only one letter or an invalid postal code.
c. Send or Next Step
Here is a bit obvious that a "Send or Next step" button should mark the empty required fields.
2. Aditional help
Some fields could have additional help like autocomplete when typing common things like places, county, probably States.
3. Validation warnings of different intensity
What I want to explore here is that some validations can be for example more subtle, for example, when typing an email the field could be a light gray color as the background, and when the email is valid (after having an @ and a final .something) change to white.
But when validating empty fields this same field can turn to a strong angry red.
4. Choose the right color scheme from start
I need to use a cell phone provider's page constantly to add some cash to my phone.
But the "modern cool sleek" color scheme they have makes a check box for an "I accept terms and conditions" almost totally invisible. I assure you, that I would mark it if I could see it from start.