78

I came across a number of login configuration settings where there is a list of allowable special characters and was wondering:

Does this limitation cater for a specific security or usability need?

Example: A list of special characters supported by Oracle Identity Manager and Microsoft Active Directory for password field :

enter image description here

Update:

Thanks everyone for the generous response!

Every time I have asked a question that involves security and usability there seems to be a clear divide between proponents on each side. However this need not be as this is one area that requires a lot of compromises and trade-offs… UX depends on it!

Improve this question
21
  • 5
    There is a good thread about this topic over here: security.stackexchange.com/questions/17192/…
    – Tyrus
    Jan 26, 2015 at 17:53
  • 36
    The limitation generally caters for implementer-laziness, because they're worried that allowing arbitrary characters might break something elsewhere (which usually means they have bigger problems). Limiting passwords to specific characters drives me crazy, and there's no (good) reason for it ever (unless you're worried that not doing it will lead to injection attacks, in which case, fix the code you're worried about injection attacks in, and then fix your password validation).
    – neminem
    Jan 26, 2015 at 19:17
  • 10
    p.s. I'm not a UX person, I'm a developer. :p But more importantly, I'm a user. It drives me crazy when sites do this - just that, as a developer, I know why they're usually doing it. :p
    – neminem
    Jan 26, 2015 at 19:51
  • 6
    @Blam Right, obviously you don't have to test every possible unicode character, you test a random sampling of them, same as you don't have to test every string length, you test 0, 1, a few, and a random large number. And yes, smart people don't protect from injection attacks by limiting characters... but that doesn't mean that people don't still try protecting from injection attacks by limiting characters (usually, as Nathan Rabe points out, the same people also most likely to end up on plaintextoffenders.)
    – neminem
    Jan 26, 2015 at 21:13
  • 9
    "The grave accent cannot be reproduced in this document" --- seriously? What half-baked markup language are they writing it in?
    – Federico Poloni
    Jan 27, 2015 at 17:21

11 Answers 11

Reset to default
120

If the user can type it then it should be allowed in their password.

Telling someone what they can and can't use in their password always feels wrong to the user. Passwords are currently the most universal way to authenticate. Preventing users from entering anything is, in essence, telling them who they can or can't be.

1. Any printable character that a user inputs should be allowed.

The following characters are okay...

'A', 'a', 'á', 'Æ', 'æ', 'Ñ', 'ñ', '-', '_', ' ' (space), '\t' (tab), '\n' (newline), ...

Just because I don't know how to submit a TAB or ENTER character as part of my password doesn't give me the right to prevent others from doing so.
(Don't worry, few people will try to submit an ENTER character as part of their password but allowing the few that do will earn their respect.)

2. Keys that don't display a printable character should not be allowed.

The reasons for this should be obvious but for completeness I will mention the following keys which can be detected but are reserved for other actions. For example, a password input shouldn't record that the shift was hit multiple times...

[ctrl], [alt], [shift], [arrow keys], [apple key], [windows key], etc.

3. Not allowing certain characters makes users question your security.

When you prevent users from putting certain characters in their password it not only annoys people but causes many of them to question what else you are doing that isn't secure.

You may as well be saying...

"Hey we don't want to fix our application to properly deal with special characters so would you mind helping us out by making your password less secure?"

The rules below will allow for secure input while preventing a user from ever getting stuck:

  • Don't show the characters that the user is typing in password fields (there are some exceptions on mobile)

  • Having the user type in their password twice is usually sufficient in letting them know that they got it right (i.e. didn't accidentally add unintended white space etc.)

  • Having a password reset mechanism is important to handle any cases of accidental lockout.

4. Encouraging a user to add more isn't the same as prohibiting characters.

One way to help a user come up with a secure password is to make a game out of it...

password strength indicator

5. The future of authentication

"The Tech That Will Kill Passwords Dead" is a pretty good gizmodo article discussing the problems we all face with passwords. It also talks about some new patterns that could possibly replace passwords one day.

Many mobile applications are starting to allow users to show or hide passwords in plain text in order to increase ease of use and remove one barrier to entry. I would still avoid this because the problem it creates is worse than the problem it solves.

Even with a very intuitive mechanism for showing/hiding a password 60% of users still say it feels wrong to see passwords in clear text.

According to that same article it appears that Touch ID is on the right track and easily wins as far as usability is concerned. Touch ID still has some major problems that make it impractical. The biggest being that it only works on select devices and has issues with one person controlling multiple accounts.

Facial recognition is another contender as an increasing number of high pixel density cameras make their way into the world but this approach often leaves people worrying about privacy.

The one problem shared by all of these new authentication attempts is this: You are the password.

It's actually a lot easier to fake who you are than what you know. In addition, once you've been compromised it's nearly impossible to change (your fingerprints for example)

Passwords are the authentication mechanism of choice for a good long while so...

Please don't place arbitrary character limitations on my password. Thanks!

Improve this answer
31
  • 68
    Sites that only allow certain characters always make me worry that they are storing/transmitting the text of my actual password instead of a hash.
    – Nathan Rabe
    Jan 26, 2015 at 17:50
  • 62
    @Blam If a site says I can't use a character like @, %, $, or ; in my password, to me that means some script will be reading my password and they know those characters will mess something up. Even if the only time they parse the characters is to generate a hash, it seems like bad coding if it was possible to inject code via a password and all they did to fix it was disallow certain characters.
    – Nathan Rabe
    Jan 26, 2015 at 18:45
  • 11
    Even tabs or other non printable characters should be allowed; most users can't type them and so won't use them, but if someone is copy pasting or it's an automated script which uses those then let them do so.
    – user42730
    Jan 26, 2015 at 21:09
  • 18
    Just because it's a key doesn't mean it's a character (ctrl, shift, arrows, etc.); but if it's a character it should generally be allowable (including tab). If I want §™╔┌çÅ╡Θ as my password, I should be permitted to do so. I would probably make exception for non-printing characters such as ␀, ␇, ␌ and such (and the backspace character, because we want people to be able to edit a mistyped password!), but "non-printing" does not include the likes of ␉! (horizontal tab)
    – Brian S
    Jan 26, 2015 at 21:23
  • 15
    @AndréDaniel What should the we do with this password: 12345\b67890 where \b is the backspace character? What about this one: 12345\067890 where \0 is the null character, which is used to terminate strings? Control characters are intended to do weird things with text that are not appropriate in a password. If a user copy/pastes something that contains strange non-printable characters, they may not even be aware that they're present. Best to exclude non-printing characters. Then the user knows exactly what's in the password, and whoever wrote that bogus nonsense can fix their stuff.
    – jpmc26
    Jan 26, 2015 at 22:37
35

If a site requires that passwords only contain certain character codes, then a user will be able to enter the password into almost any device which is capable of producing those characters. If the password contains character codes which may be entered on some devices but not on others, then a user who creates a password on a device which could enter the codes contained therein but then later needs to log in with a device that can't, would be effectively locked out of his account.

On almost any reasonable platform, the 94 printable ASCII characters will be clearly distinct. Even if a font annoyingly uses identical glyphs for I and l, or for 0 and O, people who enter such characters will generally have no doubt about which they entered. By contrast, on some platforms a user might think he's entering a character like ɸ when he's actually entering a φ; if such a user moves to a machine where characters are entered differently, he may be unable to access his account unless or until he can figure out what characters he might have used in entering his password.

Things get further complicated if one factors in things like combining diacritical marks. Some characters like ë have two legitimate representations--either a single "Latin Small Letter E With Diaeresis" [code 0x000EB] or a "Combining Diaeresis" [code 0x00308] followed by "Latin Small Letter E" [0x00065]. Some devices may not allow the user to control which form is entered. Ideally the password would be converted to a known normalized form prior to hashing, but it's far from certain that all code which tries to "normalize" Unicode strings will always work the same way (even if it's specified to, that doesn't mean it actually will).

Improve this answer
24
  • 23
    I understand this point, however, just because you allow all printable characters doesn't mean people will do it. Preventing the few who try isn't a very good user experience... "Sorry that character can't be typed on other devices and is not allowed in your password" - Dang it I don't have any other device!
    – DaveAlger
    Jan 26, 2015 at 20:41
  • 6
    @DaveAlger: Although code could accept most Unicode characters without difficulty, some really shouldn't be, and there's no easy way to determine which ones those are. Even if code had a list of 70,000 characters which were known to be problem-free, a list of 94 permissible characters will probably be easier for users to work with than would be a list of 70,000.
    – supercat
    Jan 26, 2015 at 21:05
  • 11
    If the user chooses to pick those characters, you can trust them to know how to enter them. By far the most common use case for such users will be copy-pasting from a password manager, anyway, where this won't be an issue.
    – sapi
    Jan 26, 2015 at 22:04
  • 10
    @sapi Many users don't realize certain characters can't be typed on different devices, so it can depend on your demographic. For example, on desktop computers | will be entered, even if the keyboard shows ¦. However, on older Android devices, ¦ would be entered instead. So, sometimes it is necessary to protect your users from themselves, although an overridable warning is better than a restriction.
    – 0b10011
    Jan 26, 2015 at 22:16
  • 3
    @sapi: If one wishes to safely allow non-ASCII characters, one approach would be to say that if a password contains any non-ASCII characters or unbalanced ASCII-brace characters, all non-ASCII characters and any ASCII brace characters will be replaced by their character code, enclosed in braces before hashing. Such behavior would generally be transparent to the user, but would provide a means by which a user whose password contained e.g. the codepoint sequence 0030B+00065 could enter it, even if his system would normally replace it with 000EB.
    – supercat
    Jan 26, 2015 at 22:29
20

I would like to add to DaveAlger's point. I, like many people, create algorithms in order to better remember passwords. I've spoken to many people (in an informal manner) about passwords and I have heard a lot of objections

  • why can't I use a part of my email or my username in my password?
  • why is there a character limit? (affects my algorithm)
  • why can't I use special characters?
  • why must I use upper case, or numbers, or special characters?

Just about everyone is frustrated when they're forced to change their password. IF the password submitted is considered insecure (example 1234, 1111 or qwerty) then tell the user that the password is rejected as it considered insecure. But make certain that the password is indeed insecure even it doesn't meet some individual requirement.

Password blueOrangeMetsWasSheaNowCiti is safer than #$78rt even though it doesn't use numbers or special characters.

Limiting does not help usability as it can only frustrate users and, while I'm not a security expert, I cannot see how limiting a character set can, in any way, aid in securing a system.

Improve this answer
1
  • 7
    good point mayo! saying "Be sure to use a mix of characters that is hard for others to guess" is different than saying, "Don't use special characters in your password!"
    – DaveAlger
    Jan 26, 2015 at 19:22
3

It can make sense from a usability and support perspective.

If the character isn't possible to type on a keyboard/phone without using alt codes or copy-pasting.

Keep in mind that the most active internet enabled devices have touch screens. Your user could create the account from their laptop, then try to access the account with their phone, which isn't capable of entering in the Æ character.

And all whitespace characters should be cleaned as a generic space, trimmed from front and back, and multiple whitespace characters back to back being ignored and treated as a single whitespace. Why? Because lots of things have issues with whitespace, especially leading, trailing, and multiple whitespace characters in a row.

While you should probably allow whitespace (people like to use phrases now), you might want to inform your user how you tidied up their password, but that they don't need to worry about it if that is how they are used to entering it.

Also allowing unicode characters that then need to be piped over HTTP is another potential support ordeal.

Improve this answer
12
  • 7
    "you might want to inform your user how you tidied up their password" is a terrible concept. Looking at the Unicode character database, there are 11 white-space characters. By changing "Medium Mathematical Space" to "SPACE", you've effectively broken the ability for the user to enter their password. On a similar note, as a website designer, you shouldn't discourage users to be copy-pasting their passwords, utilities such as Keepass lead to greater user-unique passwords which is better for our entire industry.
    – AWinkle
    Jan 26, 2015 at 21:56
  • 8
    If my computer is set up to easily input Æ,ß,ž or 말, then nowadays you can be pretty sure that my phone can also do the same because I'll be using those characters in my everyday communication including the phone.
    – Peteris
    Jan 26, 2015 at 22:03
  • 2
    @Blam Are you kidding me? Of course my phone can easily enter all the non-ascii symbols of my native language and Korean users can easily enter the korean symbols on any non-ancient phone, etc. It's not something that needs to be specifically tested - almost every person with a smartphone outside of USA/UK is successfully entering non-ASCII symbols in their webbrowser, SMS, google queries, every day everywhere for many years now. In fact, in some environments it's harder to enter ascii symbols than non-ascii (a reason why numeric web domain names with no english letters are popular in China)
    – Peteris
    Jan 27, 2015 at 1:03
  • 8
    @Blam my point is that if a person has chosen to have a password in, for example, Russian cyrillic or Greek or Korean or some other alphabet, then it's a reasonable assumption that they will be able to use this alphabet in all their input devices and many of them will, in fact, strongly prefer to use that alphabet instead of the latin alphabet.
    – Peteris
    Jan 27, 2015 at 1:09
  • 13
    @Blam You can't assume that all people can use latin letters. There are millions of users online who don't know any languages with latin alphabet, and they don't use those letters - i.e., they authorize with a facebook account (linked to phone number instead of an email that they don't have), and a password in their native language. For them, remembering any password made up from latin letters would be as easy as remembering a random string in chinese for me - bad UX. People can and do use online services without knowing that in some strange faraway countries 'a' and 'A' are the same letter.
    – Peteris
    Jan 27, 2015 at 1:46
3

It depends.

If you've got reasonably strong control over the password input mechanism (keyboard layouts, software stacks, etc.), then letting users freely input anything they want is a good idea, because it maximizes the available password space. Someone attacking an English-language site probably won't try even obvious things like "كلمة المرور" (which Google Translate assures me is Arabic for "password"). In such a case, encoding mix-ups don't really matter, since any mix-up will be the same across all systems, canceling itself out.

On the other hand, if you're trying to support as broad a range of systems as possible, you should restrict passwords to the 95 printable ASCII characters, in order to keep programming and support nightmares to a minimum. Supporting everything means dealing with homoglyphs (Α, А, and A look identical to a human, but have different byte values), duplicate characters (the "micro sign" µ and the "Greek lowercase mu" μ represent the same character, but are encoded with different byte values), different composition forms (ñ and ñ look the same, but the first is an "n" followed by a combining tilde, while the second is a single precomposed character), and different ordering of combining charcters (ế can be expressed as either "e + acute accent + circumflex accent" or "e + circumflex accent + acute accent", which a computer sees as different) -- and that's just within Unicode. Garbled encoding transformations can mean that somebody's attempt to enter "Pässword" on an ISO 8859-1 system gets interpreted as "Pδssword" by an ISO 8859-7 system.

Improve this answer
2
  • 3
    Unicode normalization available in all non-broken implementations should ensure that strings obtained from user devices with varying composition forms and different ordering of combining characters in the end are equal and hash to the same value. Non-unicode encodings doesn't have any good solutions, but are they still a problem nowadays? 10 years ago it was a serious issue, but in the last few years generally even the oldest obsolete living systems seem to interface with the outside world by 100% unicode or ascii.
    – Peteris
    Jan 27, 2015 at 0:56
  • 4
    @Peteris, key word there: should. I've been in software development too long to trust "should" until it's been thoroughly tested and subjected to a few years of hammering in a large-scale deployment. As for encodings, ISO-8859-1 and Windows-1252 are still common enough to worry about -- but since they look like ASCII as commonly used, most people can get away with assuming they are.
    – Mark
    Jan 27, 2015 at 2:01
3

One problem that I've seen with non-ASCII passwords is that some systems deal with characters and others deal with encoding-specific code points. The "א" character might be represented in different ways depending on encoding, and sometimes the same character may be represented in any of multiple ways ("é" could equally be U00E9 or U0065 U0301).

Consider the Python2 -> Python3 transition. Serializing "שלום" in Python 2 and then comparing to the same value serialized with Python 3 will result in FALSE as Python 3 due to the changes in the way strings are represented internally in Python.

PHP has a similar mishap: it deals with bytes, not characters. So a user who has input "שלום" on a page with CP-1252 encoding may not be able to log in on a page with UTF-8 encoding. Combine this with the host of encoding issues that were inherent in using the mysql_* drivers (less so in the PDO drivers, which make it easier) and the problem is compounded.

In PHP I've dealt with the issue on non-English websites by ensuring that I'm properly connecting to the database via UTF-8 (Much easier since PHP 5.3.3 but problematic before that, even with PDO, so much so that I still remember the critical version number), using prepared queries and proper hashing, and that I'm always serving the page as UTF-8. However I've seen no end to the problems that my less pedantic colleagues face with the issue.

Improve this answer
5
  • @JanDvorak: Nice! Unfortunately PHP was only the example. Perl had it even worse.
    – dotancohen
    Jan 28, 2015 at 14:22
  • 1
    Perl wasn't meant for web development, was it? TBH, it looks a bit like an esoteric language that got a bit too popular to me. I do agree that poor unicode support in a language meant primarily for text processing is unfortunate. Reference?
    – John Dvorak
    Jan 28, 2015 at 14:28
  • 2
    The database part shouldn't cause problems with passwords, since they get hashed before touching the database in most applications.
    – CodesInChaos
    Jan 29, 2015 at 14:50
  • @JanDvorak Do you mean don't use Wikipedia or Facebook or a particular hobby community's forum? I imagine few people are likely to accept those "solutions".
    – Damian Yerrick
    Mar 8, 2016 at 20:16
  • @DamianYerrick I meant "don't use PHP", not "don't use web pages made in PHP". Wikipedia seems to cope pretty well with unicode support, and I'm well okay with recommending to ditch Facebook :-D
    – John Dvorak
    Mar 8, 2016 at 20:20
2

Yes, unfortunatelly, the restrictions on characters the user can use in the password have sense from the UX perspective.

If you're operating a worldwide service, you'd like your users to be able to log from all places of the world. If it's the case, you must take into account, that unfortunatelly, the keyboards are not standarized.

Event the layout of the basic characters is variable. For example, in Germany, gods know why, they've exchanged 'Z' with 'Y' and they've done a complete mish-mash with other characters.

Event if you, as a user, manage to find given button on the keyboard, it's still most likely you won't be able to type 'national' characters of your choice, because every country has own keyboard layout (virtual, or in extreme cases, like in Germany, even physical) and allowing any user to choose any keyboard layout is not an option, at least not in Windows machines.

Please note, that most users are not aware of that issues.

So there are practical reasons for limiting the choice of the characters one can have in the password to the set that can be (probably) typed by any user from any machine.

Improve this answer
1
  • How common is it for a given user to need to log in using a device with a different layout than one they are used to? Anyone who travels with a laptop/tablet/smartphone will not need to worry about this, nor anyone who doesn't travel internationally. (Nor, obviously, anyone using a password manager.)
    – Nathan Tuggy
    Jan 14, 2016 at 2:38
1

A few guide lines:

  • Let user enter any character in the ASCII range of 32 (space) to 126 (~) - these should be the same in any character code.

  • Limiting your users to less characters will only frustrate them and force them to choose less secure or harder for them to remember passwords.

  • Characters bellow ASCII 32 (and ASCII 127 = Delete) have specially meaning e.g. ESC, Enter (submit form), Tab (jump field) and other characters that are not meant to be typed and therefore should not be accepted.

  • Characters above ASCII 127 may not be typable on some keyboards or devices (may prevent login via phone or via other PCs while abroad) and require unicode storage (less of an issue, however, you need to be aware of it)

  • Don't limit the length too much - e.g. let users enter 10s of characters e.g. up to 50 or to 100.

More details on passwords in my answer here.

Improve this answer
0
1

Authentication and security are critical. A security breach will kill you.

Do you have any idea what goes on between a keyboard and server? You have normalization, encoding, ambiguities in Unicode, serialization, NAT, man-in-the-middle, and other measures. That is a secure end-to-end transaction that is used for the entire session. Bad guys want to take advantage of any of that stuff.

A common security practice is to limit the attack surface and protect it like a soldier.

This is not programmers being lazy: it is about protecting a very sensitive and critical function that lots of bad guys are tying to exploit.

To say “I want to use any character, but protect me” is like saying “accept any form of ID but assure me that they are who they say they are”. I cannot get on a plane with my school ID for a reason: it is not as secure.

The bottom line is that a functional secure password can be created from 128 characters. There is no security reason to support more. To support all Unicode is a needless security vulnerability.

Improve this answer
9
  • 3
    "Reducing attack surface" and "maximize available keyspace" are here directly opposed. Why do you consider that the former takes such precedence?
    – Nathan Tuggy
    Jan 27, 2015 at 7:13
  • @NathanTuggy No they are not. More unique characters is more attack surface. Why do you think you need more than 128 characters to create a valid password?
    – paparazzo
    Jan 27, 2015 at 14:00
  • 2
    You're just reiterating what you already said, without explaining why they aren't opposed. The keyspace with two valid characters is adequate, in a technical sense, since you can simply construct enormously long passwords; that doesn't mean that's a secure choice, though.
    – Nathan Tuggy
    Jan 27, 2015 at 19:48
  • 3
    Characters 0-32 and 127 are problematic too - that leaves less than 128. Regarding security breaches - injection is an issue if the code is bad, however, basic characters e.g. ' which should be OK for passwords falls under this condition. If you save the encrypted password and not-plain text (which you should be doing anyway (including salt)), then SQL injection is not an issue.
    – Danny Varod
    Jan 27, 2015 at 23:33
  • 1
    Honestly, NAT? What possible relevance does that have? Telling your Chinese, Arabic, Korean, Russian, Hebrew, Greek, et cetera ad nauseam users that they must use Latin letters in their password is completely unacceptable in the modern age, and is not necessary for security.
    – David Conrad
    Jan 28, 2015 at 19:48
0

What if it takes two weeks to get the password reset by a letter being sent to my home address and I can’t access my bank on holiday due to their being a character in my password that I can’t type on my iPhone.

Will I blame the bank, will I consider changing banks……

What if my back decides at some later stage they wish to use drop down lists to input a password rather than a text field….

What if the keyboard I am using today outputs a different character when I press £?

Given the risk in resetting passwords, are we just creating a different risk by allowing users to have password that are more likely to need resetting?

However if the password can be reset just by the website sending me an email, it should allow anything in the password.

Improve this answer
5
  • 2
    If a password reset requires a physical letter sent to your home then they are still doing it wrong and you should absolutely blame the bank. Ditto for using anything non-standard like drop down lists.
    – NotMe
    Jan 28, 2015 at 15:20
  • @ChrisLively, drop down lists makes it a LOT harder for some password stealing virus. Doing the reset by a "back chancel" make it a lot harder for a fishing site to trick someone into resetting their password. Remember that is my be my Mother in Law that is being targeted, not one of us...
    – Ian
    Jan 28, 2015 at 17:05
  • 1
    I hope you're kidding, because you've now moved into "seriously bad info" territory. Drop down lists don't make it harder on anyone other than the person trying to enter their password.
    – NotMe
    Jan 28, 2015 at 17:49
  • @ChrisLively, they do, for example they stop keyboard loggers from working.
    – Ian
    Jan 28, 2015 at 18:07
  • 1
    There is a very simple answer to your last question. Simply allow users to use a wide variety of characters, and see whether the number of password reset requests actually goes up by a statistically significant amount. If not, don't borrow trouble.
    – David Conrad
    Jan 28, 2015 at 19:52
0

Limiting allowed characters in passwords to a sane subset of printable characters is a good idea. More flexibility is not always better. That's why we have speed limits on roads. Frequently, usability is about protecting users from their natural aptitude for shooting themselves in the foot.

From a server-side security standpoint, there is no problem in supporting full unicode passwords. Even if you don't want to handle crazy characters on the server side, a little bit of javascript coding can easily preconvert all passwords to hexadecimal notation before they are sent and handled by the server. But, for the reasons given above, one should do this -and- also limit to a sane subset of printable characters.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.