 For instance, the lexical grammar for many programming languages specifies that a string literal starts with a " character and continues until a matching " is found (escaping makes this more complicated), that an identifier is an alphanumeric sequence (letters and digits, usually also allowing underscores, and disallowing initial digits), and that an integer literal is a sequence of digits.So in the following character sequence "abc" xyz1 23 the tokens are "string", "identifier" and "number" (plus whitespace tokens) because the space character terminates the sequence of characters forming the identifier.Further, certain sequences are categorized as keywords – these generally have the same form as identifiers (usually alphabetical words), but are categorized separately; formally they have a different token type.