Tokens

 Tokens  :  A token is smallest individual unit in a program. 

1. Keywords

2. Identifiers

3. Literals

4. Operators

5. Special Symbols

1.    Keywords: Keywords are the reserved words. We cannot use a keyword as variable name, function name or any other identifier. They are used to define the syntax and structure of the Python language. In Python, keywords are case sensitive.

Python    has  the following  keywords

False           class              from                  or

None           continue        global                pass

True            def                 if                       raise

and              del                 import               return

as                elif                 in                      try

assert          else                is                       while

async          except            lambda             with

await          finally             nonlocal            yield                             

break          for                   not                       

       2.    Identifiers: Identifier is the name given to entities like class, functions, variables etc.     In Python, It helps in differentiating one entity from another.

        Rules for writing identifiers in Python:

1. Identifiers can be a combination of letters in lowercase (a to z) or uppercase (A to Z) or digits (0 to 9) or an underscore (_).                                                                          Names like student, sno, marks, stu_mno, all are valid example.

2. An identifier cannot start with a digit.                                                                                   1variable is invalid,  but variable1 is perfectly fine.   
3. Keywords cannot be used as identifiers.
4. We cannot use special symbols like! , @, #, $, % etc. in our identifier.
5. Identifier can be of any length.

3. Literals :  Literals are constant values

Ex:   subject = “Computer Science”

         Pi = 3.14

         “\n” - newline

         “\t” -   tab space

 

4.  Operators: Symbol that used to perform arithmetic and logical operations

Ex : + - < and in << >>



5. Special Symbols: ‘ “ # . \ * + ? [ ] ( ) { } etc.