What are tokens in C?

A token is a character, word or something that a programmer uses to create a program. It's like bricks, cement, sand and other things that will be used to make the structure. Similarly, tokens are their own predefined types or functions in the compilation that make syntax for a program.

 Whenever we are talking about the English language, punctuation, conjunction and words are tokens. In the C programming language, tokens are the very small unit in C and it is very important to the compiler. Basically, tokens are building blocks in the C  programming language. For example int main ( ) { int a=10, b=20, sum; sum=a+b; printf ("sum=%d", sum); return 0; } In the above program, tokens are main - identifiers { }, ( ) - special symbols int - keywords a, b, sum - identifiers 10, 20 - constant =   assignment Operator +   addition operator Types of tokens in C

CATEGORIZATION OF TOKENS IN C

Constant

•  Constant whose value is fixed, we cannot change their value throughout the program.

• Constant is also known as literals.

a. Integer Constant

• Integer Constant are decimal, octal and hexadecimal.

• Octal integer constant (0): 0 is added before a number like 022 (it means given or mention number is octal).

• Hexadecimal integer constant (0x or 0X): 0x or 0X is added before a number like 0x345 or 0X345 (it means given number is hexadecimal).

• Decimal integer constant: 223 is an decimal number.

Octal number is lie from 0 to 7 (01,2,3,4,5,6,7)

Hexadecimal number is lie from 0 to 16 (0,1,2,3,4,5,6,7,8,9,A(10),B(11),C(12),D(13),E(14),F(15)).

Decimal values lie from 0 to 9.

Space, commas and special characters are not permitted between digits.

b. Float Constant

Float Constant are also known as real constant. Decimal values in mathematics are called float constant in the C programming language.

For example: 17.4456, -89.67, 0.65e4.

c. Character Constant

• The single character constant is enclosed in single quotes like 'A', 'c' or '5'.

'5' and 5 both are different.

• printf ("%c", a);
This statement will print the character 'a'.

• printf ("%d", a);
This statement will print the value of character 'a' that is 97.

d. String Constant

• A string is an array of character and always ended with a null character ['\0'].

• String constant is enclosed in double quotes.

• "Hello World" is a string constant

• It can contain any special characters, space, numbers etc. like "Welcome 98", "123457654", "123%^\$@#".

2. Keywords

• Keywords are reserved words in the C programming language. We cannot use as variable, constant or string. It has a special/unique meaning for the compiler.

• 32 Keywords are defined and we can not redefine these keywords in our C programming language. These 32 keywords are mentioned below:

 Keywords

3. Identifiers

• Identifiers are the variable name for example array name, string name and function name etc.

• Identifiers consist of alphanumeric characters (A-Z), underscore, digits. Identifiers first character should be a letter or underscore ( _ ).

• Identifiers are case sensitive. We cannot use keywords as identifiers.

For example

#include <stdio.h>
int main()
{
int RollNo=10;
char Name_of_Student[ ] = "ABCDE";
float marks=12.5;
int arr[ 5]={1,2,3,4,5};
char row='A';
printf("%d\n%s\n%f\n%c\n", RollNo, Name_of_Student, marks, row);

//Note: RollNo,  Name_of_Student, marks, arr, row are identifiers.

return 0;
}

4. Operator

Operator is symbols those tell the compiler that which type of operation is performed.

Type of operator

1.   Arithmetic operator
2.   Logical operator
3.   Relational operator
4.   Bitwise operator
5.   Unary operator
6.   Conditional operator
7.   Assignment operator

For example

output = a + b;

= Assignment operator

+ Addition operator (Arithmetic operator)

a and b are operands

5. Special symbols

They have some special meaning in the C programming language for the compiler and it is used to perform some special function/task in C.

a. Brackets

• Square brackets [ ]: It is used to define the size of an array, string etc.

For example int arr[5];

The integer data type is of 4 bytes and arr[5] means array arr contain 5 integer data. How many bytes arr[5] occupy is 5*4=20 bytes. 20 bytes are allocating in memory.

• Curly brackets { }: These brackets are used to define the scope of block.

{   <====  Opening brackets

} <====   Closing brackets

Statement/code/argument is written in between of opening and closing brackets.

For example

#include <stdio.h>
int main( )

{                    // Opening bracket

printf ("Welcome to My blog");           // statement
return 0;

}                // Closing bracket

• Round brackets ( ): These brackets are used to defined function like main ( ), for ( ), strcpy ( ), strlen ( ) etc. and function parameters are written inside these brackets.

For example

for (i=0; i<10; i++)

b. Semicolon (;): In the English language, we use a full stop (.) to terminate from the statement/line. Similarly, a semicolon is used to terminate from statements in the C programming language. This symbol tells to the compiler, it is the end of the statement and moves to the next statement.

c. Comma (,): Comma is used to separate more than one statement. It is not used for the termination of the statement. Comma separate more than one statement like separating parameters in the function call.

For example:

int a=20, b=30, c,i;

d. Assignment operator (=):  This operator is used to transfer the data from one variable to another variable.

For example:

g=17;
The value 17 is transfer to value g.

or

int k=20;
g=k;

The value of k i.e. 20 is transferred to g with the help of the assignment operator.

e. Asterisk (*):  This symbol is used for multiplication and to create pointer variable.

Please write comments if you find anything incorrect or you want to share more information about the topic discussed above.