Stringizing and Token Pasting Operators in C programming

Last Updated on: June 29th, 2022

C language has lots of advantages. In this tutorial, we are going to see Stringizing and Token Pasting Operators in C programming. Most of them don’t aware of this topic. Let’s start. Before that, we should know what is token in C Programming.

You can also read, bitwise operators in c, pointers in c, container_of macro in c, macro vs inline, and embedded interview topics.

Stringizing and Token Pasting Operators in C programming

Tokens in C

Every C program is a collection of instructions and every instruction is a collection of some individual units. Every smallest individual unit of a c program is called as a token. Every instruction in a c program is a collection of tokens. Tokens are used to construct c programs and they are said to the basic building blocks of a c program.

Example

int main()
{
   int a, b, final;
   a = 10, b = 20;
   final = a + b;
   printf ("Total = %d \n", final);
}

In the above program these are the Tokens —-> main, {, }, (, ), int, a, b, final

So, now we have an idea of Tokens. Let’s continue our topic.

Token Pasting Operator (##) in C

The token pasting operator ## defined by ANSI enables us to combine two tokens within a macro definition to form a single token. In another word, The token pasting (##) operator simply eliminates any white space around it and concatenates (joins together) the non-whitespace characters together. It is used to create new tokens.

Limitations

• It can only be used in a macro definition.
• It may not be the first or last characters in the replacement text.
• We can only use this method for valid tokens only.

Valid tokens are:

• identifiers (variable names, function names, etc)
• keywords (int, while, volatile, etc)
• literals (strings, numbers, characters, true or false)
• operators and punctuators (+, -, *, (, etc)

See the below example program, then you will get a clear picture.

Example 1

#include <stdio.h>

#define PASTE(arg1,arg2) arg1##arg2

main()
{
  int value_1=1000;
  printf("value_1 = %d\n", PASTE(value_,1));
}

Output

value_1 = 1000

Example 2

#include <stdio.h>

#define CONCAT(A,B) A##B

int main()
{
  printf("value1: %d\n",CONCAT(12,20));
  printf("value2: %d\n",CONCAT(12,20)+10);  
  return 0;
}

Output

value1: 1220
value2: 1230

Example 3

#include <stdio.h>

#define MY_DEVICE_BASE_ADDR  0x40004000
#define MY_DEVICE_REG_CTRL_OFS  0x20
#define MY_DEVICE_REG_STATUS_OFS 0x30
#define MY_DEVICE_REG_INTR_OFS  0x40

#define REG(name) ((unsigned int)(MY_DEVICE_BASE_ADDR) + MY_DEVICE_REG_##name##_OFS)

int main()
{
 printf("Example to use ## Pre-Processor!\n");

 printf("*** My-Device Register ***\n");
 printf("Control-Register Addr = 0x%x\n", REG(CTRL));
 printf("Status-Register Addr = 0x%x\n", REG(STATUS));
 printf("Interrupt-Register Addr = 0x%x\n", REG(INTR));

 return 0;
}

Output

Example to use ## Pre-Processor!
*** My-Device Register ***
Control-Register Addr = 0x40004020
Status-Register Addr = 0x40004030
Interrupt-Register Addr = 0x40004040

These will be done at the preprocessing time.

Why We need this Token Pasting Operator?

• It is used for Logs in our Program

#define dbg_print(fmt,args...) printf("Debug: "fmt ,##args)

• It is used to reduce repetitive typing.
• This can be taken further and be applied to even an abstract data type, such as a linked list. I’ve constructed myself a generic linked list.

#define LINKED_LIST(A) struct list##_##A {\
                              A value; \
                              struct list##_##A *next; \
                              };

In this case, LINKED_LIST(int) would give you,

struct list_int {
int value;
struct list_int *next;
};

Stringizing Operator (#) in C

Sometimes you may need to convert a macro’s argument to a string. Parameters are not replaced inside string constants, but you can use the ‘#’ preprocessing operator instead. When a macro parameter is used with a leading ‘#’, the preprocessor replaces it with the literal text of the actual argument, converted to a string constant. Unlike normal parameter replacement, the argument is not macro-expanded first. This is called stringizing.

#define print_string(y) #y

• This will convert whatever is passed as a parameter y into a string.
• This will delete Leading and trailing white space.
• If we pass the macro, it won’t expand. It will print the string.
• \ will convert into \\ and " is converted into \"
• Some characters cannot be stringized – notably, the comma , because it is used to delimit parameters and the right parenthesis ) because it marks the end of the parameter list.

Example

Consider the below code.

#define print_string(y) #y

#define DEF welcome

I’m using the print_string macro like below.

1. print_string(Testing);
2. print_string( Testing Program );
3. print_string("Testing" "Program");
4. print_string(Testing DEF Program); (Here DEF is a macro)

This macro will expand like below.

1. "Testing"
2. "Testing Program" (leading and trailing spaces were trimmed and space between words was compressed to a single space character)
3. "\"Testing\" \"Program\""(the quotes were automatically converted)
4. "Testing DEF Program" (DEF macro won’t expand)

Can we use both Stringize and the token-pasting operators in a single MACRO?

Yup, We can. In the below example, we are going to use both operators in a single MACRO.

#define tokenpaster(n) printf ("token" #n " = %d", token##n)

tokenpaster(34);

This example results in the following actual output from the preprocessor:

printf ("token34 = %d", token34);

Both the stringize and the token-pasting operators are used in this example.

You can also read the below tutorials.

Linux Device Driver Tutorials	C Programming Tutorials
FreeRTOS Tutorials	NuttX RTOS Tutorials
RTX RTOS Tutorials	Interrupts Basics
I2C Protocol – Part 1 (Basics)	I2C Protocol – Part 2 (Advanced Topics)
STM32 Tutorials	LPC2148 (ARM7) Tutorials
PIC16F877A Tutorials	8051 Tutorials
Unit Testing in C Tutorials	ESP32-IDF Tutorials
Raspberry Pi Tutorials	Embedded Interview Topics
Reset Sequence in ARM Cortex-M4	BLE Basics
VIC and NVIC in ARM	SPI – Serial Peripheral Interface Protocol
STM32F7 Bootloader Tutorials	Raspberry PI Pico Tutorials
STM32F103 Bootloader Tutorials	RT-Thread RTOS Tutorials
Zephyr RTOS Tutorials - STM32	Zephyr RTOS Tutorials - ESP32
AUTOSAR Tutorials	UDS Protocol Tutorials
Product Reviews	STM32 MikroC Bootloader Tutorial
VHDL Tutorials