Scenario x64_3: Converting integers to hexadecimal text

In this scenario, we will modify the conversion library, adding another function which should convert integer input into a hexadecimal representation.

Prerequisites
Go through previous scenarios where the conversion library and the main program are described, and the initial code of both is presented. You should be familiar with writing, assembling and running programs in Visual Studio Community or using simple MASM tools. Please read the chapter “Programming in Assembler for x64” for a detailed description of how to prepare the software environment.

Scenario
Extend the conversion library described in “Creating static libraries” with a function which converts the integer argument into hex. Display the result in a console.

Result
The converted hex string is displayed in a console.

Start
Make a copy of the int_to_ascii function and introduce some simple modifications.

Step 1
First, we need to divide the input value by 16, not by 10.

   mov rbx, 16

Step 2
After each division operation, we will obtain the remainder from the range 0-15. We can't convert this into an ASCII digit the same way as in decimal, because the digits 0-9 and letters A-F do not form a continuous range. We can deal with this situation in different ways. One approach is to check if dl is bigger than 9 and shift it to point to letter characters if true.

    cmp dl, 9          ; test if dl > 9
    jna zero_to_nine   ; if not jump over adjustment
    add dl, "A"-"9"-1  ; adjust dl with the distance between A and 9
zero_to_nine:
    add dl, "0"        ; convert to ASCII

Step 3
Assemble the program and observe results.

Step 4
Another approach is to define the table of characters (lookup table) in the data section containing all digits and letters, and pick the correct character using the mov with proper indirect addressing mode.

.data
hex_digits db "0123456789ABCDEF"
 
.code
...
    lea rcx, hex_digits        ; load address of lookup table
    and rdx, 0000000Fh         ; limit the range to 15
    mov byte ptr dl, [rcx+rdx] ; convert remainder into ASCII
...

In the second approach, we use indirect addressing with the use of the sum of the rcx and rdx registers. The base address of a table must be loaded to rcx with the use of the lea instruction, not used as a constant. This is because an instruction we could use in 32-bit mode:

    mov byte ptr dl, hex_digits[rdx]   ; This instruction is NOT VALID in 64-bit mode

used in 64-bit long mode will signal an error. The address of the lookup table is a 64-bit number, but the constant encoded in the used form of the mov instruction can't exceed 32 bits.

Step 5
It is possible to pick up the character from a lookup table by using the xlatb instruction. To use this instruction, we have to preserve the rax before conversion. We will do it by storing it temporarily in rcx. We need to handle the rbx in a different way. In each iteration, set it to 16 before dividing, and to the lookup table address before xlatb.

.code
...
    mov rbx, 16          ; prepare divisor
    div rbx		 ; rax / 16 → remainder in rdx
    mov rcx, rax         ; store temporarily rax
    lea rbx, hex_digits  ; load address of lookup table
    and rdx, 0000000Fh   ; limit the range to 15
    mov al, dl           ; prepare index in al
    xlatb                ; convert remainder into ASCII
    mov [rdi], al        ; put character to resulting table
    mov rax, rcx         ; restore rax

Result validation
All versions of a code should print the same results in a console window.

Possible optimisation
To improve the performance of our code, in the case of hexadecimal numbers, it is possible to replace the time-consuming division instruction with an instruction to shift the number by four bit positions to the right. We leave the implementation of this optimisation to the reader.

If it does not assemble?
Observe the messages passed by the assembler and linker programs. Refer to the documentation and instruction set.