Optimisation

Optimisation strongly depends on the processor's microarchitecture. Some optimisation recommendations change with new processor versions. Producers usually publish the most up-to-date recommendations. The last release of the Intel documentation is “Intel® 64 and IA-32 Architectures Optimization” ¹⁾. AMD published the document “Software Optimization Guide for the AMD Zen5 Microarchitecture” ²⁾. A selection of specific optimisation recommendations is described in this section.

It is recommended to place variables in the memory at their natural boundaries. It means that if the data is 16 bytes, the address should be evenly divisible by 16. For 8-byte data, the address should be divisible by 8.

It is recommended to use registers rather than memory for scalar data whenever possible. Keeping data in registers eliminates the need to load and store it in memory.

It is natural for programmers to use inc or dec instructions to increment or decrement the variable. These instructions are simple and intuitively appear to be executed faster than addition and subtraction with a constant “1”. The inc and dec are single-byte instructions, while add and sub with the argument as a constant consume at least one byte more. The problem with inc and dec instructions is that they do not modify all flags, whereas add and sub do. Modifying all flags frees the processor from waiting for previously executed instructions to complete their flag modifications. Intel recommends replacing inc and dec with add and sub instructions, but compilers do not always consider this recommendation.

While new extensions are introduced, several new instructions appear. In addition to advanced data processing instructions, simple logic instructions are also implemented. Previous versions of the instructions are extended to operate on the latest, larger registers. This may lead to confusion about which instruction to use, especially when the instructions perform the same operation and yield the same result. Let's consider three logic XOR instructions pxor, xorps and xorpd. All of them can operate on 128-bit XMM registers, performing the bit-wise logic XOR function. At first sight, the instruction choice is meaningless - the result will be the same. In reality, the selection of the instruction matters. The performance analysis shows that, in different situations, the execution time can be longer or shorter. A deeper analysis reveals that when previous calculations are performed with integers, it is better to use integer operation pxor; if the data is floating-point, it is better to use the floating-point version xorps or xorpd. There is a section in the Intel optimisation manual about mixing SIMD data types. It is recommended to use packed-single instead of packed-double when possible.

It is a common method to pause the program execution and wait for an event for a short period in a spin loop. In the case of a brief waiting period, this method is more efficient than calling an operating system function that waits for an event. In modern processors, the pause instruction should be used inside such a loop. It helps the processor's internal mechanisms temporarily allocate hardware resources to another logical processor.

In modern microarchitectures, the cache memory is essential for improving performance. In general, the processor handles cache memory as efficiently as possible; however, it is easy to write a program that prevents the processor from utilising this mechanism effectively. The cache works on two main principles:

• temporal locality
• spatial locality.

The term temporal locality refers to the fact that if a program recently used a certain portion of data, it is likely to need it again soon. It means that if data is used, it remains in the cache for a certain period until other data is loaded into it. It is efficient to keep data in a cache rather than reload it. The term spatial locality refers to whether a program has recently accessed data at a particular address; it is likely to need data at the next address soon. The cache helps the program run faster by automatically prefetching data and code that will likely be used or executed soon. It is recommended to write the programs in any programming language, keeping these rules in mind. Some recommendations are listed below:

• The program should do as much work as possible on one small area of code and data; after doing the job, it can move to the next part.
• The program should avoid frequent jumping over distant regions of memory.
• While processing big multidimensional data arrays, keep in mind their placement in memory (row-wise or column-wise), which is specific to the programming language.
• Object-oriented programming helps to utilise cache because members of the class are grouped.

Cache temporal locality is the feature that improves performance when a program repeatedly uses the same variables, e.g., in a loop. If the processed data exceeds half the size of a level 1 cache, it is recommended to use the non-temporal data move instructions movntq and movntdq to store data from registers to memory. These instructions are hints to the processor to omit the cache if possible. It doesn't mean that the data is immediately stored directly in memory. It can remain in the internal processor's buffers, and, likely, the last version is not visible to other units of the computer. It is the programmer's responsibility to synchronise the data using the sfence (Store Fence) instruction.

Some instructions allow the programmer to support the processor's cache utilisation.

• movntq - saving the contents of the MMX register, bypassing cache
• movntps - write the contents of the SSE register, bypassing cache
• maskmovq - write selected bytes from the MMX register, bypassing cache
• movntdqa - non-temporal aligned move

Fence instructions guarantee that the load and/or store instructions before the fence are completed before the corresponding instruction after the fence.

• sfence - force the memory–cache synchronisation after store instructions
• lfence - force the memory–cache synchronisation after load instructions
• mfence - force the memory–cache synchronisation after load and store instructions

Some instructions are hints to the processor indicating that the programmer expects the data to be stored in cache rather than in memory, or that the programmer no longer expects to use the data in cache.

• prefetch - a hint to the processor, which indicates that the memory area should be considered higher in the hierarchy cache
• clflush - flushes a cache line from all levels of cache.

The essential readings in an optimisation topic are the vendors' optimisation guides mentioned at the beginning of this section.

An exceptional position about optimisation in x64 processors is by Agner Fog³⁾. This is a must-read for programmers who want to optimise their programs. Thanks to the author's extensive knowledge and hard work, this guide documents various tricks and the execution times of individual processor instructions. It is mentioned in almost all online articles and blog posts about optimisation.

Interesting Understanding Windows x64 Assembly tutorial ⁴⁾ not only about optimisation but also about using low-level programming in Windows.