MMX, SSE and AVX Extensions

At some point in the evolution of personal computers, it became clear that they would be used not only for professional use, for example, in companies, financial institutions, and education, but also would be used as centres of home entertainment systems, enabling users to play games, watch videos, and listen to music. This led to the empowerment of processors to process multimedia data. As stereo sound is represented as a series of samples and images are often represented as a matrix of colour pixels, one way to improve the performance of multimedia processing is to introduce parallelism. At the processor level, the answer is SIMD - Single Instruction Multiple Data, which allows the execution unit to perform the same operation on many data units at the same time. More formally, one stream of instructions performs operations on multiple data streams. The first SIMD instructions introduced in the x86 family that follow this idea are the MMX (MultiMedia eXtension) instructions.

The MMX instruction set operates on 64-bit packed data types. Packed means that the 64-bit data can contain 8 bytes, 4 words, or 2 doublewords. Based on this, the new data types were defined. Packed data types are also called vectors. Please refer to the section “Integer vector data types” for details. The MMX instructions operate on eight 64-bit registers named MM0-MM7.

To copy data from memory or between registers, two new data transfer instructions were introduced.

• movd - instruction allows copying 32 bits of data between MMX registers and memory or between MMX registers and general-purpose registers of the main processor.
• movq - instruction allows copying 64 bits of data between MMX registers and memory or between two MMX registers.

In all MMX instructions except data transfer, the first operand, which is a destination operand, is always an MMX register.

The main idea of vector data processing is shown in figure 1. It shows the example of an operation performed with packed word vector data.

When performing arithmetic operations, the main processor stores additional information in the FLAG register's flags. The MMX unit does not have flags for each calculated result, so some other approach should be used. Key information for arithmetic operations is the carry in addition and the borrow in subtraction. The simplest solution is to omit the carry, which, if the maximum value is exceeded, will result in truncation of the oldest bits and a reduction in the result. In subtraction, the situation is reversed, and the result will be larger than expected. For multimedia operations, a better solution is to limit the result to a maximum or minimum value. This approach is called saturation and comes in signed and unsigned versions. This means that, for example, when a pixel reaches its maximum brightness, it will no longer be brightened further. This way, information about brightness differences is lost, but the resulting image looks natural. Let's consider the addition operation on four arguments, each of size a word, in three versions. In figure 2, the packed word addition with wraparound paddw is shown. In figure 3, the packed word addition with signed saturation paddsw is presented, and finally, the packed word addition with unsigned saturation paddusw is shown in figure 4.

The last letter in the instruction specifies the size of arguments and results. MMX addition and subtraction instructions are shown in the table 1

The multiplication operation requires twice as much space for the result as the arguments. The solution for this issue is to split the operation into two multiplication instructions, storing the higher and lower halves of the results.

• pmulhw - packed multiply with storing higher halves of the results
• pmullw - packed multiply with storing lower halves of the results

Later halves can be joined to form full results with unpacking instructions.

• punpckhwd - unpack from higher halves words to doublewords
• punpcklwd - unpack from lower halves words to doublewords

The whole algorithm is shown in figure 5.

The code that calculates the presented multiplication can look as follows:

Numbers DW  01ACh, 2112h, 03F3h, 00A4h,
            0006h, 0137h, 0AB7h, 00D8h
lea         ESI, Numbers
movq        MM0, [ESI]          ; MM0 = 00A4 03F3 2112 01AC
movq        MM1, [ESI+8]        ; MM1 = 00D8 0AB7 0137 0006
movq        MM2, MM0
pmullw      MM0, MM1            ; MM0 = 8A60 50B5 2CDE 0A08
pmulhw      MM1, MM2            ; MM1 = 0000 002A 0028 0000
movq        MM2, MM0
punpcklwd   MM0, MM1            ; MM0 = 0028 2CDE 0000 0A08
punpckhwd   MM2, MM1            ; MM2 = 0000 8A60 002A 50B5

The MMX set of instructions also contains the multiply and add packed words to doublewords instruction.

• pmaddwd - multiply and add packed words to doublewords

It computes the products of the corresponding signed word operands. The four intermediate doubleword products are summed in pairs to produce two doubleword results. Its behaviour is shown in figure 6. This instruction can simplify the multiplication process when multiplying two pairs of word arguments, while the other two pairs yield zero.

The set of comparison instructions allows for comparing values in two vectors. The result is stored as a mask of bits, with all ones at the element of the vector where the comparison result is true, and all zeros in the opposite case. There are six compare instructions as shown in table 2.

An example of a comparison instruction for the equality of two word vectors is shown in figure 7.

The unpack instructions presented in figure 5 are not the only ones. There exist unpack instructions for high-order data elements and for low-order data elements.

• punpckhbw - unpack high-order bytes to words
• punpckhwd - unpack high-order words to doublewords
• punpckhdq - unpack high-order doublewords to quadwords
• punpcklbw - unpack low-order bytes to words
• punpcklwd - unpack low-order words to doublewords
• punpckldq - unpack low-order doublewords to quadwords

The figure 8 presents unpacking of high-order bytes into words, and figure 9 presents unpacking of low-order bytes into words.

The pack instructions are used to shrink the size of arguments and pack them into smaller data. Only three pack instructions are implemented in MMX extension:

• packsswb - pack words into bytes with signed saturation
• packssdw - pack doublewords into words with signed saturation
• packuswb - pack words into bytes with unsigned saturation.

The example of pack instruction is shown in figure 10.

Packed shift instructions perform shift operations and elements of the specified size. All elements of the vector are shifted separately. In a logical shift, empty bits are filled with zeros; in arithmetical shift right, the higher bit is copied to preserve the sign of values. There are eight shift instructions, as presented in table 3

MMX logical instructions operate on the 64-bit data as a whole. They perform bitwise operations as shown in table 4.

MMX instructions use the same physical registers as the FPU. As a result, mixing FPU and MMX instructions in the same fragment of the code is not possible. Switching between FPU and MMX in a code requires executing the emms instruction, which resets the FPU and MMX units.

• emms - empty mmx state

Fortunately, newer extensions (SSE, AVX) introduce a separate set of registers for improved flexibility.

The SSE is a large set of instructions that implement SIMD processing for floating-point calculations and increase the size and number of registers. The abbreviation SSE comes from the name Streaming SIMD Extensions. As the number of instructions introduced across all SSE versions exceeds a few hundred, we present a general overview of each SSE version and detailed information on selected instructions of interest. The first group of SSE instructions defines a new vector data type containing four single-precision floating-point numbers. It's easy to calculate that it requires the 128-bit registers. These new registers, named XMM0-XMM7, are distinct from previously implemented registers, so SSE floating-point operations do not conflict with MMX and FPU operations.

In modern processors, it is very important to transfer data from and to memory effectively. The memory management unit can perform data transfer much faster if the data is aligned to a specific address. For SSE instructions, an address must be evenly divisible by 16. In the SSE extension, two versions of data transfer instructions were implemented.

• movups - copies packed single-precision data from any address
• movaps - moves data from an aligned address
• movss - instruction moves a single-precision scalar value. It doesn't have to be aligned.
• movhps - copy data between the upper half of the XMM register and memory
• movlps - copy data between the lower half of the XMM register and memory
• movhlps - copy data from the lower to the higher half of XMM register
• movlhps - copy data from the higher to lower half of the XMM register
• movmskps - instruction copies the most significant bits of single-precision floating-point values to a general-purpose register. It allows us to create a bit mask based on the sign bits of the vector's elements.

The SSE performs vector and scalar operations on single-precision floating-point numbers. No prefix for instruction names operating on floating-point numbers was added, but the mnemonic suffix describes the type. PS (packed single) - action on vectors, SS (scalar single) - operation on scalars. If the instructions operate on halves of XMM registers (i.e. either refer to bits 0..63 or 64..127), the instruction mnemonics contain the letter L or H. The idea of vector and scalar operations is shown in figure 11 and figure 12, respectively.

In the SSE extension, mathematical calculations on single-precision floating-point numbers are implemented in both vector (packed) and scalar versions. These instructions are summarised in table 5.

In addition to math calculations, there are instructions for comparing vector cmpps and scalar cmpss values. As a result, we obtain the all-ones or all-zeros fields as in MMX. The condition of comparison is encoded as the third 8-bit immediate argument. Assemblers usually implement a set of pseudoinstructions which automatically choose the constant value. The scalar version of these pseudoinstructions is presented in table 6

Using the comiss instruction, it is possible to compare scalars and set the flags in the FLAG register directly based on the comparison result.

There are four logical instructions which operate on all 128 bits of the XMM register.

• andps - packed logical and
• andnps - packed logical and not
• orps - packed logical or
• xorps - packed logical exclusive or

Two unpack instructions operate similarly to unpack instructions known already from MMX. Because the source and destination data are packed single-precision floating-point values, unlike in MMX, these instructions are not used to form longer data types but instead change the positions of two elements in two vectors.

• unpcklps - unpack the single-precision packed values
• unpckhps - unpack the single-precision packed values

They are presented in figure 13.

The more universal is the shuffle instruction.

• shufps - shuffle packed single-precision value

It selects two out of four single-precision values from the source argument and rewrites them to the bottom half of the destination argument. The upper half is filled with two single-precision values from the destination register. Which values ​​will be taken is determined by the third, 8-bit immediate argument. Each two-bit field of the immediate determines the number of packed single values. For 11 - it is X3 or Y3, for 10 - X2 or Y2, for 01 - X1 or Y1 and for 00 - X0 or Y0. It is presented in figure 14.

Together with new data registers, an additional control register appeared in the processor. It is named MXCSR and is similar in meaning to the FPU control register. New instructions are implemented:

• stmxcsr - save the MXCSR to memory
• ldmxcrs - restore MXCSR from memory
• fxsave - store the state of both the x87 unit and the SSE extension
• fxrstor - restore the state of both the x87 unit and the SSE extension.

Some additional MMX instructions were introduced together with the SSE extension. In the SSE, the first set of data conversion instructions was implemented. The summary of all these instructions will be presented in the following chapter. Also, cache supporting instructions were added. These instructions will be described in the chapter on optimisation.

The SSE2 instruction set implements integer vector operations using the XMM registers. In general, the same instruction mnemonics defined for MMX can be used with XMM registers, supporting twice-as-long vectors. Additionally, the floating-point calculations are complemented with vector operations on the double-precision data type. In XMM registers, vectors of two double-precision values can be processed. SSE2 uses the same software environment (eight 128-bit XMM registers) as SSE. In the SSE2 extension, the denormals-are-zeros mode was introduced. The processor automatically converts all unnormalised floating-point arguments to zero. In such a case, a flag indicating the denormalised argument is not set, and an exception is not raised. The denormals-are-zeros mode is not compliant with IEEE Standard 754, but it allows implementation of faster algorithms for advanced audio and video processing. The arithmetic instructions are similar to SSE, but they possess the suffix of pd - packed double or sd - scalar double instead of ps and ss, respectively.

In figure 15, we present the type-conversion instructions. They enable conversion between integer and floating-point data of various sizes and in different registers. The green arrows and instruction nodes represent the conversion from single-precision floating-point to integers, pink represents the conversion from double-precision floating-point to integers, blue represents the conversion from integers to floating points, and orange represents the conversion between single and double precision floating-point.

The SSE3 is a set of 13 instructions. The main innovation in SSE3 is the implementation of horizontal instructions. These instructions perform calculations on the elements of a vector within the same register. There are four such instructions.

• haddpd - performs horizontal addition of double-precision values
• hsubpd - is a horizontal subtraction of double-precision values
• haddps - is a horizontal addition of single-precision values
• hsubps - is a horizontal subtraction of single-precision values

All horizontal instructions operate similarly. The lower (bottom) part of the resulting vector is the result of operation on the bottom and top elements of the first (destination) operand; the higher (top) part of the resulting vector is the result of operation on the second (source) operand's bottom and top. The best way to present the principles of horizontal operations is a picture. Because in the subtraction operation the order of arguments is important, the hsubpd instruction is shown in figure 16.

While there are more than two elements of source vectors, like in the hsubps instruction, it is also important to know the order of the elements in the resulting vector. Please look at the figure 17.

This abbreviation stands for Supplemental Streaming SIMD Extension 3. It is a set of 16 instructions introduced in the Core 2 architecture. It implements integer horizontal operations on XMM registers. The principles are the same as in horizontal instructions in SSE3, but instructions can process vectors of doublewords or words. They are summarised in the table 7.

Two data shuffle instructions are worth mentioning.

• pshufb - packed shuffle bytes
• palignr - packed align right

The pshufb instruction makes copies of bytes from the first 128-bit operand based on the control information taken from the second 128-bit operand. Each byte in the control operand determines the corresponding byte at that position.

• bit 7 is 1 - byte is cleared
• bit 7 is 0 - byte contains a copy of the source byte
• bits 0-3 - a number of the source byte to be copied

The illustration is shown in figure 18.

The palignr instruction combines bytes from two source operands as shown in figure 19. The position of the byte split is specified as third immediate. In the figure, the immediate is equal to 2.

The SSE4 is composed of SSE4.1 and SSE4.2. These groups include instructions supplementing previous extensions. For example, eight instructions expand support for determining the minimum and maximum of packed integers, or twelve instructions that improve packed integer format conversions with sign and zero extensions. The dpps and dppd instructions calculate the dot product of four single-precision and two double-precision operands, respectively.

• dpps - dot product of packed single-precision values
• dppd - dot product of packed double-precision values

Additionally, the arguments are controlled with the third immediate operand. The example showing the dppd is presented in figure 20.

There are also advanced shuffle, insert, and extract instructions that enable manipulation of the positions of data of various types. The type of the data is specified with the suffix of the mnemonic: b - bytes, w - words, d - doublewords, q - quadwords, ps - single precision and pd - double precision elements. Although these instructions behave the same for the integer and floating-point data elements, formally, those operating with integers begin with the letter “P”. A few examples are shown in the following figures.

The blending instructions copy elements of vectors, mixing two sources into the destination.

• blendps - blend packed single-precision values
• blendpd - blend packed double-precision values
• pblendw - blend packed words

They conditionally copy elements from vector X or Y. The mask is specified as the third, immediate value. The behaviour of blendpd is shown in figure 21

The instructions blendvps, blendvpd and pblendvb operate similarly, but the condition is specified as the sign bit of the corresponding elements of the third implied argument stored in XMM0.

• blendvps - variable blend single-precision packed values
• blendvpd - variable blend double-precision packed values
• pblendvb - variable blend packed bytes

The behaviour of blendvpd is shown in figure 22

The set of extract instructions includes:

• pextrb - extract byte
• pextrw - extract word
• pextrd - extract doubleword
• pextrq - extract quadword
• extractps - extract packed floating-point values

They take one element of the vector from the XMM register and store it in a CPU register or in memory. The offset of the element is specified with an immediate constant. The behaviour of extractps is shown in figure 23

The insert instructions are:

• pinsrb - insert byte
• pinsrd - insert doubleword
• pinsrq - insert quadword
• insertps - insert single-precision value

They operate in an opposite way to extract instructions. They take an element from memory or a general-purpose register and insert it into the XMM register at the position specified with a constant immediate. The behaviour of pinsrd is shown in figure 24

The insertps is one of the most complex. inserts a scalar single-precision floating-point value with the position of the vector's element in source and destination controlled with an 8-bit immediate. The example showing the insertps instruction is presented in figure 25. In this example, the immediate contains the bit value of 10011000b.

In SSE4.2, the set of string compare instructions was added. As the XMM registers can contain sixteen bytes, it is much more efficient to implement string processing algorithms with bigger XMM registers than with registers in the main processor with the use of string instructions. There are four string-compare instructions (see table 8), each of which can be configured to provide different functionality. The length of strings can be explicit or implicit. Explicit length means that the length of the first operand is specified with the RAX register, and the length of the second operand is specified with the RDX register. Implicit length means that both operands contain null-terminated strings. Instructions can produce two kinds of results. Index means that the index of the first or last result is returned. Mask means that the bit mask is returned (one bit for every two elements compared) or a mask of the same size as the elements (similar to MMX compare).

The third, immediate operand encodes the comparison method and result encoding. Details in tables 9 and 10.

The SSE4.2 string compare instructions are advanced, powerful means for processing byte or word strings. The detailed explanation of SSE4.2 string instructions behaviour together with illustrations can be found on ¹⁾.

AVX is the abbreviation of Advanced Vector Extensions. The AVX implements larger 256-bit YMM registers as extensions of XMM. In 64-bit processors, the number of YMM registers is increased to 16. Many SSE instructions are extended to support operations on new, larger data types without changing the mnemonics. The most important improvement in the instruction set of x64 processors is the implementation of RISC-like instructions in which the destination operand can differ from two source operands. A three-operand SIMD instruction format is called the VEX coding scheme. The AVX2 extension implements additional SIMD instructions for operations on 256-bit registers. The AVX-512 extends the register size to 512 bits. An interesting, comprehensive description of a variety of x64 AVX instructions is available on the website ²⁾.