A performance model for integrated layer processing

A Performance Model for Integrated Layer ProcessingBengt AhlgrenSwedish Institute of Computer ScienceBox1263,S-16428Kista,SwedenE-mail:Bengt.Ahlgren@sics.seAbstractIntegrated Layer Processing is an implementation technique for data manipulation functions in communication protocols.The purpose of this technique is to increase communication perfor-mance.It reduces the number of memory accesses and thus relieves the memory bandwidth bottleneck.Integrated Layer Processing can however,in some situations,substantially increase the number of memory accesses,and therefore instead reduce performance.The main reason is contention for processor registers.We present a performance model that captures the memory behavior of data manipulation functions for both integrated and sequential implementations.By comparing the model to mea-surements of real and synthetic data manipulation functions,we show that the model accurately predicts the performance.The model can be used to assess whether an integrated implementation will perform better or worse than a sequential implementation.The situations where integration would reduce performance can then be avoided without spending a lot of effort on a more complex integrated implementation.KeywordsILP,Integrated Layer Processing,performance modeling,protocol implementation techniques2A Performance Model for Integrated Layer Processing 1INTRODUCTIONThe communication performance of workstation class computers is limited by the performance of their primary memory system(Druschel,Abbott,Pagels and Peterson,1993;Smith and Traw,1993).The reason is that software implementation of protocol data manipulation,such as checksum calculation,needs to access every byte of message data in memory.Since the access time of memory chips are decreasing at a much slower pace than CPU speeds are increasing(Patterson and Hennessy,1994;Wulf and McKee,1995),the bottleneck is getting more and more severe relative to the CPU performance of future computer systems. Integrated Layer Processing,ILP,is an implementation technique for data manipulation functions in communication protocols presented by Clark and Tennenhouse(1990).The purpose with ILP is to relieve the memory bottleneck by reducing the number of memory accesses for message data and thus increase communication performance.The potential reduction comes from combining the manipulation functions of several protocol layers into a pipeline in a single processing loop.For each iteration of the loop the manipulations are run in succession on a small quantity of data,typically a word or a couple of words.Message data is pipelined via registers between the manipulations and need not be accessed from memory by each function.ILP has been shown to considerably improve throughput of simple functions(Abbott and Peterson,1993;Gunningberg,Partridge,Sirotkin and Victor,1991;Partridge and Pink,1993) and to improve throughput less for more complicated functions and when used in complete systems(Braun and Diot,1995;Gunningberg,Partridge,Sirotkin and Victor,1991).In a previous paper(Ahlgren,Bj¨o rkman and Gunningberg,1996)we have shown that ILP also can substantially decrease performance compared to a conventional sequential implementation. The conclusion in that paper was that the use of CPU registers play a very important role for the performance gain,or loss,for an integrated implementation compared to a sequential. We found that when the aggregated state size of the data manipulation functions does notfit in processor registers,the integrated implementation quickly becomes much slower than the sequential implementation.In this paper we continue to study the performance behavior of integrated and sequential implementations of data manipulation functions.Since the purpose of ILP is to reduce the number of memory accesses,we are particularly interested in the memory behavior and the role of the CPU registers.We present a performance model that predicts the performance of the two implementation techniques.The model calculates the number of clock cycles per message byte given a set of architecture parameters and a set of function parameters describing the computer system and the data manipulation functions,respectively.We show that the predicted performance from the model compare well with measurements.The important contribution of our performance model is that it does not only consider memory accesses for message data,but also memory accesses for function state that does notfit in processor registers.The motivation behind this work is to fully understand how different parameters affect the performance of the two implementation techniques.This knowledge can be useful for tools(“ILP compilers”(Braun and Diot,1996b))that automatically generate integrated implementations.Related work3input bufferreadwriteoutput bufferFigure1A data manipulation function“”.2RELATED WORKWe have studied Integrated Layer Processing using a technique with synthetic data manipulation functions.These results are presented in a previous paper(Ahlgren,Bj¨o rkman and Gunningberg, 1996).We concluded that the key factor is whether the aggregated function statefits in the available CPU registers or not.We showed experimentally that an integrated implementation can be slower than the corresponding sequential.Abbott and Peterson(1993)also recognizes that“the key limiting factor is register pressure”when the number of integrated functions is increased.They also predict that the integrated implementation can be slower than the sequential but do not show it analytically or verify it experimentally.They present a performance model,but it only includes memory references to message data.In their model,the integrated implementation is therefore always faster than the sequential implementation.We compare our model with their model in a later section of this paper.Integrated Layer Processing has also been studied by a number of other researchers(Braun and Diot,1995;Braun and Diot,1996a;Gunningberg,Partridge,Sirotkin and Victor,1991).They have however not presented any performance model for comparing an integrated implementation with a sequential.Previous work have paid quite a lot of attention to the practical problem of applying ILP to a real protocol stack(Abbott and Peterson,1993;Braun and Diot,1996a).We are aware of these problems but do not address them in our work.These problems are independent of whether integration as such has the potential of producing a faster implementation for a specific set of data manipulation functions or not.3DATA MANIPULATION FUNCTIONS AND ILPCommunication protocol processing can be divided in two parts:protocol control and data manipulation.The protocol control part processes the information in message headers and maintains the state needed by the protocol,for example,connection state.The control processing is usually specified and implemented as afinite state machine.The data manipulation part processes the data part of the messages.Examples of data manipulation functions are checksums, encryption and presentation encoding.They all have in common that they process each byte of the message data and possibly produce new data.This is illustrated in Figure1.The data to be manipulated is stored in buffers.The input and output buffers are often the same buffer in a real implementation,but to be more general we illustrate them as separate buffers.The data manipulation function is implemented as a loop which reads a block of data from the input4A Performance Model for Integrated Layer Processing1input bufferread write2output bufferread writeinput buffer output bufferreadwriteFigure 2Composition of data manipulation functions,sequential (left)and integrated (right).buffer,computes the function value and writes the result to the output buffer.The minimum size of the block depends on the function.3.1Sequential vs.integrated implementationsIn a traditional implementation of a protocol stack,the data manipulation functions are exe-cuted independently in their respective layer,as illustrated to the left in Figure 2.We call this implementation a sequential implementation.Each function follows sequentially and reads and,possibly,writes all data in the buffer.The functions need not be immediately adjacent as in the figure.They can even be located in different address spaces.An example of this is the imple-mentation of the TCP/IP protocol stack with an RPC protocol in a typical Unix system.It has presentation layer encoding in the application and the UDP or TCP checksum in the operating system kernel.The idea with Integrated Layer Processing (Clark and Tennenhouse,1990)is to combine the data manipulation functions of several protocol layers into one processing loop as illustrated to the right in Figure 2.The purpose is to avoid reading and writing the message data in the buffers several times.The performance gain has shown to be substantial (Abbott and Peterson,1993).We call this implementation an integrated implementation.It reads a block of data from the input buffer and applies all functions in succession on the data.The assumption is that the transformed data is stored in processor registers between the functions and need not be accessed from memory by each function.After the last manipulation function the resulting block of data is written to the output buffer and the loop continues with the next block until the input buffer is finished.In this particular example with three functions each producing new data,the number of memory accesses for message data is reduced to 1Data manipulation functions and ILP5 data manipulation functions at a suitable level of abstraction.Since the purpose with an integrated implementation is to save memory accesses,we must capture the characteristics that define the memory behavior of the function.The following paragraphs describe the parameters we have identified as relevant.Input data block The size of the input block is fundamental to a data manipulation function. The whole block must be present in order to do the processing.The function processes one block of data in each iteration.For example,the TCP/IP checksum has an input block size of2 bytes,the DES encryption has8bytes and the MD-5message digest has16bytes.Output data block Depending on the particular function,the size of the output data block may be the same,less than or greater than the size of the input data block.A data compression function is an example of a function that produces less data than input.Some functions,like checksums,do not produce output data at all.State Many data manipulation functions keep a state between iterations.This is the case for the TCP/IP checksum and the DES encryption algorithm in CBC mode.The checksum algorithm adds a16bit data block to the accumulated sum of previous blocks,i.e.,the sum is the state. The state need to be stored somewhere—preferable in registers,since it is accessed for each iteration.The size of the state is therefore important for the memory behavior of the function. Thefinal state is usually the result of functions that do not produce output data.Constants Data manipulation functions can use constant values in its computation.A constant is explicit in the code and is accessed at least once per loop iteration.A constant is therefore a candidate for register allocation by the compiler.As such,constants compete for register space with the function state.A difference is that constants in memory need only be read,while the state is updated and must be both read and written.A constant can however be used more than once,so the compiler must analyze the number of references to be able to make a good register allocation.Table look-ups Tables are similar to constants in that they contain constant values.The difference is that only a fraction of the entries in a table are accessed for each iteration.A table is also usually too large for the processor registers and must therefore be accessed from memory. The interesting parameters are the number of table lookups per iteration and the size of the table. Temporaries A data manipulation function will use a number of temporary variables for intermediate results.The temporaries need not be explicit in the original source code.The compiler usually puts temporary variables in registers.Instructions We distinguish between three logical categories of machine level instructions: computation,memory and loop overhead.The computation instructions are arithmetic-logic and control instructions used to compute the state or the output data of the data manipulation function. The memory instructions transfer data between the CPU registers and the memory or the cache. The loop overhead instructions consist of instructions for incrementing the buffer pointers, decrementing and testing the loop variable and a conditional branch back to the beginning of the loop.The particular machine instructions and the number of them in an implementation depend both on the machine architecture and the compiler used to produce the code.The computation instructions,however,are pretty well defined by the data manipulation function.The number of memory instructions depend on register allocation,except for the accesses to the input and output data buffers which always are referenced in memory.The loop overhead instructions depend mainly on the processor instruction set,but also on the compiler.6A Performance Model for Integrated Layer ProcessingTable1Model parameters.number of available CPU registers(word size)access time offirst level cache(clocks/word)access time of memory(clocks/word)CPI cycles per instructionloop overhead(instructions/loop)CPU clock frequencyword size(bytes)Function parameters4THE PERFORMANCE MODELIn this section we present a performance model for data manipulation functions.The model computes the cost in time per data byte both for an integrated implementation and for a sequential implementation of the same combination of data manipulation functions.The costs of the implementations can then be compared to see which performs best.4.1Model parametersTable1lists the input parameters of the model.The parameters are of two kinds.The architecture parameters specify the characteristics of the target computer system.The function parameters specify the characteristics of the desired data manipulation functions.The number of CPU registers()denotes the number of registers that the compiler has available for the function state.This parameter is thus compiler dependent in addition to being CPU dependent.We assume that this number is invariant when the other parameters of the model are varied.The memory access time()is the average access time for reading and writing memory sequentially through the cache(s)of the system.The effects of cache linefills and cache line write-backs are included in this average.We assume that the read and the write access times are the same.The CPI parameter models the superscalarity of the system.A superscalar processor can execute more than one instruction in a single clock cycle.Non-superscalar processors have CPI 1.Superscalar processors can have CPI1,but the value is usually very dependent on the mix of instructions.The loop overhead()depends on the instruction set of the CPU,but can also depend on the compiler.As defined in the previous section,the loop overhead consist of incrementing two buffer pointers(or one pointer for checksum-like functions),decrementing and testing the loop variable,and a conditional branch back to the beginning of the loop.To make the model as simple as possible,yet still capturing the desired behavior,we haveThe performance model7 chosen to not include constants and tables of the function parameters described in the previous section.We believe that this does not significantly affect the modeled memory behavior of the respective implementation techniques.The constants have almost the same influence on the memory behavior and total performance as the function state has,since the constants also com-pete for the available registers and overflows on the stack.The tables add a random component of scattered memory accesses which is very similar,if not identical,for both implementation techniques.Another limitation in the model is the selection of the output block size.A function must either not produce output at all(checksum-like functions),or produce the same amount of output as input,i.e.,the output block size must be the same as the input block size.4.2Model formulasThe purpose of the performance model is to accurately model the register,cache and memory behavior of the integrated and sequential implementations of a set of data manipulation functions. The number of memory accesses are therefore central in the model.There are two kinds:accesses for message data,denoted DA(data accesses),and accesses for function state,denoted SA(state accesses).Both of these are expressed as the number of word accesses(load or store)per byte of message data.DA for the integrated case is simply:DA ILP21out0:1out0:2(2)If out is zero,the function only reads the message data(1access).Otherwise it writes as well(2 accesses).The number of state accesses for the integrated implementation is:SA ILP28A Performance Model for Integrated Layer Processing We assume that the state in memory has to be loaded from and stored back to memory once per iteration of the ILP loop,thus the“2”in the numerator of the fraction in Equation3.ILP in the denominator is the block size common for all functions in the integrated implementation.It is the least common multiple of the block sizes of all manipulation functions:ILP lcm in1(5)The SA formula for the sequential implementation is a sum over the functions:2SA SEQ1LO DA SA(7)inThe second term is the time to access the message data and the state from the cache(is the cache access time parameter).Thefirst term is the number of clock cycles for the non-memory instructions.The sum is the number of computation instructions per byte for all functions.LO is the number of instructions of loop overhead.It is computed with the following formulas:LO ILP(9)inComparison with measured results9 The sum of the computation instructions and the loop overhead is then multiplied by CPI,the cycles per instruction parameter,to get the time to execute all instructions.The total formula then represents the number of clock cycles to execute the instructions and access the memory. The non-cached,or memory,case is similar to the cached case,but the data accesses are multiplied by,the memory access time:clocks(memory)CPI1(11)5COMPARISON WITH MEASURED RESULTSIn this section we compare the modeled performance to the measurements from a previous paper(Ahlgren,Bj¨o rkman and Gunningberg,1996).In that paper we measured the performance of integrated and sequential implementations of“synthetic”data manipulation functions.The synthetic functions are generated from the same set of function parameters as we use in the model.The synthetic implementations process data and use memory and registers according to the parameters,but does not produce any“useful”output.Synthetic functions makes it easy to study the behavior when one parameter(e.g.,the size of the state)is varied and all other parameters are kept constant.5.1The Abbott&Peterson loopThefirst comparison is both with a real implementation of the BSW AP/PES/CKSUM function combination used by Abbott and Peterson(1993)(see their Figure2)and with a synthetic implementation with the same characteristics.The BSW AP/PES/CKSUM loop consists of a byte-swap,a“pseudo-encryption”and the Internet checksum.We have made one change to their original integrated implementation.We use a block size of two words which saves the“if”statement and gives better performance.We discuss the block size further in Section6.Table2presents the measured performance of the real and synthetic implementations com-pared with the modeled performance for three computer systems.The modeled performance values compare well to the measured real implementation for all three systems.The largest deviation is about10%(for the HP,sequential with warm cache).Tables3and4lists the model parameters used in the comparison.The available CPU registers have been determined by looking at the machine code produced by the respective compilers from synthetic functions with varying state size.The compilers used are the stock“cc”compilers for both operating systems(SunOS4.1.x and HP-UX A.9.5). The SunOS compiler is not particularly good at register allocation.The value of the register parameter therefore varies with the situation.Of the memory accessfigures,the value for the SPARCstation2is calculated from the actual memory access time.For the SPARCstation20/71,the value is calculated from the maximum10A Performance Model for Integrated Layer Processing Table2Comparison of the measured performance ofthe real and synthetic implementations with the mod-eled performance using the BSW AP/PES/CKSUM loop.System impl.type cold cache warm cacheSeq.ILP Seq.ILPHP9000-735/99real95495437synthetic95495437modeled87524937SPARCstation real78395432model20/71synthetic82395733modeled81405831Architecture parameter HP735SS-2SS-20available CPU registers1388cache access(clocks/word)131memory access(ave.clocks/word)47 2.4CPI cycles per instruction110.5loop overhead(instr/loop)155clock frequency(MHz)994075word size(bytes)444Function parameter BSW AP PES CKSUMstate size(words)001constants(words)322instructions per input block564in input block size(bytes)484out output block size480Model analysis110501001502000123456789n s /b y t e State size (words)Message data accessed from cacheSequential, modeled Integrated, modeled Sequential, measured Integrated, measured 0501001502000123456789n s /b y t e State size (words)Message data accessed from memory Sequential, modeled Integrated, modeled Sequential, measured Integrated, measured Figure 3Comparison of measured performance with modeled performance for the HP 9000-735/99.As previously mentioned,for simplicity we chose not to include the constants parameter in the model.It does not affect the result in this case,because all of the state and the constants fits in the available registers.But it leads to a weakness in the comparison.Since the state fits in the registers,the register exhaustion part of the model is not exercised.In the next section we therefore also compare with synthetic loops with varying state sizes.5.2Synthetic loopsThe second comparison is with measurements of synthetic data manipulation functions on the HP 9000-735/99.We have varied the state size and the number of functions.The other function parameters have been kept constant:3functions,4byte input and output block sizes and 10computation instructions per input block.The architecture parameters used are those for the HP in Table 3.As can be seen in Figure 3,the measured and modeled performance correspond very well.We can see that the model captures the register exhaustion just as intended for the integrated implementation when the state size grows above 4words.For a state size of 5words,the aggregated state is 3515words,which does not fit in the available 13registers.6MODEL ANALYSISHow do different factors affect the performance of an integrated implementation compared to a sequential?In this section we will discuss the role of the parameters in the model and how they affect the performance of the two implementation techniques.6.1Abbott and Peterson’s modelBut first we compare the performance model with the model presented by Abbott and Peter-son (1993)(see Figure 4).Their model computes the number of clock cycles for loop overhead,computation and data accesses.This basic structure is very similar to our model.We also count12A Performance Model for Integrated Layer ProcessingLoop overhead Computation Data accesslayers computation data cycles num writeaccessaccessModel analysis13a) Message data accessed from cache 12345051015010203040Functions State size (words)Clock cyclesSequential Integrated b) Message data accessed from memory 12345051015010203040Functions State size (words)Clock cycles Sequential Integrated Figure 5Modeled performance for different state sizes (per function)and number of integrated functions.tions having a larger natural block size are in their model recoded to collect the required number of one-word blocks before doing its computation.The following functions are then repeatedly invoked with a one-word input.Braun and Diot (1996a)advocate that the least common multiple of the functions is used as the block size of the integrated implementation.We agree with their approach,but with a different motivation.The main drawback with the word filter approach is that it introduces additional conditional branches and state variables in the inner integrated loop.In a simple experiment with the previously mentioned BSW AP /PES /CKSUM combination of functions,we received between 25%(for cached message data)and 35%(for non-cached message data)improvement in performance when we increased the block size to two words from the original one word.There are also other considerations for the block size in the case when the function state is too large for the processor registers.When the block size is increased,the additional loads and stores of the function state is amortized over the larger block size.This effect is seen in Figure 6.The slope above the knees is decreasing rapidly when the block size increases.For the case in the figure,the integrated implementation is faster than the sequential when the block size reaches 32bytes,regardless of state size.In this example,the input and output block sizes are the same.The number of available registers are 10for a block size of 4bytes.For all larger block sizes,the number of available registers is decreased with the number of additional registers needed for the larger input block.The other model parameters used to generate this graph are:3instructions per 4bytes of message data,3functions,1cycle cache access time,1CPI and 1instruction of loop overhead.The figure shows the case when message data is in the cache.When the message data is not present in the cache,the absolute difference between the surfaces becomes larger with a constant,but the shapes of the surfaces are the same.Computation instructions In a previous paper (Ahlgren,Bj¨o rkman and Gunningberg,1996)we showed that the number of computation instructions does not affect the absolute performance difference between an integrated and a sequential implementation.The model also has this behavior,since the computation instructions add the same number of clock cycles for both implementation types.14A Performance Model for Integrated Layer Processing 05101548163205101520State size (words)Block size (bytes)Clock cyclesSequential Integrated Figure 6Modeled performance for different state sizes (per function)and block sizes.6.3Architecture parametersNumber of registers The number of processor registers available for function state defines the threshold point when the excess function state is allocated on the stack.Processors with more available registers increase the applicability of the ILP technique to more functions and to functions with larger state.As an example,the MD-5message digest function has a state (the chaining variables)that needs 4registers.MD-5has,however,a 64byte block size,which potentially uses another 16registers,and 64unique constants.Another example is the Safer family of encryption algorithms.It has an 8byte block size,but processes the message data a byte at a time and therefore needs 8registers to hold the input block.The whole expanded encryption key is used for each input block.The expanded key is 104bytes for the minimum recommended 6“rounds”of the algorithm,which potentially use 26registers.Actually,because of the byte operations,one register per byte is desirable.Both of these functions are examples where the CPU registers can not hold everything that is needed in the inner processing loop.If these functions are integrated with another function having state,the registers will overflow even more.Memory access time and cache hit rate The memory access time and the cache hit rate are very important for the performance difference between the integrated and sequential implementations.The slower the memory and the lower the cache hit rate,the more is gained with integration.The actual performance experienced in a real system depends heavily on the cache hit rate for accesses to message data.Sequential implementations are more sensitive to varying cache hit rates than integrated implementations,since sequential implementations access data several times.Conversely,in a real system integrated implementations result in lower cache hit rates than sequential implementations.This is simply the result of that the message data is accessed only once.In extreme cases with low cache hit rates (for message data)and slow memory,the integrated implementation may always be faster than the sequential implementation,also in the cases when the function state overflows the registers.This is illustrated in Figure 7.。