藏在系统核心芯片中的DRAM控制器

藏在系统核心芯片中的DRAM控制器

Ron Wilson，总编辑，Altera公司

DRAM控制器藏在您的系统核心芯片系统(SoC)中——可能有两个，甚至是四个。有一些精心制作的逻辑小模块，用于连接SoC内部和外部DRAM，它们并没有引起系统设计人员的注意。它们有可能造成很大的问题，浪费带宽，占用太多的能耗，甚至导致数据被破坏。

DRAM控制器能否正常工作会使得系统有很大的不同，有的系统能够满足其设计要求，而有的系统则运行缓慢，过热，甚至失败。不论哪种情况，最终是由系统设计团队承担责任，他们一般很少掌握控制器的信息。

成功还是失败都源自我们要求DRAM控制器所做的工作。模块不仅仅是一个接口。在高级系统设计中，DRAM控制器必须很好的处理SoC体系结构复杂而又难以预测的存储器申请，以及一侧的系统软件申请，还有另一侧DRAM芯片设计复杂的时序和约束要求。能否处理好这些关系会在多个方面影响DRAM吞吐量：这很容易在系统性能上体现出来。

为解释这些问题——以及系统设计人员能够对此做什么，我们需要回答三个主要问题。首先，我们应检查DRAM芯片提出的要求。然后，需要讨论SoC体系结构对存储器访问模式的影响，第三，研究一个高级DRAM控制器的结构和功能。通过这三部分，我们得出系统设计的一些结论。

DRAM需要什么

系统规划对外部存储器的要求是确定性随机访问：任何时候来自任何位置的任意字，具有固定延时。但是，确定性随机访问恰恰是现代DDR3DRAM所不能提供的。

相反，DRAM提供任何您需要的字，但是具有复杂的时序约束，因此，很难知道数据究竟什么时候出现。图1中“简化的”状态转换图简单解释了为什么会这么复杂。这种复杂度也意味着，命令到达DRAM芯片的顺序会对时序以及带宽有很大的影响。要理解这一点，我们需要深入了解DDR3DRAM。

图1.DDR DRAM芯片“简化的”状态图显示了控制器设计人员所面临的复杂问题。

DRAM芯片将数据存储在电容阵列中。当您读写数据时，您并不会直接访问阵列。而是在读写之前，您激活阵列中的某一行。激活命令使得DRAM读取该行中的所有列的所有比特，将其送入传感放大器块，它实际上用作该行的本地寄存器文件。然后，您可以对传感放大器上的数据发出读写命令。通过这种方式，能够非常快的读写已经激活的行：一般是三到五个时钟来开始一次突发传送，然后，在突发期间传送每个字节需要一个时钟。例如，这种时序安排使得DDR3DRAM非常适合L2高速缓存数据交换。

但是，如果您不使用已经激活的行，那么会非常复杂。改变行时，即使是一个字节，您也必须对当前行去激活，然后激活一个新行。这一过程需要确定已经在一段时间内激活了当前行。由于读取DRAM单元是破坏性的，因此需要最小延时：您激活了一行后，DRAM实际上是将最新到达传感放大器上的数据复制回比特单元阵列中，然后刷新行。您可以在此期间读写当前行，但是，要确定在您改变行之前完成了这一过程。

即使满足了这一要求，也还有其他问题。您必须对阵列预充电。预充电命令使得传感放大器中的数据无效，提升阵列和传感放大器输入之间导线上的电压，使得电压值位于逻辑0和逻辑1电平之间。这种准备是必要的，比特单元电容上很小的电荷都会传送到导线上，以某种方式提示传感放大器。

对导线进行预充电之后，您必须向新行发送一个激活命令，等待操作完成，然后，您最终可以发送一个读操作新命令。加上所有涉及到的延时后，即，读取字节序列的最差情况，每一字节都来自不同的行，这要比读取来自一个新行连续位置相同数量字节的时间慢十倍。

这种不同还只是部分问题。如图2所示，DDR DRAM有多个块：与比特单元无关的阵列。DDR3DRAM中有八个块，每一块都有自己排列成行的传感放大器。因此，原理上，您可以通过激活每一个块中的一行，读写较长的突发，然后，对每一激活后的行进行读写操作——实际上是对块进行间插操作。唯一增加的延时是连接每一块的传感放大器和芯片内部总线的缓冲的切换时间。这一延时要比对相同块中一个新行进行预充电和激活的时间短得多。

图2.一个典型的DDR DRAM结构图。一个DDR3器件会有8个块，而不是4个。

这就是原理。实际中，您可以对块进行间插处理，但是有一个限制，不是基于DRAM逻辑，而是芯片能够承受的热量。这种限制可以通过著名的“滚动四块访问窗口”，即，tRAW来表达：您一次能够有四个激活块的最长时间。这一规则实际上有例外，只要您从一个块转向下一块之前，在一个块上保持一定的时间，那么，您可以有连续激活的8个块。但是您应该知道：这比较复杂。

建立一个控制器

与前面所述不同的是DRAM时序非常复杂，接近混沌。从DRAM芯片设计人员的角度看，这非常合理，但是，很难满足多核SoC的需求。DRAM序列或者时序命令上看起来无关紧要的小改动会导致您访问存储器的带宽的巨大变化。由于存储器带宽通常是关键任务的瓶颈所在，因此，带宽的变化很快就会影响系统性能。然而，命令序列和时序来自应用程序和系统软件之间，以及系统硬件各种单元之间复杂的交互——包括缓存控制器、存储器管理器、直接存储器访问(DMA)控制器和加速器，以及DRAM控制器。

SoC的功能越来越强大，这种情况会更加复杂。目前，一个多核系统级IC会有同时运行的两个甚至更多的多线程CPU，导致共享L2高速缓存来读取指令线，随机对数据线进行读写操作。同时，计算加速器以自己的方式遍历数据结构。一个器件可以处理流视频，另一个用于矩阵乘法预读取，第三个执行路由表的随机访问。增加一个散射收集DMA控制器，处理光纤接口、硬盘和显示器之间的数据，结果是，在DRAM控制器的系统侧会有些不协调。

如果DRAM控制器只是按照系统接收顺序进行操作，那么，优化DRAM操作的工作会同等落在规划人员、设计人员和软件开发人员上——这是很难做到的。Altera公司战略市场经理Argy Krikelis提醒说：“特别是多核设计，规划人员遇到定位和性能问题。”责任落在DRAM控制器上，那么，尽可能利用其信息消除这种不协调，转换为经过优化的命令流。

深入了解DRAM控制器就会知道，这些模块的设计人员怎样处理这些难题。您可以认为一个现代DRAM控制器有三个主要模块——物理接口、命令处理器以及事物处理器——如图3所示。

图3.一个现代DRAM控制器涉及到事物处理器、命令处理器和物理接口。

物理接口连接DRAM芯片或者存储器模块。它读取来自命令处理器的一个命令流，将具有正确时序的命令发送至DRAM芯片，管理相关的数据字节流。接口收发器、命令和数据同步缓冲，以及产生正确命令和数据时序的状态机都含在这一模块中。而且，还有用于进行复杂的初始化操作的状态机，校准DDR3DRAM规范设定的序列，如图1所示。此外，某些应用的物理接口还会包括自测试、诊断和误码探测以及纠错硬件。当您改变DRAM的容量或者速率等级时，必须调整物理接口。

物理接口的上游是命令处理器。这一模块跟踪DRAM的状态，将到达总线读写周期转换为相应的DRAM命令序列。例如，命令处理器会找到通过其输入队列散射连续字的总线读序列，然后，向其输出队列发出预充电，激活，以及模块读命令。对此，命令处理器必须知道当发出新命令后，将打开哪一芯片的哪一块的哪一行。在某些设计中，命令处理器还处理地址重新映射，在多个块上扩展一个连续的数据结构。

随着对带宽需求的增长，命令处理器的复杂度也在不断提高。例如，处理器会提前处理其输入队列，重新安排操作，尽可能保持在激活的行上，重叠预充电读操作，或者对块进行间插操作。最重要的是，处理器会尽量避免一个块的行之间出现乒乓效应。必须确定所有这些调整，并且随时能够进行调整。

这方面的努力会有其回报。Krikelis说：“我们看到在某些应用中，分组和重新排序能够实现92%的理论最大DRAM带宽。”

最后，事物处理器位于命令处理器和SoC的其他部分之间。一般有一些通道连接至SoC的高速中心交换结构上。事物处理器的主要工作是将到达的各种通道的读写数据流进行组合，加上优先级，这样，每一通道得到了所需的延时和带宽——因此，每一高速缓存控制器、DMA引擎或者这些通道另一端的加速器也得到了所需的延时和带宽。

在动态环境中选择这种优先级方案并不容易。如果您不能精确的预测每一通道的数据流特性，那么，这会非常困难。理想情况下，工作负荷是固定的，因此，您可以为其优化优先级方案。或者，会有一些清晰的访问模式，随着数据流的变化而提供动态调整优先级。Krikelis 说，系统规划人员和控制器设计人员研究了这一问题，使用了从表格到商用DRAM仿真工具的所有工具。但，还是无法让工作更简单一些。

在某些情况下，应用的特征很明显，事物处理器会承担更多的工作。Krikelis说，可以对最近的DRAM行进行高速缓存操作，或者控制器中经常被激活的行进行高速缓存操作。而且，在某些情况下，设计人员可以针对某些通道进行一些特定任务的重新排序或者某些读写操作。

高级DRAM控制器中的三个主要模块协同工作，能够使复杂多核SoC尽可能接近最大理论DRAM带宽。但是提高带宽可能需要牺牲延时，最高优先级线程除外。某些控制器设计会有32或者64深命令序列，意味着，低优先级访问会长时间停留在序列中。一般而言，DRAM 控制器能够进行的工作越多，它处理的SoC体系结构和组合任务就越具体。这就把难题留给了系统设计人员。

回到系统级

您可能会说：“很有趣。对此，我应该做什么?”正如我们在开始所阐述的，软件、系统硬件以及控制器之间的交互会决定您从DRAM那里能够得到的实际带宽。作为一名系统设计人员，您的确有一定的自由度。

最好的方法一般是采用SoC供应商的参考设计。参考设计团队完成了他们的工作。理想情况下，您完全按照设计人员所希望的方式来使用SoC。Krikelis提醒说：“如果您购买了ASSP，那就没有太多的选择。DRAM控制器和芯片中的其他模块会针对特定的应用进行整体优化。”

参考设计中的这些软件也是在知道了这些优化后才编写的。例如，经验丰富的编程人员会尽可能保持存储器参考位于行中，可以同时打开，以便减少高速缓存未命中和DRAM行未命中等问题。他们能够熟练的在块上分配数据结构，采用间插操作。他们可以安排CPU内核、加速器和DMA的工作，避免控制器可能解决不了的冲突问题。他们知道，对于控制器中未处理器的命令，DRAM、高速缓存以及命令队列中的数据，数据一致性是他们要解决的关键问题。采用这类参考设计的系统设计人员的工作是尽量不打破这种一致性。

但是有些时候，系统设计人员会有更大的自由度。Krikelis指出，如果DRAM物理接口是可配置的，您可以通过简单的使用更大的DRAM来提高存储器的有效带宽。在某些情况下，可以调整一些DRAM控制器的内部参数，例如，分配给通道的优先级、重新排序算法，以及命令队列的深度等。

但是，在某些情况下，仅仅进行调整是不够的。Krikelis提醒说：“没有一个简单的答案来满足所有人的规划需求。有时候您需要建立自己的访问抽象层。”

对于资金雄厚的有影响的设计团队，这意味着，与ASSP供应商合作，修改事物处理器，甚至是命令处理器。对于其他规模较大的工程，DRAM带宽需求会满足开发ASIC的要求。

对于不能满足ASIC前端成本的设计，替代方案是系统级FPGA。通过这些方法，系统设计人员在控制器的某些部分采用现有的知识产权(IP)，设计尽可能多的定制操作和命令处理操作，以满足其需求。

即使系统团队选择不去修改DRAM控制器，他们理解其功能也很重要。很多选择都能够实现与DRAM控制器的互操作，从DRAM芯片选择到数据怎样在系统中输入输出，线程怎样分配给处理器，应用程序怎样将数据结构映射到物理存储器中等。难点是怎样高效的使用DARM带宽，最终目的是提高整个系统的性能和能效。

DRAM Controllers for System Designers

By Ron Wilson,Editor-in-Chief,Altera Corporation

Buried somewhere inside the system-on-a-chip(SoC)at the heart of your system is a DRAM controller—or maybe there are two,or four.They are carefully-crafted,tiny blocks of logic that quietly go about their business of connecting the internals of the SoC to external DRAM,requiring no attention from system designers.Or,they wreak havoc,wasting bandwidth,burning energy needlessly,and even allowing data to be corrupted.

The proper operation of the DRAM controller can make the difference between a system that meets its design requirements and a system that runs too slow,overheats,or fails.Either way,ultimately the system design team—who often have little access to information about the controller—will bear the responsibility.

The difference between success and failure arises from the job we ask the DRAM controller to do.The block is far more than just an interface.In advanced system designs,the DRAM controller must mediate between the complex and unpredictable patterns of memory requests dictated by the SoC architecture and system software on one side,and the byzantine timing and obstructive constraints of the DRAM chip design on the other.The difference between good and bad mediation can be a factor of two or more in effective DRAM throughput:a change easily visible in system performance.

To explain these issues—and what system designers can do about them—we need to address three topics.First, we will examine the requirements imposed by the DRAM chip.Then we will discuss the influences of the SoC architecture on memory access patterns,and third,we will explore the structure and function of an advanced DRAM controller.These three sections will lead us to some conclusions about system design.

What DRAMs Want

All a system architect asks of external memory is deterministic random access:any word from any location at any time,with a fixed latency.But deterministic random access is exactly what the modern DDR3DRAM does not offer.

Instead,DRAM offers any word you want,but with a labyrinth of timing constraints that make it difficult to know when the data will actually appear.The“simplified”state-transition diagram in Figure1captures some idea of this complexity.This complexity also means that the order in which commands arrive at the DRAM chip can have a huge influence over timing,and hence,bandwidth.To understand,we must look inside the DDR3DRAM.

simplified””state diagram for a DDR DRAM chip suggests the complexity faced by Figure1.The“simplified

controller designers.

The DRAM chip stores data in an array of capacitors.When you read or write data,you don’t go directly to the array.Instead,before reading or writing,you activate a particular row in the array.The activate command causes the DRAM to read all the bits from all the columns in that row into a bank of sense amps,which in effect then serve as a local register file for the row.Then you can issue read or write commands that act on the data in the sense amps.In this way reads and writes to an already-active row can be very fast:typically three to five clocks to begin a burst transfer,and then one clock per byte during the burst.This timing makes the DDR3 DRAM wonderful for exchanging data with L2caches,for example.

But if you leave the row you have activated,things get more complicated.To change rows—even for one byte—you must deactivate the current row and activate the new one.This process requires making sure that the current row has been active for a specified minimum time.This minimum delay is necessary because reading

DRAM cells is destructive:as soon as you activate a row,the DRAM copies the data newly arrived at the sense amps back into the bit-cell array,refreshing the row,in effect.You can do reads and writes on the current row while this is happening,but you have to be sure that this process is completed before you change rows.

Even after this requirement is met,there’s more.You must precharge the array.The precharge command invalidates the data in the sense amps,and it pumps up the voltage on the wires between the array and the sense-amp inputs to halfway between the logic-0and logic-1levels.This preparation is necessary so that the tiny charge—or lack thereof—on the bit cell capacitors can transfer into the wires and tip the sense amps one way or the other.

Once you have precharged the wires,you must send an activate command for the new row,wait for the operation to complete,and then you can finally send a new read command.After adding up all the delays involved,the worst case for reading a sequence of bytes,each from a different row,can be ten times slower than reading the same number of bytes from consecutive locations in one row.

This difference is only part of the story.As Figure2illustrates,DDR DRAM has multiple banks:independent arrays of bit cells.Each bank,of which there are eight in DDR3DRAMs,has its own row of sense amps.So,in principle,you could read or write a longer burst by activating a row in each of several banks,and then reading or writing a burst in each activated row—in effect interleaving the banks.The only additional delay would be the switching time of the buffers that connect each bank’s sense amps to the chip’s internal bus.That delay would be far shorter than the time to precharge and activate a new row within the same bank.

Figure2.A typical DDR DRAM block diagram.A DDR3device would have eight banks instead of four.

That is the principle.In practice you actually can interleave banks,but there is a limitation,based not on the DRAM logic but on the amount of heat the chip can withstand.The limit is expressed as the semi-famous“rolling four-bank access window,”or tRAW:the longest time during which you can have four banks activated at one time.There are actually exceptions to this rule,allowing you to have all eight banks active continuously so long as you spend the right amount of time in one bank before moving to the next one.But you get the idea:it’s complicated.

Creating a Controller

The point of the foregoing excursion is that DRAM timing is intricate,bordering on chaotic.It makes perfect sense from the point of view of a DRAM chip designer,but is hopelessly inappropriate to the needs of a multicore SoC.A small,seemingly inconsequential change in the sequence or timing of commands to the DRAM can result in a huge change in the bandwidth you can actually get through the memory.With memory bandwidth often the bottleneck for critical tasks,the change in bandwidth quickly becomes a change in system performance. Unfortunately,the command sequence and timing come from a complex interaction of the application and

system software,the various elements of the system hardware—including cache controllers,memory managers, direct memory access(DMA)controllers,and accelerators—and the DRAM controller.

The situation only gets more complicated as SoCs grow more powerful.Today a multicore system-level IC may have two or more multithreaded CPUs running concurrently,causing a big shared L2cache to fetch lines of instructions and to read and write lines of data seemingly at random.At the same time,computational accelerators may be traversing data structures,each in its own way.One device may be streaming video while another prefetches for a matrix multiplication and a third executes random accesses to a routing table.Add a scatter-gather DMA controller pumping data between a fiber interface,a disk,and a display,and the result is that,at the system side of the DRAM controller,you have cacophony.

If the DRAM controller simply processed operations in the order received from the system,the responsibility for optimizing DRAM transactions would fall equally upon architects,hardware designers,and software developers—a pretty hopeless situation.“Especially with multicore designs,architects run into localization and performance issues,”warns Argy Krikelis,strategic marketing manager at Altera Corporation.The responsibility falls on the DRAM controller,then,to use the information available to it to turn cacophony into an optimized stream of commands as best it can.

A look inside the DRAM controller shows how designers of these blocks are responding to this predicament.You can think of a modern DRAM controller as having three main blocks—the physical interface,the command processor,and the transaction processor—as shown in Figure3.

Figure3.A modern DRAM controller involves a transaction processor,a command processor,and a physical interface.

The physical interface connects to the DRAM chips or memory modules.It takes in a single stream of commands from the command processor,sends the commands with proper timing to the DRAM chip,and manages the

associated flows of data bytes.The interface transceivers,synchronous buffers for commands and data,and a state machine to generate proper command and data timing are all included in this block.Also,there is a state machine for the complex initialization and calibration sequences specified in the DDR3DRAM spec and alluded to in Figure1.In addition,the physical interface in some applications may include self-test,diagnostic,and error-detection and correction hardware.The physical interface must be adjusted when you change to a different size or speed-grade of DRAM.

Upstream from the physical interface is the command processor.This block keeps track of the state of the DRAM, and translates incoming bus read and write cycles into the proper sequence of DRAM commands.For example, the command processor might find a series of bus reads of consecutive words scattered through its input queue, and,in response,issue a precharge,an activate,and a block-read command to its output queue.In order to do this,the command processor must know which rows in which banks of which chips will be open when the new commands get issued.In some designs,the command processor also does address remapping,in effect spreading one contiguous data structure across multiple blocks.

As the need for bandwidth increases,the complexity of the command processor must increase,too.For example, the processor might look ahead through its input queue and attempt to reorder operations to stay on an activated row as long as possible,to overlap reads with precharges,or to interleave banks.Above all,the processor will seek to avoid ping-ponging between rows within one bank.All of these adjustments must be identified and made on the fly.

This effort can have its rewards.“We’ve shown that in some applications,grouping and reordering can achieve 92percent of the theoretical maximum DRAM bandwidth,”Krikelis says.

Finally,the transaction processor sits between the command processor and the rest of the SoC.It typically has a number of channels connecting it to the SoC’s high-speed central switch.The main job of the transaction processor is to blend together the streams of reads and writes coming in on the various channels,imposing a priority scheme so that each channel—and,therefore,each cache controller,DMA engine,or accelerator on the other ends of those channels—gets the latency and bandwidth it needs.

Selecting such a priority scheme in a dynamic environment is not easy.It is more difficult if you can’t accurately predict the characteristics of the traffic on each channel.Ideally,the workload would be fixed so you could optimize a priority scheme for it.Or there would be several clearly identified modes of access,and provision for dynamically adjusting priorities as the traffic shifted.Krikelis says that system architects and controller designers study this problem to exhaustion with tools ranging from spreadsheets to commercial DRAM simulation tools.But nothing makes the job easy.

In some cases where the application is well characterized,the transaction processor may take on even more responsibility.“It can make sense to cache the most recent DRAM row,or a row that is frequently activated,in the controller,Krikelis says.Also,in some cases,designers may do some task-specific reordering or speculative reads or writes for a particular channel.

Working together,the three major blocks in an advanced DRAM controller can move even a complex multicore SoC closer to the theoretical maximum DRAM bandwidth.But bandwidth may come at the expense of latency for

all but the highest-priority threads.Some controller designs may have32-or64-deep command queues, meaning that a low-priority access may be lodged in the queue a long time.In general,the more aggressive the DRAM controller is,the more specifically it must be tuned to a particular SoC architecture and task mix.And therein lays the challenge for the system designer.

Back to System Level

“Fascinating,”you say.“What does that have to do with me?”As we said at the beginning,interaction of software,system hardware,and the controller will determine the actual bandwidth you get from your DRAMs. As a system designer,you do have some degrees of freedom.

The best approach often is to adopt the SoC vendor’s reference design.The reference design team has—one hopes—done their homework.And ideally,you are using the SoC in the exact way that the designers envisioned.“If you are buying an ASSP,you don’t have many options,”Krikelis warns.“The DRAM controller and the other blocks in the chip have probably been optimized together for a specific application.”

The software in the reference design has probably been written with knowledge of those optimizations,too.For example,skilled programmers will try to keep memory references localized to rows that are likely to be open simultaneously in order to minimize both cache misses and DRAM row misses.They will skillfully distribute data structures across banks to exploit interleaving.They will choreograph operation of CPU cores,accelerators,and DMA to avoid contention issues the controller could not resolve.And they will be aware that with many pending commands queued in the controller and versions of the data spread among the DRAM,caches,and command queues,data coherency is a critical issue that is up to them to solve.The system designer’s job in applying such a reference design is to not break it.

But sometimes the system designer has more freedom.If the DRAM physical interface is configurable,Krikelis points out,you can often add effective memory bandwidth by simply using a bigger DRAM.And in some cases, some of the internal parameters of the DRAM controller,such as the priorities assigned to the channels,the reordering algorithms,or the depth of the command queue,may be adjustable.

For some situations,though,a few knobs are not enough.“There is no single answer to everyone’s scheduling needs,”Krikelis asserts.“Sometimes you have to create your own access abstraction layer.”

For rich and influential design teams,this can mean working with the ASSP vendor to modify the transaction processor and maybe the command processor.For other large projects,the need for DRAM bandwidth may justify developing an ASIC.For designs that can’t justify the up-front cost of an ASIC,the alternative increasingly is a system-level FPGA.These approaches allow system designers to exploit existing intellectual property(IP)for some portions of the controller and to design as much custom transaction and command processing as they require.

Even if the system team chooses not to modify the DRAM controller at all,it is still important for them to understand its function.Many choices--from DRAM chip selection to how data moves in and out of the system, to how threads are allocated to processors,to how applications map data structures into physical memory—all

can interact with the operation of the DRAM controller.At stake is the efficient use of DARM bandwidth and, ultimately,the performance and energy efficiency of the entire system.