Reliable verification using symbolic simulation with scalar values

Reliable Verification Using Symbolic Simulation withScalarValuesChris Wilson Computer Systems LaboratoryStanford UniversityStanford,CA,94035 chriswi@David L.Dill Computer Systems Laboratory Stanford UniversityStanford,CA,94035dill@ABSTRACTThis paper presents an algorithm for hardware verification that uses simulation and satisfiability checking techniques to determine the correctness of a symbolic test case on a circuit.The goal is to have coverage greater than that of random testing,but with the ease of use and predictability of directed testing.The user uses symbolic variables in simple directed tests to increase the input space that is explored.The algorithm,which is called quasi-symbolic simu-lation,simulates these tests using only scalar(0,1,X)values inter-nally causing potentially conservative values to be generated at the outputs.Divide and conquer of the symbolic input space is used to resolve this conservativeness.In the best case,this method is as efficient as symbolic simulation using BDDs and,in the worst case,gives coverage and predictability at least as good as directed testing.1.INTRODUCTIONThe ability to verify hardware designs has not scaled with the in-crease in design size over time.Although a lot of new methods have been proposed,they have not replaced directed and random simula-tion in most designs as the primary verification method.One reason for this is that new methods concentrate on improving efficiency with less consideration put on meeting the reliability requirements of mainstream verification.Reliability is the ability to produce at least some results,including predictable coverage and useful feedback,when memory or time limits are exceeded.Efficiency is the number of input patterns that can be verified with a given cost.We define cost to include man-power effort,computer resource usage,and project schedule time allocated.The basic methods used in verification can be characterized by their reliability and efficiency.First,directed tests are reliable becauseScalar simulation is simulation in whichvalues,,range over thedomain .Binary simulation is the subset of scalar simulation which ranges over thedomain,.Verification consists of defining the total input space that needs to be simulated to ensure the design is correct.The input space is defined such that a single binary test covers one point of the input space.The total input space that needs to be verified is usually too large to cover completely,so directed and random testing work by sampling the input space.Symbolic simulation verifies a set of scalar tests in the input space with a single symbolic test.Since each scalar test may require a different value on each input,symbolic functions encoded using BDDs are used to represent all the possible values on each input.This is the method used in Symbolic Trajectory Evaluation (STE)[8].The advantage of STE is that it can cover large input spaces efficiently and completely and that it is design size independent allowing scaling to large designs.However,the use of BDDs means that it is not reliable due to the well known BDD blow up problem [3].When BDD blow up occurs,no coverage is obtained since the simulation does not finish and these cases can be hard to debug.1.2Quasi-Symbolic SimulationIn quasi-symbolic simulation,values are not allowed to take on ar-bitrary functions and,consequently,BDDs are not used.Instead,the scalar domain is extended to include multiple,unique unknowns called symbolic variables .Test cases apply values from this domain to the simulator to produce output values to be checked.The algorithm consists of two parts.First,after input values are ap-plied,a simulation engine quickly computes a potentially conser-vative scalar value at each internal node.Second,a decision pro-cedure looks at the result of each run and divides the input space to resolve any unwanted conservativeness;multiple runs are per-formed to cover the entire input space.The rationale behind this is that,in large circuits,most internal node values do not propagate to outputs that are checked and so it is wasteful to generate exact values for these nodes.When the circuit is re-evaluated,only those nodes that actually propagate to checked outputs need be computed more exactly.The key to making this effective is to search the in-put space efficiently.This paper describes a search method that uses case splitting and depth-first search to get the desired reliability and efficiency.Outline We first describe the lattice that forms the basis of our abstraction method and then the basic simulation algorithm and search procedure.We then describe how this procedure is extended to handle reactivity and report on some experiments done to test our methodology.2.BOOLEAN FORMULASA circuit composed with a simulation relation creates a boolean for-mula that the simulator tries to satisfy.A boolean formula consists of strings over variables and theconnectives,.Formulas are evaluated over thelatticewhereand represent 1a unique symbolic variable and its com-plement having the variable identifier,.The lattice is definedas,,and .For soundness,the boolean connectives are required to be monotonic;that is,the rela-tion must hold.Table 1shows amonotonicXQuasi-symbolic simulation evaluates the simulation relation over the circuit directly rather than by conversion to clausal form.A variant of the Davis-Putnam(DP)[6]procedure is used to search over the symbolic input variables.Forward simulation combined with searching over input variables means that leaf nodes of the search give some amount of input space coverage.DP uses depth-first searching which gets to leaf nodes quickly.If time out occurs, it is likely to have reached at least some number of leaf nodes,and the longer the timeout,the more leaf nodes that have been reached. This gives us the reliability we are looking for.The basic DP algorithm is as follows:1.Determine satisfiability by simulating the circuit.2.If satisfiable,terminate with an error.If unsatisfiable,backtrack to a previous decision.Otherwise3.Perform unit propagation of the symbolic variables if allowed.4.Select a symbolic variable,,from the sub-domain,,tocase split on.5.Recursively do these steps with and.Unit propagation is an implication procedure that reduces the space that must be searched.For example,if a formula has the form, ,where is a variable,then must be set to true to satisfy,eliminating the branch in the search tree.DP allowsunit propagation to occur at any time,however,we will show later that our method must disallow unit propagation at certain times in order for the method to be complete.This is the only difference between our method and classical DP.The simulator uses a non-clausal unit propagation algorithm due to Dalal[5],called PCP(Propositional Constraint Propagation.)In Dalal’s method,each sub-formula is evaluated in a bottom up man-ner and two sets,called the C-set and D-set,are associated with each sub-formula.If a sub-formula can be represented as a con-junction of literals2with some sub-formula,,then it has the C-set,.D-sets are constructedsimilarly from variables that are disjuncted with the sub-formula. Once the formula has been evaluated,if the D-set is non-empty, the formula is known immediately to be satisfiable.If the C-set is non-empty,all literals in the C-set are unit propagated to be true. The value of a sub-formula is denoted as if is the set ofliterals in the D-set or if is the set of literals in the C-set.Aliteral,,in this context has both a C-set and D-set exactly equalto the literal and is equivalent to our notion of symbolic variables since the value of this subformula is exactly or when the literalis or respectively.3.2Variable Selection HeuristicThe algorithm uses a heuristic to select the best variable to split that minimizes the size of the search tree.The heuristic starts by associating a variable with each internal signal in the circuit.This variable is selected from the inputs to the function driving the signal as follows:creating a copy of the thread,called a virtual thread which starts ex-ecuting at the statement after the wait and is guarded with the wait condition being true.The original thread continues to wait,guarded with the wait expression being false.As long as the wait expres-sion evaluatesto ,the simulator forks off new virtual threads and continue to wait in the original thread.The semantics of stopping a test are that if any thread executes a stop statement,the entire test is stopped.Again,to maintain mono-tonicity,if the thread guard valueis when a stop statement is encountered,then the test must both stop and continue executing.For correctness,it is necessary to stop the simulation only when a stop statement is encountered.Since the guard condition for a stop could alsobe ,the decision procedure needs to prove that the test really stops at this point by case splitting on this guard condition before case splitting on any error conditions.Unit propagation cannot be used in this case since this is incom-plete.The problem is that the stop guard condition case splitting is trying to prove that the simulation can stop in this particular cycle ,not whether it may stop in this or some future cycle.Since the test case should stop eventually for all case split values,by not allowing unit propagation,the entire search tree will be enumerated,allow-ing the simulator to reach all cycles in which the test can possibly stop.To illustrate this,consider the following example.A four bit counter is initially loaded with a value and then decremented.The counter outputsa signal when the count reaches zero.For any binary vector loaded into the counter,the output will always eventually be asserted.Thus,if a test consists of loading the counter,waiting for the output to be asserted,and then stopping,the test case will always eventually terminate.Now consider a symbolic vector representing the set of all possible count values being loaded into the counter.The initial state and output after simulating one cycle will be asfollows:Since the stop condition is non-zero,the simulator stops at this point and case splits.If unit propagation was allowed,all vari-ables would be set to 0as indicated by the stop condition C-set.In the nextevaluation in the first cycle which means the test case has been proved to actually stop in the first cycle.The decision procedure now backtracks,but since all variables were unit prop-agated,the search would terminate immediately and would have only tested the case in which the test stops after one cycle.But,clearly,the test can stop at all lengths up to the maximum counter value.CompletenessReactive tests are inherently incomplete,even when limited to the binary domain.For example,a wait statement may wait forever for a signal to be asserted due to a bug in either the circuit or test case.This extends to the symbolic domain since symbolic tests represent sets of binary tests.What we are interested in showing is that if all binary tests repre-sented by a symbolic test are complete,then the symbolic test is complete.By disabling unit propagation in the stop condition casesplitting,we correctly explore all test case lengths.The search will also terminate since in any leaf node,a scalar stop value will be pro-duced,and in any abstract node,an value will be produced for the stop condition at a valid stop point.The search is guaranteed to terminate because there are a finite number of symbolic variables.Therefore,the test will always terminate and be complete.5.EXPERIMENTSWe have implemented a Verilog based quasi-symbolic simulator that supports hierarchical,gate level models,is event-based,and is semantically equivalent to a scalar Verilog simulator.RTL simula-tion is supported by synthesizing hierarchical RTL descriptions into equivalent gate level models.We implemented our own test case language because Verilog does not support the creation of symbolic variables and the RTL subset does not have some of the language constructs needed for test benches.We have performed two experiments,the first measures the effi-ciency of our method in finding a bug and the second demonstrates its reliability.The simulator runs 3at a speed of 85K events/second and four time frames/second for the circuit we used.Test lengths ranged from around 30to 100time frames.The simulator plus de-sign required 36MB of main memory and this value did not grow significantly as a test ran.5.1Test Design DescriptionThe design we are using is an industrial bus bridge chip called the MCU [9].We are modeling the bus interface section of the chip which has approximately 140K gates and 2,402state bits.This chip was taped-out,went through bringup and subsequent follow-on versions were designed.In the process of verifying these follow-on designs,bugs were found in the original design that were not caught in the original verification or in bringup.The bug we are targeting in our first experiment is in this category.5.2Experiment 1:EfficiencyOur method improves efficiency compared to directed and random testing because of its completeness by allowing the user to rule out large parts of the input space quickly when looking for bugs.However,most of the effort in verification is spent in writing and debugging test cases.We expect this to be no different for symbolic simulation and so we need to show that our method does not cause significant bottlenecks when bugs are found.In this experiment,we developed a test case to find a known,but not well characterized bug.We wrote an initial test case and then recorded the number of evaluations (nodes in the search tree)and execution time of each run as we tried to debug it.After performing a number of runs,it became apparent that there were basically four possible results:1.(SAT)A test case protocol violation in driving the requests onto the bus would occur.These typically were found with a few evaluations and required no backtracking.2.(TIMEOUT)The test would timeout waiting for a response from the circuit.These were also typically due to test case protocol violations.case evals rand evalsSAT 3.8 1.12249.0LEAK78 1.07.1UNSAT8k53-886.64UNSAT23111-420.4UNSAT2753-352.0UNSAT3337-143.5random simulator would,in all likelihood,set all variables tobothand at some point during a test.However,this problem is easily overcome by re-running the test with a different variable order if it times out.5.4ConclusionsWe have stated that reliability is an important important factor in mainstream verification.Our goal is to improve mainstream meth-ods and our strategy is to ensure reliabilityfirst,and then work on improving efficiency.Symbolic simulation can efficiently search large input spaces and its completeness allows ruling out large in-put spaces,which is a significant advantage over random simula-tion when looking for bugs.It can also use short tests to give the same coverage as long random tests,making tests easier to debug. However,traditional symbolic simulation based on BDDs does not meet our reliability criteria.Quasi-symbolic simulation uses algo-rithms that use scalar values internally and satisfiability checking methods to search over the set of symbolic test cases allowing it to degrade gracefully in the face of resource limits.The combination of predictability and simple test case debugging make our method comparable to directed testing in its ease of use.In our experiments,we showed that our method had better effi-ciency than random testing in ruling out input spaces to search for bugs and that it does not introduce significant bottlenecks when bugs are encountered.We also showed that our method does de-grade gracefully when resource limits are encountered.In summary,quasi-symbolic simulation has the ease of use and predictability of directed testing and the efficiency of random test-ing while meeting the reliability goals of mainstream verification. Thus,we believe that quasi-symbolic simulation can be used as one of the primary methods offinding the majority of bugs in large de-signs with the advantage that it can perform verification tasks not possible with either random or directed testing.6.REFERENCES[1]A.Biere et al.Verifying safety properties of a powerpcmicroprocessor using symbolic model checking without bdds.In Proceedings,Computer Aided Verification(CAV’99),LNCS 1633,pages60–71,1999.[2]V.Boppana et al.Model checking based on sequential atpg.InProceedings,Computer Aided Verification(CAV’99),LNCS1633,pages418–430,1999.[3]R.E.Bryant.Graph-based algorithms for boolean functionmanipulation.IEEE Transactions on Computers,C-35(8):677–691,August1986.[4]K.-T.Cheng.Gate-level test generation for sequential circuits.ACM Trans.Design Automation of Electronic Systems,1(4):406–442,October1996.[5]M.Dalal.Efficient propositional constraint propagation.InProc.of the Tenth National Conf.on Artificial Intelligence(AAAI-92),pages409–414,1992.[6]M.Davis,G.Logemann,and D.Loveland.Machine programfor munications of the ACM,5(7):394–397,1962.[7]M.Ganai,A.Aziz,and A.Kuehlman.Augmenting simulationwith symbolic algorithms.In Proc.of36th Design Automation Conf.,pages385–390,1999.[8]C.-J.Seger and R.E.Bryant.Formal verification by symbolicevaluation of partially-ordered trajectories.Formal Methods in System Design,6(2):147–189,1995.[9]W.-D.Weber et al.The mercury interconnect architecture:Acost-effective infrastructure for high-performance servers.In Proc.of the24th Annual Intl.Symp.on Computer Architecture (ISCA97),1997.。