SystemVerilog Reference Verification Methodology: VMM Adoption

Thomas Anderson, Janick Bergeron and Eduard Cerny (Synopsys, Inc.) Alan Hunter and Andrew Nightingale (ARM Ltd.)

The larger and more complex that system-on-chip (SoC) designs grow, the more verification dominates the development process. In fact, effective design reuse puts even more pressure on the verification team to reduce their part of the schedule. The best solution to this dilemma is the adoption and deployment of a reuse-oriented, coverage-driven methodology that yields more efficient verification, while also increasing the likelihood of first-silicon success. This is the last in a series of four articles outlining a reference verification methodology that meets these goals for both RTL and system-level verification. This methodology is enabled by the SystemVerilog hardware design and verification language standard and is documented in the Verification Methodology Manual (VMM) for SystemVerilog, a book jointly authored by ARM and Synopsys. This article focuses on ways to adopt the VMM methodology and deploy it quickly throughout an entire SoC project. Methodology Adoption Advanced verification techniques are not always adopted easily; verification teams are not just sitting around looking for something new to try. Instead, they investigate new approaches when older methods break down due to designs growing larger and more complex. Techniques such as constrained-random stimulus generation, coverage-driven verification, assertions, and formal analysis moved from theory to practice precisely because they were needed for real designs in real projects. Thus, adoption of the techniques recommended in the VMM for SystemVerilog may occur when the most recent SoC project has suffered serious bugs in silicon. However, many verification teams are well aware of the limitations of older methods and do recognize the challenges of new projects before disaster strikes. In either case, the goal is to get up to speed on the VMM methodology as quickly as possible. There are really two forms of adoption required in order to benefit fully from the book. One is embracing the general verification techniques that the reference methodology supports. For example, engineers unfamiliar with the concept of adding assertions or properties need to become comfortable with this approach independent of whatever assertion language or library they may use. Similarly, adopting constrained-random stimulus generation requires familiarity with the role of constraints, regardless of the language used. The object-oriented nature of the VMM methodology may seem to be the biggest hurdle for some of the engineers considering adoption. Encapsulation, classes, inheritance, extensions, and other aspects of object-oriented programming do require thinking about the verification environment differently than with a traditional testbench. Fortunately, many engineers have had some exposure to languages such as C++ and Java, so they just need to carry their object-oriented concepts into the verification domain. The other form of adoption is using the SystemVerilog-specific techniques in the book, which is aided greatly by the building-block libraries specified in the appendices. While it's true that comfort with the general verification concepts can help with adoption of the methodology for SystemVerilog, it's also the case that the SystemVerilog libraries help users understand the concepts. For example, the assertion-checker library suggests concrete places to place assertions and the base-class library is a great starting point for using object-oriented verification. The VMM for SystemVerilog specifies four libraries that help accelerate and ease adoption: The VMM Standard Library, a set of SystemVerilog testbench base and utility classes The VMM Checker Library, a set of SystemVerilog assertion checkers The XVC Standard Library, a set of SystemVerilog system-level base and utility classes The Software Test Framework, a C library for software verification Using these libraries allows development teams to get started more quickly and easily when setting up their verification environments. Users of these libraries have reported saving at least several months in the early phases of projects. Further, adoption of standard libraries establishes consistency among different users, across multiple projects, and with vendors providing VMM-compliant verification components. The following sections provide brief overviews of the four libraries. The VMM Standard Library As described in the second article of this series, the VMM for SystemVerilog defines a layered testbench architecture that supports advanced verification and fosters reuse. The control of this testbench and the execution of testcases involve a series of well-defined steps under the control of "virtual methods." The VMM Standard Library defines the vmm_env base class to control the run of each test case, as shown in (Fig. 1). 1. The vmm_env class defines a series of virtual methods to execute each test case. The process shown in (Fig 1) involves generating the test case configuration, building the testbench, resetting the design under test (DUT), configuring the DUT, running the test, performing cleanup at the end, and generating reports. The virtual methods in the vmm_env base class provide the infrastructure for each of these steps, however many of the methods have to be "extended" to perform actions specific to the DUT. The vmm_log class provides an interface to the VMM message service so that all messages, regardless of their sources, can have a common "look and feel." The message service describes and controls messages based on several concepts: Message source (transactor, generator, test case, etc.) Message filters (promote, demote, or suppress messages) Message type (failure, timing error, debug information, etc.) Message severity (fatal error, non-fatal error, warning, etc.) Simulator message handling (continue, abort, invoke debugger, etc.) The vmm_data base class is the basis for all transaction descriptors and data models in the testbench. This class can be extended to build any model appropriate for the particular testbench, for example an Ethernet MAC frame data model or the description of a packet on a serial bus. Transactions are modeled by transaction descriptors, also extended from the vmm_data class. This makes it easier to create streams of random transactions than is the case with traditional procedure calls. Other classes in the VMM Standard Library include: vmm_channel: provides a generic transaction-level interface mechanism vmm_broadcast: replicates transactions on multiple channels vmm_notify: interfaces to synchronization for concurrent execution threads vmm_xactor: serves as the basis for all transactors Together, all of these classes provide essential building blocks for the overall verification environment, accelerating testbench development while providing the hooks for a wide range of customization to meet the verification needs for the particular design under test. The extensibility of predefined base classes is a key to the object-oriented approach; each verification team can customize the testbench environment and its operation without having to modify the base classes themselves. SystemVerilog source code for the base classes can be helpful when developing proprietary classes, so Synopsys offers a no-cost license for its implementation of the VMM Standard Library. The VMM Checker Library The role of assertions in finding more bugs, more quickly has been well-documented for many years. Assertions capture critical designer intent in executable form, isolate design errors close to the source, speed up debug time, and enable formal analysis to find corner-case bugs that might be missed by simulation. Given these advantages, it is surprising that not all design and verification teams use assertions as a matter of course. One past barrier to assertion adoption is that engineers had to learn special languages and buy expensive tools. Now that SystemVerilog includes powerful assertion constructs that are directly supported by leading simulators, adoption is much easier. In fact, recent surveys have shown a significant increase in assertion usage since SystemVerilog support became available. However, it may still take time for engineers to become comfortable with assertions; it can be hard to "think of what to check." An assertion-checker library is an excellent way to make it easier to add assertions to RTL designs. The checkers are designed to map to common design elements, such as FIFOs, stacks, arbiters, memories, state machines, and handshake interfaces. Engineers don't have to think about what sort of assertions might be appropriate for these structures; they can simply connect an arbiter's request and grant signals to an arbiter checker or the memory address bus and control signals to a memory checker. Surveys have also shown the value of assertion-checker libraries; the Open Verification Library (OVL) from the language standards body Accellera has been widely adopted. The VMM for SystemVerilog specifies a superset of OVL that adds support for types of design elements"including arbiters, FIFOs, synchronous and asynchronous memories, and stacks"not covered by OVL. Fig 2 shows the complete set of 50 assertion checkers defined in the VMM Checker Library. 2. The VMM Checker Library extends OVL's assertion capabilities These checkers are implemented as SystemVerilog modules, so they can be placed anywhere in the design or testbench that is legal for module instantiation. Usage is quite simple; the user simply connects the clock, reset, and signals to be checked. For example, the following checker instantiation ensures that two signals, hot and cold, are mutually exclusive (never on at the same time): assert_mutex temperature_check(reset_n,clock,hot,cold); Synopsys recently donated its SystemVerilog implementation of the VMM Checker Library to Accellera with the expectation that a future version of OVL will include the full set of assertion checkers defined in the VMM for SystemVerilog. The XVC Standard Library As described in the third article of this series, the VMM for SystemVerilog defines the extensible verification component (XVC), a system-level transactor extended from a block-level transactor or a combination of block-level transactors. The book also specifies the XVC standard library, a set of classes for use in building an XVC for system-level verification as shown in (Fig 3). 3. An XVC can be built using classes from the VMM Standard Library and the XVC Standard Library. The XVC manager is an optional verification component responsible for the high-level synchronization of XVCs. The synchronization and XVC control mechanisms can be user-defined according to the need of the system or a specific test. The XVC Standard Library specifies the xvc_manager base class, as well as the pre-defined XVC manager vmm_xvc_manager class, which is implemented as an extension to the base class. The vmm_xvc_manager class provides several additional elements beyond xvc_manager, including notifications to control actions, a pre-defined command file structure, and a command format that includes action control, event control, interrupts, execution, and logging. Because the command files describe the test scenarios, it is not necessary to recompile the testbench or DUT to run different scenarios. The user can reuse or modify the command files as needed. The xvc_xactor base class is used to implement XVC-compliant transactors. It is derived from the vmm_xactor class but offers additional features, including trace logging, an input channel (xvc_action_channel) for actions to be executed in order, another input channel for interrupts, and a notification service interface. Finally, the xvc_action base class is used to define the commands and the actions they produce. The Software Test Framework Verification using embedded software is an important part of any system verification infrastructure in which a host processor directs application data and controls memories and peripherals. For system-level testing where a CPU or DSP is part of the system design being tested, it is desirable to have a verification environment that supports the execution of test software to demonstrate that the system can successfully support the execution of an operating system, application software or a DSP control algorithm. The VMM for SystemVerilog defines a software test environment to complement the hardware-centric testbench infrastructure described in the book and covered in the previous articles in this series. This environment is used in place of an operating system in a CPU-centric system design. System verification can thus be conducted prior to the operating system being available on the system itself. XVCs in the verification environment work in concert with the software test framework such that both external and software-internal stimulus can be generated and synchronized to create interesting and relevant system conditions to meet the hardware/software verification requirements. The VMM for SystemVerilog defines a C library that can be used to implement a VMM-compliant software verification environment and tests. The elements of this library include: System Descriptor: array of peripheral device descriptors Test Actions: macros, declarations and functions to support the specification and execution of test actions on specific peripherals Low-Level Services: functions to issue messages, identify the software verification environment used to execute the tests, skip tests that cannot be executed in this environment, access memory-mapped registers, manage system resources, manage caches, control interrupts, generate random numbers, specify assertions, and provide software-XVC connectivity Conclusions When coupled with an appropriate methodology, SystemVerilog provides all the constructs and features necessary to build a complete RTL and system-level (ESL) verification environment. Interaction with SystemC models and with a C-based software test environment is also fully supported. The Verification Methodology Manual for SystemVerilog encompasses this complete scope, describing a robust, proven, reference methodology and specifying building-block libraries to accelerate adoption and project-wide deployment. This series of four articles has provided an outline of the reference methodology and libraries from the VMM for SystemVerilog. The most obvious next step is to read the complete book, but other forms of information are also available. In fact, the industry has embraced the VMM approach: a Japanese translation has been published, two more VMM-related books are now available, training classes are offered by several vendors in addition to Synopsys, and there's even a VMM college course through the University of California Extension, Santa Cruz. www.vmm-sv.org provides more information on industry support for the VMM methodology. Adopting the methodology and recommendations of the VMM for SystemVerilog is a proven way to address the verification challenges of today's huge chips. Built on countless years of experience by leading industry experts from ARM, Synopsys, and their customers, this book makes their expertise available to any development team. The reward for adoption will be a more efficient and more thorough verification process, resulting in a much greater chance for first-silicon success.