Introduction

The DaCapo Benchmark Callback provides a convenient way to gather a summary of multiple benchmark invocations in a single file. For each benchmark invocation, the DaCapo Benchmark Callback must be registered as a callback by supplying the --callback org.scalabench.dacapo.callback.XmlSummaryCallback command line option to the harness. Furthermore, the callback JAR as well as its dependencies must be on the system classpath.

After each benchmark iteration, the generated summary file is updated by the callback. By default, this file is called summary.xml and is placed in the current directory. This default can be overridden by setting the org.scalabench.dacapo.callback.xml.summary system property to the desired location of the summary file. The file’s format is described below.

Summary File Format

The DaCapo Benchmark Callback gathers the summary in a straight‐forward XML format, exemplified below for two invocations of the scalac benchmark.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<summary xmlns="http://www.dacapo.scalabench.org/modules/dacapo-benchmark-callback/xmlns/dacapo-benchmark-summary"
    benchmark="scalac">
  <vmName>OpenJDK Client VM</vmName>
  <vmVendor>Sun Microsystems Inc.</vmVendor>
  <vmVersion>19.0-b09</vmVersion>
  <invocation size="small">
    <iteration>
      <valid>true</valid>
      <warmup>true</warmup>
      <duration>7919</duration>
    </iteration>
    <iteration>
      <valid>true</valid>
      <warmup>false</warmup>
      <duration>2101</duration>
    </iteration>
  </invocation>
  <invocation size="default">
    <iteration>
      <valid>true</valid>
      <warmup>true</warmup>
      <duration>30750</duration>
    </iteration>
    <iteration>
      <valid>true</valid>
      <warmup>false</warmup>
      <duration>14489</duration>
    </iteration>
  </invocation>
</summary>

For each benchmark iteration it is noted whether it is a warm‐up iteration and whether its output has been successfully validated. Furthermore, the iteration’s duration (in milliseconds) is noted.