criterion performance measurements
overview
want to understand this report?
| linear | |
| map | |
| monoidal | |
| tree | |
| static-tree | |
| static-linear |
linear/front
25.0 ns 25.5 26.0 26.5 27.0 27.5 28.0 28.5
mean |
25.0 ns 25.5 26.0 26.5 27.0 27.5 28.0 28.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 25.25 ns | 25.28 ns | 25.41 ns |
| Standard deviation | 101.5 ps | 326.1 ps | 729.1 ps |
Outlying measurements have slight (5.7%) effect on estimated standard deviation.
linear/middle
354 ns 356 358 360 362
mean |
354 ns 356 358 360 362
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 353.7 ns | 353.8 ns | 354.1 ns |
| Standard deviation | 549.9 ps | 937.0 ps | 1.859 ns |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
linear/back
665 ns 670 675 680 685 690
mean |
665 ns 670 675 680 685 690
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 668.2 ns | 668.9 ns | 670.1 ns |
| Standard deviation | 2.843 ns | 4.476 ns | 6.407 ns |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
map/front
80 ns 85 90 95
mean |
80 ns 85 90 95
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 77.48 ns | 77.69 ns | 78.53 ns |
| Standard deviation | 412.8 ps | 1.904 ns | 4.431 ns |
Outlying measurements have moderate (18.1%) effect on estimated standard deviation.
map/middle
80 ns 82 84 86 88
mean |
80 ns 82 84 86 88
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 79.41 ns | 79.55 ns | 79.94 ns |
| Standard deviation | 468.2 ps | 1.080 ns | 2.315 ns |
Outlying measurements have slight (6.6%) effect on estimated standard deviation.
map/back
66 ns 68 70 72 74 76 78
mean |
66 ns 68 70 72 74 76 78
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 65.87 ns | 66.03 ns | 66.49 ns |
| Standard deviation | 527.9 ps | 1.275 ns | 2.718 ns |
Outlying measurements have moderate (12.3%) effect on estimated standard deviation.
monoidal/front
42.5 ns 43.0 43.5 44.0 44.5 45.0
mean |
42.5 ns 43.0 43.5 44.0 44.5 45.0
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 42.44 ns | 42.48 ns | 42.59 ns |
| Standard deviation | 101.7 ps | 300.2 ps | 650.7 ps |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
monoidal/middle
1.250 μs 1.275 1.300 1.325 1.350 1.375
mean |
1.250 μs 1.275 1.300 1.325 1.350 1.375
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 1.251 μs | 1.253 μs | 1.258 μs |
| Standard deviation | 7.627 ns | 15.88 ns | 32.56 ns |
Outlying measurements have slight (3.7%) effect on estimated standard deviation.
monoidal/back
2.40 μs 2.45 2.50 2.55 2.60 2.65 2.70
mean |
2.40 μs 2.45 2.50 2.55 2.60 2.65 2.70
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 2.426 μs | 2.430 μs | 2.441 μs |
| Standard deviation | 19.38 ns | 35.13 ns | 67.87 ns |
Outlying measurements have slight (7.5%) effect on estimated standard deviation.
tree/front
66.5 ns 67.0 67.5 68.0 68.5
mean |
66.5 ns 67.0 67.5 68.0 68.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 66.78 ns | 66.82 ns | 66.90 ns |
| Standard deviation | 189.9 ps | 281.1 ps | 425.4 ps |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
tree/middle
66 ns 67 68 69 70 71 72 73
mean |
66 ns 67 68 69 70 71 72 73
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 66.65 ns | 66.76 ns | 67.01 ns |
| Standard deviation | 408.4 ps | 792.3 ps | 1.563 ns |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
tree/back
67.5 ns 68.0 68.5 69.0 69.5
mean |
67.5 ns 68.0 68.5 69.0 69.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 67.47 ns | 67.51 ns | 67.58 ns |
| Standard deviation | 133.3 ps | 232.4 ps | 466.2 ps |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
static-tree/front
15.0 ns 15.5 16.0 16.5 17.0 17.5 18.0 18.5
mean |
15.0 ns 15.5 16.0 16.5 17.0 17.5 18.0 18.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 15.21 ns | 15.25 ns | 15.40 ns |
| Standard deviation | 81.05 ps | 340.9 ps | 787.3 ps |
Outlying measurements have moderate (15.2%) effect on estimated standard deviation.
static-tree/middle
16.000 ns 16.025 16.050 16.075 16.100 16.125
mean |
16.000 ns 16.025 16.050 16.075 16.100 16.125
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 16.04 ns | 16.05 ns | 16.05 ns |
| Standard deviation | 29.14 ps | 32.88 ps | 36.69 ps |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
static-tree/back
16.5 ns 17.0 17.5 18.0 18.5 19.0 19.5
mean |
16.5 ns 17.0 17.5 18.0 18.5 19.0 19.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 16.66 ns | 16.70 ns | 16.82 ns |
| Standard deviation | 111.7 ps | 318.5 ps | 706.7 ps |
Outlying measurements have moderate (12.3%) effect on estimated standard deviation.
static-linear/front
12.5 ns 13.0 13.5 14.0 14.5 15.0 15.5
mean |
12.5 ns 13.0 13.5 14.0 14.5 15.0 15.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 12.42 ns | 12.46 ns | 12.58 ns |
| Standard deviation | 91.40 ps | 308.5 ps | 680.3 ps |
Outlying measurements have moderate (18.1%) effect on estimated standard deviation.
static-linear/middle
52.5 ns 53.0 53.5 54.0 54.5 55.0 55.5
mean |
52.5 ns 53.0 53.5 54.0 54.5 55.0 55.5
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 52.74 ns | 52.78 ns | 52.90 ns |
| Standard deviation | 115.1 ps | 309.0 ps | 690.6 ps |
Outlying measurements have no (1.0%) effect on estimated standard deviation.
static-linear/back
102 ns 104 106 108 110
mean |
102 ns 104 106 108 110
|
| lower bound | estimate | upper bound | |
|---|---|---|---|
| Mean execution time | 101.4 ns | 101.6 ns | 102.0 ns |
| Standard deviation | 778.9 ps | 1.272 ns | 2.032 ns |
Outlying measurements have slight (1.3%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. In each section, we display two charts, each with an x axis that represents measured execution time. These charts are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. Measurements are displayed on the y axis in the order in which they occurred.
Under the charts is a small table displaying the mean and standard deviation of the measurements. We use a statistical technique called the bootstrap to provide confidence intervals on our estimates of these values. The bootstrap-derived upper and lower bounds on the mean and standard deviation let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.