As a contributor, changing a single line to test the desired version does not warrant a separate file for each type of report.
The only argument I can see for keeping the gem versions is that users are likely reporting a bug for a specific version of Rails, typically during an upgrade. As few people are probably testing their applications against Rails main branch.
Additionally, having the gem version templates means that we are testing both main and a stable release of Rails in CI, which has some benefit.
By removing the edge version templates, we also lose that coverage in CI, but I think it's something we can fix in buildkite-config to replace the line in those guides before running them later.
IMO, the cost of separate files and the confusion it will create if we want to add more templates outweighs the benefits mentioned here.
My motivation is that I want to add more report templates, like for additional adapters, and this list will grow to be unmanageable if we keep the gem versions.
This reverts commit c1b8833edbc55cdbea34b572e8759a92ed9a4660.
Revert "Lock benchmark-ips version < 2.8"
This reverts commit 62bfa50ff2db093508575b442ac50b125dd0850b.
LoadError is fixed in benchmark-ips 2.8.2.
https://github.com/evanphx/benchmark-ips/pull/101
Given that Bundler 1.10 was released back in 2015, we can assume
that people are at least on this version or a higher one so there
is no need to ask people to upgrade.
Also, given that Rails 5.0 supports Ruby 2.2+ and given that this
version come with Minitest 5.4.3 bundled, we don't need to ensure
backward compatibility with Minitest 4.
This replaces boilerplate in the “benchmark your code” section of the
contributors’ guide with an executable template. I also amended the text
to encourage best practices and codified it in the template.
For now this is only good for relatively self-contained changes that can
be inlined into a simple script. In the future, this can be expanded to
cover how to measure the difference between two commits.
The output looks like this:
```
==================================== Empty =====================================
Warming up --------------------------------------
blank? 225.963k i/100ms
fast_blank? 238.147k i/100ms
Calculating -------------------------------------
blank? 8.825M (± 6.4%) i/s - 44.063M in 5.014824s
fast_blank? 9.311M (± 6.3%) i/s - 46.439M in 5.009153s
Comparison:
fast_blank?: 9310694.8 i/s
blank?: 8824801.7 i/s - same-ish: difference falls within error
================================= Single Space =================================
Warming up --------------------------------------
blank? 56.581k i/100ms
fast_blank? 232.774k i/100ms
Calculating -------------------------------------
blank? 813.985k (±16.7%) i/s - 4.017M in 5.076576s
fast_blank? 9.547M (± 5.2%) i/s - 47.719M in 5.013204s
Comparison:
fast_blank?: 9547414.0 i/s
blank?: 813985.0 i/s - 11.73x slower
================================== Two Spaces ==================================
Warming up --------------------------------------
blank? 58.265k i/100ms
fast_blank? 244.056k i/100ms
Calculating -------------------------------------
blank? 823.343k (±16.2%) i/s - 4.020M in 5.014213s
fast_blank? 9.484M (± 4.9%) i/s - 47.347M in 5.005339s
Comparison:
fast_blank?: 9484021.6 i/s
blank?: 823343.1 i/s - 11.52x slower
=============================== Mixed Whitspaces ===============================
Warming up --------------------------------------
blank? 53.919k i/100ms
fast_blank? 237.103k i/100ms
Calculating -------------------------------------
blank? 763.435k (±16.8%) i/s - 3.720M in 5.018029s
fast_blank? 9.672M (± 5.8%) i/s - 48.369M in 5.019356s
Comparison:
fast_blank?: 9672467.2 i/s
blank?: 763435.4 i/s - 12.67x slower
=============================== Very Long String ===============================
Warming up --------------------------------------
blank? 34.037k i/100ms
fast_blank? 240.366k i/100ms
Calculating -------------------------------------
blank? 409.731k (± 8.9%) i/s - 2.042M in 5.028235s
fast_blank? 9.794M (± 4.3%) i/s - 49.035M in 5.016328s
Comparison:
fast_blank?: 9794225.2 i/s
blank?: 409731.4 i/s - 23.90x slower
```
