This code block was removed in a previous PR, but it had
a side effect of causing certian timestamp fields to be
changed from `datetime(6)` to `datetime` when performing a schema
migration in the 6.1 version of the migration class. This is due
to `options[:precision]` being set to `nil` and therefore
not being set to the default of 6 later on in the code path.
Instead of hosting the connection logic in the command object, the
database adapter should be responsible for connecting to a console session.
This patch moves #find_cmd_and_exec to the adapter and exposes a new API to
lookup the adapter class without instantiating it.
Co-authored-by: Paarth Madan <paarth.madan@shopify.com>
Most model attribute types try to cast a given value before serializing
it. This allows uncast values to be passed to finder methods and still
be serialized appropriately. However, when persisting a model, this
cast is unnecessary because the value will already have been cast by
`ActiveModel::Attribute#value`.
To eliminate the overhead of a 2nd cast, this commit introduces a
`ActiveModel::Type::SerializeCastValue` module. Types can include this
module, and their `serialize_cast_value` method will be called instead
of `serialize` when serializing an already-cast value.
To preserve existing behavior of any user types that subclass Rails'
types, `serialize_after_cast` will only be called if the type itself
(not a superclass) includes `ActiveModel::Type::SerializeCastValue`.
This also applies to type decorators implemented via `DelegateClass`.
Benchmark script:
```ruby
require "active_model"
require "benchmark/ips"
class ActiveModel::Attribute
alias baseline_value_for_database value_for_database
end
VALUES = {
my_big_integer: "123456",
my_boolean: "true",
my_date: "1999-12-31",
my_datetime: "1999-12-31 12:34:56 UTC",
my_decimal: "123.456",
my_float: "123.456",
my_immutable_string: "abcdef",
my_integer: "123456",
my_string: "abcdef",
my_time: "1999-12-31T12:34:56.789-10:00",
}
TYPES = VALUES.to_h { |name, value| [name, name.to_s.delete_prefix("my_").to_sym] }
class MyModel
include ActiveModel::API
include ActiveModel::Attributes
TYPES.each do |name, type|
attribute name, type
end
end
TYPES.each do |name, type|
$attribute_set ||= MyModel.new(VALUES).instance_variable_get(:@attributes)
attribute = $attribute_set[name.to_s]
puts "=" * 72
Benchmark.ips do |x|
x.report("#{type} before") { attribute.baseline_value_for_database }
x.report("#{type} after") { attribute.value_for_database }
x.compare!
end
end
```
Benchmark results:
```
========================================================================
Warming up --------------------------------------
big_integer before 100.417k i/100ms
big_integer after 260.375k i/100ms
Calculating -------------------------------------
big_integer before 1.005M (± 1.0%) i/s - 5.121M in 5.096498s
big_integer after 2.630M (± 1.0%) i/s - 13.279M in 5.050387s
Comparison:
big_integer after: 2629583.6 i/s
big_integer before: 1004961.2 i/s - 2.62x (± 0.00) slower
========================================================================
Warming up --------------------------------------
boolean before 230.663k i/100ms
boolean after 299.262k i/100ms
Calculating -------------------------------------
boolean before 2.313M (± 0.7%) i/s - 11.764M in 5.085925s
boolean after 3.037M (± 0.6%) i/s - 15.262M in 5.026280s
Comparison:
boolean after: 3036640.8 i/s
boolean before: 2313127.8 i/s - 1.31x (± 0.00) slower
========================================================================
Warming up --------------------------------------
date before 148.821k i/100ms
date after 298.939k i/100ms
Calculating -------------------------------------
date before 1.486M (± 0.6%) i/s - 7.441M in 5.006091s
date after 2.963M (± 0.8%) i/s - 14.947M in 5.045651s
Comparison:
date after: 2962535.3 i/s
date before: 1486459.4 i/s - 1.99x (± 0.00) slower
========================================================================
Warming up --------------------------------------
datetime before 92.818k i/100ms
datetime after 136.710k i/100ms
Calculating -------------------------------------
datetime before 920.236k (± 0.6%) i/s - 4.641M in 5.043355s
datetime after 1.366M (± 0.8%) i/s - 6.836M in 5.003307s
Comparison:
datetime after: 1366294.1 i/s
datetime before: 920236.1 i/s - 1.48x (± 0.00) slower
========================================================================
Warming up --------------------------------------
decimal before 50.194k i/100ms
decimal after 298.674k i/100ms
Calculating -------------------------------------
decimal before 494.141k (± 1.4%) i/s - 2.510M in 5.079995s
decimal after 3.015M (± 1.0%) i/s - 15.232M in 5.052929s
Comparison:
decimal after: 3014901.3 i/s
decimal before: 494141.2 i/s - 6.10x (± 0.00) slower
========================================================================
Warming up --------------------------------------
float before 217.547k i/100ms
float after 298.106k i/100ms
Calculating -------------------------------------
float before 2.157M (± 0.8%) i/s - 10.877M in 5.043292s
float after 2.991M (± 0.6%) i/s - 15.203M in 5.082806s
Comparison:
float after: 2991262.8 i/s
float before: 2156940.2 i/s - 1.39x (± 0.00) slower
========================================================================
Warming up --------------------------------------
immutable_string before
163.287k i/100ms
immutable_string after
298.245k i/100ms
Calculating -------------------------------------
immutable_string before
1.652M (± 0.7%) i/s - 8.328M in 5.040855s
immutable_string after
3.022M (± 0.9%) i/s - 15.210M in 5.033151s
Comparison:
immutable_string after: 3022313.3 i/s
immutable_string before: 1652121.7 i/s - 1.83x (± 0.00) slower
========================================================================
Warming up --------------------------------------
integer before 115.383k i/100ms
integer after 159.702k i/100ms
Calculating -------------------------------------
integer before 1.132M (± 0.8%) i/s - 5.769M in 5.095041s
integer after 1.641M (± 0.5%) i/s - 8.305M in 5.061893s
Comparison:
integer after: 1640635.8 i/s
integer before: 1132381.5 i/s - 1.45x (± 0.00) slower
========================================================================
Warming up --------------------------------------
string before 163.061k i/100ms
string after 299.885k i/100ms
Calculating -------------------------------------
string before 1.659M (± 0.7%) i/s - 8.316M in 5.012609s
string after 2.999M (± 0.6%) i/s - 15.294M in 5.100008s
Comparison:
string after: 2998956.0 i/s
string before: 1659115.6 i/s - 1.81x (± 0.00) slower
========================================================================
Warming up --------------------------------------
time before 98.250k i/100ms
time after 133.463k i/100ms
Calculating -------------------------------------
time before 987.771k (± 0.7%) i/s - 5.011M in 5.073023s
time after 1.330M (± 0.5%) i/s - 6.673M in 5.016573s
Comparison:
time after: 1330253.9 i/s
time before: 987771.0 i/s - 1.35x (± 0.00) slower
```
Prior to this change
t.virtual :column_name, type: :datetime
would erroneously produce the same result as
t.virtual :column_name, type: :datetime, precision: nil
This is because the code path for `virtual` skipped the default lookup:
- `t.datetime` is delegated to the `column` method
- `column` sees `type == :datetime` and sets a default `precision` is none was given
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `t.virtual` is delegated to the `column` method
- `column` sees `type == :virtual`, not `:datetime`, so skips the default lookup
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :virtual`, so sets `type = options[:type]`
By moving the default lookup, we get consistent code paths:
- `t.datetime` is delegated to the `column` method
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :datetime` and sets a default `precision` is none was given
- `t.virtual` is delegated to the `column` method
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :virtual`, so sets `type = options[:type]`
- `new_column_definition` sees `type == :datetime` and sets a default `precision` is none was given
Active Record erroneously uses a precision of `nil` instead of the default
precision, when adding a virtual datetime column.
This tests that behavior ahead of fixing it.
Problem:
Though, the `MessageVerifier` that powers `signed_id` supports both
`expires_in` and `expires_at`, `signed_id` only supports `expires_in`.
Because of this, generating signed_id that expires at a certain time is
somewhat tedious. Imagine issuing a coupon that is valid only for a
day.
Solution:
Add `expires_at` option to `signed_id` to generate signed ids that
expire at the given time.
Building on the work done in #44576 and #44591, we extend the logic that automatically
reconnects broken db connections to take into account a timeout limit. This ensures
that retries + reconnects are slow-query aware, and that we don't retry queries
if a given amount of time has already passed since the query was first tried.
This value will default to 5 seconds, but can be adjusted via the `connection_retry_timeout`
config.
Even if tests are "skipped" by `Minitest::Assertions#skip`
`ensure` is executed because tests methods are Ruby methods.
Adding conditions outside of the test methods, entire test methods are
executed only when they are necessary.
This PR calls `ActiveRecord::Base` directly on `establish_connection`
and `connection` rather than delegating. While this doesn't have much
effect right now, I'm working on moving the database tasks away from
their reliance on Base and eventually we'll need to pass a class through
to these adapter tasks. This change prepares these adapter tasks for
that change.
This avoids problems when complex data structures are mutated _after_
being handed to ActiveRecord for processing. For example false hits in
the query cache.
Fixes#46044
In https://github.com/rails/rails/pull/45796 I overlooked that
`ActiveRecord::LogSubscriber#sql` wasn't only logging the SQL query
but was also responsible for collecting the `:db_runtime` metric.
Ultimately I think it is cleaner to move this concern to `RuntimeRegistry`.
Since changing internal metadata to no longer inherit from Base
in #45982 I accidentally changed the behavior when a key's value is the
same. Prior to this change the record would not be updated if the
environment key was found an the value had not changed. So instead of
just checking whether we have an entry here we also need to check if the
value should actually be updated, otherwise we should return the entry.
Following #45924 sometimes these calls will throw deprecation warnings
depending on the test order and whether there are connections left
behind from other tests. I attempted to prevent leaking connections but
when using fixtures it's pretty hard to avoid because the
`teardown_fixture_connections` will often put connections back in place.
It is eaiser to ensure these calls opt into the new behavior to avoid
deprecation warnings.
Followup to #45908 to match the same behavior as SchemaMigration
Previously, InternalMetadata inherited from ActiveRecord::Base. This is
problematic for multiple databases and resulted in building the code in
AbstractAdapter that was previously there. Rather than hacking around
the fact that InternalMetadata inherits from Base, this PR makes
InternalMetadata an independent object. Then each connection can get it's
own InternalMetadata object. This change required defining the methods
that InternalMetadata was depending on ActiveRecord::Base for (ex
create!). I reimplemented only the methods called by the framework as
this class is no-doc's so it doesn't need to implement anything beyond
that. Now each connection gets it's own InternalMetadata object which
stores the connection.
This change also required adding a NullInternalMetadata class for cases
when we don't have a connection yet but still need to copy migrations
from the MigrationContext. Ultimately I think this is a little weird -
we need to do so much work to pick up a set of files? Maybe something to
explore in the future.
Aside from removing the hack we added back in #36439 this change will
enable my work to stop clobbering and depending directly on
Base.connection in the rake tasks. While working on this I discovered
that we always have a ActiveRecord::InternalMetadata because the
connection is always on Base in the rake tasks. This will free us up
to do less hacky stuff in the migrations and tasks.
Both schema migration and internal metadata are blockers to removing
`Base.connection` and `Base.establish_connection` from rake tasks, work
that is required to drop the reliance on `Base.connection` which will
enable more robust (and correct) sharding behavior in Rails..
Previously, only bold text was supported via a positional argument.
This allows for bold, italic, and underline options to be specified
for colored logs.
```ruby
info color("Hello world!", :red, bold: true, underline: true)
```
Previously, SchemaMigration inherited from ActiveRecord::Base. This is
problematic for multiple databases and resulted in building the code in
AbstractAdapter that was previously there. Rather than hacking around
the fact that SchemaMigration inherits from Base, this PR makes
SchemaMigration an independent object. Then each connection can get it's
own SchemaMigration object. This change required defining the methods
that SchemaMigration was depending on ActiveRecord::Base for (ex
create!). I reimplemented only the methods called by the framework as
this class is no-doc's so it doesn't need to implement anything beyond
that. Now each connection gets it's own SchemaMigration object which
stores the connection. I also decided to update the method names (create
-> create_version, delete_by -> delete_version, delete_all ->
delete_all_versions) to be more explicit.
This change also required adding a NullSchemaMigraiton class for cases
when we don't have a connection yet but still need to copy migrations
from the MigrationContext. Ultimately I think this is a little weird -
we need to do so much work to pick up a set of files? Maybe something to
explore in the future.
Aside from removing the hack we added back in #36439 this change will
enable my work to stop clobbering and depending directly on
Base.connection in the rake tasks. While working on this I discovered
that we always have a `ActiveRecord::SchemaMigration` because the
connection is always on `Base` in the rake tasks. This will free us up
to do less hacky stuff in the migrations and tasks.
This PR enables `Minitest/AssertRaisesWithRegexpArgument` cop
and it suppresses the new warning below.
```console
% bundle exec rubocop
(snip)
Offenses:
activerecord/test/cases/adapters/sqlite3/sqlite3_adapter_test.rb:111:9: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises(ActiveRecord::StatementInvalid, /TypeError/) do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
activerecord/test/cases/adapters/sqlite3/sqlite3_adapter_test.rb:628:9: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises(ActiveRecord::StatementInvalid, /SQLite3::ReadOnlyException/) do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:307:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:311:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:336:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:361:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
The last argument of `assert_raises` is a custom message string to help explain failures.
So, it's not the argument that `exception.message` is compared to.
`assert_raises` returns a raised exception and can be used to match against a regular expression.
And it updates the dependency version of rubocop-minitest in the Gemfile.
Because `Minitest/AssertRaisesWithRegexpArgument` cop was introduced in minitest 0.22.
https://github.com/rubocop/rubocop-minitest/releases/tag/v0.22.0
Following on #45924 I realized that `all_connection_pools` and
`connection_pool_list` don't make much sense as separate methods and
should follow the same deprecation as the other methods on the handler
here. So this PR deprecates `all_connection_pools` in favor of
`connection_pool_list` with an explicit argument of the role or `:all`.
Passing `nil` will throw a deprecation warning to get applications to
be explicit about behavior they expect.
Previously when I implemented multiple database roles in Rails there
were two handlers so it made sense for the methods
`active_connections?`, `clear_active_connections!`,
`clear_reloadable_connections!`, `clear_all_connections!`, and
`flush_idle_connections!` to only operate on the current (or passed)
role and not all pools regardless of role. When I removed this and moved
all the pools to the handler maintained by a pool manager, I left these
methods as-is to preserve the original behavior.
This made sense because I thought these methods were only called by
applications and not called by Rails. I realized yesterday that some of
these methods (`flush_idle_connections!`, `clear_active_connections!`,
and `clear_reloadable_connections!` are all called on boot by the
Active Record railtie.
Unfortunately this means that applications using multiple databases
aren't getting connections flushed or cleared on boot for any connection
but the writing ones.
The change here continues existing behavior if a role like reading is
passed in directly. Otherwise if the role is `nil` (which is the new
default` we fall back to all connections and issue a deprecation
warning. This will be the new default behavior in the future. In order
to easily allow turning off the deprecation warning I've added an `:all`
argument that will use all pools but no warning. The deprecation warning
will only fire if there is more than one role in the pool manager,
otherwise we assume prior behavior.
This bug would have only affected applications with more than one role
and only when these methods are called outside the context of a
`connected_to` block. These methods no longer consider the set
`current_role` and applications need to be explicit if they don't want
these methods to operate on all pools.
