Because we're using a class_attribute to track all of the attribute_aliases,
each subclass was regenerating the parent class methods, causing some unexpected
deprecation messages.
Instead, we can use a class instance variable to track the attributes aliased locally
in that particular subclass. We then walk up the chain and re-define the attribute methods
if they haven't been defined yet.
Pluses cannot be used to create code blocks when the content includes a
space.
Found using a regular expression:
```bash
$ rg '#\s[^+]*\+[^+]*\s[^+]*\S\+'
```
If you have already loaded the relation when using in_batches, you
would reload each batch from the database. Since we already have the
records available, we can avoid these queries and return the records
as-if the query was run.
Despite the inconvenience of double-backslashing, the backslash was chosen
because it's a very common char to use in escaping across multiple programming
languages. A developer, without looking at documentation, may intuitively try
to use it to achieve the desired results in this scenario.
Fixes#37779
AbstractAdapter#schema_version uses migration_context.current_version by
default, but some adapters might redefine this method to use a custom
schema version number. This fix ensures that generated schema cache
files use this method and that it is also used when validating a file's
version.
This PR fixes a bug introduced in https://github.com/rails/rails/pull/48761
which leads to a `NoMethodError` when a non Active Record object passed
to `include?` or `member?` since only Active Record objects
respond to `composite_primary_key?`.
Followup: https://github.com/rails/rails/pull/48743
After careful consideration, unless users have a schema cache dump loaded
and `check_schema_cache_dump_version = false`, we have no choice but
to arbitrate between resiliency and performance.
If we define attribute methods during boot, we allow them to be shared
between forked workers, and prevent the first requests to be slower.
However, by doing so we'd trigger a connection to the datase, which
if it's unresponsive could lead to workers not being able to restart
triggering a cascading failure.
Previously Rails used to be in some sort of middle-ground where
it would define attribute methods during boot if for some reason
it was already connected to the database at this point.
But this is now deemed undesirable, as an application initializer
inadvertantly establishing the database connection woudl lead to
a readically different behavior.
`include` and `member?` delegate to `exists?` with the record's primary
key to determine if an unloaded relation contains a given record. If
the primary key is composite, `exists?` belives we are passing
where-style conditions and fails.
This commit fixes the issue by turning the record's composite primary key
into a hash of column / value pairs that `exists?` can accept.
The callback when the value is generated. When called with `on:
:initialize`, the value is generated in an `after_initialize` callback,
otherwise the value will be used in a `before_` callback. It will
default to `:create`.
This commit changes bodies of methods generated by `alias_attribute`
along with generating these methods lazily.
Previously the body of the `alias_attribute :new_title, :title` was
`def new_title; title; end`. This commit changes it to
`def new_title; attribute("title"); end`.
This allows for `alias_attribute` to be used to alias attributes named
with a reserved names like `id`:
```ruby
class Topic < ActiveRecord::Base
self.primary_key = :title
alias_attribute :id_value, :id
end
Topic.new.id_value # => 1
```
#48530 introduced a problem for the system that extends AR queries involving deterministically encrypted attributes. The problem is that this option added the same "previous scheme" for all the attributes, when the only intended ones were the "non deterministic" ones. A side effect of this is that, when encrypting query param values, it was using the wrong encryption scheme.
Fixes https://github.com/rails/rails/issues/48204#issuecomment-1623301440
Followup: https://github.com/rails/rails/pull/48716
`model.connection_pool.schema_reflection` is never falsy, so that `if`
was pointless.
Instead we more properly check if the schema cache contains that table.
I also added some more comments to explain why the initializer tries
so hard not to touch the database.
Partially Fixes#48164. Followup to #48200.
`within_new_transaction` currently discards connections if an error
is raised which may have left the connection in a transaction. This change
updates that logic to discard connections in an `ensure` block to handle
cases where an error is not raised, for instance if the thread is killed.
A big problem with the current `SchemaCache` implementation is that it holds
a reference to a connection, and uses it to lazily query the schema when the
information is missing.
This cause issues in multi-threaded contexts because all the connections
of a pool share the same schema, so they are constantly setting themselves
as the connection the schema cache should use, and you can end up in
situation where Thread#1 query the schema cache, which end up using the
connection currently attributed to Thread#2.
For a very long time this worked more or less by accident, because all
connection adapters would have an internal Monitor.
However in https://github.com/rails/rails/pull/46519 we got rid of this
synchronization, revealing the pre-existing issue. Previously it would
work™, but still wasn't ideal because of unexpected the connection
sharing between threads.
The idea here is to refactor the schema cache so that it doesn't hold a
reference onto any connection.
And instead of a `SchemaCache`, connections now have access to a
`SchemaReflection`. Now the connection that needs a schema information
is always provided as an argument, so that in case of a miss, it can be
used to populate the cache.
That should fix the database thread safety issues that were witnessed.
However note that at this stage, the underlying `SchemaCache` isn't
synchronized, so in case of a race condition, the same schema query
can be performed more than once. It could make sense to add synchronization
in a followup.
This refactoring also open the door to query the cache without a connection,
making it easier to eagerly define model attribute methods on boot without
establishing a connection to the database.
Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
Co-Authored-By: zzak <zzakscott@gmail.com>
Fix when multiple `belongs_to` maps to the same counter_cache column.
In such situation `inverse_which_updates_counter_cache` may find the
wrong relation which leads into an invalid increment of the
counter_cache.
This is done by releying on the `inverse_of` property of the relation
as well as comparing the models the association points two.
Note however that this second check doesn't work for polymorphic
associations.
Fixes#41250
Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
A developer may not always want to specify query constraints. If, for
example they would like to use a non-composite primary key in an
asscoaition, they need to use primary_key/foreign_key options instead.
Co-authored-by: Nikita Vasilevsky <nikita.vasilevsky@shopify.com>
Support assignment of belongs_to associations with query constraints.
This improves errors messages for composite key mismatches, and some
edge cases with query constraint lists.
Co-authored-by: Nikita Vasilevsky <nikita.vasilevsky@shopify.com>
Catches more bugs in CPK assocaitions by validating the shape of keys on
both ends of the association.
Co-authored-by: Nikita Vasilevsky <nikita.vasilevsky@shopify.com>
The check for `Base.connected?` was added in
4ecdf24bde.
At the time this change was made, we were calling
`Base.connection.reset_runtime` so it made sense to check if the
database was connected. Since
dd61a817de
we have stored that information in a thread instead, negating the need
to check `Base.connected?`.
The problem with checking `Base.connected?` is that in a multi-db
application this will return `false` for any model running a query not
inheriting from `Base`, leaving the `db_runtime` payload `nil`. Since
the information we need is stored in a thread rather than on the
connection directly we can remove the `Base.connected?` check.
Fixes#48687
Fix: https://github.com/rails/rails/issues/45017
Ref: https://github.com/rails/rails/pull/29333
Ref: https://github.com/ruby/timeout/pull/30
Historically only raised errors would trigger a rollback, but in Ruby `2.3`, the `timeout` library
started using `throw` to interupt execution which had the adverse effect of committing open transactions.
To solve this, in Active Record 6.1 the behavior was changed to instead rollback the transaction as it was safer
than to potentially commit an incomplete transaction.
Using `return`, `break` or `throw` inside a `transaction` block was essentially deprecated from Rails 6.1 onwards.
However with the release of `timeout 0.4.0`, `Timeout.timeout` now raises an error again, and Active Record is able
to return to its original, less surprising, behavior.
Previously, when merging persisted and in-memory records, we were using
`record[name]` to get the value of the attribute. For records with a
composite primary key, the full CPK would be returned, which was
incompatible with `#_write_attribute`: we'd attempt to assign an Array
to the `id` column, which would result in the column being set to nil.
When dealing with a record with a composite primary key, we need to
write each part of the primary key individually via `#_write_attribute`.
The `name` argument is not useful as `remove_connection` should be called
on the class that established the connection. Allowing `name` to be
passed here doesn't match how any of the other methods behave on the
connection classes. While this behavior has been around for a long time,
I'm not sure anyone is using this as it's not documented when to use
name, nor are there any tests.
If anyone calls a cypher in the console it will show the secret of the
encryptor.
By overriding the `inspect` method to only show the class name we can
avoid accidentally outputting sensitive information.
Before:
```ruby
ActiveRecord::Encryption::Cipher::Aes256Gcm.new(secret).inspect
"#<ActiveRecord::Encryption::Cipher::Aes256Gcm:0x0000000104888038 ... @secret=\"\\xAF\\bFh]LV}q\\nl\\xB2U\\xB3 ... >"
```
After:
```ruby
ActiveRecord::Encryption::Cipher::Aes256Gcm(secret).inspect
"#<ActiveRecord::Encryption::Cipher::Aes256Gcm:0x0000000104888038>"
```
Allows building of records from an association with a has_one through a
singular association with inverse. For belongs_to through associations,
linking the foreign key to the primary key model isn't needed.
For has_one, we cannot build records due to the association not being mutable.
The existing process was attempting to detect duplicates by storing added records in `Set.new`.
When a record is added to `Set.new`, the identity is calculated using `ActiveRecord::Core#hash`, but this value changes before and after saving, so duplicates were not detected before and after saving even for the same object.
This PR fixed the problem by using the `#object_id` of the record to detect duplicates.
Note that when storing a new object obtained by `ActiveRecord::Base.find` etc., duplicates are not eliminated because the `#object_id` is different. This is the same behavior as the current `ActiveRecord::Associations::CollectionProxy#<<`.
Adds support for building records in has_many through has_one
composite primary key associations.
Also updates query constraints on associations affected by rails/rails#48564.
Fixes 2 bugs in parallel testing.
The first bug was related to changes made in #45450 which meant that we were
no longer replacing the connection in parallel testing because the
config object is equal (we simply merge a new db name but the object id
of the config stays the same). This bug only manifested in mysql and
sqlite3 interestingly. It would fail on the internal metadata tables
because they were missing in the schema version check.
To fix this I introduced a `clobber: true` kwarg onto the connection
handler that allows us to bypass the functionality that won't make a new
connection if the config is the same. This is an easy way to fall back
to the old behavior from before this change. I only added `clobber`
to the `reconstruct_from_schema` call because we need to actually
replace the connection for these. It's not safe to add everywhere since
we don't always want to replace the connection.
After implementing this fix I was still seeing failures in the mysql
demo app I made due to the fact that `purge` was not re-establishing the
connection to a config that had a database defined. Neither sqlite3 or
postgresql were missing this.
I added a test for mysql2 so we don't have regressions in the future. I
think this was missed because sqlite3 only demonstrates the bug if it
was never successful on that worker and postgresql was fine.
Fixes#48547
The change in PR #43302 introduced a bug where unsaved records are not displayed in pretty_print.
The following code will not show unsaved records until this PR is merged.
```ruby
post = Post.create!
post.comments.build
pp(post.comments) #=> expected "[#<Post:0x000000014c0b48a0 ...>]", got "[]"
```
Fixed to call `#load_target` before display as well as `#inspect`.
Fixes#48398
Prepared Statements and Query Logs are incompatible features due to query logs making every query unique.
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
Currently it is a bit unclear which dirty methods can be called on
Active Record models. You have to know that methods from ActiveModel::Dirty
are included.
It also unclear if methods can be invoked in the form of
`saved_change_to_name?` unless you read the documentation of the
`saved_change_to_attribute?` method.
By adding an introduction to the module we can show which methods are
defined specifically for Active Record, and how to call them, very
similar to the ActiveModel::Dirty introduction.
Linking to ActiveModel::Dirty makes it's also easier to find methods
defined there.
This reverts commit 6264c1de762cbd7fda6eddc825d6de3027ed5e42, reversing
changes made to 6c80bcdd20eb0c71d53511b16a932bbed4f7da1f.
Reason: Still discussion about the feature. We want to make it opt-in
but we need to better understand why people would want to opt-in to
this behavior.
When eager_load is enabled and something (like the frozen_record gem)
tries to load yaml files that use `ActiveRecord::FixtureSet.identify(...)`
we'll get an uninitialized constant error.
Move requires for constants in the ActiveRecord::FixtureSet namespace
to below the creation of the class to avoid circular requires.
See ActiveSupport::Deprecation for a similar instance of this:
7dd67dc285/activesupport/lib/active_support/deprecation.rb (L35-L54)
Support primary key dirty tracking when ID is a composite
primary key with an id column.
Introduces `ActiveRecord::AttributeMethods::Query#_query_attribute`
to bypass id reader.
* Make sure active record encryption configuration happens after initializers have run
Co-authored-by: Cadu Ribeiro <mail@cadu.dev>
* Add a new option to support previous data encrypted non-deterministically with a hash digest of SHA1
There is currently a problem with Active Record encryption for users updating from 7.0 to 7.1 Before
#44873, data encrypted with non-deterministic encryption was always using SHA-1. The reason is that
`ActiveSupport::KeyGenerator.hash_digest_class` is set in an after_initialize block in the railtie config,
but encryption config was running before that, so it was effectively using the previous default SHA1. That
means that existing users are using SHA256 for non deterministic encryption, and SHA1 for deterministic
encryption.
This adds a new option `use_sha1_digest_for_non_deterministic_data` that
users can enable to support for SHA1 and SHA256 when decrypting existing data.
* Set a default value of true for `support_sha1_for_non_deterministic_encryption` and proper initializer values.
We want to enable the flag existing versions (< 7.1), and we want it to be false moving by
default moving forward.
* Make sure the system to auto-filter params supports different initialization orders
This reworks the system to auto-filter params so that it works when encrypted
attributes are declared before the encryption configuration logic runs.
Co-authored-by: Cadu Ribeiro <mail@cadu.dev>
---------
Co-authored-by: Cadu Ribeiro <mail@cadu.dev>
Composite primary key records need to conditionally unset multiple
column attributes from the associated record in order to properly
support dependent: nullify.
This reverts commit 8b36095881435e996db16604c52737e144b6bff3, reversing
changes made to e05245db878077097d666f7667c0f9057f767583.
Railties tests have been failing since this change. The issue is that
calling `primary_key` as the model is loaded requires either a
connection to the database or a populated schema cache. This becomes an
issue when an app loads models that do not have underlying tables, as
shown in the failing Railties tests.
When eager loading an app using `rails/all`,
`ActionMailbox::InboundEmail` will be loaded whether or not `rails g
action_mailbox:install` has been run. This means the `primary_key` for
`InboundEmail` will not be in the schema cache and a database connection
will be required to boot the app.
* Improve the spacing of various paragraphs
* Link to AS::TestCase.fixture_paths= API
* Add a note about needing to require "rails/test_help" to get fixture_paths
* Expanded briefly on YAML default map unordering property
* Improve heading weight consistency
* Unlink several RDoc autolinked things
This removes the deprecated .fixture_path and .fixture_path= methods
from public documentation, and creates a class method for the getter and
setters.
Interestingly because these methods were always `class_attribute` based,
which RDoc doesn't parse, they weren't technically "public"
documentation.
Fixes https://github.com/rails/rails/issues/48524
The test case in the issue breaks because `value.respond_to?(:id)` returns true [here](51f2e2f80b/activerecord/lib/active_record/relation/predicate_builder.rb (L58)). This effectively adds a default scope to queries where it shouldn't.
There might be a way to fix this in Active Record but I'd be surprised if nothing else breaks from defining `id` instance and class methods. I think it is simpler to not allow it as a value since it really should be treated as a reserved method.
Minor fixes to the deprecation messages around TestFixtures.fixture_path,
TestFixtures.fixture_path=, and TestFixtures#fixture_path to clarify
which method is deprecated and which should be used instead.
If a has_one association uses a composite primary key, and part of the
composite primary key changes on the owner, these changes need to be
reflected on the belonging object's foreign key.
This was not working previously, because `#_record_changed?` was not
equipped to handle composite primary key associations, so we were not
recognizing that the belonging object's foreign key needed to be updated
when the owner's primary key changed.
Checking whether a record has a foreign key for a composite primary key
association requires us to check whether all parts of the foreign key
are present. Otherwise, the inverse association will not be set.
This commit stops issuing the
"Active Record does not support composite primary key" warning
and allows `ActiveRecord::Base#primary_key` to be derived as an `Array`
`#scope_for_create` for singular associations removes the primary key
from the scope so that we don't assign PK columns when building an
association.
However, removing the primary key from the scope doesn't currently
handle composite primary keys. This commit fixes that.
When find_each/find_in_batches/in_batches are performed on a table with composite primary keys, ascending or descending order can be selected for each key.
```ruby
Person.find_each(order: [:desc, :asc]) do |person|
person.party_all_night!
end
```
This commit handles destroying CPK associations when autosave is set
and the parent association is marked for destruction. It does so
by ensuring that all parts of the foreign key (the parent's CPK)
are set to nil before destroying the parent record.
Anytime an exception is raised from an adapter we now provide a
`connection_pool` along for the application to further debug what went
wrong. This is an important feature when running a multi-database Rails
application.
We chose to provide the `connection_pool` as it has relevant context
like connection, role and shard. We wanted to avoid providing the
`connection` directly as it might accidentally be used after it's
returned to the pool and been handed to another thread.
The `ConnectionAdapters::PoolConfig` would also have been a reasonable
option except it's `:nodoc:`.
While we had hoped to turn prepared statements on for Rails 7.2, the bug
that's preventing us from doing that is still present. See #43005.
Until this bug is fixed we should not be encouraging applications
running mysql to change the `prepared_statements` in the config to
`true`. In addition to this bug being present, Trilogy does not yet
support `prepared_statements` (although work is in progress).
It will be better to implement this deprecation when mysql2 and trilogy
can both handle `prepared_statements` without major bugs.
This commit extends Active Record creation logic to allow for a database
auto-populated attributes to be assigned on object creation.
Given a `Post` model represented by the following schema:
```ruby
create_table :posts, id: false do |t|
t.integer :sequential_number, auto_increment: true
t.string :title, primary_key: true
t.string :ruby_on_rails, default: -> { "concat('R', 'o', 'R')" }
end
```
where `title` is being used as a primary key, the table has an
integer `sequential_number` column populated by a sequence and
`ruby_on_rails` column has a default function - creation of
`Post` records should populate the `sequential_number` and
`ruby_on_rails` attributes:
```ruby
new_post = Post.create(title: 'My first post')
new_post.sequential_number # => 1
new_post.ruby_on_rails # => 'RoR'
```
* At this moment MySQL and SQLite adapters are limited to only one
column being populated and the column must be the `auto_increment`
while PostgreSQL adapter supports any number of auto-populated
columns through `RETURNING` statement.
If an application is using sharding, they may not want to use `default`
as the `default_shard`. Unfortunately Rails expects there to be a shard
named `default` for certain actions internally. This leads to some
errors on boot and the application is left manually setting
`default_shard=` in their model or updating their shards in
`connects_to` to name `shard_one` to `default`. Neither are a great
solution, especially if Rails can do this for you. Changes to Active
Record are:
* Simplify `connects_to` by merging `database` into `shards` kwarg so we
can do a single loop through provided options.
* Set the `self.default_shard` to the first keys in the shards kwarg.
* Add a test for this behavior
* Update existing test that wasn't testing this to use `default`. I
could have left this test but it really messes with connections in the
other tests and since this isn't testing shard behavior specifically, I
updated it to use `default` as the default shard name.
This is a slight change in behavior from existing applications but
arguably this is fixing a bug because without this an application won't
boot. I originally thought that this would require a huge refactoring to
fix but realized that it makes a lot of sense to take the first shard as
they default. They should all have the same schema so we can assume it's
fine to take the first one.
Fixes: #45390
This removes the following previously internal API:
* AR::Associations::Preloader
* AR::Associations::Preloader::Association
* AR::Associations::Preloader::Association::LoaderQuery
* AR::Associations::Preloader::Association::LoaderRecords
Because there was a `:nodoc:` on both Preloader and Preloader::Association,
the assumption is that everything underthat is also private.
However, RDoc did not do the right thing.
* Fix RDoc autolinks where it makes sense
* Fixed-width font for database_tasks option
* Remove duplicate "Also aliased as blank?"
* Link to AR::DatabaseConfigurations from config classes
* Link to AR::Base.configurations from DatabaseConfigurations
* Move options doc to initializer
When `destroy` is called twice on a persisted record inside a
transaction, the after_commit callbacks fail to run. The second
time `destroy` is called, `@_trigger_destroy_callback` is set to false
because the record has already been deleted, even though the
transaction hasn't completed yet, and the after_commit callbacks haven't
had a chance to run from the first `destroy`.
SelectManager#with currently accepts As and TableAlias nodes.
Neither of these support materialization hints for the query
planner. Both Postgres and SQLite support such hints.
This commit adds a Cte node that does support materialization
hints. It continues to support As and TableAlias nodes by
translating them into Cte nodes.
Introduces a new engine initializer, `add_fixture_paths`, which
automatically adds a `test/fixtures` folder under the engine root to
`fixture_paths` if such a folder exists.
The recent [refactor][1] to the MySQL DatabaseStatements classes renamed
Mysql::DatabaseStatements to Mysql2::DatabaseStatements. That commit
also updated most of the references to the top level Mysql2 to be
explicit since Ruby will now assume they refer to
ActiveRecord::ConnectionAdapters::Mysql2. However, Mysql2::Error was not
updated, and that rescue will currently raise:
```
NameError: uninitialized constant ActiveRecord::ConnectionAdapters::Mysql2::Error
```
if the execute raises any error.
To fix this, Mysql2 must be changed to explicitly refer to the top level
namespace.
[1]: 93b5fc1f95e04b6a51f3e8dd2c885ba9ddd139a6
This is basically a multi-db aware version of `ActiveRecord::Base.connection.disconnect!`.
It also avoid connecting to the database if we weren't already.
This can be useful to reset state after `establish_connection` has been used.
