This allows users to use write protection if they want, but no longer treats it as a catch all. Write protection is not accurate enough to classify all cases correctly. Instead users should configure their replicas to not allow writes and rely on the database erroring if the query is not allowed.
Co-authored-by: Alex Ghiculescu <alexghiculescu@gmail.com>
This allows to have a different behavior on update and create.
For instance it might be desirable to disable partial inserts
to protect against a concurrent migration removing the default
value of a column, which would cause processes with an outdated
schema cache to fail on insert.
Ruby master ships with Psych 4.0.0 which makes `YAML.load`
defaults to safe mode (https://github.com/ruby/psych/pull/487).
However since these YAML files are trustworthy sources
we can parse them with `unsafe_load`.
`cattr_accessor` rely on class variables which has terrible
performance on long ancestor chains. See https://bugs.ruby-lang.org/issues/17763
for a detailed description of the problem.
In comparison `class_attribute` on `ActiveRecord::Base` is almost 7x
faster:
```
Calculating -------------------------------------
logger 1.700M (± 0.9%) i/s - 8.667M in 5.097595s
clogger 11.556M (± 0.9%) i/s - 58.806M in 5.089282s
Comparison:
clogger: 11555754.2 i/s
logger: 1700280.4 i/s - 6.80x (± 0.00) slower
```
This is because `ActiveRecord::Base.ancestors.size == 62`.
Most databases order tables with the `NULL` value first, having it before
all other data values. Postgres has `NULLS` last.
Fortunately, ANSI SQL has an option to allow the database to specify where NULLS
come out in this sort order
ORDER BY column ASC NULLS FIRST
MS SQL, SQLite, Oracle, and Postgres all follow this syntax. Unfortunately, MySql
does not.
Before:
PostgreSQL: both `.nulls_first()` and `.nulls_last()` work as designed.
Others: both raise a runtime error.
After:
MySQL: `.nulls_first()` works as designed.
MySQL: `.nulls_last()` raises a runtime error
Others: both work as designed
#one? and #many? generate a query like `SELECT COUNT(*) FROM posts` under the hood, and then compare if the result is equal (or greater) to 1. That count operation can be really slow for large tables or complex conditions, but there's no need to count all the records in these cases. It's much faster just by adding a limit, like `SELECT COUNT(*) FROM posts LIMIT 2`
Logs a warning message when running SQlite in production.
The warning can be disabled by setting `config.active_record.sqlite3_production_warning=false`.
Closes#34715
This is similar to the `disable_joins` option on `has_many :through`
associations applied to `has_one :through` associations. When
`disable_joins` is set Rails will create 2 or more queries to get
associations instead of generating a join.
```ruby
class Person
belongs_to :dog
has_one :veterinarian, through: :dog, disable_joins: true
end
```
Then instead of generating join SQL, two queries are used for `@person.veterinarian`:
```
SELECT "dogs"."id" FROM "dogs" WHERE "dogs"."person_id" = ? [["person_id", 1]]
SELECT "veterinarians".* FROM "veterinarians" WHERE "veterinarians"."dog_id" = ? [["dog_id", 1]]
```
Co-authored-by: Eileen M. Uchitelle <eileencodes@gmail.com>
In https://github.com/rails/rails/issues/21126 it was suggested to make "timestamp with time zone" the default type for datetime columns in PostgreSQL. This is in line with PostgreSQL [best practices](https://wiki.postgresql.org/wiki/Don't_Do_This#Don.27t_use_timestamp_.28without_time_zone.29). This PR lays some groundwork for that.
This PR adds a configuration option, `ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.datetime_type`. The default is `:timestamp` which preserves current Rails behavior of using "timestamp without time zone" when you do `t.datetime` in a migration. If you change it to `:timestamptz`, you'll get "timestamp with time zone" columns instead.
If you change this setting in an existing app, you should immediately call `bin/rails db:migrate` to ensure your `schema.rb` file remains correct. If you do so, then existing columns will not be impacted, so for example if you have an app with a mixture of both types of columns, and you change the config, schema dumps will continue to output the correct types.
This PR also adds two new types that can be used in migrations: `t.timestamp` and `t.timestamptz`.
```ruby
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.datetime_type = :timestamp # default value is :timestamp
create_table("foo1") do |t|
t.datetime :default_format # "timestamp without time zone"
t.timestamp :without_time_zone # "timestamp without time zone"
t.timestamptz :with_time_zone # "timestamp with time zone"
end
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.datetime_type = :timestamptz
create_table("foo2") do |t|
t.datetime :default_format # "timestamp with time zone" <-- note how this has changed!
t.timestamp :without_time_zone # "timestamp without time zone"
t.timestamptz :with_time_zone # "timestamp with time zone"
end
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:my_custom_type] = { name: "custom_datetime_format_i_invented" }
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.datetime_type = :my_custom_type
create_table("foo3") do |t|
t.datetime :default_format # "custom_datetime_format_i_invented"
t.timestamp :without_time_zone # "timestamp without time zone"
t.timestamptz :with_time_zone # "timestamp with time zone"
end
```
**Notes**
- This PR doesn't change the default `datetime` format. The default is still "timestamp without time zone". A future PR could do that, but there was enough code here just getting the config option right.
- See also https://github.com/rails/rails/pull/41395 which set some groundwork (and added some tests) for this.
- This reverts some of https://github.com/rails/rails/pull/15184. https://github.com/rails/rails/pull/15184 alluded to issues in XML serialization, but I couldn't find any related tests that this broke.
- Case
- CurrentRow
- Distinct
- InfixOperation
- NotRegexp
- Regexp
- UnaryOperation
- With
Also add test coverage for When and Else for which support was just
added in a previous commit in this changeset.
Also use assert_edge test helper for Arel::Visitors::Dot.
Both MySQL and PostgreSQL support dropping multiple columns in a single
SQL statement.
For tables that are very large, dropping a column can be time consuming.
When multiple columns are involved, dropping them all at once can reduce
the total overhead when compared to dropping each column individually.
SQLite3 does not support this feature so its adapter overrides the
remove_columns method to workaround SQLite3's ALTER TABLE limitations.
The already written method remove_columns_for_alter creates the ALTER
TABLE SQL fragments to execute.
The remove_timestamps method also happens to drop multiple columns, so
it now uses the updated remove_columns to take advantage of the
optimization.
In a multiple database application, associations can't join across
databases. When set, this option tells Rails to make 2 or more queries
rather than using joins for associations.
Set the option on a has many through association:
```ruby
class Dog
has_many :treats, through: :humans, disable_joins: true
has_many :humans
end
```
Then instead of generating join SQL, two queries are used for `@dog.treats`:
```
SELECT "humans"."id" FROM "humans" WHERE "humans"."dog_id" = ? [["dog_id", 1]]
SELECT "treats".* FROM "treats" WHERE "treats"."human_id" IN (?, ?, ?) [["human_id", 1], ["human_id", 2], ["human_id", 3]]
```
This code is extracted from a gem we use internally at GitHub which
means the implementation here is used in production daily and isn't
experimental.
I often get the question "why can't Rails do this automatically" so I
figured I'd include the answer in the commit. Rails can't do this
automatically because associations are lazily loaded. `dog.treats` needs
to load `Dog`, then `Human` and then `Treats`. When `dog.treats` is
called Rails pre-generates the SQL that will be run and puts that
information into a reflection object. Because the SQL parts are pre-generated,
as soon as `dog.treats` is loaded it's too late to skip a join. The join
is already available on the object and that join is what's run to load
`treats` from `dog` through `humans`. I think the only way to avoid setting
an option on the association is to rewrite how and when the SQL is
generated for associations which is a large undertaking. Basically the
way that Active Record associations are designed, it is currently
impossible to have Rails figure out to not join (loading the association
will cause the join to occur, and that join will raise an error if the
models don't live in the same db).
The original implementation was written by me and Aaron. Lee helped port
over tests, and I refactored the extraction to better match Rails style.
Co-authored-by: Lee Quarella <leequarella@gmail.com>
Co-authored-by: Aaron Patterson <aaron@rubyonrails.org>
When saving a record, autosave adds callbacks to save its' associations.
Since the associations can have similar callbacks for the inverse,
endless loops could occur.
To prevent these endless loops, the callbacks for `has_many` and
`belongs_to` are defined as methods that only execute once.
This is implemented in the `define_non_cyclic_method` method.
However, this wasn't used for the `has_one` callbacks.
While `has_one` association callbacks didn't result in endless loops,
they could execute multiple times.
For example for a bidirectional `has_one` with autosave enabled,
the `save_has_one_association` gets called twice:
class Pirate < ActiveRecord::Base
has_one :ship, autosave: true
def save_has_one_association(reflection)
@count ||= 0
@count += 1 if reflection.name == :ship
super
end
end
class Ship < ActiveRecord::Base
belongs_to :pirate, autosave: true
end
pirate = Pirate.new(catchphrase: "Aye")
pirate.build_ship(name: "Nights Dirty Lightning")
pirate.save!
# this returns 2 instead of 1.
assert_equal 1, pirate.instance_variable_get(:@count)
This commit changes `has_one` autosave callbacks to be non-cyclic as
well. By doing this the autosave callback are made more consistent for
all 3 cases: `has_many`, `has_one` and `belongs_to`.
Calling save on a record with cyclic autosave callbacks, can call other
callbacks and hooks multiple times. This can lead to unexpected
behaviour.
For example `save` gets called twice on Post in the following example.
This results in `changes_applied` getting called twice.
class Post < ApplicationRecord
belongs_to :postable, polymorphic: true, inverse_of: :post
end
class Message < ApplicationRecord
has_one :post, as: :postable
end
post = Post.create!(postable: Message.new(subject: "Hello, world!"))
# the following would return false when true is expected
post.id_previously_changed?
`save` gets called twice because Post autosaves Message, which
autosaves Post again.
Instead of calling `changes_applied` everytime `save` is called,
we can skip it if it has already been called once in the current saving
cycle. This requires us to track the `@_saving` state of a record.
if `@_saving` is true we know we the record is being saved.
To track if a method has already been called we reuse the
@_already_called hash that is already used for this purpose.
This deprecates `legacy_connection_handling` via the
`connection_handlers` setter. This is called from the ActiveRecord
Railtie on boot and since most applications don't set this themselves
this will prevent the deprecation from being raised multiple times for a
test run or in development.
I've also updated the guides to include a migration path for
applications using the deprecated methods. The majority of applications
won't need to make any changes.
Commit a1a5d37749964b1e1a23914ef13da327403e34cb doesn't properly set the
foreign key on associations. Instead of trying to patch the change
revert it for now, so we can investigate a better solution.
This reverts commit a1a5d37749964b1e1a23914ef13da327403e34cb
It also reverts the follow-up commits:
* Revert "Rename internal `@saving` state to `@_saving`"
This reverts commit 2eb5458978f3f993ccc414b321b35fb1aef1efd2
* Revert "Add `_` prefix for the internal methods"
This reverts commit 12c0bec15275fa458bc0ddd6b57f7a0ae7881bd5.
* Revert "protected :can_save?"
This reverts commit 8cd3b657f8795aedb9bc97e725642cbd04dc09b8.
* Revert "Exclude #saving? from API docs"
This reverts commit 35d3923ea0c51b3a628e913e3b35f85124de8ac8.
Add an optional mode argument to
Core#strict_loading! to support n_plus_one_only
mode. Currently, when we turn on strict_loading
for a single record, it will raise even if we are
loading an association that is relatively safe to
lazy load like a belongs_to. This can be helpful
for some use cases, but prevents us from using
strict_loading to identify only problematic
instances of lazy loading.
The n_plus_one_only argument allows us to turn
strict_loading on for a single record, and only
raise when a N+1 query is likely to be executed.
When loading associations on a single record,
this only happens when we go through a has_many
association type. Note that the has_many
association itself is not problematic as it only
requires one query. We do this by turning
strict_loading on for each record that is loaded
through the has_many. This ensures that any
subsequent lazy loads on these records will raise
a StrictLoadingViolationError.
For example, where a developer belongs_to a ship
and each ship has_many parts, we expect the
following behaviour:
developer.strict_loading!(mode: :n_plus_one_only)
# Do not raise when a belongs_to association
# (:ship) loads its has_many association (:parts)
assert_nothing_raised do
developer.ship.parts.to_a
end
refute developer.ship.strict_loading?
assert developer.ship.parts.all?(&:strict_loading?)
assert_raises ActiveRecord::StrictLoadingViolationError do
developer.ship.parts.first.trinkets.to_a
end
Usually we add `_` prefix for newly added short term living (used)
internal state (e.g. ae02898, d1107f4, dcb8259), and also `@saving`
might be already used in users' code.
Autosave will double save records for some cyclic associations.
For example a child record with a parent association.
If save is called on the child before the parent has been saved,
the child will be saved twice.
The double save of the child will clear the mutation tracker on
the child record resulting in an incorrect dirty state.
```
# pirate has_one ship
ship = Ship.new(name: "Nights Dirty Lightning")
pirate = ship.build_pirate(catchphrase: "Aye")
ship.save!
ship.previous_changes # => returns {} but this should contain the changes.
```
When saving ship the following happens:
1. the `before_save` callbacks of the ship are called
2. the callbacks call `autosave_associated_records_for_pirate`
3. `autosave_associated_records_for_pirate` saves the pirate
4. the `after_save` callbacks of the pirate are called
5. the callbacks call `autosave_associated_records_for_ship`
6. `autosave_associated_records_for_ship` saves the ship
7. the ship is saved again by the original save
`autosave_associated_records_for_ship` saves the ship because the ship
association is set by inverse_of in a `has_one` on the pirate. This does
not happen with a `has_many` by default because the inverse is not set.
If setting the inverse on the `has_many` the problem occurs as well.
----------------------------
This commit adds a @saving state which tracks if a record is currently being saved.
If @saving is set to true, the record won't be saved by the autosave callbacks.
With this commit the following happens when saving a ship:
1. @saving is set to true
2. the `before_save` callbacks of the ship are called
3. the callbacks call `autosave_associated_records_for_pirate`
5. `autosave_associated_records_for_pirate` saves the pirate
6. the `after_save` callbacks of the pirate are called
6. `autosave_associated_records_for_ship` skip saving the ship
8. the ship is saved.
9. @saving is set to false
One disadvantage of this approach is the following...
While the child is no longer saved in the autosave, similar children
could be autosaved. This will result in unexpected order when creating
new records, as similar children will be commited first.
After assigning string "infinity" to datetime attribute with postgresql adapter, reading it back gets Float::INFINITY.
But assigning Float::INFINITY to datetime attribute results in getting nil value when ActiveRecord::Base.time_zone_aware_attributes is true.
This is due to TimeZoneConverter not handling Float::INFINITY appropriately.
Related to #35336.
The notable thing about #41516 is that unknown labels will no longer
match 0 on MySQL.
Matching unknown labels to 0 was not by design, but rather almost like a
bug, people should not rely on that behavior.
Closes#41595.
In #41495 I added an `ImmediateExecutor` for applications that haven't
configured a thread pool executor. After some thought and consideration
I don't think this is correct. While it's essentially a no-op it will be
confusing for users who see that queries that ran in the foreground
marked as running in the background. At the moment we don't have any
code that calls `load_async` internally from Rails but we may in the
future and we should ensure that those calls run in the foreground
and marked as such until an application has opted into async behavior by
configuring which async executor they want.
In this PR I've replaced the `ImmediateExecutor` with `nil`.
I then added a check for `async_enabled?` that checks whether
the executor is set to `nil` and if the adapter supports concurrent
queries. This is a slight change in behavior because it ensures that
adapters that can't run async queries don't log those queries are async.
Co-authored-by: John Hawthorn <john@hawthorn.email>
