Commit Graph

22442 Commits

Author SHA1 Message Date
Alex Watt
d0570697f4 ActiveRecord: Improve find_db_config performance
I noticed an application spending ~5ms on `find_db_config`, with a lot
of time spent on sorting the database configs and comparing the arrays
(`Array#<=>`). I updated `find_db_config` to avoid the sort entirely.

Here is a benchmark script I used to measure the improvement:

```

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem "rails", github: "rails/rails", branch: "main"
  gem "benchmark-ips"
end

require "active_record"
require "active_record/database_configurations"

module ActiveRecord
  class DatabaseConfigurations
    def fast_find_db_config(env)
      current_env_configs, other_configs = configurations.partition(&:for_current_env?)
      [*current_env_configs, *other_configs].find do |db_config|
        db_config.env_name == env.to_s ||
          (db_config.for_current_env? && db_config.name == env.to_s)
      end
    end
  end
end

def generate_configs(adapter, count)
  count.times.to_h do |i|
    [i == 0 ? "primary" : "config_#{i}", { "adapter" => adapter }]
  end
end

small_config = ActiveRecord::DatabaseConfigurations.new(
  **generate_configs("sqlite3", 3)
)

large_config = ActiveRecord::DatabaseConfigurations.new({
  **generate_configs("randomadapter", 100),
  ActiveRecord::ConnectionHandling::DEFAULT_ENV.call => generate_configs("sqlite3", 100)
})

SCENARIOS = {
  "Empty"                   => ActiveRecord::DatabaseConfigurations.new({}),
  "A few connections"       => small_config,
  "Hundreds of connections" => large_config,
}

SCENARIOS.each_pair do |name, value|
  puts
  puts " #{name} ".center(80, "=")
  puts

  Benchmark.ips do |x|
    x.report("find_db_config") { value.find_db_config("primary") }
    x.report("fast_find_db_config") { value.fast_find_db_config("primary") }
    x.compare!
  end
end
```

The results show a consistent speedup, especially for many configs:

==================================== Empty =====================================

Warming up --------------------------------------
      find_db_config    82.849k i/100ms
 fast_find_db_config   172.141k i/100ms
Calculating -------------------------------------
      find_db_config    830.202k (± 1.9%) i/s -      4.225M in   5.091388s
 fast_find_db_config      1.633M (± 6.6%) i/s -      8.263M in   5.082794s

Comparison:
 fast_find_db_config:  1633426.8 i/s
      find_db_config:   830201.9 i/s - 1.97x  slower

============================== A few connections ===============================

Warming up --------------------------------------
      find_db_config    25.356k i/100ms
 fast_find_db_config    47.260k i/100ms
Calculating -------------------------------------
      find_db_config    248.648k (± 2.7%) i/s -      1.268M in   5.102833s
 fast_find_db_config    475.184k (± 3.0%) i/s -      2.410M in   5.077268s

Comparison:
 fast_find_db_config:   475184.1 i/s
      find_db_config:   248647.6 i/s - 1.91x  slower

=========================== Hundreds of connections ============================

Warming up --------------------------------------
      find_db_config   361.000  i/100ms
 fast_find_db_config     2.400k i/100ms
Calculating -------------------------------------
      find_db_config      3.622k (± 1.9%) i/s -     18.411k in   5.085694s
 fast_find_db_config     24.073k (± 2.0%) i/s -    122.400k in   5.086726s

Comparison:
 fast_find_db_config:    24073.0 i/s
      find_db_config:     3621.5 i/s - 6.65x  slower
2023-04-08 12:12:44 -04:00
Paarth Madan
996d5023f0 Support composite primary keys during transaction rollback 2023-03-22 19:16:05 -04:00
Eileen M. Uchitelle
2cef3ac45b
Merge pull request #47729 from Shopify/pm/cpk-where-syntax
Introduce query-by-tuple syntax
2023-03-22 17:08:42 -04:00
Vipul A M
047a76e37f
Merge pull request #47702 from shouichi/class-cache-nodoc
Add nodoc to ActiveRecord::FixtureSet::ClassCache [skip ci]
2023-03-23 01:54:01 +05:30
Paarth Madan
350b397917 Delete Cpk::Book in non-transactional test
Having this record persisted across test runs was leading to flakey behaviour in tests that were creating Cpk::Book with the same primary key.
2023-03-22 16:16:41 -04:00
Paarth Madan
77b9cddbbe Use tuple query syntax to avoid branching 2023-03-22 13:56:48 -04:00
Paarth Madan
e39e013cf7 Specify where clauses by mapping columns to tuples
With the introduction of composite primary keys, a common usecase is querying for records with tuples representing the composite key. This change introduces new syntax to the where clause that allows specifying an array of columns mapped to a list of corresponding tuples. It converts this to an OR-joined list of separate queries, similar to previous implementations that rely on grouping queries.
2023-03-22 13:56:47 -04:00
Eileen M. Uchitelle
355fd59290
Merge pull request #47731 from Shopify/fix-CPK-models-equality
Fix `#hash` and `#==` for composite pk models
2023-03-22 10:28:45 -04:00
Nikita Vasilevsky
7b1a617c21 Fix #hash and #== for composite pk models
Given a model with a composite primary key, for example:
`TravelRoute.primary_key = [:from, :to]`

and two objects of the given model, objects `a` and `b`, should be
equal to each other and have the same `hash` value if they have the same
values for the composite primary key, like:

```ruby
a = TravelRoute.new(from: "NYC", to: "LAX")
b = TravelRoute.new(from: "NYC", to: "LAX")
a == b # => true

a.hash == b.hash # => true
```

At the same time, two objects of the given model should not be equal to
each other and should have different `hash` values if they have
different primary key values or no primary key being set, like:

```ruby
a = TravelRoute.new(from: "NYC", to: "LAX")
b = TravelRoute.new(from: "NYC", to: "SFO")
a == b # => false
a.hash == b.hash # => false

TravelRoute.new == TravelRoute.new # => false
TravelRoute.new.hash == TravelRoute.new.hash # => false
```
2023-03-22 13:46:20 +00:00
Eileen M. Uchitelle
ba19dbc499
Merge pull request #47721 from Shopify/fix-nullifying-has-many-through-association
Fix nullification of has_many :through association with `query_constraints
2023-03-21 11:57:28 -04:00
Nikita Vasilevsky
300c853494 Fix nullification of has_many :through association with query_constraints
Given a `has_many :through` association with `query_constraints`:

```ruby
BlogPost.has_many :posts_tags
BlogPost.has_many :tags,
  through: :posts_tags,
  query_constraints: [:blog_id, :post_id]
``

It is possible to nullify the association like `blog_post.tags = []`
which results in deletion of records from the `posts_tags` joining table.
2023-03-21 13:51:30 +00:00
Eileen M. Uchitelle
4b183e4942
Merge pull request #47720 from Shopify/pm/cpk-id-accessor
Accept composite primary key in `id=`
2023-03-21 09:50:01 -04:00
Paarth Madan
10e86244ce Accept composite primary key in id= 2023-03-20 19:00:43 -04:00
Nick Borromeo
6bd100711c Allow SQL Warnings to be ignored using error codes
In https://github.com/rails/rails/pull/46690 the `db_warnings_action` and `db_warnings_ignore` configs where added. The
`db_warnings_ignore` can take a list of warning messages to match against.

At GitHub we have a subscriber that that does something like this but also filters out error codes. There might also be
other applications that filter via error codes and this could be something they can use instead of just the explicit
messages.

This also refactors the adapter tests in order for mysql2 and postgresql adapters to share the same helper when setting
the db_warnings_action and db_warnings_ignore configs
2023-03-20 13:06:22 -07:00
Matthew Draper
bc081a55be
Merge pull request #47715 from adrianna-chang-shopify/ac-pg-fix-conn-param
PostgreSQL adapter uses :host instead of :hostname for connection params
2023-03-21 04:27:38 +10:30
Adrianna Chang
b3d6c6496f
PostgreSQL adapter uses :host instead of :hostname for connection parameters 2023-03-20 13:09:10 -04:00
sampatbadhe
d9d8d8a4d2 Correct typos in activerecord changelog and querylogs docs [ci-skip] 2023-03-18 19:07:41 +05:30
Rafael Mendonça França
e23e1718ea
Merge pull request #47690 from andrewn617/add-on-load-callback-for-active-record-fixtures
Run a load hook when TestFixtures is included
2023-03-17 14:10:52 -04:00
Andrew Novoselac
49283adc16 Run a load hook when TestFixtures is included 2023-03-17 13:36:48 -04:00
Shouichi Kamiya
0ad120b72e Add nodoc to ActiveRecord::FixtureSet::ClassCache [skip ci]
I believe this is an implementation detail.
2023-03-17 15:21:05 +09:00
Nikita Vasilevsky
95a3e6cd0a Allow querying by whole record for composite query constraints associations
Given an association defined with composite query constraints like:
```ruby
BlogPost.has_many :comments, query_constraints: [:blog_id, :blog_post_id]
```

it is possible to query blog posts by whole `comments` objects like:

```ruby
comments = Comment.first(2)
BlogPost.where(comments: comments).to_a
```
2023-03-16 17:02:50 +00:00
Jonathan Hefner
9f399a6b6b Rely on AM::ForbiddenAttributesProtection autoload
`ActiveModel::ForbiddenAttributesProtection` is autoloaded, so this
`require` is not necessary.
2023-03-15 13:23:00 -05:00
Jonathan Hefner
8986050c41
Merge pull request #47683 from yskkin/patch-3
Remove redundant require: follow #10776
2023-03-15 13:16:20 -05:00
Matthew Draper
0dc919b544
Merge pull request #47684 from rails/revert-47493-quote_binary_strings
Revert "Quote binary strings in Arel"
2023-03-16 01:40:13 +10:30
Eileen M. Uchitelle
3b7e4134e5
Merge pull request #47664 from Shopify/support-composite-identifier-in-find
Extend `ActiveRecord::FinderMethods#find` with support for composite pk values
2023-03-15 10:57:34 -04:00
Nikita Vasilevsky
7375ff2494 Introduce ActiveRecord::PrimaryKey#composite_primary_key? abstraction
Initial implementation falls back to `primary_key.is_a?(Array)` so
it can be immediately used in place of direct `is_a?` checks.
Though implementation may be changed to rely on a pre-initialized
`@composite_primary_key` ivar in the future.
2023-03-15 13:45:23 +00:00
Nikita Vasilevsky
b05321d31a Extend ActiveRecord::FinderMethods#find with support for composite pk values
`ActiveRecord::FinderMethods#find` now supports passing sets of
composite primary key values like:

```ruby
Cpk::Book.find([1, 1])
Cpk::Book.find([[1, 1]])
Cpk::Book.find([1, 1], [1, 2])
Cpk::Book.find([[1, 1], [1, 2]])
```

and treats values as values of the composite primary key columns but
only for models with the `primary_key` being an `Array`.
2023-03-15 13:44:54 +00:00
Eileen M. Uchitelle
9527e5e307
Merge pull request #47668 from Shopify/pm/destroy-async-composite-keys
Destroying associations asynchronously respect query constraints
2023-03-15 09:18:42 -04:00
Rafael Mendonça França
74554cde8b
Merge PR #47675 2023-03-15 13:05:44 +00:00
Matthew Draper
bb7f3be138
Revert "Quote binary strings in Arel" 2023-03-15 19:50:50 +10:30
Yoshiyuki Kinjo
db944ce3b2
Remove redundant require: follow #10776
active_model/forbidden_attributes_protection is not used since #10776
2023-03-15 18:16:51 +09:00
Jean Boussier
335733b858
Merge pull request #47649 from fatkodima/retry-flaky-secure_password-test 2023-03-15 08:56:39 +00:00
Yasuo Honda
126d2169d0
Merge pull request #47655 from alpaca-tc/deferrable_exclusion_constraint
Adds support for deferrable exclude constraints in PostgreSQL.
2023-03-15 08:27:01 +09:00
Paarth Madan
a270108bf6 Destroy associations async respect query constraints
The codepaths related to destroying associations asynchronously now
consider when query constraints are present. In most cases, this means
interpreting the primary key as an array of columns, and identifying
associated records by a tuple of these columns, where previously this
would've been a single ID. In each of the callsites, we use branching.
This is done to maintain backwards compatibility and ensure the
signature of the destroy job remains stable: it has consumers outside of
Rails.
2023-03-14 19:09:09 -04:00
Andrew Novoselac
6902cbce1b Introducs TestFixtures#fixture_paths.
Multiple fixture paths can now be specified using the `#fixture_paths` accessor.
2023-03-14 19:02:56 -04:00
Hormoz Kheradmand
34d8a65969
add nodoc to valid_*_options methods 2023-03-14 12:32:58 -07:00
eileencodes
0603d512dc
Pass self directly to connection_class
Returning `self` and `db_config` from `resolve_config_for_connection` is
hold over from legacy behavior when we returned `self.name` or
`Base.name` rather than just `self`. We can simplify this by passing
`self` directly` to the handlers `establish_connection`.
2023-03-14 11:26:11 -04:00
alpaca-tc
cbc7b59749 Adds support for deferrable exclude constraints in PostgreSQL.
By default, exclude constraints in PostgreSQL are checked after each statement.
This works for most use cases, but becomes a major limitation when replacing
records with overlapping ranges by using multiple statements.

```ruby
exclusion_constraint :users, "daterange(valid_from, valid_to) WITH &&", deferrable: :immediate
```

Passing `deferrable: :immediate` checks constraint after each statement,
but allows manually deferring the check using `SET CONSTRAINTS ALL DEFERRED`
within a transaction. This will cause the excludes to be checked after the transaction.

It's also possible to change the default behavior from an immediate check
(after the statement), to a deferred check (after the transaction):

```ruby
exclusion_constraint :users, "daterange(valid_from, valid_to) WITH &&", deferrable: :deferred
```

*Hiroyuki Ishii*
2023-03-14 23:29:35 +09:00
Jason Karns
1f61fd65ae
Delegated Type supports customizeable foreign_type column 2023-03-14 09:19:30 -04:00
Alexander Ryazantsev
8381a503c1
Typo fix on create_table description 2023-03-13 23:22:28 +03:00
Eileen M. Uchitelle
deeb50f3dd
Merge pull request #47616 from szymonlipka/fix-drop-table-invert
Fix 47614 issue with inverting create_table/drop_table migration
2023-03-13 10:46:41 -04:00
Eileen M. Uchitelle
3c0569ec17
Merge pull request #47635 from Shopify/generate-composite-pk-in-fixtures
Fix fixtures id generation for composite primary keys
2023-03-13 10:26:06 -04:00
Szymon Lipka
d8b564f7e7 Fix drop table invert when if_exists option is present 2023-03-13 15:07:37 +01:00
fatkodima
fe97e26785 Optimize slow tests in activerecord 2023-03-13 14:07:28 +02:00
fatkodima
2d5508a7cf Retry flaky secure password test 2023-03-12 23:24:28 +02:00
Yasuo Honda
3025eaa091
Merge pull request #46192 from alpaca-tc/support_unique_constraints
Add support for unique constraints (PostgreSQL-only).
2023-03-12 18:31:38 +09:00
Matthew Draper
511ffed752
Merge pull request #47438 from benedikt/arel-documentation
Adds documentation for Arel::Nodes::Node
2023-03-12 03:34:06 +10:30
Nikita Vasilevsky
059627cbbd Fix fixtures id generation for composite primary keys 2023-03-10 21:27:39 +00:00
Nikita Vasilevsky
8b43c13e53 Use composite primary_key value as a fallback for query_constraints_list
Given a model with a composite primary key, the `query_constraints_list`
should equal the `primary_key` value to enable capabilities managed by
`query_constraints_list` such as `destroy`, `update`, `delete` and others.
2023-03-10 18:31:02 +00:00
Eileen M. Uchitelle
62e36149c5
Merge pull request #47625 from Shopify/add-model-with-a-composite-primary-key
Define `ActiveRecord::Base#id` API for composite primary key models
2023-03-10 10:48:44 -05:00