Commit Graph

10619 Commits

Author SHA1 Message Date
Jean Boussier
67c0f9aa25
Merge pull request #41930 from Shopify/optimize-hstore-parser
Optimize the HStore parser
2021-04-13 15:46:26 +02:00
Andrew White
98bf64bcb9 Use StringScanner to parse Hstore payloads 2021-04-13 15:32:09 +02:00
John Bampton
33baf0dabd
Fix spelling of EnvelopeEncryptionPerformanceTest 2021-04-13 09:17:11 -04:00
Jorge Manrubia
170bc08eae Remove fixed "id" in encrypted book fixtures
Hopefully this will help with flaky encrypted_fixture_test failing from
time to time due to not finding book with id=1.
2021-04-13 08:55:44 -04:00
Jorge Manrubia
ae7f693647 Fix: assign previous encryption schemes via previous: config option.
The previous: config option was being skipped because it was checking
the existence of a reader method instead of the accessor itself.

This also adds a test for the .configure option that was missing.
2021-04-13 08:55:44 -04:00
John Bampton
97cf9c21fb
Fix spelling in EncryptionSchemesTest 2021-04-13 08:39:01 -04:00
Rafael Mendonça França
14bca25975
Make sure establish_connection with symbol work properly
Before when calling `preventing_writes?` in the connection the code
would raise a `NoMethodError` when calling `current_preventing_writes`
in a String.

This problem only happen when `establish_connection` is called on the
`ConnectionHandler` and not in the model itself.

This also removes the conditional from the `PoolConfig`.
2021-04-12 22:27:16 +00:00
Rafael França
32db8149ed
Merge pull request #41926 from jbampton/fix-favorite
chore: fix spelling change `favourite` to the more used `favorite`
2021-04-12 16:14:56 -04:00
Vladimir Dementyev
4bef82217c Force Arel.sql for returning and on_duplicate 2021-04-12 20:23:46 +03:00
Vladimir Dementyev
8f3c12f880 Add update_sql option to #upsert_all 2021-04-12 19:09:17 +03:00
Vladimir Dementyev
c7613dde53 Support string returning clause for AR#insert_all
(cherry picked from commit 15b3310a4d6b2044da4ff79737e2b19fef9b6267)
2021-04-12 18:08:04 +03:00
Ryuta Kamizono
9aed3dcdfe Remove unused Category.has_and_belongs_to_many :popular_grouped_posts definition
This was added at 97403ad but it was never used.
2021-04-12 15:05:42 +09:00
John Bampton
11557e6cec chore: fix spelling change favourite to the more used favorite 2021-04-12 12:35:12 +10:00
Ryuta Kamizono
b8c67807b4
Merge pull request #41898 from kamipo/delete_all_payload_name
Improve the payload name for `delete_all` to more appropriate
2021-04-12 07:34:11 +09:00
John Bampton
13b1d9dc35 chore: fix grammar and spelling 2021-04-12 05:30:44 +10:00
Jean Boussier
9ef7f406e6 Optimize the HStore parser
Test string from taken from the the test suite: `"\"a\\\\b\"=>\"b\\\\ar\", \"1\\\"foo\"=>\"2\""`

Bench:

```
Warming up --------------------------------------
            original     6.896k i/100ms
             patched    15.787k i/100ms
Calculating -------------------------------------
            original     68.176k (± 1.2%) i/s -    344.800k in   5.058270s
             patched    157.786k (± 1.0%) i/s -    789.350k in   5.003144s

Comparison:
             patched:   157786.0 i/s
            original:    68176.1 i/s - 2.31x  (± 0.00) slower

```

Memory before: allocated 4376 bytes (55 objects)
Memory after: allocated: 1880 bytes (23 objects)
2021-04-11 21:11:21 +02:00
Jean Boussier
7abd9b5f66 Precompute Inheritance.base_class
This saves checking the inehritance chain on
each access.
2021-04-11 08:40:51 +02:00
Ryuta Kamizono
99049262d3 Clear @cache_keys cache even when eager loading
Follow up to #41789.
2021-04-10 21:28:28 +09:00
Ryuta Kamizono
899ecd4186 Improve the payload name for delete_all to more appropriate
The payload name for `delete_all` was named "Destroy" in #30619 since
`delete_all` was used in `record.destroy` at that time.

Since ea45d56, `record.destroy` no longer relies on `delete_all`, so now
we can improve the payload name for `delete_all` to more appropriate.
2021-04-10 20:19:53 +09:00
Ryuta Kamizono
b576dd02af Fix user-defined self.default_scope to respect table alias
If a model which has a user-defined `self.default_scope` is joined with
table alias, a user-defined `self.default_scope` ignores table alias.

This problem has potentially existed for a long time, but it has not
become apparent because it is difficult to meet this condition.

Since #40106, table alias is easily used if association names are used
in `where`.

So user-defined `self.default_scope` should be evaluated in the current
aliased relation.

Fixes #41857.
2021-04-09 19:43:56 +09:00
John Hawthorn
82b958a222 Add more methods to NullPool
This commit adds more empty methods to a connection pool.

In normal usage, a connection is always be associated with a pool, but
for testing (we do this a few places in the AR test suite) it can be
convenient to build a connection and use it without associating a pool.

In most cases, this worked fine, but there were a few corner cases which
would hit NoMethodError on the NullPool. In particular in some cases
raising in a transaction would error in connection.throw_away!.

To make these "standalone" or "pool-less" connections work consistently,
this commit adds the methods which a connection could call on it's pool
(but not any other of ConnectionPool's public interface).
2021-04-08 10:58:07 -07:00
Aaron Patterson
f95c0b7e96
Merge pull request #41825 from rails/refactor-scope-registry
Refactor scope registry
2021-04-08 08:53:36 -07:00
Ryuta Kamizono
7fd8079ef2
Merge pull request #41789 from kamipo/update_cache_key_after_mutation
Clear `@cache_keys` cache after `update_all`, `delete_all`, `destroy_all`
2021-04-07 16:47:17 +09:00
Ryuta Kamizono
bc488431ae
Merge pull request #41788 from pbstriker38/check_table_name_constraint_mariadb
Add support for check_constraints with the same name on different tables
2021-04-07 09:16:59 +09:00
eileencodes
634bf89df3
Deprecate legacy_connection_handling
This deprecates `legacy_connection_handling` via the
`connection_handlers` setter. This is called from the ActiveRecord
Railtie on boot and since most applications don't set this themselves
this will prevent the deprecation from being raised multiple times for a
test run or in development.

I've also updated the guides to include a migration path for
applications using the deprecated methods. The majority of applications
won't need to make any changes.
2021-04-06 18:57:22 -04:00
John Hawthorn
23996030ba Fix build 2021-04-06 11:13:33 -07:00
John Hawthorn
0ff395e1b1
Merge pull request #41790 from jhawthorn/preloader_smart_batching
"Smart" ActiveRecord Preloader batching
2021-04-05 10:14:07 -07:00
George Claghorn
432bc68a2c Fix more Book references.
Follow-up to 434fb39.
2021-04-03 08:31:59 -04:00
George Claghorn
b4120d0a30 Fix lingering references to Book
Follow-up to 434fb39.
2021-04-03 08:20:42 -04:00
Ricardo Díaz
434fb39d47
Add new "encrypted_books" table to the schema
Reusing the "books" one could cause interferences when fixtures are
loaded in a very specific order such as:

https://buildkite.com/rails/rails/builds/76217#ee4ce591-e6c1-4a0d-a7db-1f83647d141e

Reproduction script:

```
activerecord $ bin/test -v --seed 23607 -n "/^(?:EagerAssociationTest#(?:test_preloading_a_regular_association_with_a_typo_through_a_polymorphic_association_still_raises)|ActiveRecord::Encryption::EncryptableFixtureTest#(?:test_fixtures_get_encrypted_automatically)|ViewWithoutPrimaryKeyTest#(?:test_attributes|test_reading))$/"
```
2021-04-03 08:00:01 -04:00
Aaron Patterson
8af85e161d
Reduce the number of times we look up the ScopeRegistry
I noticed in profiles of simple queries (like `Post.where(id: 1).first`)
we're spending lots of time looking up the current thread.  I couldn't
find any way to speed this up in Ruby, so I thought maybe we could call
`Thread.current` fewer times per query.

This patch should eliminate 4 calls to `Thread.current` per query.

For this benchmark:

```ruby
  StackProf.run(mode: :wall, out: 'out.dump') do
    8000.times { Post.where(id: id).first }
  end
```

`Thread.current` goes from 7% to 4.7% of time:

```
==================================
  Mode: wall(1000)
  Samples: 1633 (0.00% miss rate)
  GC: 51 (3.12%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
       140   (8.6%)         140   (8.6%)     String#sub!
       114   (7.0%)         114   (7.0%)     Thread.current
```

```
==================================
  Mode: wall(1000)
  Samples: 1719 (0.00% miss rate)
  GC: 51 (2.97%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
       134   (7.8%)         134   (7.8%)     String#sub!
        99   (5.8%)          99   (5.8%)     Module#===
        81   (4.7%)          81   (4.7%)     Thread.current
```

This isn't huge, but I think we need to find more sources of
Thread.current.  It's surprising to me that we spend so much time
looking up the current thread when doing a query that is so "easy"
2021-04-02 12:31:05 -07:00
Aaron Patterson
73c18888ad
Use methods to get ScopeRegistry values rather than symbols
We're spending time validating symbol parameters of the ScopeRegistry.
It's an internal class, and we can stop validating symbols by converting
to methods (you'll automatically get an error if you try to call a
method that doesn't exist).  Second, since we only have 3 things to keep
track of, rather than keep those things in a hash, just break it out in
to 3 instance variables.  (This is absolutely not a memory bottleneck,
but technically this patch will save some memory as the 3 ivars will be
embedded in the object rather than require a full st_table for the
original wrapper hash)
2021-04-02 12:14:57 -07:00
Ricardo Díaz
27b0180232 Remove print statement / uncomment relevant assertion 2021-04-02 01:03:43 -05:00
Jorge Manrubia
5a6352c072 Fix deterministic queries that were broken after #41068
This is adding yet another patch to make them work. This system needs to
be reworked as it's currently very brittle.
2021-04-01 22:10:59 +02:00
John Hawthorn
20b9bb1de0 Intelligent batch preloading
This examines all the association branches we are being asked to preload
and will delay loading an association if it's likely that we find a
similar association later and can batch them together.

For example, when loading

    Author.preload(:posts, favorite_authors: :posts).first

The preloader now knows to delay loading the top level posts so that it
can load both the top level :posts and the :posts from the favourite
authors associations together.

Co-authored-by: Dinah Shi <dinahshi@github.com>
2021-04-01 12:52:43 -07:00
Jorge Manrubia
e24fb5524a Validate that proper keys are configured when declaring attributes
This enables to disable deterministic encryption by just not setting
deterministic_key.
2021-04-01 18:20:54 +02:00
Jorge Manrubia
a61692cf41 Add support for uniqueness validations 2021-04-01 15:02:15 +02:00
Jorge Manrubia
f78a480818 Encourage deterministic encryption to remain unchanged
This implements several changes to encourage deterministic encryption to
remain unchanged. The main motivation is letting you define unique
indexes on deterministically-encrypted columns:

- By default, deterministic encryption will always use the oldest
encryption scheme to encrypt new data, when there are many.
- You can skip this default behavior and make it always use the current
encryption scheme with:

```ruby
deterministic: { fixed: false } # using this should be a rare need
```

- Deterministic encryption still supports previous encryption schemes
normally. So they will be used to add additional values to queries, for
example.
- You can't rotate deterministic encryption keys anymore. We can add
support for that in the future.

This makes for reasonable defaults:

- People using "deterministic: true" will get unique indexes working out
of the box.
- The system will encourage keeping deterministic encryption stable:
  - By always using oldest encryption schemes
  - By forbidding configuring multiple keys

But you can still opt-out of the default if you need to.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
7a1fb99302 Add support to declare previous encryption schemes globally 2021-04-01 15:02:15 +02:00
Jorge Manrubia
28145c3cee Rename master_key => primary_key 2021-04-01 15:02:15 +02:00
Jorge Manrubia
c275b10a0e Show performance tests results now that they are excluded from the build 2021-04-01 15:02:15 +02:00
Jorge Manrubia
c41b354bf0 Remove mass encryption
I don't think this belong to the library yet. It's more like an util class we used
for building some mass-encryption tasks in HEY.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
26ceb126b9 Override accessors with module
This way users can override them and call #super

https://github.com/rails/rails/pull/41659#discussion_r592624914
2021-04-01 15:02:15 +02:00
Jorge Manrubia
c0fa9428e5 Fix test 2021-04-01 15:02:14 +02:00
Jorge Manrubia
9bfca80641 Only run limit-validation tests when the limit exists
SQLite and PostgreSQL won't add a default limit of 255 for string types
2021-04-01 15:02:14 +02:00
Jorge Manrubia
826ec6a473 Rename previous_types => previous_encrypted_types
This makes it clear what they are. It was confusing with "cast_type" around.
2021-04-01 15:02:14 +02:00
Jorge Manrubia
36c4787469 Move encryption performance tests out of the main tests pipeline
This adds new rake tasks for these tests. This way, it prevents these
tests from making the build fail.

https://github.com/rails/rails/pull/41659#discussion_r592551649
2021-04-01 15:02:14 +02:00
Jorge Manrubia
4461ba1e27 Test fixes 2021-04-01 15:02:14 +02:00
Jorge Manrubia
81afcabd19 Extract encrypted models to their own files
We are not encrypting attributes when loading models with the table
missing. This way we make sure we only load the encrypted models when
necessary during the encryption tests and prevent the problem of missing
encrypted attributes due to having cached the class without them encrypted.
2021-04-01 15:02:14 +02:00
Jorge Manrubia
1163154a0b Make performance helper silent by default 2021-04-01 15:02:14 +02:00