Commit Graph

20628 Commits

Author SHA1 Message Date
Aaron Patterson
f95c0b7e96
Merge pull request #41825 from rails/refactor-scope-registry
Refactor scope registry
2021-04-08 08:53:36 -07:00
Jean Boussier
57cbf4136f
Merge pull request #41866 from etiennebarrie/simplify-active-model-record-registry
Simplify ActiveModel & ActiveRecord Type::Registry
2021-04-07 21:31:37 +02:00
Étienne Barrié
81d0653f84 Simplify ActiveModel & ActiveRecord Type::Registry
ActiveRecord::Type::Registry doesn't need to inherit from
ActiveModel::Type::Registry, and it makes both classes more simple.

Co-authored-by: Adrianna Chang <adrianna.chang@shopify.com>
2021-04-07 09:59:55 -04:00
eileencodes
b8b1c9eba9
Fix missing link in deprecation 2021-04-07 08:15:48 -04:00
Ryuta Kamizono
7fd8079ef2
Merge pull request #41789 from kamipo/update_cache_key_after_mutation
Clear `@cache_keys` cache after `update_all`, `delete_all`, `destroy_all`
2021-04-07 16:47:17 +09:00
Ryuta Kamizono
bc488431ae
Merge pull request #41788 from pbstriker38/check_table_name_constraint_mariadb
Add support for check_constraints with the same name on different tables
2021-04-07 09:16:59 +09:00
eileencodes
634bf89df3
Deprecate legacy_connection_handling
This deprecates `legacy_connection_handling` via the
`connection_handlers` setter. This is called from the ActiveRecord
Railtie on boot and since most applications don't set this themselves
this will prevent the deprecation from being raised multiple times for a
test run or in development.

I've also updated the guides to include a migration path for
applications using the deprecated methods. The majority of applications
won't need to make any changes.
2021-04-06 18:57:22 -04:00
John Hawthorn
23996030ba Fix build 2021-04-06 11:13:33 -07:00
John Hawthorn
0ff395e1b1
Merge pull request #41790 from jhawthorn/preloader_smart_batching
"Smart" ActiveRecord Preloader batching
2021-04-05 10:14:07 -07:00
Jorge Manrubia
1fff866dd1 Exclude encryption performance tests from isolated runs
Encryption performance tests were meant to be excluded from the builds,
however they were still being executed when running `isolated:` tests
for active record adapters.

See https://buildkite.com/rails/rails/builds/76249#e7605d50-87be-445e-a9ae-a5761c996597/1512-1531
2021-04-03 23:11:30 +02:00
George Claghorn
432bc68a2c Fix more Book references.
Follow-up to 434fb39.
2021-04-03 08:31:59 -04:00
George Claghorn
b4120d0a30 Fix lingering references to Book
Follow-up to 434fb39.
2021-04-03 08:20:42 -04:00
Ricardo Díaz
434fb39d47
Add new "encrypted_books" table to the schema
Reusing the "books" one could cause interferences when fixtures are
loaded in a very specific order such as:

https://buildkite.com/rails/rails/builds/76217#ee4ce591-e6c1-4a0d-a7db-1f83647d141e

Reproduction script:

```
activerecord $ bin/test -v --seed 23607 -n "/^(?:EagerAssociationTest#(?:test_preloading_a_regular_association_with_a_typo_through_a_polymorphic_association_still_raises)|ActiveRecord::Encryption::EncryptableFixtureTest#(?:test_fixtures_get_encrypted_automatically)|ViewWithoutPrimaryKeyTest#(?:test_attributes|test_reading))$/"
```
2021-04-03 08:00:01 -04:00
Aaron Patterson
8af85e161d
Reduce the number of times we look up the ScopeRegistry
I noticed in profiles of simple queries (like `Post.where(id: 1).first`)
we're spending lots of time looking up the current thread.  I couldn't
find any way to speed this up in Ruby, so I thought maybe we could call
`Thread.current` fewer times per query.

This patch should eliminate 4 calls to `Thread.current` per query.

For this benchmark:

```ruby
  StackProf.run(mode: :wall, out: 'out.dump') do
    8000.times { Post.where(id: id).first }
  end
```

`Thread.current` goes from 7% to 4.7% of time:

```
==================================
  Mode: wall(1000)
  Samples: 1633 (0.00% miss rate)
  GC: 51 (3.12%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
       140   (8.6%)         140   (8.6%)     String#sub!
       114   (7.0%)         114   (7.0%)     Thread.current
```

```
==================================
  Mode: wall(1000)
  Samples: 1719 (0.00% miss rate)
  GC: 51 (2.97%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
       134   (7.8%)         134   (7.8%)     String#sub!
        99   (5.8%)          99   (5.8%)     Module#===
        81   (4.7%)          81   (4.7%)     Thread.current
```

This isn't huge, but I think we need to find more sources of
Thread.current.  It's surprising to me that we spend so much time
looking up the current thread when doing a query that is so "easy"
2021-04-02 12:31:05 -07:00
Aaron Patterson
73c18888ad
Use methods to get ScopeRegistry values rather than symbols
We're spending time validating symbol parameters of the ScopeRegistry.
It's an internal class, and we can stop validating symbols by converting
to methods (you'll automatically get an error if you try to call a
method that doesn't exist).  Second, since we only have 3 things to keep
track of, rather than keep those things in a hash, just break it out in
to 3 instance variables.  (This is absolutely not a memory bottleneck,
but technically this patch will save some memory as the 3 ivars will be
embedded in the object rather than require a full st_table for the
original wrapper hash)
2021-04-02 12:14:57 -07:00
Jorge Manrubia
224c9b34cb
Add Active Record encryption to CHANGELOG 2021-04-02 07:40:00 -04:00
Ryuta Kamizono
7e1820d71b
Merge pull request #41818 from ricardotk002/fix-missing-require
Fix seemingly wrong require "lib"
2021-04-02 19:56:11 +09:00
Ricardo Díaz
27b0180232 Remove print statement / uncomment relevant assertion 2021-04-02 01:03:43 -05:00
Ricardo Díaz
b11645eb2b Fix wrong require
This library is used here

https://github.com/rails/rails/blob/1251703/activerecord/lib/active_record/encryption/encryptor.rb#L121

and here

https://github.com/rails/rails/blob/1251703/activerecord/lib/active_record/encryption/encryptor.rb#L135

Reference https://buildkite.com/rails/rails/builds/76198#de7db4af-b8ca-46de-986f-0db34b1d4ddc
2021-04-02 01:00:37 -05:00
Sampat Badhe
2032daa415
Fix typos encryption doc (#41817)
* fix typo

[ci skip]
2021-04-02 14:24:43 +09:00
Jorge Manrubia
5a6352c072 Fix deterministic queries that were broken after #41068
This is adding yet another patch to make them work. This system needs to
be reworked as it's currently very brittle.
2021-04-01 22:10:59 +02:00
John Hawthorn
20b9bb1de0 Intelligent batch preloading
This examines all the association branches we are being asked to preload
and will delay loading an association if it's likely that we find a
similar association later and can batch them together.

For example, when loading

    Author.preload(:posts, favorite_authors: :posts).first

The preloader now knows to delay loading the top level posts so that it
can load both the top level :posts and the :posts from the favourite
authors associations together.

Co-authored-by: Dinah Shi <dinahshi@github.com>
2021-04-01 12:52:43 -07:00
Jorge Manrubia
e24fb5524a Validate that proper keys are configured when declaring attributes
This enables to disable deterministic encryption by just not setting
deterministic_key.
2021-04-01 18:20:54 +02:00
Jorge Manrubia
2035138115 Use proper hook for loading fixtures extension 2021-04-01 15:02:15 +02:00
Jorge Manrubia
a61692cf41 Add support for uniqueness validations 2021-04-01 15:02:15 +02:00
Jorge Manrubia
f78a480818 Encourage deterministic encryption to remain unchanged
This implements several changes to encourage deterministic encryption to
remain unchanged. The main motivation is letting you define unique
indexes on deterministically-encrypted columns:

- By default, deterministic encryption will always use the oldest
encryption scheme to encrypt new data, when there are many.
- You can skip this default behavior and make it always use the current
encryption scheme with:

```ruby
deterministic: { fixed: false } # using this should be a rare need
```

- Deterministic encryption still supports previous encryption schemes
normally. So they will be used to add additional values to queries, for
example.
- You can't rotate deterministic encryption keys anymore. We can add
support for that in the future.

This makes for reasonable defaults:

- People using "deterministic: true" will get unique indexes working out
of the box.
- The system will encourage keeping deterministic encryption stable:
  - By always using oldest encryption schemes
  - By forbidding configuring multiple keys

But you can still opt-out of the default if you need to.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
7a1fb99302 Add support to declare previous encryption schemes globally 2021-04-01 15:02:15 +02:00
Jorge Manrubia
d35905ccae Add Scheme, a container for encrypted attribute configs
This adds a new class `Scheme` that encapsulates the encryption
properties for a given attribute. This adds a proper code representation
for a domain concept we keep referring to: encryption schemes.

It's in charge of processing the user-passed config properties when
declaring the attribute, as well as of validating those.

This removes the concern of processing properties from
`EncryptableRecord` and `EncryptableAttributeType`. It's also nice to
have a place to group attribute encryption options, versus passing
lists of hashes around.

This is in preparation to upcoming changes that will add new config
options and support for previous encryption schemes.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
28145c3cee Rename master_key => primary_key 2021-04-01 15:02:15 +02:00
Jorge Manrubia
c275b10a0e Show performance tests results now that they are excluded from the build 2021-04-01 15:02:15 +02:00
Jorge Manrubia
bcf6a5fafb Fix test
The .encrypts declaration in `ActionText::Encryption` was making this test
fail. It was failing to load a MySQL error that was being raised when trying
to determine the column limit.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
e4dae63013 Remove before_initialize, I think it's not needed 2021-04-01 15:02:15 +02:00
Jorge Manrubia
a786e34ab2 Move to on_load hook to see if it helps with railty test failing 2021-04-01 15:02:15 +02:00
Jorge Manrubia
bf75702b1c Inline prepend 2021-04-01 15:02:15 +02:00
Jorge Manrubia
9556720527 Wait for active record to be loaded to extend it 2021-04-01 15:02:15 +02:00
Jorge Manrubia
588c972d21 Removes the Action Text dependency from Active Record
Move the Action Text stuff to its own concern in `actiontext`.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
c41b354bf0 Remove mass encryption
I don't think this belong to the library yet. It's more like an util class we used
for building some mass-encryption tasks in HEY.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
26ceb126b9 Override accessors with module
This way users can override them and call #super

https://github.com/rails/rails/pull/41659#discussion_r592624914
2021-04-01 15:02:15 +02:00
Jorge Manrubia
dfa875fec5 Tweak class description 2021-04-01 15:02:15 +02:00
Jorge Manrubia
5cec2eced6 Remove preset env vars for encryption keys
These can be configured as +ActiveRecord::Encryption.config+ keys if you want,
so you can just use your own ENV vars in a config file if you prefer not to use
credentials.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
f9514727af Update activerecord/RUNNING_UNIT_TESTS.rdoc
Co-authored-by: Mark Oleson <fusion2004@hey.com>
2021-04-01 15:02:15 +02:00
Jorge Manrubia
d5625c97df Review class comments
Co-authored-by: Rafael Mendonça França <rafaelmfranca@gmail.com>
2021-04-01 15:02:15 +02:00
Jorge Manrubia
3c379393c3 Split concern inclusion in two lines for clarity
Order is inverted when inlined, which can make things difficult to read.

https://github.com/rails/rails/pull/41659#discussion_r592604925

In this case it doesn't really matter and I think it's better for `Configurable`
to go first, so the change is inverting the previous order deliberately.

[skip ci]
2021-04-01 15:02:15 +02:00
Jorge Manrubia
eada873e4c Make ActiveRecord::Encryption load eagerly along with ActiveRecord
https://github.com/rails/rails/pull/41659#discussion_r592598084
https://github.com/rails/rails/pull/41659#discussion_r592601747

[skip ci]
2021-04-01 15:02:14 +02:00
Jorge Manrubia
c0fa9428e5 Fix test 2021-04-01 15:02:14 +02:00
Jorge Manrubia
7945d4fde3 Make silent by default to prevent output during tests 2021-04-01 15:02:14 +02:00
Jorge Manrubia
9bfca80641 Only run limit-validation tests when the limit exists
SQLite and PostgreSQL won't add a default limit of 255 for string types
2021-04-01 15:02:14 +02:00
Jorge Manrubia
dcbe9cd5e4 Replace class var with class_attribute to prevent child class conflicts with it 2021-04-01 15:02:14 +02:00
Jorge Manrubia
ead2b2bec9 No need to register a type after 75929dfb1c
This was making a test fail, and can make existing code fail too without a good reason.
2021-04-01 15:02:14 +02:00
Jorge Manrubia
826ec6a473 Rename previous_types => previous_encrypted_types
This makes it clear what they are. It was confusing with "cast_type" around.
2021-04-01 15:02:14 +02:00