Commit Graph

10585 Commits

Author SHA1 Message Date
Ricardo Díaz
434fb39d47
Add new "encrypted_books" table to the schema
Reusing the "books" one could cause interferences when fixtures are
loaded in a very specific order such as:

https://buildkite.com/rails/rails/builds/76217#ee4ce591-e6c1-4a0d-a7db-1f83647d141e

Reproduction script:

```
activerecord $ bin/test -v --seed 23607 -n "/^(?:EagerAssociationTest#(?:test_preloading_a_regular_association_with_a_typo_through_a_polymorphic_association_still_raises)|ActiveRecord::Encryption::EncryptableFixtureTest#(?:test_fixtures_get_encrypted_automatically)|ViewWithoutPrimaryKeyTest#(?:test_attributes|test_reading))$/"
```
2021-04-03 08:00:01 -04:00
Ricardo Díaz
27b0180232 Remove print statement / uncomment relevant assertion 2021-04-02 01:03:43 -05:00
Jorge Manrubia
5a6352c072 Fix deterministic queries that were broken after #41068
This is adding yet another patch to make them work. This system needs to
be reworked as it's currently very brittle.
2021-04-01 22:10:59 +02:00
Jorge Manrubia
e24fb5524a Validate that proper keys are configured when declaring attributes
This enables to disable deterministic encryption by just not setting
deterministic_key.
2021-04-01 18:20:54 +02:00
Jorge Manrubia
a61692cf41 Add support for uniqueness validations 2021-04-01 15:02:15 +02:00
Jorge Manrubia
f78a480818 Encourage deterministic encryption to remain unchanged
This implements several changes to encourage deterministic encryption to
remain unchanged. The main motivation is letting you define unique
indexes on deterministically-encrypted columns:

- By default, deterministic encryption will always use the oldest
encryption scheme to encrypt new data, when there are many.
- You can skip this default behavior and make it always use the current
encryption scheme with:

```ruby
deterministic: { fixed: false } # using this should be a rare need
```

- Deterministic encryption still supports previous encryption schemes
normally. So they will be used to add additional values to queries, for
example.
- You can't rotate deterministic encryption keys anymore. We can add
support for that in the future.

This makes for reasonable defaults:

- People using "deterministic: true" will get unique indexes working out
of the box.
- The system will encourage keeping deterministic encryption stable:
  - By always using oldest encryption schemes
  - By forbidding configuring multiple keys

But you can still opt-out of the default if you need to.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
7a1fb99302 Add support to declare previous encryption schemes globally 2021-04-01 15:02:15 +02:00
Jorge Manrubia
28145c3cee Rename master_key => primary_key 2021-04-01 15:02:15 +02:00
Jorge Manrubia
c275b10a0e Show performance tests results now that they are excluded from the build 2021-04-01 15:02:15 +02:00
Jorge Manrubia
c41b354bf0 Remove mass encryption
I don't think this belong to the library yet. It's more like an util class we used
for building some mass-encryption tasks in HEY.
2021-04-01 15:02:15 +02:00
Jorge Manrubia
26ceb126b9 Override accessors with module
This way users can override them and call #super

https://github.com/rails/rails/pull/41659#discussion_r592624914
2021-04-01 15:02:15 +02:00
Jorge Manrubia
c0fa9428e5 Fix test 2021-04-01 15:02:14 +02:00
Jorge Manrubia
9bfca80641 Only run limit-validation tests when the limit exists
SQLite and PostgreSQL won't add a default limit of 255 for string types
2021-04-01 15:02:14 +02:00
Jorge Manrubia
826ec6a473 Rename previous_types => previous_encrypted_types
This makes it clear what they are. It was confusing with "cast_type" around.
2021-04-01 15:02:14 +02:00
Jorge Manrubia
36c4787469 Move encryption performance tests out of the main tests pipeline
This adds new rake tasks for these tests. This way, it prevents these
tests from making the build fail.

https://github.com/rails/rails/pull/41659#discussion_r592551649
2021-04-01 15:02:14 +02:00
Jorge Manrubia
4461ba1e27 Test fixes 2021-04-01 15:02:14 +02:00
Jorge Manrubia
81afcabd19 Extract encrypted models to their own files
We are not encrypting attributes when loading models with the table
missing. This way we make sure we only load the encrypted models when
necessary during the encryption tests and prevent the problem of missing
encrypted attributes due to having cached the class without them encrypted.
2021-04-01 15:02:14 +02:00
Jorge Manrubia
1163154a0b Make performance helper silent by default 2021-04-01 15:02:14 +02:00
Jorge Manrubia
54b58ef796 Prevent being able to encrypt when in protected mode
You save encrypted ciphertexts in this scenario!
2021-04-01 15:02:14 +02:00
Jorge Manrubia
1406ac294f Remove new lines 2021-04-01 15:02:14 +02:00
Jorge Manrubia
0c2415f036 Tweak performance thresholds 2021-04-01 15:02:13 +02:00
Jorge Manrubia
87a2451d11 Fix helper reference in test 2021-04-01 15:02:13 +02:00
Jorge Manrubia
eb81a4ea3d Move encryption helper code to the general helper
If not, it will fail when processing encrypts: declaration
2021-04-01 15:02:13 +02:00
Jorge Manrubia
6b4c957470 Get rid of the nested context: option
Context settings can be passed as first level options when
declaring attributes now.
2021-04-01 15:02:13 +02:00
Jorge Manrubia
1ad8bf5303 We want to exclude sqlite not target mysql 2021-04-01 15:02:13 +02:00
Jorge Manrubia
83ed6058b7 Fix tests 2021-04-01 15:02:13 +02:00
Jorge Manrubia
79173314de Move tests to namespace 2021-04-01 15:02:13 +02:00
Jorge Manrubia
51aba3e0b3 Adjust performance thresholds 2021-04-01 15:02:13 +02:00
Jorge Manrubia
638a92f734 Initial extraction from active_record_encryption gem 2021-04-01 15:02:13 +02:00
Ryuta Kamizono
6ce14ee4bc Address intermittent CI failure due to non-determined sort order
https://buildkite.com/rails/rails/builds/76112#1aa6023d-f58e-4c3e-96cc-027e36f2f415/973-984
2021-03-30 02:18:53 +09:00
Ryuta Kamizono
a0097cd701 Add test case for class level strict_loading_mode 2021-03-28 10:49:15 +09:00
Rafael França
30ef29289b
Merge pull request #41760 from Shopify/register-task-precedence
Give precedence to the DatabaseTasks registered last
2021-03-25 17:28:46 -04:00
Jean Boussier
f35ca0faf0 Give precedence to the DatabaseTasks registered last 2021-03-25 13:08:37 +01:00
Petrik
1d574ae0d9 Revert "Prevent double save of cyclic associations"
Commit a1a5d37749964b1e1a23914ef13da327403e34cb doesn't properly set the
foreign key on associations. Instead of trying to patch the change
revert it for now, so we can investigate a better solution.

This reverts commit a1a5d37749964b1e1a23914ef13da327403e34cb

It also reverts the follow-up commits:

* Revert "Rename internal `@saving` state to `@_saving`"
  This reverts commit 2eb5458978f3f993ccc414b321b35fb1aef1efd2

* Revert "Add `_` prefix for the internal methods"
  This reverts commit 12c0bec15275fa458bc0ddd6b57f7a0ae7881bd5.

* Revert "protected :can_save?"
  This reverts commit 8cd3b657f8795aedb9bc97e725642cbd04dc09b8.

* Revert "Exclude #saving? from API docs"
  This reverts commit 35d3923ea0c51b3a628e913e3b35f85124de8ac8.
2021-03-25 10:46:07 +01:00
Svyatoslav Kryukov
cb95c1b14f
ActiveRecord: Optimize cache_key computation (#41741)
* Optimize cache_key computation

* After review

[Svyatoslav Kryukov + Rafael Mendonça França]
2021-03-24 22:42:34 -04:00
Dinah Shi
5ec1cac9f2 Add n_plus_one_only mode to Core#strict_loading!
Add an optional mode argument to
Core#strict_loading! to support n_plus_one_only
mode. Currently, when we turn on strict_loading
for a single record, it will raise even if we are
loading an association that is relatively safe to
lazy load like a belongs_to. This can be helpful
for some use cases, but prevents us from using
strict_loading to identify only problematic
instances of lazy loading.

The n_plus_one_only argument allows us to turn
strict_loading on for a single record, and only
raise when a N+1 query is likely to be executed.
When loading associations on a single record,
this only happens when we go through a has_many
association type. Note that the has_many
association itself is not problematic as it only
requires one query. We do this by turning
strict_loading on for each record that is loaded
through the has_many. This ensures that any
subsequent lazy loads on these records will raise
a StrictLoadingViolationError.

For example, where a developer belongs_to a ship
and each ship has_many parts, we expect the
following behaviour:

  developer.strict_loading!(mode: :n_plus_one_only)

  # Do not raise when a belongs_to association
  # (:ship) loads its has_many association (:parts)
  assert_nothing_raised do
    developer.ship.parts.to_a
  end

  refute developer.ship.strict_loading?
  assert developer.ship.parts.all?(&:strict_loading?)
  assert_raises ActiveRecord::StrictLoadingViolationError do
    developer.ship.parts.first.trinkets.to_a
  end
2021-03-24 20:55:18 -04:00
Ryuta Kamizono
2eb5458978 Rename internal @saving state to @_saving
Usually we add `_` prefix for newly added short term living (used)
internal state (e.g. ae02898, d1107f4, dcb8259), and also `@saving`
might be already used in users' code.
2021-03-25 03:08:06 +09:00
Rafael França
7a0e5afdea
Merge pull request #41745 from dylanahsmith/ar-class-inspection-filter
Move @inspection_filter to the class to avoid marshal dumping it
2021-03-24 01:05:41 -04:00
Dylan Thacker-Smith
097e4afcba Add an assertion to the regression test
Co-authored-by: Ryuta Kamizono <kamipo@gmail.com>
2021-03-23 21:41:36 -07:00
Ryuta Kamizono
12c0bec152 Add _ prefix for the internal methods
Add `_` prefix for the internal methods since `saving?`, `can_save?`,
and `saving` are too shorter naming for the internal methods, it can
easily be overridden by the user defined methods.

Also, I've removed `saving?` since the internal method is used only for
testing.
2021-03-24 13:39:42 +09:00
Ryuta Kamizono
8cd3b657f8 protected :can_save?
And tweak tests added at #41714.
2021-03-24 10:54:16 +09:00
Dylan Thacker-Smith
1e1fe3f7bb Move @inspection_filter to the class to avoid marshal dumping it
Previously, this was causing Marshal.dump to file to dump a record
with a Proc in the filter_attributes after inspect was called on the
record.
2021-03-23 16:48:58 -07:00
Petrik
a1a5d37749 Prevent double save of cyclic associations
Autosave will double save records for some cyclic associations.
For example a child record with a parent association.
If save is called on the child before the parent has been saved,
the child will be saved twice.
The double save of the child will clear the mutation tracker on
the child record resulting in an incorrect dirty state.

```
    # pirate has_one ship
    ship = Ship.new(name: "Nights Dirty Lightning")
    pirate = ship.build_pirate(catchphrase: "Aye")
    ship.save!
    ship.previous_changes # => returns {} but this should contain the changes.
```

When saving ship the following happens:
1. the `before_save` callbacks of the ship are called
2. the callbacks call `autosave_associated_records_for_pirate`
3. `autosave_associated_records_for_pirate` saves the pirate
4. the `after_save` callbacks of the pirate are called
5. the callbacks call `autosave_associated_records_for_ship`
6. `autosave_associated_records_for_ship` saves the ship
7. the ship is saved again by the original save

`autosave_associated_records_for_ship` saves the ship because the ship
association is set by inverse_of in a `has_one` on the pirate. This does
not happen with a `has_many` by default because the inverse is not set.
If setting the inverse on the `has_many` the problem occurs as well.

----------------------------

This commit adds a @saving state which tracks if a record is currently being saved.
If @saving is set to true, the record won't be saved by the autosave callbacks.

With this commit the following happens when saving a ship:
1. @saving is set to true
2. the `before_save` callbacks of the ship are called
3. the callbacks call `autosave_associated_records_for_pirate`
5. `autosave_associated_records_for_pirate` saves the pirate
6. the `after_save` callbacks of the pirate are called
6. `autosave_associated_records_for_ship` skip saving the ship
8. the ship is saved.
9. @saving is set to false

One disadvantage of this approach is the following...
While the child is no longer saved in the autosave, similar children
could be autosaved. This will result in unexpected order when creating
new records, as similar children will be commited first.
2021-03-23 22:01:27 +01:00
Ryuta Kamizono
f8dba9e64a Fix WhereClause#extract_attributes to work it with a string where clause
Fixes #41696.
2021-03-23 10:47:10 +09:00
Rafael Mendonça França
21e0c20667
Batch queries that are the same but are using different extension
If an association scope would generate the same query but have different
values for `extending`, `skip_query_cache` or `strict_loading` we should
consider them the same for the purpose of generating the preload
batches.
2021-03-22 21:51:18 +00:00
Ryuta Kamizono
eb6a43a798 Fix the test case for #14855 to catch a future regression correctly
I've found the test doesn't catch a future regression since it doesn't
assert the result of the preload.

Also, add extra assertions to assert the difference of the value type.
2021-03-23 04:55:33 +09:00
Rafael Mendonça França
6388e300e5
Revert "Fix schema for members"
This reverts commit 8da6ba9cae21beae1ee3c379db7b7113d2731c9b.

The type is not standard to make sure preload work when the types don't
match.

See #14855.
2021-03-22 19:45:09 +00:00
Ryuta Kamizono
7c67b94986 Make infinity handling symmetrical in cast and deserialize
Related: #41716, 30391e9ddba745d6bdc0b23f526ecf432dfe6adf.
2021-03-22 15:47:20 +09:00
Ryuta Kamizono
bbac68d428
Merge pull request #41716 from shunichi/fix-postgresql-infinity-datetime
Fix Float::INFINITY assignment to datetime attributes
2021-03-22 15:16:07 +09:00
Shunichi Ikegami
800d6bd1df Fix Float::INFINITY assignment to datetime attributes
After assigning string "infinity" to datetime attribute with postgresql adapter, reading it back gets Float::INFINITY.
But assigning Float::INFINITY to datetime attribute results in getting nil value when ActiveRecord::Base.time_zone_aware_attributes is true.
This is due to TimeZoneConverter not handling Float::INFINITY appropriately.
2021-03-22 13:56:06 +09:00