In #42843 and #42846, several config settings were added to control the
default serializer for `MessageEncryptor` and `MessageVerifier`, and to
provide a migration path from a default `Marshal` serializer to a
default `JSON` serializer:
* `config.active_support.default_message_encryptor_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.default_message_verifier_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.fallback_to_marshal_deserialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
* `config.active_support.use_marshal_serialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
This commit unifies those config settings into a single setting,
`config.active_support.message_serializer`, which supports `:marshal`,
`:json_allow_marshal`, and `:json` values. So, for example,
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = true
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json_allow_marshal
```
and
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json
```
This commit also replaces `ActiveSupport::JsonWithMarshalFallback` with
`ActiveSupport::Messages::SerializerWithFallback`, which implements a
generic mechanism for serializer fallback. The `:marshal` serializer
uses this mechanism too, so
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = true
```
becomes
```ruby
config.active_support.message_serializer = :marshal
```
Additionally, the logging behavior of `JsonWithMarshalFallback` has been
replaced with notifications which include the names of the intended and
actual serializers, as well as the serialized and deserialized message
data. This provides a more targeted means of tracking serializer
fallback events. It also allows the user to "silence" such events, if
desired, without an additional config setting.
All of these changes make it easier to add migration paths for new
serializers such as `ActiveSupport::MessagePack`.
There are multiple points of failure when processing a message with
`MessageEncryptor` or `MessageVerifier`, and there several ways we might
want to handle those failures. For example, swallowing a failure with
`MessageVerifier#verified`, or raising a specific exception with
`MessageVerifier#verify`, or conditionally ignoring a failure when
rotations are configured.
Prior to this commit, the _internal_ logic of handling failures was
implemented using a mix of `nil` return values and raised exceptions.
This commit reimplements the internal logic using `throw` and a few
precisely targeted `rescue`s. This accomplishes several things:
* Allow rotation of serializers for `MessageVerifier`. Previously,
errors from a `MessageVerifier`'s initial serializer were never
rescued. Thus, the serializer could not be rotated:
```ruby
old_verifier = ActiveSupport::MessageVerifier.new("secret", serializer: Marshal)
new_verifier = ActiveSupport::MessageVerifier.new("secret", serializer: JSON)
new_verifier.rotate(serializer: Marshal)
message = old_verifier.generate("message")
new_verifier.verify(message)
# BEFORE:
# => raises JSON::ParserError
# AFTER:
# => "message"
```
* Allow rotation of serializers for `MessageEncryptor` when using a
non-standard initial serializer. Similar to `MessageVerifier`, the
serializer could not be rotated when the initial serializer raised an
error other than `TypeError` or `JSON::ParserError`, such as
`Psych::SyntaxError` or a custom error.
* Raise `MessageEncryptor::InvalidMessage` from `decrypt_and_verify`
regardless of cipher. Previously, when a `MessageEncryptor` was using
a non-AEAD cipher such as AES-256-CBC, a corrupt or tampered message
would raise `MessageVerifier::InvalidSignature` due to reliance on
`MessageVerifier` for verification. Now, the verification mechanism
is transparent to the user:
```ruby
encryptor = ActiveSupport::MessageEncryptor.new("x" * 32, cipher: "aes-256-gcm")
message = encryptor.encrypt_and_sign("message")
encryptor.decrypt_and_verify(message.next)
# => raises ActiveSupport::MessageEncryptor::InvalidMessage
encryptor = ActiveSupport::MessageEncryptor.new("x" * 32, cipher: "aes-256-cbc")
message = encryptor.encrypt_and_sign("message")
encryptor.decrypt_and_verify(message.next)
# BEFORE:
# => raises ActiveSupport::MessageVerifier::InvalidSignature
# AFTER:
# => raises ActiveSupport::MessageEncryptor::InvalidMessage
```
* Support `nil` original value when using `MessageVerifier#verify`.
Previously, `MessageVerifier#verify` did not work with `nil` original
values, though both `MessageVerifier#verified` and
`MessageEncryptor#decrypt_and_verify` do:
```ruby
encryptor = ActiveSupport::MessageEncryptor.new("x" * 32)
message = encryptor.encrypt_and_sign(nil)
encryptor.decrypt_and_verify(message)
# => nil
verifier = ActiveSupport::MessageVerifier.new("secret")
message = verifier.generate(nil)
verifier.verified(message)
# => nil
verifier.verify(message)
# BEFORE:
# => raises ActiveSupport::MessageVerifier::InvalidSignature
# AFTER:
# => nil
```
* Improve performance of verifying a message when it has expired and one
or more rotations have been configured:
```ruby
# frozen_string_literal: true
require "benchmark/ips"
require "active_support/all"
verifier = ActiveSupport::MessageVerifier.new("new secret")
verifier.rotate("old secret")
message = verifier.generate({ "data" => "x" * 100 }, expires_at: 1.day.ago)
Benchmark.ips do |x|
x.report("expired message") do
verifier.verified(message)
end
end
```
__Before__
```
Warming up --------------------------------------
expired message 1.442k i/100ms
Calculating -------------------------------------
expired message 14.403k (± 1.7%) i/s - 72.100k in 5.007382s
```
__After__
```
Warming up --------------------------------------
expired message 1.995k i/100ms
Calculating -------------------------------------
expired message 19.992k (± 2.0%) i/s - 101.745k in 5.091421s
```
Fixes#47185.
This commit factors tests related to `ActiveSupport::Messages::Rotator`
into `MessageEncryptorRotatorTest` and `MessageVerifierRotatorTest`
classes, with a shared `MessageRotatorTests` module. In particular,
this:
* Replaces the `test_rotating_serializer` test in the
`MessageEncryptorTest` class because it did not actually rotate the
serializer.
* Removes the `test_on_rotation_is_called_and_returns_modified_messages`
test from the `MessageEncryptorWithJsonSerializerTest` class because
its only purpose is to override a test of the same name in the
`MessageEncryptorTest` parent class, because the expected value must
use string keys instead of symbol keys due to the serializer.
* Removes the `test_on_rotation_can_be_passed_at_the_constructor_level`
test from the `MessageEncryptorWithJsonSerializerTest` class for the
same reason as above.
* Removes the `test_on_rotation_option_takes_precedence_over_the_one_given_in_constructor`
test from the `MessageEncryptorWithJsonSerializerTest` class for the
same reason as above.
* Removes the `test_on_rotation_is_called_and_verified_returns_message`
test from the `JsonSerializeMarshalFallbackMessageVerifierTest` class
because it is rotating the secret and digest, and is not specific to
the serializer.
* Removes the `test_on_rotation_is_called_and_verified_returns_message`
test from the `DefaultMarshalSerializerMessageVerifierTest` class for
the same reason as above.
* Adds thorough test coverage for the `on_rotation` option for both
`MessageVerifier` and `MessageEncryptor` (previous test coverage was
mostly just for `MessageEncryptor`), including coverage for when the
`on_rotation` proc should _not_ be called.
* Adds test coverage for rotating the `url_safe` option, for both
`MessageVerifier` and `MessageEncryptor`.
Prior to this commit, there were several places to potentially add a new
message metadata test:
* `SharedMessageMetadataTests` module
* `MessageEncryptorMetadataTest` class
(includes `SharedMessageMetadataTests`)
* `MessageEncryptorMetadataMarshalTest` class
(subclasses `MessageEncryptorMetadataTest`)
* `MessageEncryptorMetadataJSONTest` class
(subclasses `MessageEncryptorMetadataTest`)
* `MessageEncryptorMetadataJsonWithMarshalFallbackTest` class
(subclasses `MessageEncryptorMetadataTest`)
* `MessageVerifierMetadataTest` class
(includes `SharedMessageMetadataTests`)
* `MessageVerifierMetadataMarshalTest` class
(subclasses `MessageVerifierMetadataTest`)
* `MessageVerifierMetadataJsonWithMarshalFallbackTest` class
(subclasses `MessageVerifierMetadataTest`)
* `MessageVerifierMetadataJsonTest` class
(subclasses `MessageVerifierMetadataTest`)
* `MessageVerifierMetadataCustomJSONTest` class
(subclasses `MessageVerifierMetadataTest`)
* `MessageEncryptorMetadataNullSerializerTest` class
(subclasses `MessageVerifierMetadataTest`)
This commit refactors the message metadata tests, reducing the list to:
* `MessageMetadataTests` module
* `MessageEncryptorMetadataTest` class (includes `MessageMetadataTests`)
* `MessageVerifierMetadataTest` class (includes `MessageMetadataTests`)
This makes it easier to add new tests, as well as new testing scenarios
(e.g. new encryptor / verifier configurations).
Additionally, this commit fixes two tests that were ineffective:
* The `test_passing_expires_in_less_than_a_second_is_not_expired` test
(which is now simply a part of the "message expires with :expires_in"
test) did not use the `with_usec: true` option. Therefore, the time
resulting from `travel 0.5.seconds` was truncated, effectively
traveling 0 seconds.
* The `test_verify_with_use_standard_json_time_format_as_false` test
(which is now named "expiration works with
ActiveSupport.use_standard_json_time_format = false") did not specify
an expiration time. Therefore, the conditional branch that it was
supposed to test was never exercised.
Although Ruby provides `Base64.urlsafe_encode64` and
`Base64.urlsafe_decode64` methods, the technical term is "URL-safe",
with "URL" and "safe" as separate words.
For better readability, this commit renames the `:urlsafe` option for
`MessageEncryptor` and `MessageVerifier` to `:url_safe`.
The `test_urlsafe` test could not fail when `urlsafe: false` because the
serialized input data did not contain a bit sequence that would encode
to a non-URL-safe character. The `test_no_padding` *could* fail when
`urlsafe: false`, except when the default serializer uses JSON, because
the serialized input data would be a multiple of 3 bytes, thus not
requiring any padding.
This commit replaces those tests with a falsifiable test using a
carefully chosen input string.
urlsafe option was introduced to MessageVerifier in
09c3f36a962a7ffd350dfda643d2f980734cb5c9 but it can generate strings
containing padding character ("=") which is not urlsafe.
Fix not to pad when base64 encode.
MessageVerifier uses Base64.strict_encode64 and generated strings are
not urlsafe. Though the goal is to make MessageVerifier generated
strings urlsafe, We can not simply switch to Base64.urlsafe_encode64
because it will be a breaking change. Thus, as a first step, urlsafe
option is added to the MessageVerifier initializer.
When `ActiveSupport.parse_json_times` is `true`,
`ActiveSupport::Messages::Medata.verify` will fail with a `TypeError` as
it does not pass a `String` to `Time.iso8601` as is expected.
This leads to hard-failure on retrieval of signed cookies with expiries.
Noticed that verifiers and encryptors never once mentioned key generators
and salts but only concerned themselves with generated secrets.
Clears up the confusing naming around raw_key and secret as well. And
makes the rotation API follow the constructor signature to the letter.
Both classes now have a rotate method where new instances are added for
each call. When decryption or verification fails the next rotation
instance is tried.
Without this, I get the following result on my machine
```
# Running:
F
Failure:
MessageVerifierTest#test_backward_compatibility_messages_signed_without_metadata [/home/deivid/Code/rails/activesupport/test/message_verifier_test.rb:91]:
--- expected
+++ actual
@@ -1 +1 @@
-{:some=>"data", :now=>2010-01-01 00:00:00 +0100}
+{:some=>"data", :now=>2010-01-01 00:00:00 +0000}
bin/test test/message_verifier_test.rb:89
```
Some `require 'openssl'` statements were surrounded by `rescue` blocks to deal with Ruby versions that did not support `OpenSSL::Digest::SHA1` or `OpenSSL::PKCS5`.
[As @jeremy explains](a6a0904fcb (commitcomment-8826666)) in the original commit:
> If jruby didn't have jruby-openssl gem, the require wouldn't work. Not sure whether either of these are still relevant today.
According to the [release notes for JRuby 1.7.13](http://www.jruby.org/2014/06/24/jruby-1-7-13.html):
> jruby-openssl 0.9.5 bundled
which means the above `rescue` block is not needed anymore.
All the Ruby versions supported by the current version of Rails provide those OpenSSL libraries, so Travis CI should also be happy by removing the `rescue` blocks.
---
Just to confirm, with JRuby:
$ ruby --version #=> jruby 1.7.16.1 (1.9.3p392) 2014-10-28 4e93f31 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_20-b26 +jit [darwin-x86_64]
$ irb
irb(main):001:0> require 'openssl' #=> true
irb(main):002:0> OpenSSL::Digest::SHA1 #=> OpenSSL::Digest::SHA1
irb(main):003:0> OpenSSL::PKCS5 # => OpenSSL::PKCS5
And with Ruby 2.1:
$ ruby --version #=> ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
$ irb
irb(main):001:0> require 'openssl' #=> true
irb(main):002:0> OpenSSL::Digest::SHA1 #=> OpenSSL::Digest::SHA1
irb(main):003:0> OpenSSL::PKCS5 #=> OpenSSL::PKCS5
This commit adds a `#verified` method to
`ActiveSupport::MessageVerifier` which will return either `false` when
it encounters an error or the message. `#verify` continues to raise an
`InvalidSignature` exception on error.
This commit also adds a convenience boolean method on `MessageVerifier`
as a way to check if a message is valid without performing the
decoding.
This broke natural order of things for `StaleSessionCheck#stale_session_check!` which tried auto_loading a class based on `ArgumentError` message , and later retrying the `Marshal#load` of class, successfully allowing auto_loading.
This PR tries to fix this behavior by forwarding `ArgumentError` 's not raised by `Base64.strict_decode64` , as is, ahead to `StaleSessionCheck#stale_session_check!`
