To use a binary-encoded string as a byte buffer, appended strings should
be force-encoded as binary. Otherwise, appending a non-ASCII-only
string will raise `Encoding::CompatibilityError`.
Fixes#48748.
This help treating caches entries as expandable.
Because Marshal will hapily serialize almost everything, It's not uncommon to
inadvertently cache a class that's not particularly stable, and cause deserialization
errors on rollout when the implementation changes.
E.g. https://github.com/sporkmonger/addressable/pull/508
With this change, in case of such event, the hit rate will suffer for a
bit, but the application won't return 500s responses.
`activesupport/lib/active_support/cache.rb` is already rather long, and
having a separate file allows subclasses of `Cache::Entry` to be defined
without first requiring `activesupport/lib/active_support/cache.rb`.
`NullCoder` is functionally equivalent to the `:passthrough` coder.
Additionally, `NullCoder` was added in c4845aa7791839fcdf723dc77e3df258e7274496
to serve as the default coder for `MemCacheStore`, but `MemCacheStore`
now uses `:passthrough` as its (legacy) default coder.
While working on fixing some deprecation warnings introduced when the
6.1 cache_format deprecation was [moved][1] to be on usage, I found that
the MemCacheStore actually has its own `default_coder` method.
This adds the warning to MemCacheStore's `default_coder` method so that
every cache store will warn on using the 6.1 format.
[1]: 088551c802bb4005a667aaa33814cfbb1feb3927
An issue was recently opened with the following error message:
```
.../activesupport-7.0.6/lib/active_support/evented_file_update_checker.rb:120:in `start': undefined method `wait_for_state' for #<Listen::Listener ...>
```
The issue is that the user was using Listen 3.0.8, however the
`wait_for_state` method was [added][1] in Listen 3.3.0
We can make the error message a little better by defining a lower bound
on Listen 3.3.0 (like we do for other optional dependencies):
```
.../bundler/rubygems_integration.rb:280:in `block (2 levels) in replace_gem': can't activate listen (~> 3.3), already activated listen-3.0.8. Make sure all dependencies are added to Gemfile. (Gem::LoadError)
```
There is definitely still room for improvement here, but this should
be much more helpful in figuring out that the issue is a bad Listen
version and not a bug in Rails.
[1]: 12b4fc54a9
Previously jsonify would call `.as_json` for Integer, nil, true, and
false, even though those types are considered "JSON-ready". Technically
a user could have overridden `.as_json` for these types but I can't
imagine a use case and I don't think we should support that.
I left the same behaviour of calling `.as_json` for generic "Numeric" as
that can have user subclasses where one may have implemented as_json.
This behaviour is also used for Float (which coerces
NaN/Infinity/-Infinity into nil).
This also adds Symbol to the list of "JSON-ready" types, to avoid
unnecessarily casting them to strings (possible as we no longer perform
escaping on input). The output of jsonify should never be user visible
before it is passed through JSON.generate, so I don't think this should
be a user facing
change.
This also corrects our handling of Hash to call to_s on all keys,
matching the behaviour of `.as_json` and JSON's requirement that keys
are Strings (Symbols are also permitted as JSON knows to convert them to
a String).
Rails performs additional escaping of strings compared to the JSON gem,
and will escape U+2028, U+2029, <, >, and &.
In JSON, the only places those characters are valid is inside of a
JSON string, so it should be equivalent (and faster) to perform the
escaping on the output.
JSONGemEncoder.encode previously would always perform two passes. First
it would call `.as_json(options)`, but then would perform a second pass
"jsonify" to recursively call `.as_json` (this time without options)
until the data converges into a "JSON-ready" representation.
When options are not given, the second pass should be equivalent to the
first, so we can detect that, and only perform the "jsonify" step.
The only user-visible effect of this should be that we will pass no
options to `as_json` instead of an empty Hash, but implementations of
`as_json` should already be expected to accept that.
A deprecation warning was [added][1] to ensure that applications
manually setting `config.active_support.cache_format_version` to `6.1`
will be aware that they need to migrate. However, if an app is not using
a `config.load_defaults` of `7.0` or greater, this warning will never be
triggered.
This commit moves the deprecation warning to where the `cache_format`
value gets used to cover both cases.
[1]: 2ba3ac29c36f4c6d23b1dd302584f15739ff2aff
Previously, `EncryptedConfiguration` was [updated][1] to use
`InheritableOptions` so that keys could be called like methods. It was
later [updated again][2] to ensure that it behaved both like a `Hash` and
`OrderedOptions`. In this second change, the `InheritableOptions`
instance was accidentally nested with another `InheritableOptions`
instance.
This continued to mostly work as expected because `InheritableOptions`
will fall back to the inner `InheritableOptions` when the outer one
doesn't have a key. However, any methods that try to treat the outer
`InheritableOptions` like it should know about all of its keys will fail
(for example, `#keys`, `#to_h`, `#to_json`, etc.)
This commit fixes the issue by removing the extraneous outer
`InheritableOptions` instance.
[1]: a6a9fed1719a3cfe47eb1566ae4d6034374fd809
[2]: 80585daf2def5bf94854be69f81e24a16ce14c55
These methods are only available when requiring the railties
"rails/test_help" file, which is included by default in the
test_helper.rb for newly generated Rails applications.
Since requiring that file is the only way, and some applications may
have removed it or not used test-unit when generating their application,
I think it's worth calling out explicitly here.
If anyone calls a message encryptor in the console it will
show the secret of the encryptor.
By overriding the `inspect` method to only show the class name we can
avoid accidentally outputting sensitive information.
Before:
```ruby
ActiveSupport::MessageEncryptor.new(secret, cipher: "aes-256-gcm").inspect
"#<ActiveSupport::MessageEncryptor:0x0000000104888038 ... @secret=\"\\xAF\\bFh]LV}q\\nl\\xB2U\\xB3 ... >"
```
After:
```ruby
ActiveSupport::MessageEncryptor.new(secret, cipher: "aes-256-gcm").inspect
"#<ActiveSupport::MessageEncryptor:0x0000000104888038>"
```
If anyone calls `Rails.application.credentials` in the console it will
show the unencrypted contents of the credentials.
By overriding the `inspect` method to only show the class name we can
avoid accidentally outputting sensitive information.
Before:
```ruby
Rails.application.credentials.inspect
"#<ActiveSupport::EncryptedConfiguration:0x000000010d2b38e8 ... @config={:secret=>\"something secret\"} ... @key_file_contents=\"915e4ea054e011022398dc242\" ...>"
```
After:
```ruby
Rails.application.credentials.inspect
"#<ActiveSupport::EncryptedConfiguration:0x000000010d2b38e8>"
```
In #48104, `:message_pack` was added as a supported value for the cache
format version because the format version essentially specifies the
default coder. However, as an API, the format version could potentially
be used to affect other aspects of serialization, such as the default
compression format.
This commit removes `:message_pack` as a supported value for the format
version, and, as a replacement, adds support for specifying
`coder: :message_pack`:
```ruby
# BEFORE
config.active_support.cache_format_version = :message_pack
# AFTER
config.cache_store = :redis_cache_store, { coder: :message_pack }
```
This changed `many?` to yield with `element, *args`. The causes elements that are arrays to be destructured which can lead to defective behaviour. Since the introduction of `element, *args` was for readability purposes, we can just use `*args` instead and get the same behaviour without the defect.
Followup: https://github.com/rails/rails/pull/47354
It does a bit more than just giving you a `.instance` method
it also change the behavior of dup and clone, we don't need
any of that, and `.instance` is deprecated anyway.
Fix: https://github.com/rails/rails/issues/48352
While we should ensure instantiating the store doesn't immediately
attempt to connect, we should eagerly process arguments so that
if they are somehow invalid we fail early during boot rather than at
runtime.
Additionally, since it's common to get pool parameters from environment
variable, we can use `Integer` and `Float` so that string representations
are valid.
Prior to this commit, `MemoryStore::DupCoder` called `Entry#dup_value!`
in both `dump` and `load` to guard against external mutation of cached
values. `Entry#dup_value!` calls `Marshal.dump` then `Marshal.load` to
dup complex objects. However, to prevent external mutation, we only
need to call `Marshal.dump` once for `DupCoder.dump` and `Marshal.load`
once for `DupCoder.load`.
This commit changes `DupCoder` to call `Marshal.dump` and `Marshal.load`
directly instead of relying on `Entry#dup_value!`, thus halving the work
done by `Marshal` when writing and reading complex objects.
__Benchmark__
```ruby
# frozen_string_literal: true
require "benchmark/ips"
require "active_support"
require "active_support/cache"
LARGE_OBJECT = 100.times.map { "x" * 100 }
LARGE_STRING = LARGE_OBJECT.join
cache = ActiveSupport::Cache.lookup_store(:memory_store)
Benchmark.ips do |x|
x.report("large string") do
cache.write("x", LARGE_STRING)
cache.read("x")
end
x.report("large object") do
cache.write("x", LARGE_OBJECT)
cache.read("x")
end
end
```
__Before__
```
Warming up --------------------------------------
large string 2.667k i/100ms
large object 332.000 i/100ms
Calculating -------------------------------------
large string 26.539k (± 1.8%) i/s - 133.350k in 5.026373s
large object 3.336k (± 0.9%) i/s - 16.932k in 5.076458s
```
__After__
```
Warming up --------------------------------------
large string 2.541k i/100ms
large object 715.000 i/100ms
Calculating -------------------------------------
large string 25.117k (± 1.7%) i/s - 127.050k in 5.059945s
large object 7.111k (± 1.2%) i/s - 35.750k in 5.028267s
```
Closes#46403.
Co-authored-by: Breno Gazzola <breno.gazzola@gmail.com>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
This method previously maintained the `html_safe?` property of a string on the return
value. Because this string has been escaped, however, not marking it as `html_safe` causes
entities to be double-escaped.
As an example, take this view snippet:
```html
<p><%= html_escape_once("this & that & the other") %></p>
```
Before this change, that would be double-escaped and render as:
```html
<p>this &amp; that &amp; the other</p>
```
After this change, it renders correctly as:
```html
<p>this & that & the other</p>
```
[Fix#48256]
Most of these are redundant because rdoc handles these itself, but
`titlecase` on `ActiveSupport::Inflector` does not exist so that one is
just incorrect.
Because `ActiveSupport::MessagePack::Serializer::SIGNATURE` includes a
non-ASCII-only byte (`"\xCC"`), it raises `Encoding::CompatibilityError`
when compared with another string that is not encoded with
`Encoding::BINARY` and also includes a non-ASCII-only byte.
To prevent that, this commit changes `AS::MessagePack#signature?` to
directly compare the first two bytes of both strings.
Fixes#48196.
Updated the ActiveSupport::InheritableOptions documentation to exemplify
how to use a parent hash that contains string keys.
parent = { "foo" => true }
child = ActiveSupport::InheritableOptions.new(parent.symbolize_keys)
child.foo # => true
This example is helpful because ActiveSupport::OrderedOptions only deals
with symbol keys and this implementation detail is hidden from the API
docs.
Co-authored-by: Jonathan Hefner <jonathan@hefner.pro>
This commit adds support for `:message_pack` as a serializer for
`MessageEncryptor` and `MessageVerifier`, and, consequently, as an
option for `config.active_support.message_serializer`.
The `:message_pack` serializer is implemented via
`ActiveSupport::Messages::SerializerWithFallback` and can fall back to
deserializing with `AS::JSON`. Additionally, the `:marshal`, `:json`,
and `:json_allow_marshal` serializers can now fall back to deserializing
with `AS::MessagePack`.
This commit also adds support for `:message_pack_allow_marshal` as a
serializer, which can fall back to deserializing with `Marshal` as well
as `AS::JSON`.
Prior to this commit, `config.load_defaults 7.1` would cause old
messages (or messages from older apps) to become unreadable, and
developers were encouraged to manually set
`config.active_support.message_serializer = :json_allow_marshal` in
order to prevent this.
This commit changes the default message serializer set by
`config.load_defaults 7.1` from `:json` to `:json_allow_marshal` so that
upgraded apps can continue to read old messages without additional
configuration.
The intention is to eventually change the default to `:json` (with no
`Marshal` fallback) in Rails 7.2, after some time has passed with apps
generating JSON-serialized messages.
Apps can opt in to JSON-only serialization before Rails 7.2 by manually
setting `config.active_support.message_serializer = :json`.
Fixes#48118.
In #42843 and #42846, several config settings were added to control the
default serializer for `MessageEncryptor` and `MessageVerifier`, and to
provide a migration path from a default `Marshal` serializer to a
default `JSON` serializer:
* `config.active_support.default_message_encryptor_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.default_message_verifier_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.fallback_to_marshal_deserialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
* `config.active_support.use_marshal_serialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
This commit unifies those config settings into a single setting,
`config.active_support.message_serializer`, which supports `:marshal`,
`:json_allow_marshal`, and `:json` values. So, for example,
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = true
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json_allow_marshal
```
and
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json
```
This commit also replaces `ActiveSupport::JsonWithMarshalFallback` with
`ActiveSupport::Messages::SerializerWithFallback`, which implements a
generic mechanism for serializer fallback. The `:marshal` serializer
uses this mechanism too, so
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = true
```
becomes
```ruby
config.active_support.message_serializer = :marshal
```
Additionally, the logging behavior of `JsonWithMarshalFallback` has been
replaced with notifications which include the names of the intended and
actual serializers, as well as the serialized and deserialized message
data. This provides a more targeted means of tracking serializer
fallback events. It also allows the user to "silence" such events, if
desired, without an additional config setting.
All of these changes make it easier to add migration paths for new
serializers such as `ActiveSupport::MessagePack`.
