When we're editing the contents of encrypted files, we should use the
`Tempfile` class because it creates temporary files with restrictive
permissions. This prevents other users on the same system from reading
the contents of those files while the user is editing them.
[CVE-2023-38037]
In #42843 and #42846, several config settings were added to control the
default serializer for `MessageEncryptor` and `MessageVerifier`, and to
provide a migration path from a default `Marshal` serializer to a
default `JSON` serializer:
* `config.active_support.default_message_encryptor_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.default_message_verifier_serializer`
* Supports `:marshal`, `:hybrid`, or `:json`.
* `config.active_support.fallback_to_marshal_deserialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
* `config.active_support.use_marshal_serialization`
* Affects `:hybrid` for both `MessageEncryptor` and `MessageVerifier`.
This commit unifies those config settings into a single setting,
`config.active_support.message_serializer`, which supports `:marshal`,
`:json_allow_marshal`, and `:json` values. So, for example,
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = true
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json_allow_marshal
```
and
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = false
```
becomes
```ruby
config.active_support.message_serializer = :json
```
This commit also replaces `ActiveSupport::JsonWithMarshalFallback` with
`ActiveSupport::Messages::SerializerWithFallback`, which implements a
generic mechanism for serializer fallback. The `:marshal` serializer
uses this mechanism too, so
```ruby
config.active_support.default_message_encryptor_serializer = :hybrid
config.active_support.default_message_verifier_serializer = :hybrid
config.active_support.fallback_to_marshal_deserialization = false
config.active_support.use_marshal_serialization = true
```
becomes
```ruby
config.active_support.message_serializer = :marshal
```
Additionally, the logging behavior of `JsonWithMarshalFallback` has been
replaced with notifications which include the names of the intended and
actual serializers, as well as the serialized and deserialized message
data. This provides a more targeted means of tracking serializer
fallback events. It also allows the user to "silence" such events, if
desired, without an additional config setting.
All of these changes make it easier to add migration paths for new
serializers such as `ActiveSupport::MessagePack`.
Follow-up to #42846.
Prior to this commit, `ActiveSupport::EncryptedFile` used the
`ActiveSupport::MessageEncryptor` default serializer. #42846 changed
the default serializer to `JSON` when using `config.load_defaults 7.1`.
Thus, encrypted files that were written with the previous default
serializer (`Marshal`) could not be read by `Rails.application.encrypted`.
That included files which were written with `bin/rails encrypted:edit`
even when using the new default, because the `bin/rails encrypted:edit`
command does not run the initializer that applies the configuration to
`MessageEncryptor`:
```console
$ bin/rails r 'puts ActiveSupport::MessageEncryptor.default_message_encryptor_serializer'
json
$ bin/rails encrypted:edit config/my_encrypted.yml.enc
...
$ bin/rails encrypted:show config/my_encrypted.yml.enc
foo: bar
$ bin/rails r 'Rails.application.encrypted("config/my_encrypted.yml.enc").read'
rails/activesupport/lib/active_support/message_encryptor.rb:241:in `rescue in _decrypt': ActiveSupport::MessageEncryptor::InvalidMessage (ActiveSupport::MessageEncryptor::InvalidMessage)
...
ruby-3.2.0/lib/ruby/3.2.0/json/common.rb:216:in `parse': unexpected token at "foo: bar (JSON::ParserError)
```
This commit hard codes the serializer for `EncryptedFile` to `Marshal`.
Prior to this commit, if `config.require_master_key` was set to true in
`config/application.rb`, the `credentials:edit` command could not
automatically generate a master key file:
```ruby
# In config/application.rb
config.require_master_key = true
```
```console
# No previously existing credentials
$ rm config/master.key config/credentials.yml.enc
```
```console
$ bin/rails credentials:edit
activesupport/lib/active_support/encrypted_file.rb:118:in `handle_missing_key': Missing encryption key to decrypt file with. Ask your team for your master key and write it to config/master.key or put it in the ENV['RAILS_MASTER_KEY']. (ActiveSupport::EncryptedFile::MissingKeyError)
```
This commit adds a `EncryptedFile#key?` method that can be used to check
for the existence of a key without raising `MissingKeyError`, and the
`credentials:edit` command now uses this method:
```console
$ bin/rails credentials:edit
Adding config/master.key to store the encryption key: ...
Save this in a password manager your team can access.
If you lose the key, no one, including you, can access anything encrypted with it.
create config/master.key
```
This commit also applies the same fix to the `encrypted:edit` command.
Prior to this commit, `EncryptedFile` would internally memoize when a
key file was non-existent. This meant that a key file generated after
checking `encrypted_file.key.nil?` would not be recognized, unless a new
`EncryptedFile` instance was used. (Though setting the key in a
designated environment variable instead would entirely bypass this
memoization.)
This commit changes `EncryptedFile` to only memoize when the key file
has been read. Consequently, this commit memoizes the `EncryptedFile`
(or, specifically, `EncryptedConfiguration`) instance used by
`CredentialsCommand`.
This commit also adds more test coverage around key file generation for
`CredentialsCommand`.
When given a key of invalid length, OpenSSL::Cipher raises a "key must
be X bytes" error. However, EncryptedFile keys are packed before they
are passed to OpenSSL::Cipher, so the actual length requirement is "2*X
characters".
This commit checks the key length, and raises a more helpful error if
the key length is invalid.
Closes#39528.
When editing credentials.yml.enc after creating a symbolic link,
the symbolic link is removed and overwritten with a normal file.
This commit changes the behavior to keep the symbolic link.
In case of using a dead symbolic link that link is revived after writing.
Fixes#36411
* WIP: Add credentials using a generic EncryptedConfiguration class
This is sketch code so far.
* Flesh out EncryptedConfiguration and test it
* Better name
* Add command and generator for credentials
* Use the Pathnames
* Extract EncryptedFile from EncryptedConfiguration and add serializers
* Test EncryptedFile
* Extract serializer validation
* Stress the point about losing comments
* Allow encrypted configuration to be read without parsing for display
* Use credentials by default and base them on the master key
* Derive secret_key_base in test/dev, source it from credentials in other envs
And document the usage.
* Document the new credentials setup
* Stop generating the secrets.yml file now that we have credentials
* Document what we should have instead
Still need to make it happen, tho.
* [ci skip] Keep wording to `key base`; prefer defaults.
Usually we say we change defaults, not "spec" out a release.
Can't use backticks in our sdoc generated documentation either.
* Abstract away OpenSSL; prefer MessageEncryptor.
* Spare needless new when raising.
* Encrypted file test shouldn't depend on subclass.
* [ci skip] Some woordings.
* Ditch serializer future coding.
* I said flip it. Flip it good.
* [ci skip] Move require_master_key to the real production.rb.
* Add require_master_key to abort the boot process.
In case the master key is required in a certain environment
we should inspect that the key is there and abort if it isn't.
* Print missing key message and exit immediately.
Spares us a lengthy backtrace and prevents further execution.
I've verified the behavior in a test app, but couldn't figure the
test out as loading the app just exits immediately with:
```
/Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `load': marshal data too short (ArgumentError)
from /Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `run'
from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest.rb:830:in `run_one_method'
from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest/parallel.rb:32:in `block (2 levels) in start'
```
It's likely we need to capture and prevent the exit somehow.
Kernel.stub(:exit) didn't work. Leaving it for tomorrow.
* Fix require_master_key config test.
Loading the app would trigger the `exit 1` per require_master_key's
semantics, which then aborted the test.
Fork and wait for the child process to finish, then inspect the
exit status.
Also check we aborted because of a missing master key, so something
else didn't just abort the boot.
Much <3 to @tenderlove for the tip.
* Support reading/writing configs via methods.
* Skip needless deep symbolizing.
* Remove save; test config reader elsewhere.
* Move secret_key_base check to when we're reading it.
Otherwise we'll abort too soon since we don't assign the secret_key_base
to secrets anymore.
* Add missing string literal comments; require unneeded yaml require.
* ya ya ya, rubocop.
* Add master_key/credentials after bundle.
Then we can reuse the existing message on `rails new bc4`.
It'll look like:
```
Using web-console 3.5.1 from https://github.com/rails/web-console.git (at master@ce985eb)
Using rails 5.2.0.alpha from source at `/Users/kasperhansen/Documents/code/rails`
Using sass-rails 5.0.6
Bundle complete! 16 Gemfile dependencies, 72 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Adding config/master.key to store the master encryption key: 97070158c44b4675b876373a6bc9d5a0
Save this in a password manager your team can access.
If you lose the key, no one, including you, can access anything encrypted with it.
create config/master.key
```
And that'll be executed even if `--skip-bundle` was passed.
* Ensure test app has secret_key_base.
* Assign secret_key_base to app or omit.
* Merge noise
* Split options for dynamic delegation into its own method and use deep symbols to make it work
* Update error to point to credentials instead
* Appease Rubocop
* Validate secret_key_base when reading it.
Instead of relying on the validation in key_generator move that into
secret_key_base itself.
* Fix generator and secrets test.
Manually add config.read_encrypted_secrets since it's not there by default
anymore.
Move mentions of config/secrets.yml to config/credentials.yml.enc.
* Remove files I have no idea how they got here.
* [ci skip] swap secrets for credentials.
* [ci skip] And now, changelogs are coming.
