Commit Graph

8326 Commits

Author SHA1 Message Date
Alexey Savin
80f59eadb4 Make Enumerable.pluck faster for single key 2020-08-06 21:22:20 +03:00
Rajesh Sharma
4a552c3ab9 Fix option not being passed with fetch_multi
In redis cache store, options to `fetch_multi` are passed correctly to
`write_multi` but not to `read_multi`. This causes cache always to be missed
when passing `namespace` option to it.
2020-07-26 08:34:30 +02:00
maxgurewitz
73ba6faa36 prevents redundant memcached compression
- Disables Dalli compression in MemCacheStore.
- Fixes issue where redundant compression in Dalli can cause values to
either be compressed twice, or compressed when they fall below the
specified compression threshold.
2020-07-25 12:59:39 -07:00
maxgurewitz
f1fb097a74 prevents raw redis and memcached compression
- Fixes issue where reads with raw: true using redis or memcached cache
store, will compress values on reads.
- Should speed up raw cache reads by preventing unnecessary cpu intensive
operation.
2020-07-24 11:56:43 -07:00
Eugene Kenny
19d8578361 Add a regression test for per-fiber tagged logging
Followup to 978421abd83d45b72e299f50a4fb324fcbdb3e00.
2020-07-02 22:37:23 +01:00
Eugene Kenny
c3913e580c Revert "Cache tags_text to avoid computing tags each time when logging"
This reverts commit 05060ddba170ccb3ba9ce48555ed45400fe1ea96.

Tags are per-fiber, so they can't be cached in an instance variable.
2020-07-02 22:30:46 +01:00
Jean Boussier
b0cc7d985e Refactor MemoryStore to use Hash ordering rather than key access times
This is mainly to simplify the code and use less memory, as large hash can use quite a lot:

```ruby
>> ObjectSpace.memsize_of(1000.times.map { |i| [i, i]}.to_h)
=> 28768
>> ObjectSpace.memsize_of(10_000.times.map { |i| [i, i]}.to_h)
=> 458848
```

The performance is mostly not impacted, if not slightly better:

```ruby
require 'benchmark/ips'
require 'active_support/all'
@store = ActiveSupport::Cache::MemoryStore.new
@store.write("small", "small")
Benchmark.ips do |x|
  x.report("read:miss") { @store.read("miss") }
  x.report("read:small") { @store.read("small") }
  x.report("write:small") { @store.write("small", "small") }
end
```

6.0.3.2:
```
Warming up --------------------------------------
           read:miss    42.466k i/100ms
          read:small    25.315k i/100ms
         write:small    17.826k i/100ms
Calculating -------------------------------------
           read:miss    426.923k (± 1.9%) i/s -      2.166M in   5.074890s
          read:small    248.518k (± 2.7%) i/s -      1.266M in   5.097049s
         write:small    180.388k (± 1.6%) i/s -    909.126k in   5.041238s
```

This branch:
```
Warming up --------------------------------------
           read:miss    42.040k i/100ms
          read:small    28.364k i/100ms
         write:small    19.361k i/100ms
Calculating -------------------------------------
           read:miss    417.814k (± 2.1%) i/s -      2.102M in   5.033186s
          read:small    278.950k (± 2.8%) i/s -      1.418M in   5.088135s
         write:small    193.384k (± 1.8%) i/s -    968.050k in   5.007446s
```
2020-07-01 12:32:55 +02:00
Haroon Ahmed
d091b9ccee Improve the docs for ordered options 2020-06-30 22:08:59 +01:00
Jean Boussier
12f3f11f61 Use URI::DEFAULT_PARSER rather than instantiate a new one 2020-06-29 23:06:34 +02:00
Eugene Kenny
cf1881237a Remove unnecessary rescue from safe_constantize
The error being handled here was removed in
01c9782fa2392729bea08409e348fe049857d9d6.
2020-06-29 16:29:50 +01:00
Jean Boussier
8044d06b4d Use LoadError#original_message if available in safe_constantize 2020-06-28 19:29:57 +02:00
colorbox
3758274b3b Fix an error for FileStore.cleanup when using Sprockets
### Summary

This PR fixes `NoMethodError` for `ActiveSupport::Cache::FileStore.cleanup` when using [Sprockets](https://github.com/rails/sprockets).

`FileStore.cleanup` assumes entry object is a `Cache::Entry`.
An entry obejct is returned from `FileStore.read_entry` method.
If `FileStore.read_entry` returns object that cannot respond to `expired?` method, `FileStore.cleanup` will fail.

Sprockets generates cache file in tmp/cache/assets.
If `FileStore.read_entry` gets these Sprocket's cache file, this method creates entry object which cannot respond to `expired?` method.

In my project, this error occured and failed to execute `ActiveSupport::Cache::FileStore.cleanup`.

This PR adds a `is_a?` checking to entry object in `read_entry` method.
2020-06-25 23:58:30 +09:00
Yasuo Honda
fdbc55b9d5 Use Array(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json and Hash(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json for Ruby 2.8.0
This pull request addresses failures at https://buildkite.com/rails/rails/builds/70219#79d96882-6c51-4854-8cab-28f50ac8bca1
According to https://bugs.ruby-lang.org/issues/16973 This is an expected change in Ruby.
These failures has been addressed by changing the order of prepend as suggested.

```diff
% git diff
diff --git a/activesupport/test/json/encoding_test.rb b/activesupport/test/json/encoding_test.rb
index 30a3b8e5a0..1328041bf7 100644
--- a/activesupport/test/json/encoding_test.rb
+++ b/activesupport/test/json/encoding_test.rb
@@ -186,6 +186,8 @@ def test_hash_should_pass_encoding_options_to_children_in_to_json
         country: "UK"
       }
     }
+    p person.method(:to_json)
+    pp person.class.ancestors
     json = person.to_json only: [:address, :city]

     assert_equal(%({"address":{"city":"London"}}), json)
@@ -287,6 +289,8 @@ def test_array_to_json_should_not_keep_options_around
     f.bar = "world"

     array = [f, { "foo" => "other_foo", "test" => "other_test" }]
+    p array.method(:to_json)
+    pp array.class.ancestors
     assert_equal([{ "foo" => "hello", "bar" => "world" },
                   { "foo" => "other_foo", "test" => "other_test" }], ActiveSupport::JSON.decode(array.to_json))
   end
%
```

* Ruby 2.8.0 without this fix uses `Array(JSON::Ext::Generator::GeneratorMethods::Array)#to_json`, which should use `Array(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json`

```
% bin/test test/json/encoding_test.rb -n test_array_to_json_should_not_keep_options_around
Run options: -n test_array_to_json_should_not_keep_options_around --seed 33311

[Array,
 JSON::Ext::Generator::GeneratorMethods::Array,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Enumerable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 JSON::Ext::Generator::GeneratorMethods::Object,
 ActiveSupport::Tryable,
 Kernel,
 BasicObject]
F

Failure:
TestJSONEncoding#test_array_to_json_should_not_keep_options_around [/Users/yahonda/src/github.com/rails/rails/activesupport/test/json/encoding_test.rb:294]:
--- expected
+++ actual
@@ -1 +1 @@
-[{"foo"=>"hello", "bar"=>"world"}, {"foo"=>"other_foo", "test"=>"other_test"}]
+["#<TestJSONEncoding::CustomWithOptions:0xXXXXXX>", {"foo"=>"other_foo", "test"=>"other_test"}]

bin/test test/json/encoding_test.rb:286

Finished in 0.015486s, 64.5745 runs/s, 64.5745 assertions/s.
1 runs, 1 assertions, 1 failures, 0 errors, 0 skips
%
```

* Ruby 2.8.0 with this fix uses `Array(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json`

```
% bin/test test/json/encoding_test.rb -n test_array_to_json_should_not_keep_options_around
Run options: -n test_array_to_json_should_not_keep_options_around --seed 12193

[ActiveSupport::ToJsonWithActiveSupportEncoder,
 Array,
 JSON::Ext::Generator::GeneratorMethods::Array,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Enumerable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 JSON::Ext::Generator::GeneratorMethods::Object,
 ActiveSupport::Tryable,
 Kernel,
 BasicObject]
.

Finished in 0.008070s, 123.9157 runs/s, 123.9157 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
%
```

* Ruby 2.8.0 without this fix uses `Hash(JSON::Ext::Generator::GeneratorMethods::Hash)#to_json`, which should use `Hash(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json`

```
% bin/test test/json/encoding_test.rb -n test_hash_should_pass_encoding_options_to_children_in_to_json

Run options: -n test_hash_should_pass_encoding_options_to_children_in_to_json --seed 18064

[Hash,
 JSON::Ext::Generator::GeneratorMethods::Hash,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Enumerable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 JSON::Ext::Generator::GeneratorMethods::Object,
 ActiveSupport::Tryable,
 Kernel,
 BasicObject]
F

Failure:
TestJSONEncoding#test_hash_should_pass_encoding_options_to_children_in_to_json [/Users/yahonda/src/github.com/rails/rails/activesupport/test/json/encoding_test.rb:193]:
--- expected
+++ actual
@@ -1 +1 @@
-"{\"address\":{\"city\":\"London\"}}"
+"{\"name\":\"John\",\"address\":{\"city\":\"London\",\"country\":\"UK\"}}"

bin/test test/json/encoding_test.rb:181

Finished in 0.015009s, 66.6267 runs/s, 66.6267 assertions/s.
1 runs, 1 assertions, 1 failures, 0 errors, 0 skips
%
```

* Ruby 2.8.0 with this fix uses `Hash(ActiveSupport::ToJsonWithActiveSupportEncoder)#to_json`

```
% bin/test test/json/encoding_test.rb -n test_hash_should_pass_encoding_options_to_children_in_to_json

Run options: -n test_hash_should_pass_encoding_options_to_children_in_to_json --seed 56794

[ActiveSupport::ToJsonWithActiveSupportEncoder,
 Hash,
 JSON::Ext::Generator::GeneratorMethods::Hash,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Enumerable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 JSON::Ext::Generator::GeneratorMethods::Object,
 ActiveSupport::Tryable,
 Kernel,
 BasicObject]
.

Finished in 0.007434s, 134.5171 runs/s, 134.5171 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
%
```

Refer
https://github.com/ruby/ruby/pull/3181
https://github.com/ruby/ruby/pull/2936
https://bugs.ruby-lang.org/issues/9573
https://github.com/rails/rails/pull/19413
2020-06-22 23:44:24 +09:00
Ryuta Kamizono
d6a9002824 Use method_defined? instead of respond_to? 2020-06-19 20:32:40 +09:00
Ryuta Kamizono
ae8f7b0d59 Use native Hash#except if it is defined
Now Ruby master has native `Hash#except`.

https://bugs.ruby-lang.org/issues/15822
2020-06-19 11:47:12 +09:00
Ryuta Kamizono
528b62e386 Address to false negative for Performance/DeletePrefix,DeleteSuffix
Follow up to c07dff72278fb7f2a3c4c71212a0773a2b25c790.

Actually it is not the cop's fault, but we mistakenly use `^`, `$`, and
`\Z` in much places, the cop doesn't correct those conservatively.

I've checked all those usage and replaced all safe ones.
2020-06-14 13:04:47 +09:00
Ryuta Kamizono
37c19f7ebc Fix Active Support failure for redis 4.2.0 with Ruby 2.8.0-dev 2020-06-10 09:25:03 +09:00
Ryuta Kamizono
c07dff7227 Auto-correct for delete_prefix/delete_suffix
Follow up to #39409.
2020-06-05 12:40:39 +09:00
Yoshiyuki Kinjo
28d1ea0e49 remove redundant require: follow #39142 2020-06-04 12:21:54 +09:00
Aaron Patterson
35fe9bc0aa
Merge pull request #39477 from p8/improve-inspect
Make custom inspect methods more consistent
2020-06-03 10:43:35 -07:00
Eugene Kenny
b93608c059
Merge pull request #39505 from jonathanhefner/alias-subclasses-as-direct_descendants
Alias direct_descendants as subclasses
2020-06-03 10:26:34 +01:00
Jonathan Hefner
8f8aa857e0 Alias direct_descendants as subclasses
This alias overrides Class#subclasses.  Thus calls to `subclasses` get a
free performance boost when the class extends DescendantsTracker.
2020-06-02 22:57:48 -05:00
Eugene Kenny
a7d2f60f3a
Merge pull request #39049 from krzysiek1507/fix/hash-with-indifferent-access-update
Use size == 1 instead of one? to check hashes count in HashWithIndifferentAccess#update
2020-06-02 02:11:53 +01:00
Kasper Timm Hansen
3b953da779
Catch Time.zone before TestHelper resets it to UTC
This is most easiest done by switching to before_setup, which fits since
we're also testing the ordering of the reset calls provided by the
TestHelper.
2020-06-02 01:31:14 +02:00
Kasper Timm Hansen
81b6ca0970
Merge pull request #39493 from rails/fix-current-attributes-reset-in-models
Reset ActiveSupport::CurrentAttributes even where executor doesn't wrap
2020-06-02 00:36:30 +02:00
David Heinemeier Hansson
36a33d721c Needlessly tight dependency spec
Let it float, unless there's a specific threat to the opposite.
2020-05-31 16:45:37 -07:00
Kasper Timm Hansen
c962409ba6
Reset ActiveSupport::CurrentAttributes even where executor doesn't wrap
Currently there's a problem with ActiveSupport::CurrentAttributes where
they don't reset unless there's a controller or a job executing.

This is because we correctly hook into the controller/job executor to
reset them.

However, we were missing plain tests, so this is that.
2020-05-31 22:26:24 +02:00
Petrik
74cb9a6f38 Make inspect look more like regular Object#inspect
Move the # outside the < > just like regular Object#inspect
2020-05-29 21:53:35 +02:00
Ryuta Kamizono
c65864cdca Prefer no allocation start/end_with? over String#[] == 2020-05-29 10:20:13 +09:00
Alexander Shemetovskiy
3d1b9389a9
Test that Array#extract_options! extracts ActiveSupport::OrderedOptions 2020-05-24 16:59:42 -04:00
Ryuta Kamizono
dc94a753d1 require "active_support/core_ext/symbol/starts_ends_with" for Ruby 2.6
And use `Symbol#[]` to strip suffix to avoid intermediate string
allocation.
2020-05-25 05:24:44 +09:00
fatkodima
e24d6ecbfd Update rubocop-performance gem and enable Performance/DeletePrefix and Performance/DeleteSuffix cops 2020-05-24 12:51:35 +03:00
Eugene Kenny
537b071002 Copy options before delegating in with_options
cd31e113c0663dabcdc293d9e7dc3b6e1392db5d switched to passing options as
keyword arguments, which always creates a new hash.

9e4ff29f748b05c3a949f0d75167950039b6cda8 removed a now-unnecessary call
to `dup`, since the options could no longer be accidentally mutated.

a55620f3fa89d957817349e5170f686d505eeee4 switched back to passing
options as a positional argument for Ruby < 2.7, but didn't restore the
call to `dup`, which meant that the same options hash was now passed
with every method call and mutations leaked from one call to another.
2020-05-23 22:47:48 +01:00
Mads Ohm Larsen
e2a2ee008b Decoding JSON dates should respect newlines 2020-05-21 13:04:01 +02:00
fatkodima
6c4f3be929 Unify raise_on_missing_translations for views and controllers 2020-05-20 02:42:59 +03:00
Ryuta Kamizono
1494fadd0d Fix test_two_classes_autoloading failure
That test requires Marshal is prepended by `MarshalWithAutoloading`.
2020-05-19 02:57:46 +09:00
Aaron Patterson
b472feddbc
Merge branch 'master-sec'
* master-sec:
  Check that request is same-origin prior to including CSRF token in XHRs
  HMAC raw CSRF token before masking it, so it cannot be used to reconstruct a per-form token
  activesupport: Avoid Marshal.load on raw cache value in RedisCacheStore
  activesupport: Avoid Marshal.load on raw cache value in MemCacheStore
  Return self when calling #each, #each_pair, and #each_value instead of the raw @parameters hash
  Include Content-Length in signature for ActiveStorage direct upload
2020-05-18 08:48:41 -07:00
Ryan Davis
3116bdceba Fixed BacktraceCleaner to never return an empty backtrace.
Empty backtraces means you didn't run any code, which isn't the case,
and goes against the contract that Minitest.backtrace_cleaner expects.
This fixes a bug I've seen in a number of reports.

It would be nice if this got backported to whatever versions are
active, as this keeps coming back on minitest issues.
2020-05-15 16:42:27 -07:00
Dylan Thacker-Smith
9b4aef4be3
activesupport: Avoid Marshal.load on raw cache value in RedisCacheStore
The same value for the `raw` option should be provided for both reading and
writing to avoid Marshal.load being called on untrusted data.

[CVE-2020-8165]
2020-05-15 15:38:33 -07:00
Dylan Thacker-Smith
0a4cf42311
activesupport: Avoid Marshal.load on raw cache value in MemCacheStore
Dalli is already being used for marshalling, so we should also rely
on it for unmarshalling. Since Dalli tags the cache value as marshalled
it can avoid unmarshalling a raw string which might have come from
an untrusted source.

[CVE-2020-8165]
2020-05-15 15:38:29 -07:00
Lee Quarella
493470acff Stop creating too many test databases
Issue #39291

In https://github.com/rails/rails/pull/38893 parallel test worker ids
were changed to `SecureRandom.uuid` to keep track of inflight work
across distributed instances, but negelected to change the worker
callbacks to match.  This caused a new set of unique test databases are
created with every test run.

This commit makes the appropriate change to worker callbacks so only one
set of of parallel test databases are created.
2020-05-15 12:31:45 -04:00
Ryuta Kamizono
d9539281bd Remove unused require "active_support/core_ext/array/extract_options"
dfa439eefc7bfaa2f00bc8d2bd712d115dd0ef33
6708f3a5b7892ffd5412eb1c1d62ea53c404c4d1
2020-05-10 04:29:18 +09:00
Ryuta Kamizono
37c7481826 Callbacks tests also uses singleton class eval 2020-05-10 04:25:26 +09:00
Ryuta Kamizono
1165401ee9 Remove unused require "active_support/core_ext/kernel/singleton_class"
`singleton_class` is in Ruby 1.9.2, and there is no use singleton class
eval in the files.
2020-05-10 03:20:02 +09:00
akinomaeni
6379f99c97 Add tests for to_s(:inspect) 2020-05-10 00:03:37 +09:00
Jean Boussier
8c0b94b995 Make NameError#missing_name work even for real Ruby NameError
On constant missing Ruby call `#inspect` on the receiver to build
the error message.

For instance, the error message for `Foo::Bar` will be `"#{Foo.inspect}::Bar"`.

And since Active Record override the model classes inspect method, this
breaks `missing_name` assumptions.

Until now it worked because missing_name was only called on errors
raised by the classic autoloader, and the classic autoloader calls
`#name` to build its error message.
2020-05-08 17:08:38 +02:00
Rafael França
9765ded048
Merge pull request #38326 from Shopify/set-callback-allocations
Reduce allocations in set_callbacks
2020-05-06 21:56:30 -04:00
Rafael França
0f60146c8f
Merge pull request #39169 from jonathanhefner/add-date_formats-inspect
Add DATE_FORMATS[:inspect]
2020-05-06 21:40:15 -04:00
Xavier Noria
f2902edc0c register the obsolescence of require_dependency in the CHANGELOG [ci skip] 2020-05-06 22:14:50 +02:00
Jonathan Hefner
2b38bf6857 Add DATE_FORMATS[:inspect]
Follow-up to #39147 and #39168.

By adding a new purpose-specific format, we avoid potential pitfalls
from concatenating format strings.  We also save a String allocation per
Time attribute per inspect.

The new format also includes a time zone offset for more introspective
inspection.
2020-05-06 15:05:02 -05:00