This pull request workarounds Active Job integration test with `resque` reported at https://buildkite.com/rails/rails/builds/72277#410533b8-0676-4ad2-900f-fa8131f9833f/1826
This fix is similar to 7cf8e30 and rails/rails#40451 which is due to https://bugs.ruby-lang.org/issues/17220
- Steps to reproduce
```
- Install Docker
$ git clone https://github.com/rails/rails
$ cd rails
$ git clone https://github.com/rails/buildkite-config .buildkite/
$ RUBY_IMAGE=rubylang/ruby:master-nightly-bionic docker-compose -f .buildkite/docker-compose.yml build base && CI=1 docker-compose -f .buildkite/docker-compose.yml run activejob runner activejob 'AJ_ADAPTER=resque AJ_INTEGRATION_TESTS=true bin/test test/integration/queuing_test.rb --seed 36344'
```
- Without this commit
```
$ RUBY_IMAGE=rubylang/ruby:master-nightly-bionic docker-compose -f .buildkite/docker-compose.yml build base && CI=1 docker-compose -f .buildkite/docker-compose.yml run activejob runner activejob 'AJ_ADAPTER=resque AJ_INTEGRATION_TESTS=true bin/test test/integration/queuing_test.rb --seed 36344'
... snip ...
+++ activejob: AJ_ADAPTER=resque AJ_INTEGRATION_TESTS=true bin/test test/integration/queuing_test.rb --seed 36344
Using resque
/usr/local/lib/ruby/gems/3.0.0/gems/resque-scheduler-4.4.0/lib/resque/scheduler/lock/base.rb:50: warning: Socket.gethostbyname is deprecated; use Addrinfo.getaddrinfo instead.
Run options: --seed 36344
.F
Failure:
QueuingTest#test_should_run_jobs_enqueued_on_a_listening_queue [/rails/activejob/test/integration/queuing_test.rb:14]:
Expected false to be truthy.
bin/test test/integration/queuing_test.rb:11
F
Failure:
QueuingTest#test_current_timezone_is_kept_while_running_perform_later [/rails/activejob/test/integration/queuing_test.rb:119]:
Expected false to be truthy.
bin/test test/integration/queuing_test.rb:110
.SSF
Failure:
QueuingTest#test_current_locale_is_kept_while_running_perform_later [/rails/activejob/test/integration/queuing_test.rb:102]:
Expected false to be truthy.
bin/test test/integration/queuing_test.rb:93
.F
Failure:
QueuingTest#test_should_run_job_enqueued_in_the_future_at_the_specified_time [/rails/activejob/test/integration/queuing_test.rb:76]:
Expected false to be truthy.
bin/test test/integration/queuing_test.rb:71
SSSSSS
Finished in 34.122641s, 0.4396 runs/s, 0.2344 assertions/s.
15 runs, 8 assertions, 4 failures, 0 errors, 8 skips
You have skipped tests. Run with --verbose for details.
$
```
- With this commit
```ruby
$ RUBY_IMAGE=rubylang/ruby:master-nightly-bionic docker-compose -f .buildkite/docker-compose.yml build base && CI=1 docker-compose -f .buildkite/docker-compose.yml run activejob runner activejob 'AJ_ADAPTER=resque AJ_INTEGRATION_TESTS=true bin/test test/integration/queuing_test.rb --seed 36344'
... snip ...
+++ activejob: AJ_ADAPTER=resque AJ_INTEGRATION_TESTS=true bin/test test/integration/queuing_test.rb --seed 36344
Using resque
/usr/local/lib/ruby/gems/3.0.0/gems/resque-scheduler-4.4.0/lib/resque/scheduler/lock/base.rb:50: warning: Socket.gethostbyname is deprecated; use Addrinfo.getaddrinfo instead.
Run options: --seed 36344
....SS...SSSSSS
Finished in 13.850658s, 1.0830 runs/s, 0.7220 assertions/s.
15 runs, 10 assertions, 0 failures, 0 errors, 8 skips
You have skipped tests. Run with --verbose for details.
$
```
This pull request addresses Active Job integration test with `sneakers` reported at https://buildkite.com/rails/rails/builds/72257#20014e66-6e08-47ae-a827-b71de7148306/2017
This fix is similar to 7cf8e30902 which is due to https://bugs.ruby-lang.org/issues/17220
- Steps to reproduce
Install ruby 3.0.0dev using ruby-build or whatever then execute these steps.
```ruby
$ git clone https://github.com/rails/rails
$ cd rails/activejob
$ bundle install
$ bundle exec rake test:integration:sneakers
```
- Result without this commit
```ruby
$ ruby -v
ruby 3.0.0dev (2020-10-24T13:53:53Z master 148961adcd) [x86_64-linux]
$ cd activejob
$ bundle exec rake test:integration:sneakers
... snip ...
/home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/adapters/sneakers.rb:49:in `rescue in start_workers': Failed to start sneakers worker (RuntimeError)
from /home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/adapters/sneakers.rb:41:in `start_workers'
from /home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/helper.rb:27:in `<top (required)>'
from /home/yahonda/src/github.com/rails/rails/activejob/test/helper.rb:12:in `require'
from /home/yahonda/src/github.com/rails/rails/activejob/test/helper.rb:12:in `<top (required)>'
from /home/yahonda/src/github.com/rails/rails/activejob/test/integration/queuing_test.rb:3:in `require'
from /home/yahonda/src/github.com/rails/rails/activejob/test/integration/queuing_test.rb:3:in `<top (required)>'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:17:in `require'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:17:in `block in <main>'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:5:in `select'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:5:in `<main>'
/home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/adapters/sneakers.rb:44:in `sleep': execution expired (Timeout::Error)
from /home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/adapters/sneakers.rb:44:in `block in start_workers'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/3.0.0/timeout.rb:112:in `timeout'
from /home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/adapters/sneakers.rb:42:in `start_workers'
from /home/yahonda/src/github.com/rails/rails/activejob/test/support/integration/helper.rb:27:in `<top (required)>'
from /home/yahonda/src/github.com/rails/rails/activejob/test/helper.rb:12:in `require'
from /home/yahonda/src/github.com/rails/rails/activejob/test/helper.rb:12:in `<top (required)>'
from /home/yahonda/src/github.com/rails/rails/activejob/test/integration/queuing_test.rb:3:in `require'
from /home/yahonda/src/github.com/rails/rails/activejob/test/integration/queuing_test.rb:3:in `<top (required)>'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:17:in `require'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:17:in `block in <main>'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:5:in `select'
from /home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb:5:in `<main>'
rake aborted!
Command failed with status (1): [ruby -w -I"lib:test" -I"/home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib" "/home/yahonda/.rbenv/versions/3.0.0-dev/lib/ruby/gems/3.0.0/gems/rake-13.0.1/lib/rake/rake_test_loader.rb" "test/integration/queuing_test.rb" ]
/home/yahonda/.rbenv/versions/3.0.0-dev/bin/bundle:23:in `load'
/home/yahonda/.rbenv/versions/3.0.0-dev/bin/bundle:23:in `<main>'
Tasks: TOP => test:integration:sneakers
(See full trace by running task with --trace)
$
```
- Result with this commit
```ruby
$ bundle exec rake test:integration:sneakers
... snip ...
15 runs, 6 assertions, 0 failures, 0 errors, 11 skips
```
PR #33995 added support for specifying the `args` argument of
`assert_enqueued_with` and `assert_performed_with` as a matcher proc.
In doing so, it added undocumented support for specifying the other
arguments as matcher procs as well. This commit officially documents
that support, and adds tests to ensure the behavior.
assert_enqueued_with with a block ignores all the jobs enqueued before
the block for its assertions by counting the number of jobs and dropping
the n first elements from the Array, but since we're now mutating the
Array in perform_enqueued_jobs without a block, it's broken.
This uses another implementation which is correct when the array is
mutated, by getting a duplicated array of jobs, then removing them from
the original array.
Similarly assert_enqueued_jobs with a block was using counts only, now
keeps track of the specific jobs to count them at the end.
This makes sure jobs don't run twice if `perform_enqueued_jobs` is
called twice without a block.
This also mimics the behavior of using `perform_enqueued_jobs` with a
block, where at the end of the block performed jobs are not in
`enqueued_jobs` but instead in `performed_jobs`.
- ### Problem
If we use `perform_enqueued_jobs` without a block, a job that
uses a retry mechanism to reeenqueue itself would get performed
right away.
This behaviour make sense when using `perform_enqueued_jobs` with
a block.
However I'm expecting `perform_enqueued_jobs` without a block to
perform jobs that are **already** in the queue not the ones that
will get enqueued afterwards.
### Solution
Dup the array of jobs given to avoid future mutation.
- ### Problem
If we use `perform_enqueued_jobs` without a block,
a job that raises an error wouldn't be appended to
the list of `performed_jobs`.
### Solution
Push the job in the array before it is actually performed.
author Aditya Narsapurkar <adityanarsapurkar@yahoo.com> 1582316102 +0530
committer Aditya Narsapurkar <adityanarsapurkar@yahoo.com> 1583159505 +0530
parent 6d0895a4894724e1a923a514daad8fb3c9ac2c28
author Aditya Narsapurkar <adityanarsapurkar@yahoo.com> 1582316102 +0530
committer Aditya Narsapurkar <adityanarsapurkar@yahoo.com> 1583159327 +0530
Randomize jitter
- This PR attempts to fix a problem with ActiveJob jitter where the `determine_jitter_for_delay` value may not always be randomized. Especially when the jitter delay multplier is between 1 and 2 it always returns 0.
- With this change, we pass a range to `Kernel.rand` beginning with 0 to the `jitter_multiplier`. With positive float values, the return value will be a random float number from the range.
- Includes test cases to verify random wait time when the jitter_multiplier is between 1 and 2.
- Updated relevant test cases stubbing the `Kernel.rand` method, refactored some by removing unwanted stubs for `Kernel.rand` method where jitter is falsey.
Fixed rubocop issue - used assert_not_equal instead of refute_equal in test case
Fixed rubocop issue - used assert_not_equal instead of refute_equal in test case
Fixed rubocop issue - used assert_not_equal instead of refute_equal in test case
Review updates - separated test cases for random wait time with default and exponentially retrying jobs
- Another test case added to make sure negative wait time does not affect the randomization
Review updates
- Instead of using Kernel.rand with range, used simple multiplication with Kernel.rand for calculating delay for jitter
- Updates to the tests according to changes
- ### Problem
In some cirumstances, the deprecation message to warn that AJ won't
run `after_(enqueue/perform)` callbacks when the chain is halted
by a `throw(:abort)` will be thrown even though no `throw(:abort)`
was thrown.
```ruby
run_callback(:foo) do
...
end
```
There is two possible way for the callback body to not be executed:
1) `before` callback throw a `abort`
2) `before` callback raises an error which is rescued by an
around callback (See associated test in this commit for
an example)
When 2) happen we don't want to output a deprecation message,
because what the message says isn't true and doesn't apply.
### Solution
In order to differentiate between 1) and 2), I have added
a `halted_callback_hook` which is called by ActiveSupport callback
whenever the callback chain is halted.
- ### Problem
Given the below example the test adapter will retry the job
indefinitely:
```ruby
class BuggyJob < ActiveJob::Base
retry_on(Exception, attempts: 2)
def perform
raise "error"
end
end
BuggyJob.perform_later
perform_enqueued_jobs
```
The problem is that when the job get retried, the
`exception_executions` variable is not serialized/deserialized,
resulting in ActiveJob to not be able to determine how many time
this job was retried.
The solution in this PR is to deserialize the whole job in the test
adapter, and reserialize it before retrying.
Fix#38391
- I made a change in 0d3aec49695 to output a log if a job was aborted
in a before callbacks. I didn't take in consideration that a job
could return a falsy value and thus it would wrongly log
that the job was aborted.
This fixes the problem by checking if the callback chain was halted
rather than the return value of the job.
- Since this is going to be the default in 6.1, let's set it in the
test suite to avoid deprecation warning.
Otherwise one has to do `AS::Deprecation.silence { }` everytime we
add a new test.
Fix#38107
* Add failing ActiveJob exceptions test for "disable retry jitter"
Thanks to @kaspth for the starting point.
* Update ActiveJob retry jitter to correctly use zero value
* Simplify "disable retry jitter" test
We don't need to repeat this many times. Fewer is shorter.
* Refactor determine_delay with jitter
* Fix indentation
* Close the curtains and give JITTER_DEFAULT some privacy
* Use .zero? instead of == to check jitter value
* Add ActiveJob test for explicit zero jitter
Co-authored-by: Kasper Timm Hansen <hey@kaspth.com>
Co-authored-by: Cliff Pruitt <cliff.pruitt@cliffpruitt.com>
The existing message only mentioned one type of before/after callback,
but the config was named generally. That mismatch is confusing and users
wouldn't necessarily know what the total effect of the config would be.
So instead of handwriting the deprecation warning in the specific instances,
consolidate it in one place and give the appropriate context. That context
is the above, but also that users shouldn't update their app config,
they should uncomment the line in the new defaults file, which now also
has more context.
I'm not totally convinced that we can't move this to when
`after_enqueue`/`after_perform` is called in the job class. Doesn't
seem worth it to blare this after every job enqueue/perform, when we
the score at boot time.
cc @Edouard-chin
-
### Problem
In rails/rails@bbfab0b33a I introduced a change which outputs
a deprecation whenever a class inherits from ActiveJob::Base.
This has the negative effect to trigger a massive amount of
deprecation at boot time especially if your app is eagerloaded
(like it's usually the case on CI).
Another issue with this approach was that the deprecation will
be output no matter if a job define a `after_perform` callbacks
i.e.
```ruby
class MyJob < AJ::Base
before_enqueue { throw(:abort) }
end
# This shouldn't trigger a deprecation since no after callbacks are defined
# The change in 6.2 will be already safe for the app.
```
### Solution
Trigger the deprecation only when a job is abort
(during enqueuing or performing) AND a `after_perform`
callback is defined on the job.
- ### Problem
```ruby
MyJob < ApplicationJob
before_enqueue { throw(:abort) }
after_enqueue { # enters here }
end
```
I find AJ behaviour on after_enqueue and after_perform callbacks
weird as they get run even when the callback chain is halted.
It's counter intuitive to run the after_enqueue callbacks even
though the job wasn't event enqueued.
### Solution
In Rails 6.2, I propose to make the new behaviour the default
and stop running after callbacks when the chain is halted.
For application that wants this behaviour now or in 6.1
they can do so by adding the `config.active_job.skip_after_callbacks_if_terminated = true`
in their configuration file.
- ### Problem
ActiveJob will always log "Enqueued MyJob (Job ID) ..." even
if the job doesn't get enqueued through the adapter.
Same problem happens when performing a Job, "Performed MyJob (Job ID) ..." will be logged even when job wasn't performed at all.
This situation can happen either if the callback chain is terminated
(before_enqueue throwing an `abort`) or if an exception is raised.
### Solution
Check if the callback chain is aborted/exception is raised, and log accordingly.
Prior to this change, exponentially_longer had adverse consequences
during system-wide downstream failures. This change adds a random value to the
back off calculation in order to prevent the thundering herd
problem, whereby all retry jobs would retry at the same time.
Specifically this change adds a jitter option to retry_on to enable users of it to
scope the randomness calculation to a reasonable amount. The default is
15% of the exponential back off calculation.
class SensitiveJob < ApplicationJob
self.log_arguments = false
def perform(my_sensitive_argument)
end
end
When dealing with sensitive arugments as password and tokens it is
now possible to configure the job to not put the sensitive argument
in the logs.
Closes#34438.
Using `self` works by essentially pointing to the included class, in
this case `ActiveJob::Base`, so that'd end up delegating to
`ActiveJob::Base.queue_adapter` instead.
This commit allows ActiveJob to serialize classes and modules without a custom serializer. This allows for workflows like:
```
class EmailJob < ApplicationJob
queue_as :default
def perform(template_class, *arguments)
template_class.new(*arguments).send!
end
end
module Email
class FooTemplate ... end
class BarTemplate ... end
end
EmailJob.perform_later(Email::FooTemplate, ...)
EmailJob.perform_later(Email::BarTemplate, ...)
```
Currently this is only achieveable through a custom serializer or through constantizing in each instance.
The `Sidekiq::Logging` module was removed in Sidekiq 6.0 in favour of
`Sidekiq.logger=`.
https://github.com/mperham/sidekiq/wiki/Logging#api-changes
> The old Sidekiq::Logging class was removed in Sidekiq 6.0. You should
> not be accessing any Sidekiq internal classes to use the logger. Use
> `logger` directly in your Worker or use `Sidekiq.logger` elsewhere.
Currently, the `perform_enqueued_jobs` helpers will also immediately
perform jobs that are scheduled via `set(wait:)` to run in the future.
This commit adds a new argument to `perform_enqueued_jobs` to make it
only run jobs scheduled at or before the passed in `Time`. This allows
testing the side effects of immediate job execution separate of jobs
delayed in the future.
`assert_enqueued_with` and `assert_performed_with` were not able to
properly test jobs with relative delay. `:at` option was asserted for
equality and test will always fail cause small fraction of time will
pass between job call and assertion.
This commit fixes that by droping microseconds from `:at` argument
assertions.
Rails 6 introduces retries per-exception, instead of a global count of
retries. Because ActiveJob 5.2 doesn't serialize the execution count
per-exception, when ActiveJob 6.0 picks up an "old" job it can't know
the exception count in the new format.
This can also be an issue if AJ 6.0 serializes a new job with
exception_executions which is later picked up by AJ 5.2, which would
clear exception_executions (since it has no knowledge of it).
Previously we handled this by resetting exception_executions, if it
wasn't defined on a job, which could result in the worst case retrying
the job 2x the times we should.
This commit changes how we handle loading a legacy job: instead of
resetting exception_executions, we instead will always use the global
executions count.
This way, jobs which only have one retry_on (and didn't have a behaviour
change in AJ 6) are backwards-and-forwards-compatible with counts
respected exactly.
Jobs with multiple retry_on will revert to the AJ5.2 behaviour if they
were ever run under AJ5.2.
Individual execution counters were introduced in #34352. However
`#determine_delay` which is used to calculate retry delay still uses the
global counter. This commit fixes it.
Motivation:
- Currently we have 2 seperate monkey patches in place for tracking
enqueded time for 2 seperate workers. It seems that activejob could be
a source of truth for how long an item has been enqued so that we can
easily use it for consistent monitoring across workers/apps to ensure
that jobs are running at an acceptable speed.
Changes:
- Add an enqueded at attribute and serilization tooling.
- Add a method to get how long a job has been enqueded for.
- Add a logging item to show how long a job was enqued prior to the
perform method firing.
Previously, by extending ActiveJob::TestCase, the test adapter provided
for tests was being used always, in all executions where supposedly
different adapters were being used. As a consequence, some bugs visible
only for some adapters might have gone undetected. This commit changes
that, skipping queue adapters for which we can't test scheduling jobs
with a delay.
Also, make tests and examples for individual execution counters
clearer, as it wasn't entierly clear what would happen in this case:
```
retry_on CustomException, OtherException, attempts: 3
```
The job would be retried at most 3 times in total, for both
CustomException and OtherException. To have the job retry 3 times at
most for each exception individually, the following retry_on
declarations are necessary:
```
retry_on CustomException, attempts: 3
retry_on OtherException, attempts: 3
```
Trusting any GlobaID object when deserializing jobs can allow
attackers to access information that should not be accessible to them.
Fix CVE-2018-16476.
* Keep executions for each specific declaration
Fixes#34337
ActiveJob used the global executions counter to control the number of
times a job should be retried. The problem with this approach was that
in case a job raised different exceptions during its executions they
weren't retried the number of times defined by their `attemps` number.
**Example:**
Having the following job:
```ruby
class BuggyJob < ActiveJob::Base
retry_on CustomException, attemps: 3
retry_on OtherException, attempts: 3
end
```
If the job raised `CustomException` in the first two executions and then
it raised `OtherException`, the job wasn't retried anymore because the
global executions counter was already indicating 3 attempts.
With this patch each `retry_on` declaration has its specific counter so
that the first two executions that raise `CustomException` don't affect
the retries count that future exceptions may have.
* Revert "clarifies documentation around the attempts arugment to retry_on"
This reverts commit 86aa8f8c5631f77ed9a208e5107003c01512133e.
- That feature is useful to enqueue or assert that jobs got enqueued
or performed based on dynamic conditions.
We will be able to leverage that feature to fix all ActionMailer
assertion helper issue when a Mailer define a custom delivery job
(see next commit).
Fixes#34366
Currently setting queue_name_prefix will not combine a prefix with the
default_queue_name; it will only affect queue names set with `queue_as`.
With this PR the prefix will affect the default_queue_name as well.
Closes#19831
Currently setting default_queue_name doesn't actually affect the
queue_name default (although default_queue_name does get used if you
pass a falsey `part_name` to `queue_as`). This PR would get
default_queue_name working as expected as well.
Because the queue_name default is now a lambda wrapping the
default_queue_name, rather than the default_queue_name itself, I had to
update one test to use the instance method `#queue_name` (which
`instance_exec`s the value) instead of the class method. I think this
change is OK, since only the instance method is documented.
There was a question about whether we want a `default_queue_name`
configuration. If we want to get rid of it, I would also be happy to
open a PR for that instead. It has been around for a while now, but it
also hasn't really worked for a while now.
r? @matthewd
since you had an opinion about this before
Since `@mutex.synchronize` is enforced in the `ensure_connection!` method,
there is no need to do so on the caller side.
c1b47f9c5d/lib/sneakers/publisher.rb (L22-L26)
Due to this, `ThreadError(deadlock; recursive locking)` has occurred.
`assert_enqueued_with` and `assert_performed_with` return a instantiated
instance of the matching job for further assertion (#21010).
Before this commit the `arguments` method on the returned instance
returns a serialized version of the arguments.
- When calling `assert_performed_with`/`assert_enqueued_with`, the
+args+ needs to match exactly what the job get passed.
Some jobs can have lot of arguments, or even a simple hash argument
has many key. This is not convenient to test as most tests doesn't
need to check if the arguments matches perfectly.
This PR make it possible to only check if a subset of arguments were
passed to the job.
Since #33751 was added `enqueue_retry.active_job` instrumentation to
the `retry_on` method, then #33897 moved the instrumentation to `retry_job`
method in order to ensure that this method publish `enqueue_retry.active_job`
notification too.
See related discussion https://github.com/rails/rails/pull/33751#discussion_r214140008
Since `enqueue_retry.active_job` moved to `retry_job`, there is no guarantee
that payload of `enqueue_retry.active_job` would have `:error`.
See test `LoggingTest#test_enqueue_retry_logging_on_retry_job` as
example of that case.
Related to https://github.com/rails/rails/pull/33897#discussion_r219707024
I think we can improve notification of `enqueue_retry.active_job`:
- If there is no `event.payload[:error]`, then publish like
"Retrying RescueJob in 3 seconds." only.
- If `event.payload[:wait]` is `nil`, then publish
"Retrying RescueJob in 0 seconds." instead of "Retrying RescueJob in nil seconds."
- If there is `event.payload[:error]`, then publish
"Retrying RescueJob in 3 seconds, due to a DefaultsError.".
- Change the type of the message from `error` to `info.`
Also, this commit removes part of messages - "The original exception was #{ex.cause.inspect}."
of `enqueue_retry.active_job`, `retry_stopped.active_job`, and `discard.active_job` since I
haven't found it useful. Please let me know whether you agree with that?
psqlrc files can affect the execution of commands in ways that can hold
up execution by blocking or otherwise cause unexpected side effects and
should best be ignored when using psql programmatically.
Currently when executing `bundle exec rake test:integration` under `activejob/`
derectory, it prints helpful info like:
```
(snip)
*** rake aj:integration:async ***
(snip)
*** rake aj:integration:delayed_job ***
(snip)
```
but there is no defined `:aj` scope in `activejob/Rakefile`,
so I think output should be like:
```
(snip)
*** rake test:integration:async ***
(snip)
*** rake test:integration:delayed_job ***
(snip)
```
By the way `rake test:integration` doesn't work if execute it without
prepending `bundle exec` to that command. It is probably what we should
fix too.
The priority wasn't being passed from ActiveJob to Backburner, despite
priority being supported. This also brings it inline with the docs,
which mark Backburner as supporting priorities in the "Backend Features"
table: https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html
If the `:queue` option is specified, then only the job(s) enqueued to a specific
queue will not be performed.
Example:
```
def test_assert_no_performed_jobs_with_queue_option
assert_no_performed_jobs queue: :some_queue do
HelloJob.set(queue: :other_queue).perform_later("jeremy")
end
end
```
If the `:queue` option is specified, then only the job(s) enqueued to a specific
queue will be performed.
Example:
```
def test_assert_performed_jobs_with_queue_option
assert_performed_jobs 1, queue: :some_queue do
HelloJob.set(queue: :some_queue).perform_later("jeremy")
HelloJob.set(queue: :other_queue).perform_later("bogdan")
end
end
```
If the `:queue` option is specified, then only the job(s) enqueued to
a specific queue will be performed.
Example:
```
def test_perform_enqueued_jobs_with_queue
perform_enqueued_jobs queue: :some_queue do
MyJob.set(queue: :some_queue).perform_later(1, 2, 3) # will be performed
HelloJob.set(queue: :other_queue).perform_later(1, 2, 3) # will not be performed
end
assert_performed_jobs 1
end
```
Follow up #33265
[bogdanvlviv & Jeremy Daer]
Currently, the execution count increments after deserializes arguments.
Therefore, if an error occurs with deserialize, it retries indefinitely.
In order to prevent this, the count is moved before deserialize.
Fixes#33344.
It can be asserted that no jobs are enqueued to a specific queue:
```ruby
def test_no_logging
assert_no_enqueued_jobs queue: 'default' do
LoggingJob.set(queue: :some_queue).perform_later
end
end
```
Rename `in_block_job` to `enqueued_job` since this variable can refer not only
to jobs that were created in the block.
See #33258.
Return back accidentally removed test to activejob/test/cases/test_helper_test.rb
See #33258.
Fix name of tests.
Example of `assert_enqueued_with` with no block
```ruby
def test_assert_enqueued_with
MyJob.perform_later(1,2,3)
assert_enqueued_with(job: MyJob, args: [1,2,3], queue: 'low')
MyJob.set(wait_until: Date.tomorrow.noon).perform_later
assert_enqueued_with(job: MyJob, at: Date.tomorrow.noon)
end
```
Example of `assert_enqueued_email_with` with no block:
```ruby
def test_email
ContactMailer.welcome.deliver_later
assert_enqueued_email_with ContactMailer, :welcome
end
def test_email_with_arguments
ContactMailer.welcome("Hello", "Goodbye").deliver_later
assert_enqueued_email_with ContactMailer, :welcome, args: ["Hello", "Goodbye"]
end
```
Related to #33243
When a job was just deserialized `arguments` is `nil` and the serialized
arguments are in the `@serialized_arguments` variable. If we try to
serialize this job again the arguments are going to be `nil` instead of
what was serialized.
The test we had was not checking this case because it was deserializing
the job in the same object that had the arguments.
To fix this, when the `@serialized_arguments` are present we return it
instead of the result of the `arguments` serialized.
Reasons are that the Qu gem wasn't compatible since Rails 5.1,
gem development was stopped in 2014 and maintainers have
confirmed its demise. See issue #32273
Record what was the current timezone in effect when the job was
enqueued and then restore when the job is executed in same way
that the current locale is recorded and restored.
The serializer serializes an instance of `ActiveSupport::TimeWithZone`.
The serializer deserializes value to `ActiveSupport::TimeWithZone` if possible.
We can speed up things for the supported types by keeping the code in the
way it was.
We can also avoid to loop trough all serializers in the deserialization by
trying to access the class already in the Hash.
We could also speed up the custom serialization if we define the class
that is going to be serialized when registering the serializers, but
that will remove the possibility of defining a serialzer for a
superclass and have the subclass serialized using it.
Now custom serialziers can register itself in the serialized hash using
the "_aj_serialized" key that constains the serializer name.
This way we can avoid poluting the hash with many reserved keys.
Currently, sidekiq integration test + Ruby 2.5.0-rc1 show exception as follows.
```
#<Thread:0x000000000670bec0@/home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:23 run> terminated with exception (report_on_exception is true):
/home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/bootsnap-1.1.7/lib/bootsnap/load_path_cache/core_ext/active_support.rb:53:in `block in load_missing_constant': uninitialized constant Sidekiq::CLI (NameError)
from /home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/bootsnap-1.1.7/lib/bootsnap/load_path_cache/core_ext/active_support.rb:8:in `without_bootsnap_cache'
from /home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/bootsnap-1.1.7/lib/bootsnap/load_path_cache/core_ext/active_support.rb:53:in `rescue in load_missing_constant'
from /home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/bootsnap-1.1.7/lib/bootsnap/load_path_cache/core_ext/active_support.rb:42:in `load_missing_constant'
from /home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.0.5/lib/sidekiq/launcher.rb:65:in `heartbeat'
from /home/travis/build/rails/rails/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.0.5/lib/sidekiq/launcher.rb:123:in `start_heartbeat'
```
https://travis-ci.org/rails/rails/jobs/317187279#L2152
The reason for this is that `Sidekiq::CLI` has not been loaded.
Sidekiq integration test launches a Sidekiq instance within
another Ruby process. In such a case, need to require 'sidekiq/cli'
in that launch code.
Ref: https://github.com/mperham/sidekiq/pull/3692#issuecomment-352032251
`BigDecimal.new` has been deprecated in BigDecimal 1.3.3
which will be a default for Ruby 2.5.
Refer
533737338d
* This commit has been made as follows:
```
cd rails
git grep -l BigDecimal.new | grep -v guides/source/5_0_release_notes.md | grep -v activesupport/test/xml_mini_test.rb | xargs sed -i -e "s/BigDecimal.new/BigDecimal/g"
```
- `activesupport/test/xml_mini_test.rb`
Editmanually to remove `.new` and `::`
- guides/source/5_0_release_notes.md
This is a Rails 5.0 release notes.
ActiveJob wraps every adapter into its own class, that is later passed
into DelayedJob which is responsible for displaying all the logs.
This change improves the logging so we can easily trace executed
jobs and see meaningful information in the logs.
Omit `rails activestorage:install` for jdbcmysql, jdbc and shebang tests
AppGeneratorTest#test_config_jdbcmysql_database
rails aborted!
LoadError: Could not load 'active_record/connection_adapters/mysql_adapter'.
Make sure that the adapter in config/database.yml is valid.
If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add
the necessary adapter gem to the Gemfile.
(compressed)
bin/rails:4:in `<main>'
Tasks: TOP => activestorage:install => environment
(See full trace by running task with --trace)
AppGeneratorTest#test_config_jdbc_database
rails aborted!
LoadError: Could not load 'active_record/connection_adapters/jdbc_adapter'.
Make sure that the adapter in config/database.yml is valid.
If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add
the necessary adapter gem to the Gemfile.
(compressed)
bin/rails:4:in `<main>'
Tasks: TOP => activestorage:install => environment
(See full trace by running task with --trace)
AppGeneratorTest#test_shebang_is_added_to_rails_file
/home/ubuntu/.rbenv/versions/2.4.1/bin/ruby: no Ruby script found in input (LoadError)
Prevent PendingMigrationError in tests
* Run `bin/rails db:migrate RAILS_ENV=test` in test_cases before start tests to prevent PendingMigrationError
* FileUtils.rm_r("db/migrate")
* --skip-active-storage
Fix failed tests in `railties/test/railties/engine_test.rb`
Related to #30111
Imporve `SharedGeneratorTests#test_default_frameworks_are_required_when_others_are_removed`
- Explicitly skip active_storage
- Ensure that skipped frameworks are commented
- Ensure that default frameworks are not commented
Fix error `Errno::ENOSPC: No space left on device - sendfile`
Since `rails new` runs `rails active_storage:install`
that boots an app.
Since adding Bootsnap 0312a5c67e35b960e33677b5358c539f1047e4e1
during booting an app, it creates the cache:
264K tmp/cache/bootsnap-load-path-cache
27M tmp/cache/bootsnap-compile-cache
* teardown_app must remove app
* Use `gem 'redis', '~> 4.0'` for new app Gemfiles
* Loosen Action Cable redis-rb dep to `>= 3.3, < 5`
* Bump redis-namespace for looser Redis version dep
* Avoid using the underlying `redis.client` directly
* Use `Redis.new` instead of `Redis.connect`
