Commit Graph

676 Commits

Author SHA1 Message Date
Matt Muller
4b44d4c0e9
Fix S3 multipart uploads when threshold is larger than file 2020-12-11 16:26:40 -05:00
Rafael França
45d1efab51
Merge pull request #40766 from rafaelfranca/rm-queue-defaults
Change default queue name of all the internal jobs to be the job adapter's default
2020-12-08 22:47:47 +00:00
Daniel Colson
8ef5bd9ced
Allow custom purpose for ActiveStorage signed IDs
This commit allows passing a custom `purpose:` keyword argument to both
`find_signed!` and `signed_id`. This matches the signature of the
`super` methods in ActiveRecord, and it is already possible with the
non-bang [`find_signed`][find_signed] method, since that one doesn't
have an override in ActiveStorage (although perhaps it should, to set
the default :blob_id purpose)

This is useful in cases where you want to further isolate different
types of blobs, for example if they have different levels of
authorization.

[find_signed]: be11d1b6e8/activerecord/lib/active_record/signed_id.rb (L42-L48)
2020-12-03 12:36:52 -05:00
Rafael Mendonça França
59f7f5889e
Start Rails 6.2 development 🎉 2020-12-03 01:35:29 +00:00
Alex Ghiculescu
76e432341c Always recommend config.asset_host in examples 2020-11-30 16:33:51 -06:00
Tahsin Hasan
bb148d822c
Permit hash on direct upload in active storage (#40662) 2020-11-28 10:33:44 +01:00
Jonathan Hefner
3397924e66
Use FFmpeg scene detection for video previews
Generating a video preview by capturing only the first frame of a video
is problematic for videos that begin with a fade in from black.  By
using keyframe and scene detection that is built in to FFmpeg, we can
generate a more representative preview.
2020-11-17 17:25:01 -05:00
David Angulo
1db02b3896 Add strict loading for active storage 2020-11-16 22:09:15 +08:00
Rafael Mendonça França
8389f9902c
Preparing for 6.1.0.rc1 release 2020-11-02 21:12:47 +00:00
Eugene Kenny
b7eaaf73fb Update comment about reloading in development.rb [ci skip]
This comment predates the existence of `reload_classes_only_on_change`.
2020-11-02 19:36:26 +00:00
Rafael Mendonça França
62bda903be
Remove deprecated support to pass :combine_options operations to ActiveStorage::Transformers::ImageProcessing 2020-10-30 00:25:48 +00:00
Rafael Mendonça França
bad3d0f6a3
Remove warning of undefined instance variable 2020-10-30 00:25:47 +00:00
Rafael Mendonça França
74476f0f90
Remove deprecated ActiveStorage::Transformers::MiniMagickTransformer
Now ActiveStorage only support mini_magick transformation through the
image_processing gem.
2020-10-30 00:25:47 +00:00
Rafael Mendonça França
bc28e37de2
Remove deprecated config.active_storage.queue 2020-10-30 00:25:45 +00:00
Rafael Mendonça França
92a8e1d7c5
Remove deprecated ActiveStorage::Downloading 2020-10-30 00:25:44 +00:00
Rafael França
9492339979
Merge pull request #38443 from jonathanhefner/plugin-fix-stack-traces
Fix stack traces when testing plugins
2020-10-08 19:06:40 -04:00
Eugene Kenny
e44b3419d4 Prefix keys in Active Storage service test
Otherwise this test fails when run concurrently.

https://buildkite.com/rails/rails/builds/71999#77923632-f943-4fa0-b478-0af38001c22b/1032-1098
https://buildkite.com/rails/rails/builds/71999#01fdec79-dc1f-4a6a-b730-ac0df134a2e7/970-1062
2020-10-07 21:47:59 +01:00
Jonathan Hefner
33fdae0584 Fix backtraces for generated plugin tests
`Minitest.plugin_rails_init` sets `Minitest.backtrace_filter` to
`Rails.backtrace_cleaner` right before tests are run, overwriting the
value set in test_helper.rb.

`Rails.backtrace_cleaner` silences backtrace lines that do not start
with `Rails.root` followed by e.g. "lib/" or "test/".  Thus when
`Rails.root` is a subdirectory of the project directory -- for example,
when testing a plugin that has a dummy app -- all lines of the backtrace
are silenced.

This commit adds a fallback such that when all backtrace lines are
silenced, the original `Minitest.backtrace_filter` is used instead.

Additionally, this commit refactors and expands existing test coverage.
2020-10-07 15:40:56 -05:00
Akira Matsuda
d71291cb3c Module#include is a public method since Ruby 2.1 2020-10-06 21:19:24 +09:00
Jonathan Hefner
fd3fd618f4 Add gitattributes for vendored and generated files [ci-skip]
The `linguist-vendored` attribute excludes the specified file from the
project's language stats on GitHub.  The `linguist-generated` attribute
does the same, and also suppresses that file in diffs on GitHub.

See https://github.com/github/linguist for more information.
2020-10-04 16:44:37 -05:00
George Claghorn
a50d2f1993 Extract ActiveStorage::Record
Permit applications to hack in custom DB config for ASt models until ASt has first-class multi-DB support:

    ActiveSupport.on_load(:active_storage_record) do
      connects_to reading: :active_storage_replica, writing: :active_storage_primary
    end

rebase
2020-09-29 11:59:14 -04:00
Pietro Moro
1e2cd7a75e
Support per-environment ASt service config 2020-09-29 09:59:52 -04:00
David Genord II
f4aa54d487
Add requires for mimemagic
This dependency was recently added, but not the require. This looked like it worked because sometimes marcel does the require for us.
2020-09-25 15:00:04 -04:00
George Claghorn
b6d86add73 Remove unused require
Follows up on a185ff8.
2020-09-23 14:39:52 -04:00
George Claghorn
a185ff86c0
Set default variant format from blob content type 2020-09-22 13:03:53 -04:00
Vincent Robert
3803671a81 Generate a preview without print margins
When a PDF is used for both printing and displaying. It will most likely
contain a crop box in order to hide print margins when displaying the PDF.

Use Poppler's parameter to automatically use the crop box (visible box)
rather than the media box (printable box) in order to remove those margins
when drawing the PDF.

See https://manpages.debian.org/testing/poppler-utils/pdftoppm.1.en.html
2020-09-17 11:59:45 -04:00
Santiago Bartesaghi
4a78dcb326
Clarify ActiveStorage::Service#url docs [ci skip]
* Public services don’t respect `:disposition`, `:filename`, or `:content_type`.
* `:expires_in` is optional.
2020-09-12 12:47:33 -04:00
George Claghorn
9055156668 Override ActiveStorage.signed_id_verifier instead of assigning
Guard against the case where ActiveStorage.verifier isn't yet initialized at load time.

Yes, you're not supposed to load AR models in initializers, but it's easy to do accidentally as long as we don't prevent it. We should be resilient against it wherever practical.
2020-09-02 08:41:15 -04:00
George Claghorn
b221a4dc43
Proxying: sanitize Content-Type and Content-Disposition
Prevent XSS where unsafe content is served inline on the application origin.

Follows up on #34477. References 06ab7b2 and d40284b.
2020-08-31 11:24:13 -04:00
Jonathan Hefner
a197d39e13
Fall back to video container for duration
Some video formats, such as WebM, do not store duration information at
the level of the video stream.  Instead, the duration is stored as part
of the container format information.

This commit modifies `VideoAnalyzer` to use the duration from the video
container when the duration from the video stream is not available.

Fixes #40130.
2020-08-31 08:48:06 -04:00
Jonathan Hefner
601006c56d
Update service metadata for updated Blobs only
Follow-up to #40013.

Newly created `Blob`s may not be uploaded yet, so do not try to update
their service metadata.
2020-08-13 10:45:46 -04:00
Jonathan Hefner
6aa26c30e2
Identify directly-uploaded blobs before saving the associated record
An Active Storage `Blob` must be identified before it can be reliably
validated.  For direct uploads, a `Blob` is identified when it is
attached, rather than when it is created.

Before this commit, the sequence of events when attaching a `Blob` was:

1. Find the `Blob`.
2. Assign the `Blob` to an `Attachment`.
3. Save the owner record.
4. Save the `Attachment`.
5. Identify the `Blob`'s true `content_type` from its file.
6. Save the `Blob`.

This meant that the owner record's validations might not see the
`Blob`'s true `content_type`.

After this commit, the sequence of events will be:

1. Find the `Blob`.
2. Identify the `Blob`'s true `content_type` from its file.
3. Assign the `Blob` to an `Attachment`.
4. Save the owner record.
5. Save the `Attachment`.
6. Save the `Blob`.

Thus the `Blob`'s true `content_type` will be available when running the
owner record's validations.
2020-08-11 18:08:09 -04:00
Lawrence Chou
eb5f5ed0be
Fix attaching blobs via nested attributes
Closes #37411.
2020-08-04 00:26:27 -04:00
Victor Perez Rodriguez
3eb48a2148 fix misleading variant test
This commit fixes the "resized variation of BMP blob" test.

By default `create_file_blob` use "image/jpeg" as content type,
since this test does not specify the correct `content_type` for
a `*.bmp` image ("image/bmp") the `ActiveStorage::Variant#specification`
consider the blob as a web image which causes the variant to return a
`*.bmp` URL and a "BMP" type, this is an incorrect behavior since if you
upload a `*.bmp` image the variant will have a PNG format with "image/png"
as content type.

After this change the test will cover the current activestorage behavior.

Changes:

* Specify correct `content_type` on "resized variation of BMP blob" test.
* Change asserts to cover the current activestorage behavior.
2020-07-29 16:32:14 -05:00
Victor Perez Rodriguez
396b43a99d touch parent model when an attachment is purged
Currently `delete` is used on the `purge` and `purge_later` methods,
that prevent any callbacks to be triggered which causes the parent
model to not be updated when an attachment is purged. This behaviour
cause issues on some caching strategies as reported here: https://github.com/rails/rails/issues/39858

Changes:

* Add `record&.touch` on `attachment#purge`
* Add `record&.touch` on `attachment#purge_later`
* Remove extra blank line on attachment.rb
* Add tests which are failing before this change and pass after the change
2020-07-26 12:22:37 -05:00
Evgeniy Rashchepkin
6af5662165
Disable template digesting for Active Storage controllers
Error entries appear in the log when we request ActiveStorage
controllers (`ActiveStorage::Representations::ProxyController#show`,
`ActiveStorage::Blobs::ProxyController#show`).

These entries look like: "Couldn't find template for digesting:
active_storage/representations/proxy/show".

These controllers use the method
`ActionController::ConditionalGet#http_cache_forever`,
and therefore `ActionController::ConditionalGet#combine_etags` method,
and therefore `ActionController::EtagWithTemplateDigest` module via
`etaggers` array.

`ActionController::EtagWithTemplateDigest` module requires a template
(view).

We have no views in ActiveStorage, so `EtagWithTemplateDigest` is
now turned off in ActiveStorage controllers by
`etag_with_template_digest` class attribute.
2020-07-05 10:51:08 -04:00
Santiago Bartesaghi
31148cd6be Use Active Record signed IDs in Active Storage 2020-07-04 21:09:00 -04:00
Ross Kaffenberger
04cbaa1475 Use ES module syntax for application.js.tt and docs
This change swaps the CommonJS require() syntax in the Webpacker
application.js pack template file and in documentation examples with ES
module import syntax.

Benefits of this change include:

Provides continuity with the larger frontend community: Arguably, one of
the main draws in adopting Webpacker is its integration with Babel to
support ES module syntax. For a fresh Rails install with Webpacker, the
application.js file will be the first impression most Rails developers
have with webpack and Webpacker.  Most of the recent documentation and
examples they will find online for using other libraries will be based
on ES module syntax.

Reduces confusion: Developers commonly add ES imports to their
application.js pack, typically by following online examples, which means
mixing require() and import statements in a single file. This leads to
confusion and unnecessary friction about differences between require()
and import.

Embraces browser-friendliness: The ES module syntax forward-looking and
is meant to be supported in browsers. On the other hand, require()
syntax is synchronous by design and not browser-supported as CommonJS
originally was adopted in Node.js for server-side JavaScript. That
webpack supports require() syntax is merely a convenience.

Encourages best practices regarding optimization: webpack can statically
analyze ES modules and "tree-shake", i.e., strip out unused exports from
the final build (given certain conditions are met, including
`sideEffects: false` designation in package.json).
2020-06-16 15:12:12 -04:00
George Claghorn
fd61bae02c ActiveStorage::Attachment#signed_id must return a signed *blob* ID
Fixes that calling e.g. `url_for @user.avatar` would return an incorrect URL resulting in a 404. The Active Storage URL helper ends up calling ActiveStorage::Attachment#signed_id, which previously delegated to ActiveStorage::Blob#signed_id but changed to return a signed *attachment* ID in 1a3dc42.

The Active Storage controllers pass signed IDs to ActiveStorage::Blob.find_signed, so we need signed blob IDs.
2020-05-28 09:20:49 -04:00
Jonathan Hefner
d02d259b61
Fix Active Storage behavior on record dup
Prior to this commit, a `dup`ed record and its originating record would
share the same `Attached` proxy objects.  Those proxy objects, in turn,
would point to the same `Attachment` associations, causing changes made
through the proxy interface to affect both records.
2020-05-25 22:03:32 -04:00
Jonathan Hefner
03f5f57af0 Test Attached::Many in Attached::Many test
This exact test exists in `one_test.rb`.  It was probably overlooked
after being copied and pasted.
2020-05-22 22:52:38 -05:00
George Claghorn
7e52d0ae58 Preserve old redirect URLs
They may be cached for existing apps upgrading to 6.1. Follows up on #34477.
2020-05-21 10:36:34 -04:00
fatkodima
6c4f3be929 Unify raise_on_missing_translations for views and controllers 2020-05-20 02:42:59 +03:00
Rafael França
a80115b7c5
Merge pull request #39204 from prathamesh-sonpatki/template-annotation
Add the configuration option for annotating templates with file names to the generated app
2020-05-19 19:11:31 -04:00
Paul Blaze
84057dab16
Update aws-sdk-s3 dependency
whitelist_headers support added in 1.48.0:
https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/CHANGELOG.md#1480-2019-08-30
2020-05-19 23:09:39 +08:00
Prathamesh Sonpatki
a673ce69e7
Rename annotate_template_file_names to annotate_rendered_view_with_filenames
- Add the configuration option for annotating templates with file names to the generated app.
- Add `annotate_rendered_view_with_filenames` option to configuring guide.
2020-05-19 09:28:14 +05:30
Ryuta Kamizono
393df7425f Fix syntax error 2020-05-19 01:04:22 +09:00
Travis Pew
c0ab9a7d29
Include Content-Length in signature for ActiveStorage direct upload
[CVE-2020-8162]
2020-05-15 14:32:19 -07:00
George Claghorn
57d926a78a Fix unintentional method redefinitions
References dfb5a82.
2020-05-14 09:46:16 -04:00
Jonathan Fleckenstein
dfb5a82b25
Active Storage: allow serving files by proxying 2020-05-11 16:21:58 -04:00