Dumping the schema is on by default for all databases in an application. To turn it off for a
specific database use the `schema_dump` option:
```yaml
# config/database.yml
production:
schema_dump: false
```
Co-authored-by: Luis Vasconcellos <vasconcelloslf@gmail.com>
In #42604 we added functionality that would sort the migrations by
timestamp so that if there were 2 migrations in 2 databases that were
dependent on one another they would be run in timestamp order rather
than in cluster order.
While debugging another issue I realized that if the timestamps are the
same that only one of the migrations will get run. This fixes the issue
by collecting all the db configs for a version so that we get the entire
list.
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Previously if there were 2 migrations in one db and 1 migration in the
other db all the migrations for db one would run and then all migrations
for db two would run. If a migration in one database depended on a
migration in another database then it could fail. This is probably
pretty rare, however in a multi-db application that's moving tables from
one db to another, running them out of order could result in a migration
error.
In this this change we collect all the versions for each migration and
the corresponding db_config so we can run them in the order they are
created rather than per-db.
Closes#41664
Related #41538
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Co-authored-by: Kiril Dokh <dsounded@gmail.com>
* Guard against using VERSION with db:rollback
I recently ran a migration that I needed to rollback, and admittedly, I often forget the proper incantation for this 😅
so the first thing I tried was to run `bin/rake db:rollback VERSION=123454679`. I had hoped that this reverted my
migration back and at first glance I thought it worked. However on closer inspection I realized that it was a different
migration, which initially confused me.
So I looked over the docs and saw that I was using the rake task incorrectly, and promptly corrected my mistake.
Proposal
Looking at the how the `:down` task is defined we see
8dc7439058/activerecord/lib/active_record/railties/databases.rake (L206-L211)
This got me thinking that maybe it would be helpful to have the opposite of this guard defined in `rollback` so that if
`VERSION` is passed it will raise an exception instead of just negecting the extra argument. This could help the user
realize that something went wrong instead of just seeing output and assuming that the rollback happened.
Change
We now raise an execption if `VERSION` is passed when attempting to rollback a migration
* update test name and fix failing test
* remove byebug
[Nick Borromeo + Kate Travers + Rafael Mendonça França]
The handling for single database applications has always set a schema.rb
or structure.sql files for loading the database schema. When we first
implemented multiple database support we intended to keep this for the
original, default database. Afterall Rails _has_ to connect to something
on boot. In development only one connection is connected on boot since
we don't eager load the app.
Originally we had thought that all applications should be required to
add a `primary` entry in the database configurations file. However,
this hasn't worked in practice and we have some code now that does not
assume there's a primary. The schema dumping/loading code however,
still assumed there was a "primary" in the configurations file.
We want the "default" database in any application to use the original
files even when converted to a multiple database application as this
reduces the need to make changes when implementing this functionality on
an existing application.
The changes here update Rails to ensure that we treat either "primary"
or the first database configuration for an environment as "default".
If there is a "primary" that will be used as the default configuration.
If there is no primary the configuration that is first for an
environment will be used as the default. For schema dump/load this means
that the default configuration (primary or first) will use `schema.rb`
as the filename and other configurations will use
`[CONFIGURATION_NAME]_schema.rb`.
This should also help us finish the pull request to infer migrations
paths since now we can say the first configuration is the default. This
is a natural assumption for application developers.
Followup to #39536
On current master with multiple DBs configured, calling db:migrate:redo fails when trying to run db:rollback.
Before:
```
» bin/rails db:migrate:redo
rake aborted!
You're using a multiple database application. To use `db:rollback` you must run the namespaced task with a VERSION. Available tasks are db:rollback:primary and db:rollback:secondary.
Tasks: TOP => db:rollback
(See full trace by running task with --trace)
```
After:
```
» bin/rails db:migrate:redo
rake aborted!
You're using a multiple database application. To use `db:migrate:redo` you must run the namespaced task with a VERSION. Available tasks are db:migrate:redo:primary and db:migrate:redo:secondary.
Tasks: TOP => db:migrate:redo
(See full trace by running task with --trace)
```
Running the namespaced version:
```
» bin/rails db:migrate:redo:secondary
== 20200728162820 CreateAnimals: reverting ====================================
-- drop_table(:animals)
-> 0.0025s
== 20200728162820 CreateAnimals: reverted (0.0047s) ===========================
== 20200728162820 CreateAnimals: migrating ====================================
-- create_table(:animals)
-> 0.0028s
== 20200728162820 CreateAnimals: migrated (0.0029s) ===========================
```
This test was incorrect. `primary` was winning for the schema cache load
but when you boot an application it's actually the first configuration
that wins (in a multi db app).
The test didn't catch this because I forgot to add a migrations_paths to
the configuration.
We updated the schema cache loader railtie as well because any
application that didn't have a `primary` config would not be able to use
the schema cache. Originally we thought we'd enforce a `primary`
configuration but no longer feel that's correct. It's simpler to say
that the first wins in a 3-tier rather than implementing a solution to
require `primary` and / or allow aliases.
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Mandrill's Inbound API checks to see if a URL exists before it creates
the webhook. It sends a HEAD request, to which we now return a 200 OK
response to indicate that the route exists.
Now we can generate inbound API calls with ease on Mandrill, without
having to shuffle around tokens in production.
Fixes#37609.
* Add a way to deliver inbound emails by source
Great for testing when you get an eml file and can just paste it in.
* Test updates
* Fix tests
* Fix spacing
* require, require_relative, load by double quotes
We're getting rid of all single quote usage, unless it serves a specific purpose, as per the general style guide.
With a multiple database application `db:rollback` becomes problematic.
We can't rollback just the primary, that doesn't match the behavior in
the other tasks. We can't rollback a migration for every database, that
is unexpected.
To solve this I handled `db:rollback` the same way I handled `:up` and
`:down`. If `db:rollback` is called for a multi-db application then it
will raise an error recommending you use `db:rollback:[NAME]` instead.
Calling `db:rollback:primary` or `db:rollback:animals` will rollback
the migration for the number of steps specified.
Closes: #38513
Follow-up to: #34078
If I had had these tests previously I would have not created PR #38658
and then promptly realize I needed to revert it.
We need to load and parse the configurations twice. Once before the
environment is loaded to create the named tasks and once after the
environment is loaded to have the real configurations. The configs
loaded before the env have the ERB stripped out and aren't valid
configs.
I have so. many. regrets. about using `spec_name` for database
configurations and now I'm finally putting this mistake to an end.
Back when I started multi-db work I assumed that eventually
`connection_specification_name` (sometimes called `spec_name`) and
`spec_name` for configurations would one day be the same thing. After
2 years I no longer believe they will ever be the same thing.
This PR deprecates `spec_name` on database configurations in favor of
`name`. It's the same behavior, just a better name, or at least a
less confusing name.
`connection_specification_name` refers to the parent class name (ie
ActiveRecord::Base, AnimalsBase, etc) that holds the connection for it's
models. In some places like ConnectionHandler it shortens this to
`spec_name`, hence the major confusion.
Recently I've been working with some new folks on database stuff and
connection management and realize how confusing it was to explain that
`db_config.spec_name` was not `spec_name` and
`connection_specification_name`. Worse than that one is a symbole while
the other is a class name. This was made even more complicated by the
fact that `ActiveRecord::Base` used `primary` as the
`connection_specification_name` until #38190.
After spending 2 years with connection management I don't believe that
we can ever use the symbols from the database configs as a way to
connect the database without the class name being _somewhere_ because
a db_config does not know who it's owner class is until it's been
connected and a model has no idea what db_config belongs to it until
it's connected. The model is the only way to tie a primary/writer config
to a replica/reader config. This could change in the future but I don't
see value in adding a class name to the db_configs before connection or
telling a model what config belongs to it before connection. That would
probably break a lot of application assumptions. If we do ever end up in
that world, we can use name, because tbh `spec_name` and
`connection_specification_name` were always confusing to me.
The schema cache tests test the following scenarios:
1) The default case works (single db, primary spec name (dev is default
to primary in 2-tier config), standard default schema cache filename)
2) Primary always wins over other entries
3) A custom schema cache filename works when set in the configuration
4) A custom schema cache filename works when set in the ENV
Cases that don't work:
1) A non-primary database entry picks up a namespaced schema cache file
This can't work currently because there's no way of knowing which cache
we actually want. In this railtie we can only load ActiveRecord::Base's
schema cache. If we grab the first config we risk loading a cache for
another connection because order is not guaranteed.
2) Multi-db schema caches
The reasons are similar to above. In addition we can't loop through the
configs, establish a connection, and load the cache because we don't
know what parent class to establish a connection to. In that case AR
Base will always get the cache and it would cause the last one to win
and therefore be loaded on the wrong connection.
The real fix for these issues is to get rid of the railtie entirely, but
for now we needed to set this back to what the behavior was before
recent changes but with the ability to pass a custom key.
Co-authored-by: Katrina Owen <kytrinyx@github.com>
Database configurations are now objects almost everywhere, so we don't
need to fake access to a hash with `#default_hash` or it's alias `#[]`.
Applications should `configs_for` and pass `env_name` and `spec_name` to
get the database config object. If you're looking for the default for
the test environment you can pass `configs_for(env_name: "test", spec_name:
"primary")`. Change test to developement to get the dev config, etc.
`#default_hash` and `#[]` will be removed in 6.2.
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Fixes https://github.com/rails/rails/issues/28827.
The steps to reproduce are as follows:
git clone git@github.com:bbuchalter/rails-issue-28827.git
cd rails-issue-28827
bundle install
bin/rails db:create
Observe that we create two databases when invoking db:create: development and test. Now observe what happens when we invoke our drop command while using DATABASE_URL.
DATABASE_URL=sqlite3://$(pwd)/db/database_url.sqlite3 bin/rails db:create
As expected, the development environment now uses the DATABASE_URL. What is unexpected is that the test environment does not.
It's unclear what the expected behavior should be in this case, but the cause of it is this: 9f2c74eda0/activerecord/lib/active_record/tasks/database_tasks.rb (L494)
Because of each_local_configuration, there seems to be no way invoke these database rake on only the development environment to ensure DATABASE_URL is respected.
The smallest scope of change I can think to make would be to conditionalize this behavior so it does not get applied when DATABASE_URL is present.
We missed these in rails/rails#38005 because deprecation warnings are
silently swallowed by these tests.
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
- The migrate task iterates and establish a connection over each db
resulting in the last one to be used by subsequent rake tasks.
We should reestablish a connection to the connection that was
established before the migrate tasks was run
- Fix#37578
Services can be configured in `config/storage.yml` with a new key
`public: true | false` to indicate whether a service holds public
blobs or private blobs. Public services will always return a
permanent URL.
Deprecates `Blob#service_url` in favor of `Blob#url`.
Convert all uses of `db_config.configuration_hash[*]` to use methods
defined on an implementation of `DatabaseConfigurations::DatabaseConfig`.
Since we want to get away from accessing properties directly on the
underlying configuration hash, we'll move here to accessing those values
via the implementations on `DatabaseConfig` (or more specifically,
`HashConfig`).
There are still codepaths that are passing around `configuration_hash`,
and follow-on PRs will address those with the goal of using
configuration objects everywhere up until the point we pass a resolved
hash over to the underlying client.
Co-authored-by: eileencodes <eileencodes@gmail.com>
Eventually we'd like to get rid of this class altogether but for now
this PR reduces the surface area by removing methods from the class and
moving classes out into their own files.
* `adapter_method` was moved into database configurations
* `initialize_dup` was removed because it was only used in tests
* Resolver is now it's own class under connection adapters
* ConnectionUrlResolver, only used by the configurations, is in a class
under DatabaseConfigurations
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Previously in some places we used symbol keys, and in some places we used
string keys. That made it pretty confusing to figure out in a particular
place what type of configuration object you were working with.
Now internally, all configuration hashes are keyed by symbols and
converted to such on the way in.
A few exceptions:
- `DatabaseConfigurations#to_h` still returns strings for backward compatibility
- Same for `legacy_hash`
- `default_hash` previously could return strings, but the associated
comment mentions it returns symbol-key `Hash` and now it always does
Because this is a change in behavior, a few method renames have happened:
- `DatabaseConfig#config` is now `DatabaseConfig#configuration_hash` and returns a symbol-key `Hash`
- `ConnectionSpecification#config` is now `ConnectionSpecification#underlying_configuration_hash` and returns the `Hash` of the underlying `DatabaseConfig`
- `DatabaseConfig#config` was added back, returns `String`-keys for backward compatibility, and is deprecated in favor of the new `configuration_hash`
Co-authored-by: eileencodes <eileencodes@gmail.com>
