We are moving to using `DatabaseConfig` objects everywhere inside of
Rails instead of passing around Hash objects.
This PR moves to using `DatabaseConfig` objects inside of the individual
database tasks adapters. In the interest of not breaking existing
adapters, we've introduced an upgrade path by which adapters can get a
hold of database configuration objects instead of Hashes by implementing
a method `self.using_database_configurations?`
Co-authored-by: eileencodes <eileencodes@gmail.com>
We're moving in the direction of getting `DatabaseConfig` to be used in
more places instead of `Hash` connections configurations being passed
around directly.
Here we're making all of `DatabaseTasks` and `databases.rake` use
database configuration objects.
A few notes:
* This also contains some tests that were missing for public methods on
`DatabaseTasks`: `charset_current` & `collation_current`
* There is a deprecation of some arguments in `schema_up_to_date?` that
are now duplicative/confusing. One of them wasn't used previously
(`environment`), and now `spec_name` can be pulled directly off of the
`db_config (DatatabaseConfig)` object.
Co-authored-by: eileencodes <eileencodes@gmail.com>
Uses of `current_config` and `current_config=` were removed when
implementing multiple datatabase Rails tasks. This means they are no
longer used internally. However, since it is a documented method it may
be used in gems or applications. This change deprecates these methods so
that we can remove them in 6.2.
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
This commit fixes a method typo that was introduced in #37199, also also
adds tests for the `DatabaseTasks#current_config` method given it's
current implementation.
While I was in here, I also made it so that if `current_config` is
called with an `env` it won't try to look up the `Rails.env`
potentially resulting in an error for non-Rails application.
IMO this method should be deprecated & removed as it's no longer used in
Rails, is confusing (for example: `test_current_config_read_after_set`),
and is currently dependent on Rails if `env` isn't passed.
Co-authored-by: eileencodes <eileencodes@gmail.com>
Previously in some places we used symbol keys, and in some places we used
string keys. That made it pretty confusing to figure out in a particular
place what type of configuration object you were working with.
Now internally, all configuration hashes are keyed by symbols and
converted to such on the way in.
A few exceptions:
- `DatabaseConfigurations#to_h` still returns strings for backward compatibility
- Same for `legacy_hash`
- `default_hash` previously could return strings, but the associated
comment mentions it returns symbol-key `Hash` and now it always does
Because this is a change in behavior, a few method renames have happened:
- `DatabaseConfig#config` is now `DatabaseConfig#configuration_hash` and returns a symbol-key `Hash`
- `ConnectionSpecification#config` is now `ConnectionSpecification#underlying_configuration_hash` and returns the `Hash` of the underlying `DatabaseConfig`
- `DatabaseConfig#config` was added back, returns `String`-keys for backward compatibility, and is deprecated in favor of the new `configuration_hash`
Co-authored-by: eileencodes <eileencodes@gmail.com>
Previously it was the responsibility of the database tasks to translate
the invalid statement from creating a duplicate database into an
ActiveRecord::Tasks::DatabaseAlreadyExists error.
It's actually easier for us to do this detection inside of the adapter,
where we already do a case statement on the return code to translate the
error.
This commit introduces ActiveRecord::DatabaseAlreadyExists, a subclass
of StatementInvalid, and updates both AbstractMysqlAdapter and
PostgresqlAdapter to return this more specific exception in that case.
Because this is a subclass of the old exception, StatementInvalid, it
should be backwards compatible with any code expecting that from
create_database.
This works for both create_database and exectute("CREATE DATABASE")
MySQLDatabaseTasks, like AbstractMysqlAdapter, should be able to operate
on any mysql adapter, not just mysql2. Errors having a .error_number
attribute is a mysql2 specific API, which we (Rails) don't control, so
we should instead use connection.error_number(err), which we do.
This also updates tests to better test how this really works, previously
it stubbed create_database to raise Tasks::DatabaseAlreadyExists, which
can never happen.
Most existing tests expects `connection_handlers` has only one default
handler, but the test added at #34779 dirties that.
We need to reset `connection_handlers` to default in that case.
Closes#35471.
* Add `ActiveRecord::Base.connection.truncate` for SQLite3 adapter.
SQLite doesn't support `TRUNCATE TABLE`, but SQLite3 adapter can support
`ActiveRecord::Base.connection.truncate` by using `DELETE FROM`.
`DELETE` without `WHERE` uses "The Truncate Optimization",
see https://www.sqlite.org/lang_delete.html.
* Add `rails db:seed:replant` that truncates database tables and loads the seeds
Closes#34765
There are two main reasons why `assert_called_with` should require
`args` argument:
1) If we want to assert that some method should be called and we don't
need to check with which arguments it should be called then we should use
`assert_called`.
2) `assert_called_with` without `args` argument doesn't assert anything!
```ruby
assert_called_with(@object, :increment) do
@object.decrement
end
```
It causes false assertions in tests that could cause regressions in the project.
I found this bug by working on
[minitest-mock_expectations](https://github.com/bogdanvlviv/minitest-mock_expectations) gem.
This gem is an extension for minitest that provides almost the same method call
assertions.
I was wondering whether you would consider adding "minitest-mock_expectations"
to `rails/rails` instead of private `ActiveSupport::Testing::MethodCallAssertions` module.
If yes, I'll send a patch - a970ecc42c
psqlrc files can affect the execution of commands in ways that can hold
up execution by blocking or otherwise cause unexpected side effects and
should best be ignored when using psql programmatically.
After #33637 some tests in `activerecord/test/cases/tasks/database_tasks_test.rb`
don't assert anything.
We used to stub `ActiveRecord::Base::configurations` method in those
tests like `ActiveRecord::Base.stub(:configurations, @configurations) {}`.
Since #33637 `ActiveRecord::Base::configurations` is a `ActiveRecord::DatabaseConfigurations`
object(not a Hash object) we can't do so anymore.
`ActiveRecord::DatabaseConfigurations` object builds during `ActiveRecord::Base::configurations=`.
We can replace `ActiveRecord::Base.stub(:configurations, @configurations) {}` to
```
begin
old_configurations = ActiveRecord::Base.configurations
ActiveRecord::Base.configurations = @configurations
# ...
ensure
ActiveRecord::Base.configurations = old_configurations
end
```
Also I fixed tests in `activerecord/test/cases/tasks/legacy_database_tasks_test.rb`
But currently It looks like duplication of
`activerecord/test/cases/tasks/database_tasks_test.rb`.
We should improve those tests or remove them.
I've tried (in `activerecord/test/cases/tasks/legacy_database_tasks_test.rb` file):
```
def with_stubbed_configurations
old_configurations = ActiveRecord::Base.configurations.to_h
ActiveRecord::Base.configurations = @configurations
ActiveRecord::Base.stub(:configurations, ActiveRecord::Base.configurations.to_h) do
yield
end
ensure
ActiveRecord::Base.configurations = old_configurations
end
```
but it causes erros in tests cases.
After discussion we decided to remove
`activerecord/test/cases/tasks/legacy_database_tasks_test.rb`
Related to #33637
While the three-tier config makes it easier to define databases for
multiple database applications, it quickly became clear to offer full
support for multiple databases we need to change the way the connections
hash was handled.
A three-tier config means that when Rails needed to choose a default
configuration (in the case a user doesn't ask for a specific
configuration) it wasn't clear to Rails which the default was. I
[bandaid fixed this so the rake tasks could work](#32271) but that fix
wasn't correct because it actually doubled up the configuration hashes.
Instead of attemping to manipulate the hashes @tenderlove and I decided
that it made more sense if we converted the hashes to objects so we can
easily ask those object questions. In a three tier config like this:
```
development:
primary:
database: "my_primary_db"
animals:
database; "my_animals_db"
```
We end up with an object like this:
```
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
```
The configurations setter takes the database configuration set by your
application and turns them into an
`ActiveRecord::DatabaseConfigurations` object that has one getter -
`@configurations` which is an array of all the database objects.
The configurations getter returns this object by default since it acts
like a hash in most of the cases we need. For example if you need to
access the default `development` database we can simply request it as we
did before:
```
ActiveRecord::Base.configurations["development"]
```
This will return primary development database configuration hash:
```
{ "database" => "my_primary_db" }
```
Internally all of Active Record has been converted to use the new
objects. I've built this to be backwards compatible but allow for
accessing the hash if needed for a deprecation period. To get the
original hash instead of the object you can either add `to_h` on the
configurations call or pass `legacy: true` to `configurations.
```
ActiveRecord::Base.configurations.to_h
=> { "development => { "database" => "my_primary_db" } }
ActiveRecord::Base.configurations(legacy: true)
=> { "development => { "database" => "my_primary_db" } }
```
The new configurations object allows us to iterate over the Active
Record configurations without losing the known environment or
specification name for that configuration. You can also select all the
configs for an env or env and spec. With this we can always ask
any object what environment it belongs to:
```
db_configs = ActiveRecord::Base.configurations.configurations_for("development")
=> #<ActiveRecord::DatabaseConfigurations:0x00007fd1acbdf800
@configurations=[
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbded10
@env_name="development",@spec_name="primary",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>,
#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007fd1acbdea90
@env_name="development",@spec_name="animals",
@config={"adapter"=>"sqlite3", "database"=>"db/development.sqlite3"}>
]>
db_config.env_name
=> "development"
db_config.spec_name
=> "primary"
db_config.config
=> { "adapter"=>"sqlite3", "database"=>"db/development.sqlite3" }
```
The configurations object is more flexible than the configurations hash
and will allow us to build on top of the connection management in order
to add support for primary/replica connections, sharding, and
constructing queries for associations that live in multiple databases.
Issue #27852 reports that when `rails db:create` fails, it causes
leaking of user's DB credentials to $stderr.
We print a DB's configuration hash in order to help users more quickly
to figure out what could be wrong with his configuration.
This commit changes message from
"Couldn't create database for #{configuration.inspect}" to
"Couldn't create '#{configuration['database']}' database. Please check your configuration.".
There are two PRs that fixing it #27878, #27879, but they need a bit more work.
I decided help to finish this and added Author of those PRs credit in this commit.
Since it is a security issue, I think we should backport it to
`5-2-stable`, and `5-1-stable`.
Guided by https://edgeguides.rubyonrails.org/maintenance_policy.html#security-issuesFixes#27852Closes#27879
Related to #27878
[Alexander Marrs & bogdanvlviv]
Six Mocha calls prove quite resistant to Minitestification. For example,
if we replace
```
ActiveRecord::Associations::HasManyAssociation
.any_instance
.expects(:reader)
.never
```
with `assert_not_called`, Minitest wisely raises
```
NameError: undefined method `reader' for class `ActiveRecord::Associations::HasManyAssociation'
```
as `:reader` comes from a deeply embedded abstract class,
`ActiveRecord::Associations::CollectionAssociation`.
This patch tackles this difficulty by adding
`ActiveSupport::Testing::MethodCallAsserts#assert_called_on_instance_of`
which injects a stubbed method into `klass`, and verifies the number of
times it is called, similar to `assert_called`. It also adds a convenience
method, `assert_not_called_on_instance_of`, mirroring
`assert_not_called`.
It uses the new method_call_assertions to replace the remaining Mocha
calls in `ActiveRecord` tests.
[utilum + bogdanvlviv + kspath]
Use attr_reader/attr_writer instead of methods
method is 12% slower
Use flat_map over map.flatten(1)
flatten is 66% slower
Use hash[]= instead of hash.merge! with single arguments
merge! is 166% slower
See https://github.com/rails/rails/pull/32337 for more conversation
Step 6 in #33162
When using Mocha like this:
`ActiveRecord::Base.expects(:establish_connection).with(some_args)`,
the expectations created look something like this:
```
@expectations=
[#<Expectation:0x561350d968e0 expected exactly once, not yet invoked: ActiveRecord::Base.establish_connection("adapter" => "mysql2", "database" => nil) >,
#<Expectation:0x561350dab8f8 allowed any number of times, not yet invoked: ActiveRecord::Base.establish_connection(any_parameters) >,
#<Expectation:0x561350dc30c0 allowed any number of times, not yet invoked: ActiveRecord::Base.connection(any_parameters) >]
```
Minitest mocking (and the way we use it in `MethodCallAssertions`)
expressly refuses to facilitate such permissiive expectations, insisting
that all calls be specified in the actual expected order.
This patch replaces such calls to `Mocha#expects` with
`ActiveSupport::Testing::MethodCallAssertions` and specifies all
expected calls in the epxected order.
Many calls to `Mocha#expects` preceded the introduction of
`ActiveSupport::Testing::MethodCallAssertions` in 53f64c0fb,
and many are simple to replace with `MethodCallAssertions`.
This patch makes all these simple replacements.
Step 5 in #33162
Remove extra stub of `ActiveRecord::Base::connection` in
`activerecord/test/cases/tasks/mysql_rake_test.rb`.
Remove extra stub of `File::exist?` in
`activerecord/test/cases/tasks/sqlite_rake_test.rb`.
`ActiveRecord::Base::establish_connection` shouldn't return `true`
in test cases.
Related to https://github.com/rails/rails/pull/33337.
Remove returning of `false` value for stubbed `lock_thread=` methods
since there aren't any needs in it.
Remove unnecessary returning of `true` for stubbed `drop_database` method.
Follow up #33309.
Related to #33162, #33326.
An entry in `ActiveRecord::Base.configurations` can either be a
connection spec ("two-level") or a hash of specs ("three-level").
We were detecting two-level configurations by looking for the `database`
key, but the database can also be specified as part of the `url` key,
which meant we incorrectly treated those configurations as three-level.
This reverts commit 16f279ebd474626577ced858e3626ac4535a33df, reversing
changes made to 6c6a30a7c357ce1eafa093d77d2b08684fe50887.
The config can be named anything, not just default (although all
generated apps will be named default). We can't just delete configs that
don't have a database because that will break three-tier configs. Oh
well.
Because of this default configuration we're constantly checking if the
database exists when looping through configurations. This is unnecessary
and we should just delete it before we need to loop through
configurations.
Rails has some support for multiple databases but it can be hard to
handle migrations with those. The easiest way to implement multiple
databases is to contain migrations into their own folder ("db/migrate"
for the primary db and "db/seconddb_migrate" for the second db). Without
this you would need to write code that allowed you to switch connections
in migrations. I can tell you from experience that is not a fun way to
implement multiple databases.
This refactoring is a pre-requisite for implementing other features
related to parallel testing and improved handling for multiple
databases.
The refactoring here moves the class methods from the `Migrator` class
into it's own new class `MigrationContext`. The goal was to move the
`migrations_paths` method off of the `Migrator` class and onto the
connection. This allows users to do the following in their
`database.yml`:
```
development:
adapter: mysql2
username: root
password:
development_seconddb:
adapter: mysql2
username: root
password:
migrations_paths: "db/second_db_migrate"
```
Migrations for the `seconddb` can now be store in the
`db/second_db_migrate` directory. Migrations for the primary database
are stored in `db/migrate`".
The refactoring here drastically reduces the internal API for migrations
since we don't need to pass `migrations_paths` around to every single
method. Additionally this change does not require any Rails applications
to make changes unless they want to use the new public API. All of the
class methods from the `Migrator` class were `nodoc`'d except for the
`migrations_paths` and `migrations_path` getter/setters respectively.
