Previously we prevent the user migrating the database in either
direction if there were missing or out-of-order migrations. However,
going down should always be safe (assuming proper down scripts were
written) and often going down is the proper way to resolve out-of-order
migrations. Going down should always be done thoughtfully (it is highly
likely to lead to data loss), and really should not be done in a
production setting.
However, in a development environment, it is not unusual to get into
states where you have missing scripts. For example, if there are test
scripts that apply test data to a development environment on top of the
formal application schema, you may need to back out the test migrations
before creating new application migrations to avoid getting into an
"inconsistent state." Consider the following migrations:
- 1-initial-schema
- 2-add-feature-x
- T1-test-user-data
If you then add *3-add-feature-y*, you will be in an inconsistent
state, as the expected ordering from db_migrate's point of view will be:
- 1-initial-schema
- 2-add-feature-x
- 3-add-feature-y
- T1-test-user-data
With the previous behavior db_migrate refuses to do anything and you
must manually back-out the test migrations before applying the new
migration. With the new behavior you can easily go down to back out the
test migrations automatically before going back up with the new migrations.
As an aside:
Another way to avoid this is to name test migrations in such a way that
they stay inline with the changes they are built on top of:
- 1-initial-schema
- 1.1-test-user-data
- 2-add-feature-x
- 2.1-feature-x-test-data
But sometimes the previous pattern is preferable, as it allows you to
have test migrations that evolve and match the "current state of test
data" rather than a developer needing to layer the contents of multiple
migrations to get a clear picture of the test data. This same issue
applies to non-test migrations, but db_migrate exists to solve the
use-case where you generally prefer to have traversible, immutable
layers that are used to manage production database schema migrations
downtown.
* The Nim binary now recognizes the following environment and allows them to
override configured values:
- `DATABASE_DRIVER`: overrides the `driver` config value, selects which
kind of database we expect to connect to.
- `MIGRATIONS_DIR`: overrides the `sqlDir` config value, sets the path to
the directory containing the migration SQL files.
- `DATABASE_URL`: overrides the `connectionString` config value, supplies
connection parameters to the database.
- `DBM_LOG_LEVEL`: overrides the `logLevel` config value, sets the logging
level of db_migrate.
* Fixes a bug in the parsing of migration files. Previously the file was split
on `;` characters and chunks that started with `--` were ignored as comments.
This was wrong in the common case where a block starts with a comment but
then contains SQL further. Such a block was being ignored. The new behavior
is to consider each line and build up queries that way.
* Fixes a logging bug when parsing the configuration. If there was an that
prevented the configuration from loading this in turn prevented logging from
being setup correctly, and so the error was not logged. Now errors that may
occur before logging is setup are hard-coded to be logged to STDERR.
* Changes the logic for creating the `migrations` table to check before
performing the query that creates the table. This is to avoid continually
printing messages about skipping this creation when the table already exists
(which is the normal case). This change is PostgreSQL-specific, but of course
the entire tool is currently PostgreSQL-specific.
jdb
Jonathan Bernard