[backend] update queries to postgres

This commit is contained in:
2025-10-20 17:41:29 +02:00
parent 9f6a88e7f8
commit 8244e2ce5f
3 changed files with 2 additions and 78 deletions
-69
View File
@@ -1,69 +0,0 @@
# Database Migration Plan: MariaDB to PostgreSQL
This document captures the current understanding of the persistence layer and lays out the plan for migrating Basango's backend from MariaDB 10.11 to PostgreSQL 16. It will evolve as the Doctrine mappings and migrations are adapted for PostgreSQL-specific capabilities.
## 1. Current Storage Footprint
* Doctrine connects to MariaDB through `DATABASE_URL` defined in `.env` and environment overrides. The URL is currently `mysql://root:root@mariadb:3306/app?serverVersion=Mariadb-10.11.11&charset=utf8mb4`.
* Custom Doctrine DBAL types (`article_id`, `open_graph`, etc.) are registered in `config/packages/doctrine.yaml`. These will continue to work with PostgreSQL as they are platform-agnostic, but their SQL declarations must be verified once the platform switches.
* Article persistence relies on MySQL-specific generated columns for `image` and `excerpt`, and maintains indexes on `hash`, `publishedAt`, and `publishedAt + id` for cursor pagination. JSON is stored via the custom `open_graph` type.
* Feed management tables (`bookmark`, `comment`, etc.) rely on boolean defaults stored as `0/1` and on cascaded relations that will need to be expressed with PostgreSQL `ON DELETE` behaviour.
## 2. Design Goals for the PostgreSQL Schema
1. Preserve all existing data while enabling PostgreSQL features needed by the product roadmap (full-text search, advanced indexing, vector similarity, etc.).
2. Replace MySQL-specific constructs:
* Convert generated columns to PostgreSQL generated columns or materialized expressions (e.g. `image` JSON path extraction can use `GENERATED ALWAYS AS ((metadata->>'image')) STORED`).
* Revisit boolean defaults (`DEFAULT FALSE/TRUE`) instead of `0/1`.
* Use `JSONB` for `open_graph` payloads.
3. Optimise pagination and search:
* Create a descending composite index on `(published_at DESC, id DESC)` to support cursor-based feeds.
* Evaluate trigram (`pg_trgm`) or GIN indexes for text-heavy fields (`title`, `body`, `comment.content`) as part of future search features.
4. Leverage UUIDs or native sequences as appropriate. Current IDs are custom types backed by strings; confirm compatibility or migrate to `UUID` columns if a future mapping change adopts them.
5. Ensure all `ON DELETE CASCADE` relationships continue working and enforce them with foreign keys in PostgreSQL.
## 3. Migration Workflow
1. **Preparation**
* Freeze application writes or enable maintenance mode.
* Capture a final MariaDB snapshot using the existing `bin/backup.sh` script (it currently runs `mysqldump`). Extend it later to dump PostgreSQL once cut-over is complete.
* Provision the target PostgreSQL cluster (version ≥ 16) with sufficient disk and I/O.
* Enable required extensions on the target database (`pg_trgm`, `unaccent`, `uuid-ossp`, `pgcrypto`, `vector`) as they become necessary for new features.
2. **Schema Translation**
* Generate Doctrine migration diffs against PostgreSQL to obtain the base schema.
* Manually adjust the generated migrations to:
- Replace column definitions for custom generated columns and JSON data.
- Introduce explicit collations if case-insensitive comparisons are required.
- Define indexes (including descending and GIN/GIST indexes) tailored to PostgreSQL.
* Version these migrations but defer execution until the data transfer is complete.
3. **Data Transfer**
* Use the new `projects/backend/bin/migration.sh` helper with the production connection strings to run `pgloader` (or equivalent) from MariaDB to PostgreSQL.
* Validate row counts and checksum critical tables after the load.
* Rebuild sequences or UUID generators if the Doctrine mappings change.
4. **Post-Load Migration**
* Execute Doctrine migrations against PostgreSQL to create indexes, generated columns, and constraints that `pgloader` could not translate.
* Run integrity checks (foreign keys, uniqueness).
* Recompute derived data if needed (e.g. if the excerpt column becomes a materialized value stored via trigger or generated column).
5. **Application Cut-over**
* Update environment variables (`DATABASE_URL`, Docker compose definitions, CI secrets) to point to PostgreSQL.
* Deploy code that contains PostgreSQL-specific mappings and migrations.
* Run `bin/console cache:clear` and smoke test core flows (ingesting articles, bookmarking, commenting, authentication).
* Monitor logs and metrics; keep the MariaDB snapshot for rollback until stability is confirmed.
## 4. Testing & Validation Strategy
* Automated: run the Symfony test suite and linters against a PostgreSQL database to surface platform-specific issues.
* Manual: verify API endpoints that rely on time-based pagination to ensure the descending index behaves as expected.
* Performance: benchmark feed queries, full-text search prototypes, and vector similarity routines once indexes are in place.
## 5. Open Questions / TODOs
* Confirm how custom DBAL types serialize/deserialise when mapped to PostgreSQL column types (e.g. ensure `open_graph` becomes `JSONB`).
* Decide whether to retain the string-based identifiers or transition to native UUID columns while switching databases.
* Determine if additional data transformations are required for timezone handling when migrating `datetime_immutable` columns.
* Document rollback procedures once the PostgreSQL migration is complete.
@@ -54,6 +54,7 @@ trait PaginationQuery
if ($keyset->date !== null) {
$qb->addOrderBy($keyset->date, $orderDirection);
}
$qb->addOrderBy($keyset->id, $orderDirection);
$cursor = PaginationCursor::decode($page->cursor);
@@ -305,15 +305,7 @@ final readonly class ImportEngine
}
if (is_string($value)) {
if (strlen($value) === 16) {
$uuid = Uuid::fromBinary($value);
return method_exists($uuid, 'toString')
? $uuid->toString()
: $uuid->toRfc4122();
}
return $value;
return Uuid::fromBinary($value)->toString();
}
return (string) $value;