This change only verifies that Alembic can upgrade and downgrade through
all the current migrations. It does not verify that the application
works correctly with PostgreSQL.
This change initializes the Sentry SDK, which enables error tracking
when the `SENTRY_DSN` environment variable is set.
Drop-in alternatives to Sentry are also supported, like GlitchTip.
Trying to run MySQL as database (instead of MariaDB) was failing due to
incompatible migrations and lack of support for the MySQL driver.
These small changes fix the issue and recover the support for MySQL as
database driver.
Support for MySQL is on a best-effort basis, as the project's main
database is MariaDB.
At the moment, 7zip files are generating memory issues and even OOM
errors on user installations. This is because the current stable release
of `py7zr` does not support decompression streaming, and RomM needs to
decompress the each 7zip file in the library into memory to be able to
calculate hashes.
This change introduces a `py7zr` fork I created to have a stable commit
SHA to refer to in case upstream gets any forced pushes. It includes the
contents of the pull request the `py7zr` creator is working on to
support decompression streaming [1].
The way decompression streaming is implemented in `py7zr` is different
than the other compression utilities. Instead of being able to provide a
`bytes` iterator, we need to provide a `Py7zIO` implementation that
will call a callback on each read and write operation.
[1] https://github.com/miurahr/py7zr/pull/620
The short-term goal is to completely typehint the IGDB API responses. This
first change adds the base structures and enums RomM currently uses.
The `ExpandableField` type will allow us to model the expansion
mechanism the IGDB API provides, where a field can include either an ID,
or the full nested structure.
This change replaces the creation of Redis URL, from a simple string
interpolation, to using `yarl.URL`. The main benefit, besides not
forgetting to set all five different variables on every Redis client
initialization, is that user credentials are correctly URL-encoded, if
present.
Up until now, if a password had special characters, it could break the
generated URL.
This change also introduces support for a `REDIS_SSL` setting, which
allows the user to specify if the Redis connection should use SSL or not.
This change improves memory usage, by only keeping a single archive's
member file in memory at a time during 7zip decompression.
The `py7zr` library does not support streaming decompression yet, so
this change is the best we can do for now.
Potential fix for #1211, but it won't improve memory usage for
single-file 7zip archives.
This change installs and configures the `mod_zip` nginx module [1],
which allows nginx to stream ZIP files directly.
It includes a workaround needed to correctly calculate CRC-32 values for
included files, by including a new `server` section listening at port
8081, only used for the file requests to be upstream subrequests that
correctly trigger the CRC-32 calculation logic.
Also, to be able to provide a `m3u` file generated on the fly, we add a
`/decode` endpoint fully implemented in nginx using NJS, which receives
a `value` URL param, and decodes it using base64. The decoded value is
returned as the response.
That way, the contents of the `m3u` file is base64-encoded, and set as
part of the response, for `mod_zip` to include it in the ZIP file.
[1] https://github.com/evanmiller/mod_zip
Pytest v8.2 introduced the `PYTEST_VERSION` environment variable [1],
that can be used to check if code is running from within a pytest run.
This way, we can avoid checking the loaded `sys` modules.
[1] https://docs.pytest.org/en/stable/changelog.html#id57
Convert `IGDBBaseHandler` methods to be asynchronous, and use an `httpx`
async client, instead of `requests` sync client.
This change also removes the direct dependency with `requests`, as the
project no longer uses it, preferring `httpx` instead.
For filesystem resource handler, `requests` calls have been replaced
with `httpx`, and file I/O has been replaced with `anyio` utils.
The existing approach to save covers and screenshots, by calling
`shutil.copyfileobj` with the raw response is no longer needed. `httpx`
does not provide a file-like object when streaming [1], so there's no
easy drop-in replacement.
However, the applied solution correctly builds the file iteratively, by
consuming the response in chunks.
[1] https://github.com/encode/httpx/discussions/2296
This change applies the guided migration process recommended by
SQLAlchemy [1], up to step 4, to have declarative ORM models that better
support Python typing.
The change was tested by running `alembic check`, which does not find
any schema changes.
Errors reported by `mypy` go down to 170, from the original 223 in the
current `master` commit.
[1] https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#migrating-an-existing-mapping
`aioredis` is abandoned, and has been replaced by `redis.asyncio`.
Currently, the project doesn't use `aioredis` directly, but depends on
it for `socketio.AsyncRedisManager`. This change is safe, as `socketio`
has been prioritizing `redis.asyncio` over `aioredis` since `v5.6.0` [1].
[1] 0e7691b776
