OpenTelemetry tries to send traces to a default `localhost:4317`
endpoint if no configuration is provided, which is not what we want if
users don't configure OpenTelemetry explicitly.
This change sets the `OTEL_SDK_DISABLED` environment variable to
`true` if no `OTEL_` prefixed environment variables are found, which
disables the OpenTelemetry SDK.
Guarantee that cache is initialized during startup, and only once,
instead of every time a `MetadataHandler` object is instantiated.
Also, improve logic to determine `fixtures` paths.
For steps that need to run before the web application starts, such as
scheduling tasks, this new `startup.py` script is introduced.
This fixes a recently introduced issue where task scheduling was not
being triggered, because of it being included in the
`if __name__ == "__main__":` block, which is not executed when
the application is run by Gunicorn in production environments.
We do not include this logic as part of FastAPI's lifespan
implementation, as running multiple workers with Gunicorn would
cause this logic to be executed multiple times.
This change introduces OpenTelemetry dependencies, and uses
`opentelemetry-instrument` auto-instrumentation to allow users to
configure OpenTelemetry settings via environment variables [1].
The only custom environment variable added is
`OTEL_SERVICE_NAME_PREFIX`, which allows users to set a prefix to the
service names included by RomM to differentiate between `api`, `worker`,
and `watcher` services.
The instrumentation of RQ workers and file watcher will be added in
subsequent pull requests.
[1] https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
* Added `linux-headers` back, but only for development stage.
* Fixed initialization script, as `uv` is not included in the final
Docker image.
* Initialize variable `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA`.
The `watchfiles` library supports event batching, which allows us to
process multiple filesystem changes in a single run.
This change also avoids database calls in the watcher as much as
possible.
This change replaces our custom `scheduler.py` script with the
`rqscheduler` command, allowing us to run the RQ scheduler as a
separate, low-memory process, by avoiding the need to maintain
the Python app in memory.
* Remove `scheduler.py` script.
* Move initialization of scheduled tasks to `worker.py`.
* Update `docker/init_scripts/init` to start the `rqscheduler`
command instead of the custom script.
* Fix scheduled tasks' `func` paths to the new project structure.
* Temporarily use a fork of `rq-scheduler` to support
username and SSL settings in the `rqscheduler` command.
The `WEB_CONCURRENCY` environment variable is a more common way to
configure the number of workers for Gunicorn [1] or other web servers.
This change maintains `GUNICORN_WORKERS` compatibility, while notifying
users that it is deprecated and should be replaced with
`WEB_CONCURRENCY`.
It would also allow us to replace Gunicorn with another web server in
the future without changing the variable name.
[1] https://docs.gunicorn.org/en/stable/settings.html#workers
`tini` does not wait for child processes to close, so all processes will be killed immediately. This is why the container stops so fast.
This fix makes the `init` script listen and handle terminate signals. It also ensures that child processes are shut down in reverse order with proper waiting for completion.
This change replaces the bundled Redis server with Valkey. No breaking
changes are introduced, as considered environment variables still
maintain the `REDIS_` prefix.
Fixes#925.
Currently, the `request.url_for` and `URLPath.make_absolute_url` methods
always build URLs with "http" scheme, even when the original requested
URL is using "https".
The reason for this is that Gunicorn does not allow IPs other than
127.0.0.1 to set secure headers by default. As regular RomM
installations don't know which frontend IPs will try to set security
headers in advance, we can disable this validation, and fix URL
building.
A simple way to test this change is to access any of the `feed` endpoints,
which generate URLs using the mentioned methods. Accessing the endpoint
using "https" scheme must generate "https" URLs.
Reference:
* https://github.com/encode/starlette/issues/538#issuecomment-2054013679
* https://docs.gunicorn.org/en/stable/settings.html#forwarded-allow-ips
This change moves the virtualenv creation in the `Dockerfile` to a
separate stage, to simplify isolating the process and reduce the need
for uninstalling build dependencies.
The approach is similar to the one explained in [1]. It relies on
building a virtualenv folder, and copying it in the final stage.
Changing the `PATH` environment variable makes the virtualenv usable by
default, without affecting the default Python installation.
Also, added Dockerfile arguments for Alpine, nginx, and Python versions,
as some of them are reused, and also simplifies testing new versions.
An extra side effect is that the image size for the final stage is
reduced from 315MB to 262MB.
[1] https://scribe.rip/@albertazzir/blazing-fast-python-docker-builds-with-poetry-a78a66f5aed0
