Running non-blocking Ray computations in the background
superduperdb
offers the possiblity to run all long running blocking jobs in the background via ray
.
Read about the ray
project here.
To configure this feature, configure:
cluster:
compute:
uri: ray://<ray_host>:<ray_port>
When this is so-configured the following functions push their computations to the ray
cluster:
db.apply
db.execute
(if data is inserted, deleted, updated)
When ray
is configured, these functions returns either a superduperdb.job.Job
object, or an iterable thereof.
jobs = db.apply(<component>)[0]