superduperdb.backends.dask package#

Submodules#

superduperdb.backends.dask.compute module#

class superduperdb.backends.dask.compute.DaskComputeBackend(address: str | None = None, serializers: Sequence[Callable] | None = None, deserializers: Sequence[Callable] | None = None, local: bool = False, **kwargs)[source]#

Bases: ComputeBackend

A client for interacting with a Dask cluster. Initialize the DaskClient.

Parameters:
  • address – The address of the Dask cluster.

  • serializers – A list of serializers to be used by the client. (optional)

  • deserializers – A list of deserializers to be used by the client. (optional)

  • local – Set to True to create a local Dask cluster. (optional)

  • **kwargs

    Additional keyword arguments to be passed to the DaskClient.

disconnect() None[source]#

Disconnect the Dask client.

property name: str#

Return the name of current compute engine

result(identifier: str) Any[source]#

Retrieves the result of a previously submitted task. Note: This will block until the future is completed.

Parameters:

identifier – The identifier of the submitted task.

shutdown() None[source]#

Shuts down the Dask cluster.

submit(function: Callable, *args, **kwargs) Future[source]#

Submits a function to the Dask server for execution.

Parameters:

function – The function to be executed.

property tasks: Dict[str, Future]#

List for all pending tasks

property type: str#

Return the type of compute engine

wait(identifier: str) None[source]#

Waits for all pending tasks to complete. :param identifier: Future task reference

wait_all() None[source]#

Waits for all pending tasks to complete.

Module contents#