Parallel

Tweak of joblib.Parallel (opens in a new tab) that propagates the scikit-learn configuration.

This subclass of joblib.Parallel (opens in a new tab) ensures that the active configuration (thread-local) of scikit-learn is propagated to the parallel workers for the duration of the execution of the parallel tasks.

The API does not change and you can refer to joblib.Parallel (opens in a new tab) documentation for more details.

Python Reference (opens in a new tab)

Constructors

constructor()

Signature

new Parallel(opts?: object): Parallel;

Parameters

Returns

Parallel

Defined in: generated/utils/parallel/Parallel.ts:25 (opens in a new tab)

Properties

_isDisposed

boolean = false

Defined in: generated/utils/parallel/Parallel.ts:23 (opens in a new tab)

_isInitialized

boolean = false

Defined in: generated/utils/parallel/Parallel.ts:22 (opens in a new tab)

_py

PythonBridge

Defined in: generated/utils/parallel/Parallel.ts:21 (opens in a new tab)

id

string

Defined in: generated/utils/parallel/Parallel.ts:18 (opens in a new tab)

opts

any

Defined in: generated/utils/parallel/Parallel.ts:19 (opens in a new tab)

Accessors

py

Signature

py(): PythonBridge;

Returns

PythonBridge

Defined in: generated/utils/parallel/Parallel.ts:30 (opens in a new tab)

Signature

py(pythonBridge: PythonBridge): void;

Parameters

Returns

void

Defined in: generated/utils/parallel/Parallel.ts:34 (opens in a new tab)

Methods

dispatch_next()

Dispatch more data for parallel processing

This method is meant to be called concurrently by the multiprocessing callback. We rely on the thread-safety of dispatch_one_batch to protect against concurrent consumption of the unprotected iterator.

Signature

dispatch_next(opts: object): Promise<any>;

Parameters

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:99 (opens in a new tab)

dispatch_one_batch()

Prefetch the tasks for the next batch and dispatch them.

The effective size of the batch is computed here. If there are no more jobs to dispatch, return false, else return true.

The iterator consumption and dispatching is protected by the same lock so calling this function should be thread safe.

Signature

dispatch_one_batch(opts: object): Promise<any>;

Parameters

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:129 (opens in a new tab)

dispose()

Disposes of the underlying Python resources.

Once dispose() is called, the instance is no longer usable.

Signature

dispose(): Promise<void>;

Returns

Promise<void>

Defined in: generated/utils/parallel/Parallel.ts:80 (opens in a new tab)

format()

Return the formatted representation of the object.

Signature

format(opts: object): Promise<any>;

Parameters

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:155 (opens in a new tab)

init()

Initializes the underlying Python resources.

This instance is not usable until the Promise returned by init() resolves.

Signature

init(py: PythonBridge): Promise<void>;

Parameters

Returns

Promise<void>

Defined in: generated/utils/parallel/Parallel.ts:43 (opens in a new tab)

print_progress()

Display the process of the parallel execution only a fraction of time, controlled by self.verbose.

Signature

print_progress(opts: object): Promise<any>;

Parameters

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:181 (opens in a new tab)