org.schmant.run
Class TaskExecutor

java.lang.Object
  org.schmant.run.TaskExecutor

All Implemented Interfaces:

Observer

public class TaskExecutor
extends Object
implements Observer

The TaskExecutor is used for running independent tasks in parallel execution threads. This often gives a nice speedup of the build since the build host's resources are used more efficiently, but it comes with the cost of added complexity for build scripts.

The added complexity mostly comes from that it is no longer possible to assume that task A will run before task B. If task B depends on the results of task A, task B must be scheduled with that information stored in a TaskDependency.

It is also no longer possible to assume that task A has been run when task B is being configured. As a consequence, those of task B's properties that are dependent on the result from task A must be configured using different kinds of FutureEntity:s and FutureProperty:s.

The task executor uses an internal queue to keep track of scheduled jobs that have not been run. By default, a job that has all of its dependencies met will be added at a random position in the queue. Jobs are thus run in a random order, which is a naïve way of trying to automatically balance I/O bound jobs with CPU bound jobs. To run jobs in the order that they are scheduled, call setUseFifoQueue(boolean) with a true argument.

A task executor must be started by calling start() before any tasks may be added to it. When all tasks have been added, call waitFor() to wait until all tasks have been run, or waitFor(TaskDependency) to wait until a certain dependency is met. After using a TaskExecutor (regardless of if all tasks were successful or not), always call shutdown() to terminate all build threads. See the JavaScript example below.

 var te = new TaskExecutor();
 ...configure the TaskExecutor
 te.start();
 try
 {
   ... add tasks
   te.waitFor();
 }
 finally
 {
   te.shutdown();
 }
 

A TaskExecutor can only be used once. Once waitFor() (or shutdown() has been called, no more tasks can be added to it.

The TaskExecutor defaults to using only one build thread. To use more threads, call setNumberOfThreads(int) before starting it.

Read the Schmant manual and see the task executor-using examples in the task factory reference for more information on how to use a task executor.

Since:

0.5

Author:

Karl Gustafsson

Field Summary

static int

DEFAULT_NUMBER_OF_THREADS A very safe default (1).

Constructor Summary

TaskExecutor()
Create the task executor and register it with the TaskExecutors class.

TaskExecutor(boolean register)
Create the task executor and optionally register it with the TaskExecutors class.

Method Summary

TaskDependency	add(Object o) Add a Task, a TaskFactory or a closure to the task executor.
TaskDependency	add(Object o, Object deps) Add a Task, TaskFactory or a closure with one or several dependencies to the executor.
void	awaitTermination(long time, TimeUnit tu) Wait the specified time, maximum for the task executor to terminate.
protected ThreadPoolExecutor	createExecutor() Create the thread pool Executor that is used for running tasks.
protected ThreadFactory	createThreadFactory() Create a thread factory for the Executor.
protected BlockingQueue<Runnable>	createWorkQueue() Create the work queue for the Executor.
int	getNumberOfThreads() Get the maximum number of threads for the task executor.
ReadOnlyTaskExecutorStatus	getStatus() Return a read only view of the task executor's status.
boolean	isUseFifoQueue() Does this task executor use a FIFO queue for scheduling tasks?
TaskExecutor	setLogDependencies(boolean b) By setting this to true, information about dependencies are logged to level INFO.
TaskExecutor	setNumberOfThreads(int no) Set the maximum number of threads for the task executor.
TaskExecutor	setUseFifoQueue(boolean b) Use a FIFO (first in - first out) queue for tasks? The default value of this property is, false, meaning that a queue enqueuing tasks in random order is used.
void	shutdown() Call this to shut down the task executor before all tasks have been executed.
void	shutdownNow() Call this to shut down the task executor before all tasks have been executed.
TaskExecutor	start() Start the task executor.
void	update(Observable obs, Object o) This is called when a dependency is met.
void	waitFor() Wait for all tasks to finish.
void	waitFor(TaskDependency td) Wait for a specific task dependency to complete.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_NUMBER_OF_THREADS

public static final int DEFAULT_NUMBER_OF_THREADS

A very safe default (1).

See Also:

Constant Field Values

Constructor Detail

TaskExecutor

public TaskExecutor()

Create the task executor and register it with the TaskExecutors class.

See Also:

TaskExecutor(boolean)

TaskExecutor

public TaskExecutor(boolean register)

Create the task executor and optionally register it with the TaskExecutors class.

This constructor should only be used by test classes.

Parameters:

register - Should the task executor be registered with the TaskExecutors class? If this is set to false, the build script must ensure that the task executor is shut down on all code paths. It is safest to set this to true.

See Also:

TaskExecutor()

Method Detail

getStatus

public ReadOnlyTaskExecutorStatus getStatus()

Return a read only view of the task executor's status. The returned object is live and will always reflect the task executor's current status.

Returns:

A read only view of the task executor's status.

setLogDependencies

public TaskExecutor setLogDependencies(boolean b)

By setting this to true, information about dependencies are logged to level INFO.

Parameters:

b - Should dependency information be logged?

Returns:

this.

setNumberOfThreads

public TaskExecutor setNumberOfThreads(int no)
                                throws IllegalStateException

Set the maximum number of threads for the task executor. This must be called before start():ing.

Parameters:

no - The maximum number of executor threads.

Returns:

this.

Throws:

IllegalStateException - If the task executor is started.

getNumberOfThreads

public int getNumberOfThreads()

Get the maximum number of threads for the task executor.

Returns:

The maximum number of threads.

setUseFifoQueue

public TaskExecutor setUseFifoQueue(boolean b)

Use a FIFO (first in - first out) queue for tasks? The default value of this property is, false, meaning that a queue enqueuing tasks in random order is used. This is believed to give slightly better performance since tasks with different performance profiles will be executed in parallel. If this is set to true, a FIFO queue will be used and tasks will be executed in the order that they were added to the queue (in the order that they were scheduled and in the order that all their dependencies were met).

Both queue types only execute tasks whose dependencies have been met.

Parameters:

b - Should the queue be FIFO?

Returns:

this

isUseFifoQueue

public boolean isUseFifoQueue()

Does this task executor use a FIFO queue for scheduling tasks?

Returns:

true if this task executor uses a FIFO queue.

See Also:

setUseFifoQueue(boolean)

createWorkQueue

protected BlockingQueue<Runnable> createWorkQueue()

Create the work queue for the Executor. If the useFifo property is set to true, a LinkedBlockingQueue is returned, if not, a PriorityBlockingQueue without any Comparator set is returned (the tasks will be executed in random order). Both queues are unlimited in their capacities.

Subclasses may override this to change the behavior.

See Also:

setUseFifoQueue(boolean)

createThreadFactory

protected ThreadFactory createThreadFactory()

Create a thread factory for the Executor. The default is to create a TaskExecutorThreadFactory. This can be overridden by subclasses to create other kinds of queues.

Returns:

A thread factory.

createExecutor

protected ThreadPoolExecutor createExecutor()

Create the thread pool Executor that is used for running tasks.

Returns:

A thread pool executor.

start

public TaskExecutor start()

Start the task executor. The task executor must be started before tasks can be scheduled.

After scheduling all tasks, call waitFor() to wait for all tasks to complete.

Returns:

this

shutdown

public void shutdown()

Call this to shut down the task executor before all tasks have been executed. Just like Executor.shutdown(), this will let all currently running tasks finish before the task executor is terminated.

After calling this method, call awaitTermination(long, TimeUnit) to wait for the executor to finish.

See Also:

waitFor()

shutdownNow

public void shutdownNow()

Call this to shut down the task executor before all tasks have been executed. Just like Executor.shutdown(), this will let all currently running tasks finish before the task executor is terminated.

After calling this method, call awaitTermination(long, TimeUnit) to wait for the executor to finish.

See Also:

waitFor()

awaitTermination

public void awaitTermination(long time,
                             TimeUnit tu)
                      throws InterruptedException

Wait the specified time, maximum for the task executor to terminate. This method should not be used by scripts. Use waitFor() instead.

Parameters:

time - The maximum time to wait.

tu - The time unit.

Throws:

InterruptedException - If the task executor has not terminated when the time is up.

waitFor

public void waitFor(TaskDependency td)
             throws InterruptedException,
                    TaskFailedException

Wait for a specific task dependency to complete.

Parameters:

td - The dependency.

Throws:

InterruptedException - If the thread that called this method is interrupted while waiting.

TaskFailedException - On other errors.

waitFor

public void waitFor()
             throws InterruptedException

Wait for all tasks to finish. After calling this method, call shutdown().

Throws:

InterruptedException - If the task executor is interrupted.

add

public TaskDependency add(Object o)
                   throws SchmantException

Add a Task, a TaskFactory or a closure to the task executor.

Parameters:

o - The task, task factory or closure.

Returns:

A dependency object for the added task.

Throws:

SchmantException - If the added object is neither a task, a task factory or a closure.

Since:

1.1

add

public TaskDependency add(Object o,
                          Object deps)
                   throws SchmantException

Add a Task, TaskFactory or a closure with one or several dependencies to the executor.

Parameters:

o - The task, task factory or closure.

deps - One or several dependencies. This may be a single TaskDependency object or an array or Collection of objects. The object list is flattened using a FlatteningList.

Returns:

A dependency object for the added task.

Throws:

SchmantException - If the added object is neither a task, a task factory or a closure.

Since:

1.1

update

public void update(Observable obs,
                   Object o)

This is called when a dependency is met. According to the contract in TaskDependency, the thread calling this method must hold the dependency's satisfied state lock.

Specified by:

update in interface Observer