Chapter 9. Advanced topics

Chapter 9. Advanced topics
Prev		Next

Schmant uses a custom class loader for loading task package classes and task package library dependency classes. The default setting is to have one shared class loader for all task packages (-c shared on the command line). This works well as long as task packages don't have conflicting dependencies, for instance such as when two different task packages depend on different versions of the same library.

When running a build script that uses task packages with conflicting dependencies, use the isolated class loaders option (-c isolated) to give each enabled task package its own class loader.

In-memory file systems

EntityFS-enabled tasks can use any type of file system implementations. One option is the in-memory file system created by the RamFileSystemBuilder object or by the SchmantFileSystems.createRamFileSystem() method. Keeping temporary files in memory can be used to speed up builds somewhat. This is certainly suggested by the EntityFS performance tests. The drawback is that debugging may be harder since the -k flag (obviously) does not make Schmant keep files stored in memory once the build process has terminated.

Consider the example Example 8.4, “Build a Jar file from the projects in an Eclipse workspace manually” from Chapter 8, Projects. If the tmpDir variable is created by calling TempFileUtil.createTempDir(), as suggested in the script, tmpDir will be on disk. If it is created as shown in Example 9.1, “Creating an in-memory temporary directory” it will be in RAM memory instead.

Example 9.1. Creating an in-memory temporary directory

import org.entityfs.ram.RamFileSystemBuilder import org.entityfs.util.Directories import org.schmant.report.SchmantReportLogAdapter def fs = new RamFileSystemBuilder(). // The default size of a file segment is 4096 bytes. (A file consists of as // many segments as is needed for containing all its data.) // setFileSegmentSizeBytes(8192). // // Integrate the file system's logging with Schmant's reports setLogAdapter(SchmantReportLogAdapter.INSTANCE). create() def root = fs.rootDirectory // This would do exactly the same: // def root = SchmantFileSystems.createRamFileSystem() // Set a default temporary files directory on the file system. If not set, // temporary files end up in java.io.tmpdir instead. fs.setTemporaryFilesDirectory( Directories.newDirectory(root, "tmp2")) // And our temporary files directory tmpDir = Directories.newDirectory(root, "tmp")

JavaScript

fs = new RamFileSystemBuilder(). // The default size of a file segment is 4096 bytes. (A file consists of as // many segments as is needed for containing all its data.) // setFileSegmentSizeBytes(8192). // // Integrate the file system's logging with Schmant's reports setLogAdapter(SchmantReportLogAdapter.INSTANCE). create(); root = fs.getRootDirectory(); // This would do exactly the same: // var root = SchmantFileSystems.createRamFileSystem(); // Set a default temporary files directory on the file system. If not set, // temporary files end up in java.io.tmpdir instead. fs.setTemporaryFilesDirectory( Directories.newDirectory(root, "tmp2")); // And our temporary files directory tmpDir = Directories.newDirectory(root, "tmp");

JRuby

# The org.entityfs.ram package is not included in the Schmant module. fs = Java::OrgEntityfsRam::RamFileSystemBuilder.new. # The default size of a file segment is 4096 bytes. (A file consists of as # many segments as is needed for containing all its data.) # setFileSegmentSizeBytes(8192). # # Integrate the file system's logging with Schmant's reports setLogAdapter(Schmant::SchmantReportLogAdapter::INSTANCE). create root = fs.rootDirectory # This would do exactly the same: # root = Schmant::SchmantFileSystems.createRamFileSystem # Set a default temporary files directory on the file system. If not set, # temporary files end up in java.io.tmpdir instead. fs.setTemporaryFilesDirectory( Schmant::Directories.newDirectory(root, "tmp2")) # And our temporary files directory tmpDir = Schmant::Directories.newDirectory(root, "tmp")

Jython

# The org.entityfs.ram classes are not imported by default from org.entityfs.ram import RamFileSystemBuilder # The default size of a file segment is 4096 bytes. (A file consists of as many # segments as is needed for containing all its data.) # # Integrate the file system's logging with Schmant's reports fs = RamFileSystemBuilder(). \ setLogAdapter(SchmantReportLogAdapter.INSTANCE). \ create() root = fs.getRootDirectory() # This would do exactly the same: # root = SchmantFileSystems.createRamFileSystem() # Set a default temporary files directory on the file system. If not set, # temporary files end up in java.io.tmpdir instead. fs.setTemporaryFilesDirectory( \ Directories.newDirectory(root, "tmp2")) # And our temporary files directory tmpDir = Directories.newDirectory(root, "tmp")

The following table shows the execution times when building the Schmant workspace using Sun JDK 1.6.0_12 on a quad-core, Ubuntu 8.10 machine. Every time is taken as the average build time of five builds.

Table 9.1. Build times for the Schmant workspace

Configuration	Average build time	Speedup
Temporary files on disk, one build thread	7.4s	-
Temporary files in memory, one build thread	6.8s	8%
Temporary files on disk, two build threads	6.3s	-
Temporary files in memory, two build threads	5.6s	11%
Temporary files on disk, four build threads	6.3s	-
Temporary files in memory, four build threads	5.8s	8%

When running the build script from Example 8.7, “Preprocess source files in Eclipse projects. Compile manually. Build Jar” instead, on the same machine, the build times are as in the table below.

Table 9.2. Build and preprocess times for the Schmant workspace

Configuration	Average build time	Speedup
Temporary files on disk, one build thread	11.3s	-
Temporary files in memory, one build thread	9.1s	19%
Temporary files on disk, two build threads	9.5s	-
Temporary files in memory, two build threads	7.5s	21%
Temporary files on disk, four build threads	9.4s	-
Temporary files in memory, four build threads	7.8s	17%

In the second example, the speedup from using temporary files is somewhat larger. This is probably because the TextReplaceTF tasks are more I/O bound than the Jdk6JavacTF tasks.

The conclusion is that using a RAM directory for temporary files can give a moderate speed boost for I/O intensive builds. The tradeoff is a higher memory usage, and that scripts may be harder to troubleshoot since temporary files cannot be saved for analyzing after the script has been run.

Note

The memory-backed directory /dev/shm can be used to speed up builds on Linux.

Prev		Next
Chapter 8. Projects	Home	Chapter 10. Best practices