Benchmarking napari-imagej is of interest, considering the way in which it crosses language barriers. This document serves as an initial survey of performance, in addition to being a template for further benchmarks.
To benchmark napari-imagej, we must consider how it performs relative to:
An equivalent Python application
An equivalent Java application
We consider both below, benchmarking on the goal of a simple Gaussian Blur.
This document assumes that you are running in a napari-imagej environment. For help in setting up this environment, see this page.
This particular data sample is nice, because it is:
Available to all
Sufficiently large to minimize startup overhead
Unfortunately, the data is stored within separate files for each focal plane and is treated by ImageJ2 as an 8-bit image. Below, we describe the pre-processing stage necessary for the remainder of the benchmarking.
Configure napari-imagej settings
To process EmbryoCE, we need to alter two different napari-imagej settings. For information on configuring napari-imagej, please see here.
First, we must increase the maximum amount of RAM available to napari-imagej. This procedure provides 32GB of RAM, but the procedure can be repeated with less RAM by using fewer focal planes of EmbryoCE.
Then, we change the backing instance of napari-imagej to sc.fiji:fiji; this gives us the Bio-Formats plugin, which will allow us to read in EmbryoCE.
Recommended settings for napari-imagej benchmarking
Once you’ve edited these settings, you’ll have to restart napari-imagej to register the changes.
Import EmbryoCE
We start by using Bio-Formats to import EmbryoCE into a single input image:
Launch napari-imagej, and once napari-imagej is ready, launch the ImageJ GUI.
Use Plugins->Bio-Formats->Bio-FormatsImporter. Navigate to the downloaded EmbryoCE, and select focal1.tif. The rest of the focal planes will be automatically included.
In Bio-FormatsImportOptions, do not change any settings, and click OK.
In Bio-FormatsFileStiching, change Axis1numberofimages to the number of focal planes you’d like to include. If your machine does not have 32GB of RAM, consider decreasing this number. Once finished, click OK.
Once finished, Bio-Formats will import EmbryoCE into Fiji as a standard image.
Many tools in the Python scientific stack automatically assume floating point math; for efficiency and speed, this is something that the ImageJ ecosystem does not do. We must convert EmbryoCE to floating-point samples to ensure uniformity.
This is easist done with the following SciJava script, written in Python:
By placing this script within the ImageJ base directory (by default, the scripts directory of the napari-imagej source), the script will automatically be discovered by ImageJ2 and will be searchable using the script’s filename.
Note that this script requires a pre-allocated output image. This allows us to make use of shared memory, increasing the speed of napari-imagej.
We run the script using the following steps:
Launch napari-imagej
Load in two copies of focal.tif, the pre-processed image created above.
Once napari-imagej is ready, search napari-imagej for gaussiannapariimagej. Launch this Module as a napari Widget with the Widget button.
To benchmark in Python, we devise a routine most similar to that performed in our prior tests. In this case, we perform a Gaussian Blur using scikit-image:
fromskimage.filtersimportgaussianfromskimage.ioimportimread,imsaveimporttimeitpath=<pathtowhereyousavedfocal.tif>img=imread(path)sigma=6.num_executions=5times=[]foriinrange(num_executions):duration=timeit.Timer(lambda:gaussian(img,sigma)).timeit(number=1)print(f"Execution {i}: {duration} seconds")times.append(duration)print(f"Average execution time over {num_executions} runs: {sum(times)/len(times)}")# save the imageout_path="./focal_gaussed.tif"gaussed=gaussian(img,sigma)imsave(out_path,gaussed)
This Python script can then be run on the command line, from within the napari-imagej Mamba environment:
To obtain suitable benchmarking results, we average each execution over 5 different runs. Each script is designed to be easily rerun:
* The SciJava scripts must be manually rerun, to give the JVM time to warm up.
* The pure Python script automatically reruns the computation, meaning it must only be run once to gather benchmarking data.
In the table below, we obtain the following data, running all programs on a machine with a Intel Core i7-10700 CPU, 64GB of memory, and running Ubuntu 22.04.5 LTS: