Brainchop: In-browser MRI volumetric segmentation and rendering

Brainchop is a web-based tool that allows scientists and clinicians to perform volumetric analysis of structural Magnetic Resonance Imaging (MRI) using pre-trained deep learning models, without the need for technical expertise or setup of AI solutions. It is the first browser-based front-end MRI segmentation tool that can process full brain volumes in a single pass, using the lightweight and reliable Meshnet model (Fedorov et al., 2017) modified to incorporate volumetric dilated convolutions (Yu & Koltun, 2016) for increased accuracy and modest computational requirements. In addition to offering extensibility through an open-source framework, Brainchop also addresses privacy concerns by performing client-side processing and leveraging hardware acceleration across different brands and architectures.


Summary
Brainchop is a web-based tool that allows scientists and clinicians to perform volumetric analysis of structural Magnetic Resonance Imaging (MRI) using pre-trained deep learning models, without the need for technical expertise or setup of AI solutions. It is the first browser-based front-end MRI segmentation tool that can process full brain volumes in a single pass, using the lightweight and reliable Meshnet model (Fedorov et al., 2017) modified to incorporate volumetric dilated convolutions (Yu & Koltun, 2016) for increased accuracy and modest computational requirements. In addition to offering extensibility through an open-source framework, Brainchop also addresses privacy concerns by performing client-side processing and leveraging hardware acceleration across different brands and architectures.

Statement of need
Accurate segmentation of brain tissue from MRI volumes is a crucial step in various brain imaging analysis pipelines, with applications ranging from surgical planning and measurement of brain changes to visualization of anatomical structures. However, few professionals possess both domain expertise and machine learning skills, as well as the necessary computational infrastructure, to take advantage of the benefits of artificial intelligence (AI)-assisted neuroimaging. This limitation is particularly pronounced in rural areas and developing countries. To address these challenges, we developed brainchop, an in-browser machine learning platform for volumetric neuroimaging that offers high accessibility, scalability, low latency, ease of use, lack of installation requirements, and cross-platform operation, while also preserving end-user data privacy. Unlike traditional web applications, brainchop operates within the browser environment to process the end-users data locally on their machines without the need for server-side processing and the associated privacy and portability issues. Currently available tools for 3D Neuroimaging such as Freesurfer (Dale et al., 1999) and SPM (Friston et al., 1994) are not web-based tools hence they lack the aforementioned features that the browser can bring to Neuroimaging applications. For many clinicians, setting up neuroimaging pipelines is a technological barrier, and providing those pipelines through the browser will help democratize these computational approaches.

Pipeline
In order to deploy the PyTorch MeshNet model in the browser, there is a need to convert it first to tensorflow.js (Smilkov et al., 2019). Brainchop has a pre-processing pipeline, full-volume and sub-volumes inference options, 3D input/output rendering, and post-processing capability as illustrated in Figure 1 for the tool high-level architecture.

Preprocessing
Brainchop is designed to support T1 weighted MRI volume segmentation. The input is read in Nifti format ("NIfTI Reader," 2017). T1 image needs to be in shape 256x256x256, scaled, and resampled to 1 mm isotropic voxels as a preprocessing step for proper results. This preprocessing can be made in brainchop by using mri_convert.js which uses Pyodide (Droettboom et al., 2022) to deploy the conform function used by FastSurfer (Henschel et al., 2020) for reshaping, scaling, and re-sampling MRI T1 raw image data as shown in Figure 2 (a). The rest of the preprocessing pipeline is to remove input noisy voxels and enhance input volume intensities to improve the segmentation accuracy Figure 2 (b). In addition, brainchop also supports MRI tissue cropping to speed up the inference process and lower memory use.

Inference Model
The advantage of MeshNet small size is due to its simple architecture and using dilated convolution in which a typical model for the segmentation task can be constructed with few layers as shown in Figure 3. While the MeshNet model has fewer parameters compared to the classical segmentation model U-Net, it also can achieve a competitive DICE score as shown in Table 1.

Results
Multiple pre-trained models are available with brainchop for full-volume and sub-volumes inference including brain masking, gray matter white matter (GMWM) segmentation models, in addition to brain atlas models for 50 cortical regions and 104 cortical and subcortical structures as shown in Figure 4. Normally 3D noisy regions can result from the inference process due to possible bias, variance and irreducible error (e.g. noise with data). To remove these noisy volumes we designed a 3D connected components algorithm to filter out those noisy regions.
Papaya (Papaya dev. team, 2019) viewer is used to visualize the input and output images, and a composite operation is also provided to subjectively verify the output image accuracy compared to the input.
Also, brainchop supports 3D real-time rendering of the input and output volume by using Three.js (Cabello et al., 2010) with the capability of Region of Interest (ROI) selection as shown in Figure 5.

Code availability
The source code is publicly accessible in a GitHub repository (https://github.com/neuroneural/ brainchop) with detailed step-by-step documentation. Built-in models also are provided with the brainchop live demo.