Chimaera GmbH
+49 9131 691 385
English

Tutorial: Building a 3D Volume Rendering App with 2D Transfer Functions Using VTK, PyQt5 & Matplotlib

This tutorial walks through how to build an interactive 2D transfer function editor with volume rendering, using:
 

VTK for 3D visualization and volume rendering

PyQt5 for the GUI

Matplotlib for 2D histogram interaction

SimpleITK for medical image processing

NumPy for numerical computations

Building a 3D Volume Rendering App with 2D Transfer Functions

The application loads a medical NIfTI volume (e.g. a CT scan), computes a 2D histogram (intensity vs. gradient magnitude), lets the user draw interactive ROIs as transfer functions, and visualizes the volume with GPU-based 2D transfer function volume rendering in real-time. [Download .zip 100 MB]

Get the complete code

What You Get

A live application where:

  • The 3D volume updates instantly as you manipulate ROIs in the 2D histogram
  • ROIs are mapped to specific tissue structures using color & transparency
  • You can control segmentation overlays using 3D STL meshes

Learning Outcomes

  • How to bridge SimpleITK ↔ VTK ↔ Matplotlib ↔ Qt
  • How to implement interactive 2D transfer functions
  • How to combine UI with volume rendering for medical imaging

Project Overview

This is a comprehensive tutorial and functional application for learning and using 2D transfer functions in medical volume rendering. The application demonstrates advanced visualization techniques used in medical imaging and scientific visualization.
2D transfer functions provide superior control over volume rendering compared to traditional 1D transfer functions by considering both intensity and gradient magnitude. This allows for better tissue differentiation and more intuitive volume design.

The application loads a medical NIfTI volume (e.g. a CT scan), computes a 2D histogram (intensity vs. gradient magnitude), lets the user draw interactive ROIs as transfer functions, and visualizes the volume with GPU-based 2D transfer function volume rendering in real-time.

Key Concepts

  1. Volume Rendering: 
    GPU-accelerated ray casting for real-time visualization
  2. 2D Transfer Functions: 
    Using intensity-gradient histograms for design
  3. Interactive Design: 
    Mouse-driven ROI manipulation and parameter adjustment
  4. Medical Image Processing: 
    Loading, orienting, and processing medical data
  5. 3D Mesh Overlays: 
    Segmentation mask visualization with opacity controls
interactive 2D transfer function editor

Prerequisites 

pip install PyQt5 vtk matplotlib numpy SimpleITK

Application Structure

The application is organized into several modules:

main_application.py: Main window and application logic
utils/: Core utilities for image processing, VTK rendering, and file I/O
widgets/: Custom Qt widgets for interactive components
nifti2stl.py: External module for mesh generation from segmentation masks

For the tutorial experience, download the full code and use main_application.py directly or the convenience function in init.py


Tutorial usage:
1. Run main_application.py
2. Or use the convenience function: from . import run_tutorial; run_tutorial()

 

1. Entry point

def main():
app = QApplication(sys.argv)
volume_file = "abdomen.nii"
mask_config = {}
roi_config_file = ""
main_window = TransferFunction2DMainWindow(
volume_file_path=volume_file,
mask_config=mask_config,
roi_config_file=roi_config_file
)
main_window.show()
sys.exit(app.exec_())
if __name__ == "__main__":
main()

2. Volume loading

def load_medical_image(file_path: str, target_orientation: str = "LPS") -> sitk.Image:
image = sitk.ReadImage(file_path)
oriented_image = sitk.DICOMOrient(image, desiredCoordinateOrientation=target_orientation)
oriented_image.SetDirection([
1.0, 0.0, 0.0,
0.0, 1.0, 0.0,
0.0, 0.0, 1.0
])
oriented_image.SetOrigin([0.0, 0.0, 0.0])
return oriented_image

This function loads medical images (typically NIfTI format) and ensures consistent orientation and coordinate system for processing and visualization.

3. 2D histogram computation

def compute_2d_histogram(volume_data, bins_intensity=512, bins_gradient=256):
intensities = vtk_to_numpy(volume_data.GetPointData().GetScalars())
gradient_filter = vtk.vtkImageGradientMagnitude()
gradient_filter.SetInputData(volume_data)
gradient_filter.Update()
gradient_data = vtk_to_numpy(
gradient_filter.GetOutput().GetPointData().GetScalars()
)
histogram, intensity_edges, gradient_edges = np.histogram2d(
intensities, gradient_data, bins=(bins_intensity, bins_gradient)
)
return histogram, intensity_edges, gradient_edges

This builds the 2D histogram of intensity vs gradient magnitude for transfer function design.
This histogram is the foundation of 2D transfer functions, which provide more control over volume rendering than traditional 1D transfer functions by considering both intensity and gradient information.

4. Interactive histogram editor (Matplotlib)

The InteractiveTransferFunctionCanvas class:

  • Visualize 2D intensity-gradient histograms
  • Create and manipulate rectangular ROIs
  • Adjust ROI properties (color, opacity, window levels)
     

Mouse interaction methods:

on_mouse_press

on_mouse_move

on_mouse_release

All update the 2D transfer function in real-time.

5. Volume Rendering Setup (VTK)

The class VolumeRenderer manages VTK volume rendering with 2D transfer functions. First initialize the volume renderer with:

volume_data (vtk.vtkImageData): The volume to render

bins_intensity (int): Resolution for intensity axis of transfer function

bins_gradient (int): Resolution for gradient axis of transfer function

class VolumeRenderer:
def __init__(self, volume_data, bins_intensity=1024, bins_gradient=512):
self.volume_data = volume_data
self.bins_intensity = bins_intensity
self.bins_gradient = bins_gradient
self.mapper = None
self.volume_property = None
self.volume = None
self.transfer_function_texture = None
self._setup_volume_rendering()

Configure VTK components for GPU-accelerated volume rendering.
def _setup_volume_rendering sets up:

  • GPU ray casting mapper for performance
  • Volume properties for lighting and shading
  • 2D transfer function texture
def _setup_volume_rendering(self):
# Create GPU-accelerated volume mapper
self.mapper = vtk.vtkGPUVolumeRayCastMapper()
self.mapper.SetInputData(self.volume_data)
self.mapper.SetUseJittering(1) # Reduce aliasing artifacts
# Initialize 2D transfer function texture
self._create_transfer_function_texture()
# Configure volume properties
self.volume_property = vtk.vtkVolumeProperty()
self.volume_property.SetInterpolationTypeToLinear()
self.volume_property.ShadeOn() # Enable lighting
# Set lighting parameters for realistic appearance
self.volume_property.SetAmbient(0.4)
self.volume_property.SetDiffuse(0.6)
self.volume_property.SetSpecular(0.3)
self.volume_property.SetSpecularPower(10.0)
# Use 2D transfer function mode
self.volume_property.SetTransferFunctionMode(
vtk.vtkVolumeProperty.TF_2D
)
self.volume_property.SetTransferFunction2D(self.transfer_function_texture)
# Create the volume actor
self.volume = vtk.vtkVolume()
self.volume.SetMapper(self.mapper)
self.volume.SetProperty(self.volume_property)

6. Updating the Transfer Function

Create an empty 2D transfer function texture. The texture is a 2D image where:

  • X-axis represents gradient magnitude
  • Y-axis represents intensity
  • RGBA values define color and opacity at each point
def _create_transfer_function_texture(self):
self.transfer_function_texture = vtk.vtkImageData()
self.transfer_function_texture.SetDimensions(
self.bins_intensity, self.bins_gradient, 1
)
self.transfer_function_texture.AllocateScalars(vtk.VTK_FLOAT, 4)
scalars = self.transfer_function_texture.GetPointData().GetScalars()
for i in range(scalars.GetNumberOfTuples()):
scalars.SetTuple4(i, 0, 0, 0, 0) # RGBA = (0,0,0,0)

Update the 2D transfer function based on region of interest definitions. Clear existing transfer function and apply each ROI to the transfer function.

def update_transfer_function(self, roi_list, extent_intensity, extent_gradient):
scalars = self.transfer_function_texture.GetPointData().GetScalars()
for i in range(scalars.GetNumberOfTuples()):
scalars.SetTuple4(i, 0, 0, 0, 0)
for roi in roi_list:
self._apply_roi_to_transfer_function(
roi, extent_intensity, extent_gradient
)
self.transfer_function_texture.Modified()

UI Features

  • Add/Remove ROI rectangles
  • Select colormap (e.g. coolwarm, viridis )
  • Save & Load ROI configurations ( .json )
  • Mask Controls: Adjust 3D overlay opacity (if masks are loaded)

The full working code is in 2DTransferFunctionApp.zip [100 MB]


Get the complete code

Marcus

Dr.-Ing. Marcus Prümmer

Am Weichselgarten 7
91058 Erlangen
Germany

 +49 (0)9131 - 691 385
 +49 (0)9131 - 691 386
pruemmer(at)chimaera.de