napari-czann-segment

Segmention using DeepLearning ONNX models packaged as *.czann files


    License PyPI Python Version napari hub

    Semantic Segmentation using DeepLearning ONNX models packaged as *.czann files.


    This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

    Train on APEER and use model in Napari

    Installation

    Before installing, please setup a conda environment. If you have never worked with conda environments, go through this tutorial first.

    You can then install napari-czann-segment via pip:

    pip install napari-czann-segment

    What does the plugin do

    The plugin allows you to:

    • Use a *.czann file containing the Deep Neural Network (ONNX) for semantic segmentation and metadata
    • Segmentation will be applied per 2D plane for all dimensions
    • Processing larger multi-dimensional images it uses the cztile package to chunk the individual 2d arrays using a specific overlap.

    What does the plugin NOT do

    Before one can actually use a model it needs to be trained, which is NOT done by this plugin.

    Therer two main ways hwo such a model can be created:

    • Train the segmentation model fully automated on APEER and download the *.czann file
    • Train your model in a Jupyter notebook etc. and package it using the czmodel python package as an *.czann

    Using this plugin

    Sample Data

    A test image and a *.czann model file can be downloaded here.

    • PGC_20X.ome.tiff --> use PGC_20X_nucleus_detector.czann to segment

    In order to use this plugin the user has to do the following things:

    • Open the image using "File - Open Files(s)" (requires napari-aicsimageio plugin).
    • Click napari-czann-segment: Segment with CZANN model in the "Plugins" menu.
    • *Select a .czann file to use the model for segmentation.
    • metadata of the model will be shown (see example below)
    ParameterValueExplanation
    model_typeModelType.SINGLE_CLASS_SEMANTIC_SEGMENTATIONsee: czmodel for details
    input_shape[1024, 1024, 1]tile dimensions of model input
    output_shape[1024, 1024, 3]tile dimensions of model output
    model_idba32bc6d-6bc9-4774-8b47-20646c7cb838unique GUID for that model
    min_overlap[128, 128]tile overlap used during training (for this model)
    classes['background', 'grains', 'inclusions']availbale classes
    model_nameAPEER-trained modelname of the model

    Napari - Image loaded and czann selected

    • Adjust the minimum overlap for the tiling (optional, see cztile for details).
    • Select the layer to be segmented.
    • Press Segment Selected Image Layer to run the segmentation.

    Napari - Image successfully segmented

    A successful is obviously only the starting point for further image analysis steps to extract the desired numbers from the segmented image. Another example is shown below demonstrating a simple "Grain Size Analysis" using a deep-learning model trained on APEER used in napari

    Napari - Simple Grain Size Analysis

    Remarks

    IMPORTANT: Currently the plugin only supports using models trained on a single channel image. Therefore make sure that during the training on APEER or somewhere else the correct inputs images are used. It is quite simple to train an single RGB image, which actually has three channels, load this image in napari and notice only then that the model will not work, because the image will 3 channels inside napari.

    • Only the CPU will be used for the inference using the ONNX runtime for the ONNX-CPU runtime
    • GPUs are not supported yet and will require ONNX-GPU runtime

    For developers

    • Please clone this repository first using your favorite tool.

    • Ideally one creates a new conda environment or use an existing environment that already contains Napari.

    Feel free to create a new environment using the YAML file at your own risk:

    cd the-github-repo-with-YAML-file
    conda env create --file conda_env_napari_czann_segment.yml
    conda activate napari_czmodel
    • Install the plugin locally

    Please run the the following command:

    pip install -e .

    To install latest development version:

    pip install git+https://github.com/sebi06/napari_czann_segment.git

    Contributing

    Contributions and Feedback are very welcome.

    License

    Distributed under the terms of the BSD-3 license, "napari-czann-segment" is free and open source software

    Issues

    If you encounter any problems, please file an issue along with a detailed description.

    Version:

    • 0.0.14

    Release date:

    • 22 July 2022

    First released:

    • 11 July 2022

    License:

    • BSD-3-Clause

    Supported data:

    • Information not submitted

    GitHub activity:

    • Stars: 5
    • Forks: 1
    • Issues + PRs: 0

    Python versions supported:

    Operating system:

    Requirements:

    • numpy
    • magicgui
    • qtpy
    • napari
    • cztile
    • czmodel
    • onnxruntime
    • aicsimageio

    Sign up to receive updates