SemiAutoAnnotator is an image annotation tool designed to streamline the annotation process by integrating active learning and transfer learning techniques. This approach significantly reduces manual effort and enhances the accuracy of annotations over time.

Methodology

We used an iterative annotation process combining manual efforts and model-assisted learning to efficiently train a computer vision model for mouse behavior analysis.

Once trained, the model performs Detection, and predictions are classified as either correct or incorrect. Incorrect predictions are identified for correction through user feedback in the Subset Selection phase, where challenging frames are selected for Re-annotation by human annotators. This iterative loop continues, with the model being retrained on newly annotated data, until the model reaches high accuracy.

The methodology leverages active learning and transfer learning techniques to minimize annotation costs while ensuring high-quality labeled data. By iteratively improving model accuracy and selectively annotating frames, we significantly reduce the time and resources required for annotation, making the process both scalable and effective for large-scale video analysis.

Key Features

• Active Learning Integration: The tool assists users by providing suggested annotations, leveraging a pre-trained model to predict and propose annotations for new images.

• Iterative Model Retraining: As users annotate more images, the model can be retrained to improve its performance. Each retraining session uses the current model as a base, incorporating its pre-trained weights as a starting point, thereby refining the model's accuracy continuously.

• Data Preprocessing and Clustering: SemiAutoAnnotator enhances efficiency by sorting images based on detection confidence, clustering them, and selecting diverse clusters. Using convex hulls, it identifies the most diverse and optimal data for annotation, ensuring a diverse and optimal dataset for retraining.

 Installation

1. Clone the Repository: 

git clone https://github.com/mohajeranilab/SemiAutoAnnotator.git

2. Install Dependencies: 

• For systems with GPU support: 

pip install -r requirements_gpu.txt

• For systems without GPU support: 

pip install -r requirements.txt 

Usage

Run the annotation_labeler.py script with optional arguments:

python annotation_labeler.py --frame_skip=50 --model_path=path_to_model --clustering=False

• --frame_skip: Number of frames to skip when extracting from a video (default: 50).  

• --model_path: Path to the pre-trained model weights (default: None).  

• --clustering: Enable or disable clustering for data preprocessing (default: False).

Note: After running the script, a file explorer window will prompt you to select the video file you want to annotate. The tool will extract frames from the video and organize them in the used_videos directory, creating a folder for each selected video containing the video file itself along with subfolders for extracted frames and JSON annotations.