TailOR: A Computer Vision-based Automated System for Mouse and Rat Behavior Tagging and Extraction
Konda, Madhu Sudhan Reddy
Citations
Abstract
Maternal care behaviours in rodents are fundamental to early life development but quantifying them requires labour-intensive manual annotation of home cage videos. We present TailOR, a computer vision system that automatically tags and extracts mouse and rat behaviour from side-view recordings. TailOR uses the Segment Anything Model 2 (SAM2) to generate high-quality masks and derives geometric features such as centroid positions, displacements and distances to key objects. These features feed a hierarchical rule-based decision system that applies domain-specific thresholds to classify behaviours including in-nest, off-nest eating, and off-nest non-eating activities. We evaluate TailOR on maternal care videos from mice and rats, achieving frame-level accuracies exceeding 90% across multiple datasets without manual intervention. The system's modular design and reliance on simple geometric features allow it to generalise across species, offering a scalable alternative to manual scoring and accelerating neuroscience studies of maternal care.
