Author ORCID Identifier

0000-0003-0877-7730

Date of Award

12-2024

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Computer Science

First Advisor

Jingyu Liu

Second Advisor

Esra Akbas

Third Advisor

Jonathan Shihao Ji

Fourth Advisor

Yu-Ping Wang

Abstract

This dissertation presents the culmination of research efforts aimed at advancing spatiotemporal analysis through diverse deep learning methodologies, focusing on complex graph-based and time-series data. Across a series of studies, we have developed innovative approaches to understand and predict dynamic patterns in spatiotemporal datasets, using resting-state functional magnetic resonance imaging (rs-fMRI) data as a primary test case.

Our research journey began with the introduction of the Brain ROI-aware Graph Isomorphism Network (BrainRGIN), an innovative graph neural network architecture specifically designed for brain graphs derived from static functional network connectivity (sFNC) matrices obtained from rs-fMRI data. By refining traditional graph convolution networks, BrainRGIN incorporates a clustering-based embedding and a graph isomorphism network within its convolutional layers to predict intelligence metrics.

Expanding beyond brain graphs, we developed the Enhanced Cluster Aware Graph Network (ECGN) to address challenges related to uniform node updates and imbalanced node classification in benchmark graph datasets, achieving over an 11% improvement compared to state-of-the-art methods. Progressing further, we created the Self-Clustering Graph Transformer (SCGT), a specialized graph transformer that introduces clustered-based attention mechanisms for graphs with subnetworks. Unlike conventional transformer architectures, SCGT employs a novel self-clustering graph attention mechanism, which we validated by classifying Alzheimer’s disease and predicting intelligence.

Building upon these foundational studies, we developed DSAM, a comprehensive and interpretable spatiotemporal framework. DSAM enables the direct extraction of both temporal and spatial information from timeseries data, facilitating the creation of task-specific functional connectivity matrices that support more personalized and accurate classifications.

Our most recent contribution introduces an enhanced approach to spatiotemporal modeling through a differentiable mutual information framework. This method optimizes the learning of nonlinear connectivity matrices in multi-channel time series data, providing a more robust mechanism for capturing complex dependencies across temporal and spatial dimensions.

This research has the potential to make significant contributions to neuroscience, particularly in the analysis of fMRI data. Additionally, the developed methodology may find broader applications in other domains that involve spatiotemporal data, including climate modeling, traffic forecasting, and epidemiology.

File Upload Confirmation

1

Download

Share

COinS