Abstract: The human brain is complex and its inner workings have long been of scientific interest but still not well understood.  Physiologists have studied and mapped the brain’s anatomy and its neural centers and pathways.  Psychologists have developed models of the mind based on experiments with human subjects.  And artificial intelligence experts have developed computer software applications that perform human-like tasks, with deep learning models now the preferred algorithms in government and commercial applications of visual pattern recognition and speech understanding.  Because these deep learning models incorporate aspects of brain processing such as edge and line detectors found in the visual cortex, some physicists believe these models can shed light on understanding the functioning of the brain.  While artificial intelligence has made great progress in performing human-like visual and auditory sensory tasks, there has been little progress related to human memory.  Therefore, the goal of this research is to model human long-term declarative episodic memory storage using deep learning methods.  A new kind of deep neural network models will be developed for supervised feature learning to achieve the best accuracy on several widely-used datasets, such as MNIST, imagenet, and videonet.  Deep learning models will first be trained using convolutional neural networks aiming to replicate human long-term declarative episodic memory storage.  Then, long short-term memory units will be constructed in layers of a recurrent neural network, and the model will be trained and evaluated.  Frameworks will be constructed on top of Tensorflow for training and testing the deep learning models for long-term declarative episodic memory storage for tasks using different datasets.

This project is modeled on the experiments and results of an ongoing research since last year.  Published papers on this research are highlighted below.

Pace University Research day Publications (2018):

• A Comparative Study of Convolutional Neural Network Models with Rosenblatt’s Brain Model Abu Kamruzzaman, Atik Khatri, Damiano Mastrandrea, Milind Ikke, Poorva Shelke, Yousef Alhwaiti, and Charles C. Tappert slides
• A Deep Learning Model for Long-term Declarative Episodic Memory Storage Abu Kamruzzaman, Teresa Brooks, Karan Thakkar, Aditi Markale, Shreya Rajwade, Shoaib Ahamed, Yousef Alhwaiti, and Charles C. Tappert slides

External Publications (2018~19):

• Abu Kamruzzaman, Yousef Alhwaiti, and Charles C. Tappert "Developing a Deep Learning Model to Implement Rosenblatt’s Experiential Memory Brain Model" Future of Information and Communication Conference (FICC) IEEE 2019
• Teresa Nicole Brooks, Abu Kamruzzaman, Avery Leider and Charles C. Tappert “A Computer Science Perspective on Models of the Mind” Intelligent Systems Conference (IntelliSys), IEEE, 2018.

• IntelliSys proceedings published in Springer series "Advances in Intelligent Systems and Computing" and submitted for indexing to ISI Proceedings, EI-Compendex, DBLP, SCOPUS, Google Scholar and Springerlink.
• FICC proceedings published in Springer series "Lecture Notes in Networks and Systems" and submitted for indexing to ISI Proceedings, MetaPress, Google Scholar and Springerlink.

Project Description:

This project involves creating a biologically inspired Neural Deep Learning Brain Model to replicate human memory storage capabilities:

This semester's extension of the project will involve:

Develop and implement a memory system with performance analysis including:

• The quantity of memory stored -- frames for video data, etc.
• The error rate on memory recall -- best to have a table of memory storage versus error rate
• discussion on the biological plausibility of the model
• comparison with other models

Model architecture:

Here is a graphic that illustrates the concepts of our deep learning model

Functioning Deep learning model:

• S-system to collect the input images/videos,
• A-System reserved for the hidden layers
• R-System for output layers

which is similar to a typical deep learning neural network model.

• The newly proposed C-system is the memory storage to be used for storing the trained images/videos and retrieve in future as needed.
• Experiments on CNN model, LSTM model and CNN LSTM model.

For those who are interested learning the background, we recommend reading the Rosenblatt's Brain Memory Model (Dr. Frank Rosenblatt was the inventor of perceptrons).

Students are expected to participate in building a deep learning model utilizing the architecture above.

• Previously models were built using CNN & LSTM separately.

Current Goal: Build storage mechanism for proposed C-system.

• CNN+LSTM combined may enable C-system storage capabilities.
and/or
• Build a framework to enable C-system storage capabilities in TensorFlow.
and/or
• Utilize a Deep Learning algorithm technique (ex: Auto Encoder etc.) to build the C-system storage mechanism.