RIT CIS Internship: Machine Learning

Today I began working on the other half of my project which is lifelong learning. Lifelong learning allows a network to improve itself continually without forgetting old data. This is useful because retraining an entire neural network takes lots of time and computational power. The lifelong model I implemented is called incremental nearest neighbor. Nearest neighbor stores all of your points in a coordinate grid and it classifies a new item by checking which known object is closest. Incremental nearest neighbor adds every new data point it classifies to the grid hopefully improving the accuracy of the model as it does. Next week I will continue to work on lifelong models as well as using VQA data on them.

Today I created an evaluation function for my VQA model. Every loop over the data set I would print the average loss and the overall accuracy. Loss measure the incorrectness of a functions guess. Lower loss is good higher is bad. My function trended toward lower loss as it continued to train on the dataset. I was getting an accuracy of around 30% which is fairly good considering I was training the model on questions that had no associated images. In the future I plan to combine the image data with the question data.

In the morning I tried debugging my VQA model. The error code was somewhat cryptic and it was frustrating trying to figure out the cause of it. With the help of Kushal I was able to locate the source of my error and continue working. When I was passing in data to the GRU I was not checking the batch size correctly causing the program to crash. Despite this setback by lunchtime I was able to get my VQA model to make predictions on questions. I have not formatted the output or run the model on test data but I am glad that the model is finally working. After lunch I traced through all of the code and rewrote and commented any part that was confusing to me. Tomorrow I plan to make a testing function that is able to evaluate the accuracy of the model.

Today I continued to work on my VQA model. I improved my helper methods that kept track of indexing of my questions. I also successfully implemented a method that would pad the data. Padding the data is replacing unknown characters with <UNK> and making sure all the sentences have the same number of words. If a sentence is too long it gets truncated after the max number of words. If it is too short a <PAD> token is added in order to keep the length consistent. It is important to have consistent lengths in order to vectorize the data. Tomorrow I plan to have the model successful make predictions.

Today started with a meeting about everyones presentation outlines. All the interns ran though their plan for the presentation. I felt everyone did a great job but I realized I had a way to go after I struggled to concisely explain some of the things I planned to present.  After the meeting I continued to create my own VQA program. I struggled in the morning to implement most of the programs. After some help from Kushal I was able to implement a function that could successfully accept the data as input and then manipulate it into a vector. I plan to try and get a model to make predictions on the data tomorrow.