RIT CIS Internship: Machine Learning

Today I learned how to create a convolutional neural network in python. I used a framework called Pytorch to build the network. I set up the neural net to have three hidden layers. Each node on the hidden layers has its own weights and biases in order to give input to the next layer or provide an output.  In simpler terms a node places its input into a function to generate an output that is used for later nodes. The way a network learns is by evaluating it accuracy after each prediction and updating each nodes function. When building a neural network a data set is necessary. I choose to work with the CIFAR-10 data set. CIFAR-10 contains 10,000 32x32 images of 10 different objects ( plane, car, bird , cat , deer, dog, frog, horse, ship, truck). After training on 9,000 of the images my neural net was able to identify the remaining 1,000 with 60% accuracy. This level of accuracy is certainly not phenomenal but it is very far from random guessing (10%). As I learn more about neural network

At the start of today I watched 2 of Stanford's lectures one on feed-forward neural networks and the other on convolutional neural networks. In a feed-forward network there is no connection to previous nodes. This  is a great visualization of the relationship between nodes and how a previous layer provides input for a future layer. The other type of neural network is a recurrent neural network. What makes a recurrent network different is that outputs on a certain layer can act as inputs for the previous layer. I have not gone into depth on recurrent networks so my knowledge is limited. A convolutional network is not another type of network but rather a specific layer on a neural network which alters the data. One example of convolution is pooling. As demonstrated below pooling takes a filter full of values and then averages or sums them creating a new array. Pooling is useful because it reduces the size of your matrix while retaining a similar value. This allows for calculations to

Today I watched a Stanford lecture on loss in machine learning and cost optimization function. Loss functions are a crucial part of neural networks. I learned the basics of feed forward neural networks from a helpful web series: ( https://www.youtube.com/watch?v=ZzWaow1Rvho&list=PLxt59R_fWVzT9bDxA76AHm3ig0Gg9S3So )  . After watching the YouTube series I switched to the Stanford lectures to learn more on loss functions. The purpose of a loss function is to update your neural networks classifying function. By giving your predicted result and comparing it to the actual result you are able to tell where your model went wrong and you can then update it to improve over the next iteration. I learned about hinge loss and the squared loss function which are two different ways to calculate your error. Using this knowledge I was able to begin to explore the soft max function. I don't fully understand the math behind it today, and I will continue to look into it tomorrow. The image below i

I started off today by learning more linear algebra and some classifiers used to group data. I learned about vector transpositions and their use in making vector more usable depending on your data set. After that Tyler and Kushal gave me a quick overview of certain classifiers. I learned about Nearest Neighbors first, as it is simplistic. Nearest neighbor just chooses the data point closest to the one you are trying to classify and sets its value to the value of that data point. I have a visual below that demonstrates how nearest neighbor and other classifiers classify data. After learning the general concepts of classification I was sent to implement them on my own. Despite some struggle initially I was able to create a program that successfully implemented:  Nearest Neighbors , Linear SVM, RBF SVM, Gaussian Process, and a MLP Neural Net. These are all very useful models but only the MLP Neural Net used deep learning . All of these models have different methods of classification and

Today I learned about the basics of machine learning from my mentor. Although it was complicated I thought I was able to understand some of it. After that I was tasked with learning Numpy a package that is used for linear algebra in python. I spent the rest of the day trying to learn the basics of Numpy and linear algebra. I learned a lot about neural networks and python today and I look forward to being able to apply that knowledge in a project.