An artificial neural network in Python
The next addition to my collection of learning algorithms implemented in Python is an artificial neural network. I spent a bit more time implementing this one compared to the others for a few reasons:
I wanted to make the code as modular as I could while still illustrating the basic concepts of forward propagation and back propagation. Artificial neural networks are by definition made up of many repeating structures, so it makes sense to me to take a similar approach when implementing one. It also makes the code easily re-usable.
I tried to vectorize many of the algorithms rather than relying on for loops. This makes many of the operations much faster by using numpy arrays and in my opinion makes the code cleaner and less verbose.
As I was working out the implementation, I was initially using the same test data as I did for my logistic regression notebook and was not having much success with fitting the neural network model as I present it in the notebook. After re-reading some notes, I realized that the network would work best on data set centered roughly around zero. I then read this post by Denny Britz that uses scikit-learn's
make_moonsdataset generator, (which I thought was pretty cool) so I decided to try it out with my impementation. Lo and behold, my implementation was able to fit the zero-centered generated data much easier.
If you'd like to follow along with the implementation you can find a jupyter notebook with the code and associated math in my ml-python repo (along with a few other common machine learning algorithms) or read it rendered in HTML here: