Demystifying Neural Networks

21 Jan 2020

First, a few definitions:

AI vs machine learning

The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

Thank you google. What I wanted to say is, AI is really just a very broad definition of tasks which we think need human intelligence to do (which is highly subjective!!)

Machine learning vs programming

The difference is, for programming you feed input and your rules (code), to get your output.

For Machine Learning, you feed input and your desired output, and you get your code! Using that code you predict other output (given some input).

Credits to:

Deep learning

Deep learning is a subset of neural networks, which are a subset of machine learning

Credits to:

Building a Neural Network with Tensorflow

There are four components to building a neural network with tensorflow:

This is a good website to showcase basic neural networks and how they work:

We start with x1 and x2, zero layers, and the dataset which looks like a doughnut with a blue center. Ideally you’d wanted a circular boundary to separate the orange and the blue points.

The resulting white separating boundary is simply just a slanted line, contributed by x1 and x2 (x1 contributes a lot more, hence the boundary is almost vertical).

As you can see, the data cannot be segregated with a single straight line.

Add one hidden layer with two nodes. The hidden layer nodes are also simply just combining patterns from previous nodes.

The final boundary is made up of two lines, and as you can see - two lines doesn’t segregate the data properly either.

We now add one hidden layer, with 3 nodes. Run ~100 epochs, and you should now see almost like a triangle forming in the centre, forming a boundary around the blue center.

This is because the three nodes, each a combination of x1 and x2, have formed three slopes, which then combined to form a triangle.

Repeat the same (1 hidden layer) with 4 nodes, and you (might) see a square appear! eventually they do converge to an almost triangle/circle kind of shape, this is really random.

I think the above has been the clearest illustration of neural networks I have seen so far.

Some random thoughts/comments:

so much more i have to learn oh my

A bonus article on weights and biases for those interested:

major credits to - i attended an introductory course by them during a meetup so all of this knowledge is from these guys!