Deep neural networks visualization



In this publication I present a "toy tool" to see how neurons on a Deep Neural Network (DNN) modify the links between them. These links are also called Weights. For those with technical knowledge, the "bias" unit is not shown (Because of lack of time, I will upgrade it later). In addition I haven't implemented yet the predict option, I would probably add it later.



Technical Notes:

  • This DNN uses standard back propagation without optimizations such as Adam optimizer or regularization parameters.  
  • I am using Math.js for the matrix operations. There are some errors I haven't figured out yet how to solve them, specially for Nan values that will appear when showing the visualization (Sometimes happens, no worry =) )
  • The last layer, the output layer is a sigmoid layer by default. Maybe later I will implement a linear function for regression analysis. And it has been implemented for binary classification by the moment.
  • The weights are randomly initialized between zero and 1. the weights you see are multiplied by 100 and round up to an integer. In addition, the weights reflect the magnitude (Absolute value) only. This is because the Sankey Google Chart doesn't allow floats nor numbers below 1 as weights.

Some instructions:
  • In the X input text box write your matrix n by m, n features and m training examples. Each row should be separated by a ; and each column by a , . There is an example in the input box. The same applies for the Y matrix, even though I only tested with a 1 by m, shape. If you try with other shape let me know what happens.
  • The Layer size input defines the DNN architecture. So if you want 3 hidden units followed by 2 hidden units in the next layer you should type 3,2 
  • The activation function option has two possibilities sigmoid or tanh. I will implement RelU soon.






Input data X. Separate samples by commas "," and features by ";". X:



Input data Y. Set the target binary class. Separate samples by commas "," and classes by ";". Y:



Learning rate:



Number of iterations:



Layer's size (Values separated by a comma ","):



Type of activation function for hidden layers



Update the graph each "n" iterations. n:



Make an iteration each "m" milliseconds. m:


Process
Stop


Comentarios

Publicar un comentario

Entradas populares de este blog

Google Chart Sankey Diagram online for non software developers

Imagen
I saw lots of tutorials of how to create a Sankey diagram but I couldn't find a quick way to see my data without installing or configuring parameters. Thus, here I present a Sankey Diagram to visualize your data. In case you are a software developer this may result to be something simple for you, but in case you don't have programming skills but want a Sankey diagram, this post is for you. You have to upload a CSV file, comma separated in the format: NodeA, NodeB, Weight An example of a CSV file could be: Root A,Branch X,3 Root B,Branch X,8 Root B,Branch Y,2 Root B,Branch Z,3 Branch X,End 1,19 Branch X,End 2,2 Branch X,End 3,4 Branch Y,End 1,10 Branch Y,End 4,17 Root B,Branch ZX,4 Branch ZX,End 1,1 This will create the following Graph: Instructions of use: Way one Click on Chose File and search for your CSV file. In the text field you will see your data loaded. You should see "NodeX1,NodeY1,N1;NodeX2,NodeY2,N2... Click in the button Process ...
Leer más

Tensorflow Batch norm not working

Have you experienced that when using  tf.contrib.layers.batch_norm during test time, the performance is diminished or is completely a mess? If yes, this post is for you. Quick summary: 1) As suggested in [1]:   you have do add this additional code:   update_ops = tf . get_collection ( tf . GraphKeys . UPDATE_OPS )   with tf . control_dependencies ( update_ops ):     train_op = optimizer . minimize ( loss ) 2) You have to use a placeholder to indicate that you are training or not, so you put this in the batch_norm param  is_training = your_place_holder, True for training. 3) If after this it doesn't show a good behavior, change the default  decay = 0.999, to a lower value such as  0.5. Other way to apply batch norm is presented in [2], for a manual setup. Nonetheless, if by using this second approach you still have a bad performance, use the point 3). If you want to use this second approach with Convolutional Neural Netw...
Leer más