Run and Outputs¶
main.py essentially guides how to run the code.
Firstly import all the modules we have created.
from lenet.trainer import trainer from lenet.network import lenet5 from lenet.dataset import mnist
Begin by creating the dataset and its placeholders,
dataset = mnist()
Create the network object and cook it.
net = lenet5(images = dataset.images) net.cook(labels = dataset.labels)
Create a trainer module that takes as input, the cooked network and the datafeed and then train it.
bp = trainer (net, dataset.feed) bp.train()
If everything went correctly, the following output would have been produced:
Extracting data/train-images-idx3-ubyte.gz Extracting data/train-labels-idx1-ubyte.gz Extracting data/t10k-images-idx3-ubyte.gz Extracting data/t10k-labels-idx1-ubyte.gz . . . . name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:02:00.0 Total memory: 7.91GiB Free memory: 7.80GiB 2017-08-15 19:58:04.465420: W tensorflow/stream_executor/cuda/cuda_driver.cc:485] creating context when one is currently active; existing: 0x3eec930 2017-08-15 19:58:04.513975: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 1 with properties: name: NVS 310 major: 2 minor: 1 memoryClockRate (GHz) 1.046 pciBusID 0000:01:00.0 Total memory: 444.50MiB Free memory: 90.00MiB 2017-08-15 19:58:04.514091: W tensorflow/stream_executor/cuda/cuda_driver.cc:485] creating context when one is currently active; existing: 0x3ee8b70 2017-08-15 19:58:04.770913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 2 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:03:00.0 Total memory: 7.91GiB Free memory: 7.80GiB . . . . Iter 0 Objective 6.67137 Cost 2.39042 Test Accuracy 0.0892 Training Accuracy 0.09062 Iter 1000 Objective 0.784603 Cost 0.205023 Test Accuracy 0.9865 Training Accuracy 0.98592 Iter 2000 Objective 0.707837 Cost 0.158198 Test Accuracy 0.9877 Training Accuracy 0.98638 Iter 3000 Objective 0.658972 Cost 0.0991117 Test Accuracy 0.9877 Training Accuracy 0.98734 Iter 4000 Objective 0.709337 Cost 0.138037 Test Accuracy 0.9882 Training Accuracy 0.9889 Iter 5000 Objective 0.687822 Cost 0.115233 Test Accuracy 0.9862 Training Accuracy 0.98782 Iter 6000 Objective 0.767473 Cost 0.192869 Test Accuracy 0.9863 Training Accuracy 0.98504 Iter 7000 Objective 0.717531 Cost 0.138536 Test Accuracy 0.9875 Training Accuracy 0.98738 Iter 8000 Objective 0.730901 Cost 0.161923 Test Accuracy 0.987 Training Accuracy 0.98738 Iter 9000 Objective 0.692139 Cost 0.127491 Test Accuracy 0.9889 Training Accuracy 0.98832 Final Test Accuracy: 0.9852
A few other prints that were unique to the systems are all skipped.
The tensorboard that is created can be setup by running,
Open a browser and enter the address
0.0.0.0:6006, this will open up tensorboard.
The tensorboard will have the following sections that are populated:
Let us go over a few of these sections, while leaving the others to the reader to interpret.
In scalars we see all of the scalars that we were tracking in summary. Two of these that are important for us to observe are the costs going down and the test accuracies going up with iterations.
In the images section, we see the filters learnt by the first layer, the confusion matrix and the images that are used in training. Note that there are sliders in all these which could be used to scroll through various levels of information.
In the graphs section, we see the graph now with the training and other auxillary nodes present.
We can see here how the training modules are setup with gradients and how the objectives are derived from all the regularizers. Tensorboard is a nice utility and for a theano user it is a miracle come true. In fact, tensorboard is the primary and pretty much the only reason I am migrating to tensorflow.