Description
1. (2 marks) Briefly give the math for feedforward and backprop steps through a single residual block in ResNets (no need for the full network; you can make necessary assumptions accordingly). 2. (2 marks) A convolutional neural network has 4 consecutive 3 × 3 convolutional layers with stride 1 and no pooling. How large is the support of (the set of image pixels which activate) a neuron in the 4th non-image layer of this network? 3. (2 marks) Consider a fully-connected deep network with one hidden layer. If the number of hidden units is increased, what effect would this have on bias and variance of the learned model? Explain briefly. 4. (3 marks) Consider a two-layer neural network as given below: yk(x, w) = σ X M j=1 w (2) kj h X D i=1 w (1) ji xi + w (1) j0 + w (2) k0 1 where h is a layer with a linear activation function, and σ(a) = {1 + exp (−a)} −1 is the sigmoid activation function. Your classmate tells that she can derive an equivalent network which computes the same function, but with the final layer activation functions replaced with tanh(a)(= e a−e−a e a+e−a ). Is she correct? Can you prove and support her argument? (Hint: Think about relating σ(a) and tanh(a), and then showing that the two networks’ parameters differ only by linear transformations.) 5. (3 marks) Consider a quadratic error defined by E(w) ≈ E(w∗ ) + 1 2 (w − w∗ ) TH(w − w∗ ), in which the Hessian matrix H, evaluated at w∗ has an eigenvalue equation given by Hui = λiui . Show that the contours of constant error are ellipses whose axes are aligned with the eigenvectors ui , with lengths that are inversely proportional to the square root of the corresponding eigenvalues λi . (Hint: Note that it is possible to write w − w∗ as P i αiui for some αis (you can assume this, if you like!). How does this help you?) 6. (3 marks) As part of improving customer service, Kaziranga National Park is designing a mobile application where a visitor can take a photo of an animal, and the application identifies the same. They have collected 20 images of all 200 species of animals in the park. They have access to a deep learning model deployed at Olympic National Park, Washington trained on 1 million images from 1000 classes. Can you help the digitization team at Kaziranga National Park to develop their deep learning model? What exact steps would you suggest? 2 Programming (35 marks) • The programming questions are shared in “Assignment 3.zip”. Please follow the instructions in the notebook. Turn-in the notebook via Google Classroom once you finish your work. • Marks breakdown is as follows: – Question 1: 17 marks (15 marks for code completion and 2 marks for reporting train & test loss, final accuracies and the plots as requested.) – Question 2: 4 marks – Question 3: 6 marks – Question 4: 8 marks 2
