It is a homework for "Data Vision" class

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrices\n", "\n", "1. Creating Matrices" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[1, 2], [3, 4]]\n" ] } ], "source": [ "arr = [1,2,3,4]\n", "arr = [[1,2],[3,4]]\n", "print(arr)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2],\n", " [3, 4]])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# convert to numpy\n", "np.array(arr)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import torch" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 2\n", " 3 4\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.Tensor(arr)\n", "# very similar to numpy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.2 Creating Matrices with default values" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 1.],\n", " [1., 1.]])" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.ones((2,2))" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.ones((2,2))" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.32728576, 0.04368428],\n", " [0.86955555, 0.50554617]])" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.rand(2,2)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 0.2412 0.9376\n", " 0.6890 0.1700\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.rand(2,2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.3 Seeds for reproducibility" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.5488135 , 0.71518937],\n", " [0.60276338, 0.54488318]])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.seed(0)\n", "np.random.rand(2,2)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.5488135 , 0.71518937],\n", " [0.60276338, 0.54488318]])" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.seed(0)\n", "np.random.rand(2,2)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.4236548 , 0.64589411],\n", " [0.43758721, 0.891773 ]])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.rand(2,2)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 0.4963 0.7682\n", " 0.0885 0.1320\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(0)\n", "torch.rand(2,2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.3 Numpy and Torch bridge" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "arr = np.ones((2,2))" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 1.],\n", " [1., 1.]])" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "arr" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<class 'numpy.ndarray'>\n" ] } ], "source": [ "print(type(arr))" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "torch_tensor = torch.from_numpy(arr)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.DoubleTensor of size 2x2]" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch_tensor" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<class 'torch.DoubleTensor'>\n" ] } ], "source": [ "print(type(torch_tensor))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "arr = np.ones((2,2), dtype = np.int8)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "ename": "RuntimeError", "evalue": "can't convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8.", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m<ipython-input-20-7613981588c9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtorch_tensor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_numpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mRuntimeError\u001b[0m: can't convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8." ] } ], "source": [ "torch_tensor = torch.from_numpy(arr)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": true }, "outputs": [], "source": [ "arr = np.ones((2,2), dtype = np.int64)\n", "torch_tensor = torch.from_numpy(arr)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.LongTensor of size 2x2]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch_tensor" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.IntTensor of size 2x2]" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "arr = np.ones((2,2), dtype = np.int32)\n", "torch_tensor = torch.from_numpy(arr)\n", "torch_tensor" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.ByteTensor of size 2x2]" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "arr = np.ones((2,2), dtype = np.uint8)\n", "torch_tensor = torch.from_numpy(arr)\n", "torch_tensor" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.4 torch to Numpy" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": true }, "outputs": [], "source": [ "torch_tensor = torch.ones((2,2))" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": true }, "outputs": [], "source": [ "np_arr = torch_tensor.numpy()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "numpy.ndarray" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(np_arr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.5 Tensor operations" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 1 1\n", " 1 1\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "a = torch.ones((2,2))\n", "print(a)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "torch.Size([2, 2])\n" ] } ], "source": [ "print(a.size())" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1\n", " 1\n", " 1\n", " 1\n", "[torch.FloatTensor of size 4]" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# resize using view\n", "a.view(4)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([2, 2])" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.view(4).size()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Element Wise Addition" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 2 2\n", " 2 2\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "a = torch.ones((2,2))\n", "b = torch.ones((2,2))\n", "# element wise addition\n", "c = a + b\n", "print(c)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 2 2\n", " 2 2\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "c = torch.add(a,b)\n", "print(c)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Old c tensor\n", "\n", " 2 2\n", " 2 2\n", "[torch.FloatTensor of size 2x2]\n", "\n", "New c tensor\n", "\n", " 3 3\n", " 3 3\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "# in-place addition\n", "print('Old c tensor')\n", "print(c)\n", "\n", "c.add_(a)\n", "\n", "print('New c tensor')\n", "print(c)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# element wise subtraction\n", "c = c - b" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 2 2\n", " 2 2\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "c" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(a.sub(b))" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "# in place subtraction\n", "print(a.sub_(b))" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(a)" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# element wise multiplication\n", "a = torch.ones((2,2))\n", "b = torch.zeros((2,2))\n", "c = a * b" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(c)" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 1 1\n", " 1 1\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "c = torch.mul(a,b)\n", "print(a)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "c = a.mul_(b)\n", "print(a)" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# element wise multiplication\n", "a = torch.ones((2,2))\n", "b = torch.zeros((2,2))\n", "c = a / b" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "inf inf\n", "inf inf\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(c)" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.div(b,a)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 0 0\n", " 0 0\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.div_(a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tensor Mean" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([10])" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.Tensor([1,2,3,4,5,6,7,8,9,10])\n", "a.size()" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 5.5000\n", "[torch.FloatTensor of size 1]" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mean(dim = 0)" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "ename": "RuntimeError", "evalue": "dimension out of range (expected to be in range of [-1, 0], but got 1)", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m<ipython-input-68-dde2f630f079>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdim\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mRuntimeError\u001b[0m: dimension out of range (expected to be in range of [-1, 0], but got 1)" ] } ], "source": [ "a.mean(dim = 1)" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([2, 10])" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.Tensor(([1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10]))\n", "a.size()" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5.5" ] }, "execution_count": 73, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mean()" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 5.5000\n", " 5.5000\n", "[torch.FloatTensor of size 2]" ] }, "execution_count": 75, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mean(dim = 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Variables and gradients" ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from torch.autograd import Variable" ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = Variable(torch.ones(2,2), requires_grad = True)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", " 1 1\n", " 1 1\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(a)" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", " 1 1\n", " 1 1\n", "[torch.FloatTensor of size 2x2]" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Not a variable\n", "torch.ones(2,2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "autograd.Variable is the central class of the package. It wraps a Tensor, and supports nearly all of operations defined on it. You can access the raw tensor through the .data attribute, while the gradient w.r.t. this variable is accumulated into .grad" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", " 3 3\n", " 3 3\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "y = a + 2\n", "print(y)" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<AddBackward0 object at 0x106e31c88>\n" ] } ], "source": [ "print(y.grad_fn)\n" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", " 27 27\n", " 27 27\n", "[torch.FloatTensor of size 2x2]\n", " Variable containing:\n", " 27\n", "[torch.FloatTensor of size 1]\n", "\n" ] } ], "source": [ "z = y * y * 3\n", "out = z.mean()\n", "\n", "print(z, out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There�s one more class which is very important for autograd implementation - a Function.\n", "let�s backprop now out.backward() is equivalent to doing out.backward(torch.Tensor([1.0]))\n" ] }, { "cell_type": "code", "execution_count": 59, "metadata": { "collapsed": true }, "outputs": [], "source": [ "out.backward()" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", " 4.5000 4.5000\n", " 4.5000 4.5000\n", "[torch.FloatTensor of size 2x2]\n", "\n" ] } ], "source": [ "print(a.grad)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You should have got a matrix of 4.5. Let�s call the out Variable �o�. We have that o=(1/4)�zi, z(i)=3*(x(i) + 2)*(x(i) + 2) and z(i)�(x(i)=1)=27. Therefore, �o�xi=3*2(x(i)+2), hence �o�xi��xi=1=9/2=4.5.\n", "\n", "\n", "You can do many crazy things with autograd!" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", "-1115.5402\n", " 291.0368\n", " -555.2755\n", "[torch.FloatTensor of size 3]\n", "\n" ] } ], "source": [ "x = torch.randn(3)\n", "x = Variable(x, requires_grad=True)\n", "\n", "y = x * 2\n", "while y.data.norm() < 1000:\n", " y = y * 2\n", "\n", "print(y)" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variable containing:\n", " 51.2000\n", " 512.0000\n", " 0.0512\n", "[torch.FloatTensor of size 3]\n", "\n" ] } ], "source": [ "gradients = torch.FloatTensor([0.1, 1.0, 0.0001])\n", "y.backward(gradients)\n", "\n", "print(x.grad)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## That's it! We are ready to build a Neural Network. For next time..!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 2 }