320 lines
17 KiB
Plaintext
320 lines
17 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 56,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# On importe les librairies\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"We want to estimate parameters of a simple linear regression (y i = wx i + b) by closed-form and GD. \n",
|
||
"\n",
|
||
"The loss function is defined on a data point by l(ŷ i , y i ) = (ŷ i − y i )² where the prediction is given by ŷ i = ŵx + b̂. The\n",
|
||
"P N\n",
|
||
"optimization problem we want to solve is min i=1 l(ŷ i , y i ). \n",
|
||
"\n",
|
||
"We assume the following data generation process:\n",
|
||
"w,b\n",
|
||
"Y ∼ wX + b + \\eta where X ∼ U[20; 40] and \\eta ∼ N (0, 1).\n",
|
||
"\n",
|
||
"# 1\n",
|
||
"\n",
|
||
"Generate N data points (x i , y i ) using the data generation process given above and store them into x vs\n",
|
||
"N\n",
|
||
"y. Note: N = 100 and start by first generating {x i } N\n",
|
||
"1 and then {y i } 1 . Do not forget to add noise!\n",
|
||
"\n",
|
||
"D'abord on va générer les x"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 57,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# On génère les x\n",
|
||
"list_x = np.random.uniform(20, 40, 100)\n",
|
||
"\n",
|
||
"# Pis les y\n",
|
||
"\n",
|
||
"def y(x):\n",
|
||
" w = 1.5\n",
|
||
" b = 5\n",
|
||
" return w * x + b + np.random.normal(0,1)\n",
|
||
"\n",
|
||
"list_y = [y(x) for x in list_x ]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 2\n",
|
||
"Plot the data points x vs y."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 58,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<matplotlib.collections.PathCollection at 0x7f543c96a128>"
|
||
]
|
||
},
|
||
"execution_count": 58,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHRFJREFUeJzt3XuQXOV55/Hvo54ZqWF0GzSABqLIKJS9lhPL1JhaL0RgcLjIFLJdJutcWOywq2WVZIGsY4Rd5cWp8sZ21otMVQRRwEAcsgbDeuUihItxhKyKDR4FgSVjr5AgNowwDRIjDWlNz+XZP/q01Gr15XTPOX05/ftUTWn69Dndj456nnn1nOd9j7k7IiLS+ea0OgAREYmGErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGLiCSEErqISEIooYuIJERPM99syZIlvnz58ma+pYhIx9uxY8cb7j5Ya7+mJvTly5czMjLSzLcUEel4ZvYvYfZTyUVEJCGU0EVEEkIJXUQkIZTQRUQSQgldRCQhlNBFRBJCCV1EJCbZ3DS7Xh0jm5tuyvs1tQ9dRKRbZHPTXLLxKd44nGPJ/D4ev+EC0n2pWN9TI3QRkRjszYzzxuEc2clp3jicY29mPPb3VEIXEYnBisF+lszvI92bYsn8PlYM9sf+niq5iIjEIN2X4vEbLmBvZpwVg/2xl1tACV1EJDbpvhTvOWNh095PJRcRkYQIldDNbJGZPWhmPzWzF8zsA2Z2i5m9amY7g681cQcrIiKVhR2hfw141N3fBbwXeCHYfqu7rwq+HoklQhGRFirtJW92b3k9atbQzWwBsBr4JIC754CcmcUbmYhIi5X2km9Zfz5rN21vam95PcKM0M8CMsDdZvasmd1pZicHz/2RmT1vZl83s8XxhSki0nylveTb9mSa3ltejzAJvQc4B7jd3d8HvA1sAG4HVgCrgP3AV8sdbGbrzGzEzEYymUw0UYuINEFpL/nqswfr7i1vZonG3L36DmanAz909+XB498ENrj7h4v2WQ487O7vqfZaw8PDrlvQiUgnyeamj+slL31c69gopv+b2Q53H661X80Ruru/BvzCzN4ZbLoY+ImZLS3a7aPArrqjFBFpc4Ve8kIiLn1cTbOn/4edWPTHwH1m1gfsAz4F3GZmqwAHXgb+cywRioi0qVqj9ULJpjBCj3v6f82SS5RUchGRpChXTgFOSPD1lGgqCVty0dR/EZEGlJZTdo+OceMDO0+olzdz+r+m/ouINKC0AwZoeUujRugiIg0oXk1xaGGafW+MM9Dfy4FxmrZcbikldBGRBqX7UqwY7D9aSz+lv49vXHsuK4fCdcFETSUXEZFZKK6lvzmeY15vqmXLASihi4jMQivuTFSJSi4iIrPQijsTVaKELiIyS82+M1ElKrmIiMxCO62PrhG6iEiDolp8KyoaoYuINKjZi2/VooQuIhJSaXmlnTpcQCUXEZFQKpVX2qXDBTRCFxEJpVJ5pZ710eOmhC4iEkK7lVfKUclFRCSEdiuvlKOELiISUrtMIKpEJRcRkYRQQhcRSQgldBFpK+00lb7TqIYuIm2j3abSdxqN0EWkbbTbVPpOEyqhm9kiM3vQzH5qZi+Y2QfMbMDMnjCzPcGfi+MOVkSSrRN6vdtZ2JLL14BH3f3jZtYHnAR8FnjS3b9kZhuADcBNMcUpIl2gE3q921nNEbqZLQBWA3cBuHvO3d8C1gL3BrvdC3wkriBFpHu001T6ThOm5HIWkAHuNrNnzexOMzsZOM3d9wMEf54aY5wiIlJDmITeA5wD3O7u7wPeJl9eCcXM1pnZiJmNZDKZBsMUkW7XaDtjN7VBhqmhvwK84u5PB48fJJ/Qf2lmS919v5ktBV4vd7C7bwY2AwwPD3sEMYtIlzkwnmPNbdsYy07V1c7YbW2QNUfo7v4a8Asze2ew6WLgJ8B3gGuCbdcAW2KJUEQ6TpSj4mxumstv28ZrhybqbmfstjbIsF0ufwzcF3S47AM+Rf6XwQNmdi3wc+CqeEIUkU4S9ah4b2acsX+dPPp4/rwejkxOk81N13zdQhtkIZakt0GGSujuvhMYLvPUxdGGIyLtLJubrtpSmM1N89ju18gcmuDI1MzRUfFsVihcMdjP4IK5ZA5NMD/dQ0/KuPquZ0L9sui2NkhN/ReRUGqNvIufn5xx5vXMiWRUXJyUj0xOc/VdzxxXQqn1y6Ldl7yNkhK6iIRSrh5dnCiLn0/3pvjzj/06l648PZJRcSEpZ3PTXVVCqZcSuoiEUqseXfp8VMm8WLeVUOpl7s3rJBweHvaRkZGmvZ+IRCtMDV3JNnpmtsPdy13HPI5G6CISWq16dDfVq9uRls8VkZpm21feTbM1W0kjdBGpKkxfebVSS7fN1mwlJXQRqapWd0uthF16/O7RMeb1plRnj4ESuohUVam7pTAqPxIk6koJv/j4gf5ebrh/J2+Oa7QeByV0EamqXKtg8aj8lP4+Bvp7OTBO2XbG2U4MkvCU0EWkptLuleIyypvjOb5x7blVyyiaGNQcSugiUrfSMszKoXB3GNLEoHgpoYt0uUYmA80mMatXPT5K6CJdbDYthUrM7UcTi0S6WLfdACLplNBFutiKwX4G+nuZ2zOHgf5eXaTscEroIl3OsOP+DENT+duTaugiXWxvZpw3x3NMTM3w5nj1vvDCxdOhhWnWbtquqfxtSAldJOGqdbGEvedm8cXTBekeDmWnNDmoDSmhiyRYrS6WsO2HxRdP3Z1FJ/UC5WeGSusooYskWK2FtSBc+2HpSH7L+vMZHctqclCbUUIXSYhypZVCIs4cmmBBuoehhemGXrvcSH6gvy/K8CUCobpczOxlM/uxme00s5Fg2y1m9mqwbaeZrYk3VBGppFBaueqOH3DJxqeOdp+k+1JsWX8+i07q5VB2irWbtjfcmVIYyWtE3r7qaVv8oLuvKrmv3a3BtlXu/kjUwYlIONUmCI2OZRkruYgpyaQ+dJEEKJRW0r2pEy5UVntOksXcvfZOZi8BBwEH/srdN5vZLcAngUPACPDf3P1gtdcZHh72kZGR2cYsImXUug2cVjjsXGa2o6Q6UlbYEfp57n4OcDnwh2a2GrgdWAGsAvYDX60QyDozGzGzkUwmE/LtRKScSjM0yyXs4n1V/+4Oobpc3H00+PN1M/s2cK67bys8b2Z/DTxc4djNwGbIj9BnHbFIl6rUU15uO6AbM3ehmiN0MzvZzOYXvgcuAXaZ2dKi3T4K7IonRBGB/IXPzKEJspPTZA5NHL24We6CaOm2x3a/pnVXukCYkstpwHYzew54Bvh7d38U+ErQyvg88EHgxhjjFEmMRhe2GlqYZnIm/5/cyRk/2lNe7qJn8bbc9AwbHnr+uHZGSaaaJRd33we8t8z2q2OJSCTBwt5QolxNfHQsS19qDtmZafpScxgdyzLQ31dx+v7jN1zAY7tfY8NDz3NkakbrrnQBtS2KNFGYG0pUmiRUrf2w3EXPdF+KS1eezuCCuWpZ7BKa+i/SRGFWN6y0/koj9/HUTZm7ixK6SBOFSbDVkn4j9/HUvT+7hxK6SJNVS7CF2rlWM5RGKKGLtInZXDAVASV0kbYRZu3ysElfupO6XERCaMZNkcMsohWmS0a6l0boIjU0a1Q82wumIkroIjWEKYVEpVZHitoQpRqVXERqaLf1xLVyolSiEbpIDRoVS6fQCF0khKhGxc24uCrdSyN0kSZRy6HETSN0kSZRy6HETQldpEna7eKqJI9KLiJNoourEjcldJEm0sqHEieVXEREEkIJXSQCakeUdqCSi8gslbYjai1zaRUldJEaaq0/XtyOmDk0wZrbtjGWnVKvuTSdErpIFdUmAxUS/dDC9NEVEBekexjLTjVlIS+RUqESupm9DBwGpoEpdx82swHgfmA58DLw2+5+MJ4wRVqj0kqLlcosQwvTrN20XcvbSkvUM0L/oLu/UfR4A/Cku3/JzDYEj2+KNDqROpWWR2Z7u7ZK64+XJvrRsezRkbh6zaVVZlNyWQtcGHx/L7AVJXRpoXKj5uLRciP17EqTgardaEK95tIqYRO6A4+bmQN/5e6bgdPcfT+Au+83s1PjClIkjNJR87Y9mUhuTFEuQWvWp7SjsAn9PHcfDZL2E2b207BvYGbrgHUAy5YtayBEkXBKR82rzx6M9XZtGolLuzF3r+8As1uAceA/ARcGo/OlwFZ3f2e1Y4eHh31kZKTRWEVqirqGLtIOzGyHuw/X2q/mTFEzO9nM5he+By4BdgHfAa4JdrsG2NJ4uCLRKL0RhW7XJt0kTMnlNODbZlbY/+/c/VEz+xHwgJldC/wcuCq+MEVEpJaaCd3d9wHvLbP9TeDiOIISCUslFZFjNFNUOlbYW7oVJ31AvwAksZTQpWNVmsVZrDjpD/T3YhhvjuuenpJMWj5XOlaYW7qVJv3M4Qnd01MSSyN06VhhJvcU96aXjtC1zookjRK6dLRak3tKkz6ohi7JpYQuiVea9DW7U5JKNXQRkYRQQhcRSQgldGlL9dx0WTdoFslTDV0i1cjMzXILaoWZMFQ4Nuy+IkmnhC6RaSS5ljsmzIShgnr2FUk6lVwkMuWSayPHhJkwVFDPviJJpxG6RKbabdnqOaaeuwHpzkEix9R9g4vZ0A0uki+KGrqIHC+yG1yI1CPMDSVKu1J0EwqRaKjkIk1RGIUPLUyzdtN2daWIxEAJXWJX3MmyIN3DoezU0Yugj+1+jUtXnq6kLhIBlVwkdsWdLAffzjF/Xg/p3hS56Rk2PPQ8F/+vrTzwo59zYDynSUIis6ARusRuxWA/i0/uJfvWNLlpJ5WCL1z5bj6/ZTdHpmYYfesIn3nox6Tm7OK0BXM5+PakyjEiDdAIXZpiavpYN9XB8UnOGuxncMFcelN2dPv0jOsGFCKzoIQusdubGefwkamjjxee1MvKoYU8fsMF3PPJ95Oak0/qqTnG4Py5miQk0iCVXCQSB8ZzbNuTYfXZgwz09x33XPHkoYXpHh75r6uPllLOO3uQH332Q0ePTfel1JMu0qDQCd3MUsAI8Kq7X2Fm9wAXAGPBLp90953Rhyjt7sB4jvf/j+8yPeOk5hg/+uyHjkvqtWZzpvtS/Nqp+e217kAkIpXVM0K/HngBWFC07U/d/cFoQ5JOs21PhumZfI18esbZtifDR953xnH7VErUWi1RJDqhauhmdibwYeDOeMORdlSrlXD12YPH1cFXnz0Y+rUbWdBLRMoLO0LfCHwGmF+y/Ytm9nngSWCDu0+UHmhm64B1AMuWLZtFqNIKB8ZzrLltG2PZqYoj6IH+vuPq4KU19GoaWdBLRMqruTiXmV0BrHH39WZ2IfDpoIa+FHgN6AM2A3vd/c+qvZYW5+osB8ZzfOjWrRx4exKAdG+Kb133geNKJ1EsrKXFuUSqC7s4V5gR+nnAlWa2BpgHLDCzv3X33w+enzCzu4FPNx6utJtsbprLvrbtaDIHWJjuOW4EHVX9WxdCRaJRs4bu7je7+5nuvhz4BPA9d//9YISOmRnwEWBXrJFKU+0eHSNz+FgFbdFJx7cbgurfIu1mNn3o95nZIGDATuC6aEKSdvSXv3NO1f5y1b9FWq+uhO7uW4GtwfcXxRCPtImVQwsZWjzvaLI+51cHTthHdwsSaS+aKSplpftSfPfGC2sma9W/RdqHErpUpGQt0lm0OJeISEIooSeMbhAh0r1UckkQrYsi0t00Qu9AlUbh6gsX6W4aoXeYaqNw9YWLdDcl9A5TbhRe6ESJsi9c66uIdB4l9A5TaxQeRauhavEinUkJvcOUG4VHPZqu9r8AEWlfSugdqHgUHsdoWrV4kc6khN7hwo6m6xnFa40Wkc6khN7hwoymS0fxW9afz+hYVmu0iCSMEnqHCzOaLh3FX37bNg5VuaWciHQmTSxqA7Odrl8YTVdKzIVRfLo3xcJ0D2P/OqnJRyIJpBF6izWjRbB4FD+0MM3aTdt1wVMkgZTQW6y0HPLY7te4dOXpsST1Qk1cFzxFkkkllxYrLofkpmfY8NDzXLLxqchWSyxXzqlVohGRzqSE3mLpvhRb1p/P+gtX0DvHODI1U1dtu1r9vVDOueqOH0T6S0JE2pNKLi2WzU2zdtN2MocmmJxx0r2p0LXtWvV3zfgU6S4aobdYIekemZqhd47x5x/79dAXRmstl1tcztEFUJHk0wi9AVGtnZLNTXNkcpqB/l4OjMOS+X11XRANs1CXLoCKdI/QCd3MUsAI8Kq7X2Fm7wC+CQwA/wxc7e65eMJsH1G1GRa/zin9fXzj2nNZOVTfhcowCVszPkW6Rz0ll+uBF4oefxm41d3PBg4C10YZWLuK6q5Axa/z5niOeb2p0Mlc9w0VkXJCjdDN7Ezgw8AXgT8xMwMuAn432OVe4Bbg9hhibCtRrUTY6OuUjuwd58D4pKbxi0jokstG4DPA/ODxKcBb7j4VPH4FOKPcgWa2DlgHsGzZssYjbRNR1aUbfZ3ikX3m8AQAE0WtjiqviHSvmiUXM7sCeN3ddxRvLrOrlzve3Te7+7C7Dw8ODjYYZnuJamJOI69T3LkyOH+uulhE5KgwI/TzgCvNbA0wD1hAfsS+yMx6glH6mcBofGFKQenIHlAXi4gAIUbo7n6zu5/p7suBTwDfc/ffA/4R+Hiw2zXAltiilOMUj+w1jV9ECmYzsegm8hdIXyRfU78rmpBERKQRdU0scvetwNbg+33AudGHJCIijdDUfxGRhFBCFxFJCCV0EZGEUEIXEUkIJfQy6lkrReuqiEi70PK5JepZTbEZN3gWEQlLI/QS9aymGNXKiyIiUejYhB5XqaOeu/zojkAi0k7MveyaWrEYHh72kZGRWb9ONjfNb936FJnDEwzOn8sTN0Zb6qjnjkRR3b1IRKQSM9vh7sO19uvIEfru0TFePZhlYmqGVw9m2T06Funr17M+itZSEZF20ZEJXURETtSRCX3l0EKGFs9jbs8chhbPY+VQYzd1KK3DqwVRRDpZR7YtpvtSfPfGC+uqXZfWuktbDresP5+1m7arBVFEOlZHJnSo72725frFS1sOt+3JnNCCqNu5iUgn6ciSS73K9YuXthyuPntQLYgi0tE6doRej0LyLozQC2WX0ps0R3Hz5wK1M4pIs3VEH3oUfeHNTLBaEkBEohS2D73tR+hRra1ST819tsqVeFSPF5G4tX0NvTQ57h4dq9ha2C5rq2hJABFphbYfoRfXv0/p7+P6+5/lwPhk2dF6uVp5K0RdjxcRCaOjauhHJqe5+q5nyE5Ok+5N8a3rPnBCKUMXI0UkaRJTQ4dj9e8D4zkWpHtw94oj8GbWykVE2knNGrqZzTOzZ8zsOTPbbWZfCLbfY2YvmdnO4GtVnIFmc9Os3bSdQ9kpFp3Uy5b152sELiJSJMwIfQK4yN3HzawX2G5m/xA896fu/mB84R2ze3SM1w9NMDE1A8DoWJZ0X0rlFRGRQM2E7vkie6FdpDf4al7hnfzo/Pr7nyU3NYMBp/T3MbQwrV5vEZEiodoWzSxlZjuB14En3P3p4KkvmtnzZnarmc2tcOw6Mxsxs5FMJtNQkHsz4xwYn8SBvp45bPz3qxgdy1ZtUdTKiSLSbUIldHefdvdVwJnAuWb2HuBm4F3A+4EB4KYKx25292F3Hx4cHGwoyOK+7lMXzGXl0MKqvd6FCUZX3fEDLtn4lJK6iHSFurpc3P0tM9sKXObu/zPYPGFmdwOfjjq4gtK+bsiP2resP5/RsewJNXTN1BSRblQzoZvZIDAZJPM08CHgy2a21N33m5kBHwF2xRlooR0xzFIA7TLBSESkmcKM0JcC95pZinyJ5gF3f9jMvhckewN2AtfFGOdRYUbfmqkpIt0oTJfL88D7ymy/KJaIagg7+tYEIxHpNh0xU7SYRt8iIuV1XEIHjb5FRMpp++VzRUQkHCV0EZGEUEIXEUkIJXQRkYRQQhcRSQgldBGRhGjqLejMLAP8S4OHLwHeiDCcqCiu+iiu+iiu+rVrbLOJ61fdvebqhk1N6LNhZiNh7qnXbIqrPoqrPoqrfu0aWzPiUslFRCQhlNBFRBKikxL65lYHUIHiqo/iqo/iql+7xhZ7XB1TQxcRkeo6aYQuIiJVtDyhm9mvmNk/mtkLZrbbzK4Ptg+Y2RNmtif4c3GF468J9tljZtfEHNNfmNlPgxtjf9vMFlU4/mUz+7GZ7TSzkShiChHbLWb2avCeO81sTYXjLzOzn5nZi2a2oQlx3V8U08vBzcbLHR/LOTOzeWb2jJk9F8T1hWD7O8zs6eBzc7+Z9VU4/ubgXP3MzC5tQlz3Be+1y8y+bma9FY6fLjqv32lCXPeY2UtF77mqwvGR/zzWiOv7RTGNmtn/rXB8LOer6PVTZvasmT0cPG7N58vdW/pF/o5I5wTfzwf+H/Bu4CvAhmD7BuDLZY4dAPYFfy4Ovl8cY0yXAD3B9i+Xiyl47mVgSZPP1y3Ap2scmwL2AmcBfcBzwLvjjKtkn68Cn2/mOSN/R63+4Pte4Gng3wIPAJ8Itt8B/Jcyx747OEdzgXcE5y4Vc1xrgucM+N/l4gqOGY/p81UprnuAj9c4Npafx2pxlezzEPAfmnm+il7/T4C/Ax4OHrfk89XyEbq773f3fw6+Pwy8AJwBrAXuDXa7l/x9S0tdCjzh7gfc/SDwBHBZXDG5++PuPhXs9kPgzNm+V1SxhTz8XOBFd9/n7jngm+TPc+xxmZkBv00+STWN540HD3uDLwcuAh4Mtlf6fK0FvunuE+7+EvAi+XMYW1zu/kjwnAPP0OTPWJXzFUYsP49h4jKz+eT/TcuO0ONkZmcCHwbuDB4bLfp8tTyhFzOz5eRvd/c0cJq774d8sgBOLXPIGcAvih6/Qvjk1khMxf4A+IcKhznwuJntMLN1UcZTI7Y/CspBX7fyJarYz1eFuAB+E/ilu++pcFhs5yz47/BO4HXySWYv8FbRL+dK5yHW81Ual7s/XfRcL3A18GiFw+eZ2YiZ/dDMyiWLOOL6YvD5utXM5pY5tGXnC/go8KS7H6pweGznC9gIfAaYCR6fQos+X22T0M2sn/x/mW6o8o9ywmFltkXWtlMpJjP7HDAF3Ffh0PPc/RzgcuAPzWx1VDFVie12YAWwCthPvrxxwmFltkXa5lTl3/F3qD46j+2cufu0u68iP9o9F/g35XYrsy3W81Ual5m9p+jpTcA2d/9+hcOXeX7W4e8CG81sRcxx3Qy8C3g/+ZLKTWUObeX5qvX5iuV8mdkVwOvuvqN4c5ldm/L5aouEHoxGHgLuc/f/E2z+pZktDZ5fSv63cqlXgF8penwmMBpjTAQXeq4Afi/4b/EJ3H00+PN14NtE9N/0arG5+y+DD/wM8NcV3jO281UprmB7D/Ax4P5Kx8Z9zoLXfgvYSr4mvCiICyqfh1jPV5m4LgMws/8ODJKvy1Y6pnC+9gXHnnAj9yjjCkpq7u4TwN204PNVLi4AMzsliOfvqxwT1/k6D7jSzF4mX8K8iPyIvTWfr9kU4KP4Iv9b6m+AjSXb/4LjL4p+pcyxA8BL5C/ALA6+H4gxpsuAnwCDVY49GZhf9P0/kf+BiPt8LS36/kbytbnSY3vIX6h6B8cuiq6MM66i8/ZUK84Z+cS4KPg+DXyf/C/kb3H8Rav1ZY5dyfEXrfYR3UXRSnH9x+Dvn65y7GJgbvD9EmAP0V3crhTX0qJ/543Al8ocG8vPY7W4gsfXAfe24nyVvM+FHLso2pLPV6R/oQZPwvnk/5vxPLAz+FpDvg71ZHDynyx8MIBh4M6i4/+A/MWEF4FPxRzTi+RrXoVtdwT7DwGPBN+fFfwjPQfsBj7XpPP1DeDHwfbvFP0AHo0teLyGfAfK3ihjqxRX8Nw9wHUl+zflnAG/ATwbxLWLoMsmeM9ngn/TbxX9wF8J/FnR8Z8LztXPgMubENdU8H6Fc1jYfvRzD/y74N/6ueDPa5sQ1/eC99oF/C3HOk5i/3msFlfw3FZKBgDNOl8l73khxxJ6Sz5fmikqIpIQbVFDFxGR2VNCFxFJCCV0EZGEUEIXEUkIJXQRkYRQQhcRSQgldBGRhFBCFxFJiP8PjD62YzoV+4EAAAAASUVORK5CYII=\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"needs_background": "light"
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"plt.scatter(list_x, list_y, s=6)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 3\n",
|
||
"Estimate parameters of the simple linear regression by closed-form. Note: store these values into w_cf\n",
|
||
"and b_cf."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 100,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# D'abord, on doit transposer nos donner pour pouvoir les utiliser dans le calcul matriciel\n",
|
||
"# Donc on génère une liste de 1 qu'on va nsuit coller à notre liste de x\n",
|
||
"\n",
|
||
"list_1 = np.ones(len(list_x))\n",
|
||
"\n",
|
||
"\n",
|
||
"# Pis on colle les deux\n",
|
||
"\n",
|
||
"X = np.stack((list_1, list_x), axis=1)\n",
|
||
"\n",
|
||
"# Enfin, on fait notre calcul de ouf\n",
|
||
"\n",
|
||
"def B(y): \n",
|
||
" X_T = np.transpose(X)\n",
|
||
" prod = np.dot(X_T, X)\n",
|
||
" inv = np.linalg.inv(prod)\n",
|
||
" return np.dot(inv, np.dot(X_T,y))\n",
|
||
"\n",
|
||
"Y = np.asarray(list_y)\n",
|
||
"beta = B(Y)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 8\n",
|
||
"Create a function loss(x_i, y_i, w, b) which is computing the regression loss on the full dataset.\n",
|
||
"What is the value of loss([1], [3], 1, 2) ?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 101,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0"
|
||
]
|
||
},
|
||
"execution_count": 101,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# On détermine la fonction de coût\n",
|
||
"\n",
|
||
"def loss(x_i, y_i, w, b):\n",
|
||
" return ((w * x_i) + b - y_i)**2\n",
|
||
"\n",
|
||
"loss(1, 3, 1, 2)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 102,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"8"
|
||
]
|
||
},
|
||
"execution_count": 102,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"def dl_dw(xi, yi, w, b):\n",
|
||
" return -2 * xi * ( yi - (w * xi + b))\n",
|
||
"\n",
|
||
"dl_dw(4, -1, 0, 0)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 103,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"2"
|
||
]
|
||
},
|
||
"execution_count": 103,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"def dl_db(xi, yi, w, b):\n",
|
||
" return -2 * ( yi - (w * xi + b))\n",
|
||
"\n",
|
||
"dl_db(4, -1, 0, 0)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 11\n",
|
||
"Implement a function update_w_and_b(x_i, y_i, w, b, lr) which is updating w and b according to the gradient compute on the full data points. What is the output of the following command of update_w_and_b([0], [3], 5, 3, 0.1) and why?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 109,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(5.0, 3.0)"
|
||
]
|
||
},
|
||
"execution_count": 109,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"def update_w_and_b(x, y, w, b, lr):\n",
|
||
" grad_w = 0\n",
|
||
" grad_b = 0\n",
|
||
" \n",
|
||
" N = len(y)\n",
|
||
" \n",
|
||
" for t in range(N):\n",
|
||
" grad_w += dl_dw(x[t], y[t], w, b)\n",
|
||
" grad_b += dl_db(x[t], y[t], w, b)\n",
|
||
" \n",
|
||
" # update\n",
|
||
" w -= (1 / float(N)) * grad_w * lr\n",
|
||
" b -= (1 / float(N)) * grad_b * lr\n",
|
||
" \n",
|
||
" return w, b\n",
|
||
" \n",
|
||
"update_w_and_b([0], [3], 5, 3, 0.1)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 12 \n",
|
||
"Estimate parameters of the simple linear regression by gradient descent from a random initialization of\n",
|
||
"w and b and with a leanring rate equals to 0.001. Note: store these values into w_gd and b_gd."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 106,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"[array([ 8.46732643, -634.60911809]), array([ 8.46732643, -20.74489117])]"
|
||
]
|
||
},
|
||
"execution_count": 106,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Maintenant on voit comment régler ça en 3 lignes\n",
|
||
"\n",
|
||
"from sklearn.linear_model import LinearRegression\n",
|
||
"...\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.7.0"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
}
|