{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Test3: (40 marks)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Your ID: 6304641464" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this excercise, we aimed at the case of customers default payments in Taiwan and compares the predictive accuracy of probability of default among data mining methods. From the perspective of risk management, the result of predictive accuracy of the estimated probability of default will be more valuable than the binary result of classification - credible or not credible clients. With the real probability of default as the response variable (Y), and the predictive probability of default as the independent variable (X), this research employed a binary variable, default payment (Yes = 1, No = 0), as the response variable. This study reviewed the literature and used the following 23 variables as explanatory variables: \\\\\n", "\n", "X1: Amount of the given credit (NT dollar): it includes both the individual consumer credit and his/her family (supplementary) credit. \\\\\n", "\n", "X2: Gender (1 = male; 2 = female). \\\\\n", "\n", "X3: Education (1 = graduate school; 2 = university; 3 = high school; 4 = others). \\\\\n", "\n", "X4: Marital status (1 = married; 2 = single; 3 = others). \\\\\n", "\n", "X5: Age (year). \\\\\n", "\n", "X6 - X11: History of past payment. We tracked the past monthly payment records (from April to September, 2005) as follows: X6 = the repayment status in September, 2005; X7 = the repayment status in August, 2005; . . .;X11 = the repayment status in April, 2005. The measurement scale for the repayment status is: -1 = pay duly; 1 = payment delay for one month; 2 = payment delay for two months; . . .; 8 = payment delay for eight months; 9 = payment delay for nine months and above. \\\\\n", "\n", "X12-X17: Amount of bill statement (NT dollar). X12 = amount of bill statement in September, 2005; X13 = amount of bill statement in August, 2005; . . .; X17 = amount of bill statement in April, 2005. \\\\\n", "\n", "X18-X23: Amount of previous payment (NT dollar). X18 = amount paid in September, 2005; X19 = amount paid in August, 2005; . . .;X23 = amount paid in April, 2005. \\\\\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn import tree\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import metrics #evaluation model.\n", "import matplotlib\n", "import numpy as np\n", "import pandas as pd\n", "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn import tree\n", "from sklearn.datasets import load_wine\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import metrics #evaluation model.\n", "from sklearn import (\n", " linear_model, metrics, pipeline, preprocessing, model_selection\n", ")\n", "import matplotlib " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Learning the Logistic Regression, KNN, in Python and Scikit-Learn \n", "\n", "There are 4 sub-question:\n", "\n", "1.1 Create the table to report the proportion of case of customers default payments in Taiwan separated by gender, education, and marital status. What is/are the interesting result/s you can draw from this table?[10 Points].\\\\" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Unnamed: 0X1X2X3X4X5X6X7X8X9...X15X16X17X18X19X20X21X22X23Y
01200002212422-1-1...000068900001
1212000022226-1200...3272345532610100010001000020001
2390000222340000...1433114948155491518150010001000100050000
3450000221370000...2831428959295472000201912001100106910000
455000012157-10-10...2094019146191312000366811000090006896790
..................................................................
2999529996220000131390000...88004312371598085002000050033047500010000
299962999715000013243-1-1-1-1...897951900183735268998129000
29997299983000012237432-1...20878205821935700220004200200031001
299982999980000131411-100...527741185548944859003409117819265296418041
299993000050000121460000...3653532428153132078180014301000100010001
\n", "

30000 rows × 25 columns

\n", "
" ], "text/plain": [ " Unnamed: 0 X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X15 X16 \\\n", "0 1 20000 2 2 1 24 2 2 -1 -1 ... 0 0 \n", "1 2 120000 2 2 2 26 -1 2 0 0 ... 3272 3455 \n", "2 3 90000 2 2 2 34 0 0 0 0 ... 14331 14948 \n", "3 4 50000 2 2 1 37 0 0 0 0 ... 28314 28959 \n", "4 5 50000 1 2 1 57 -1 0 -1 0 ... 20940 19146 \n", "... ... ... .. .. .. .. .. .. .. .. ... ... ... \n", "29995 29996 220000 1 3 1 39 0 0 0 0 ... 88004 31237 \n", "29996 29997 150000 1 3 2 43 -1 -1 -1 -1 ... 8979 5190 \n", "29997 29998 30000 1 2 2 37 4 3 2 -1 ... 20878 20582 \n", "29998 29999 80000 1 3 1 41 1 -1 0 0 ... 52774 11855 \n", "29999 30000 50000 1 2 1 46 0 0 0 0 ... 36535 32428 \n", "\n", " X17 X18 X19 X20 X21 X22 X23 Y \n", "0 0 0 689 0 0 0 0 1 \n", "1 3261 0 1000 1000 1000 0 2000 1 \n", "2 15549 1518 1500 1000 1000 1000 5000 0 \n", "3 29547 2000 2019 1200 1100 1069 1000 0 \n", "4 19131 2000 36681 10000 9000 689 679 0 \n", "... ... ... ... ... ... ... ... .. \n", "29995 15980 8500 20000 5003 3047 5000 1000 0 \n", "29996 0 1837 3526 8998 129 0 0 0 \n", "29997 19357 0 0 22000 4200 2000 3100 1 \n", "29998 48944 85900 3409 1178 1926 52964 1804 1 \n", "29999 15313 2078 1800 1430 1000 1000 1000 1 \n", "\n", "[30000 rows x 25 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#put your code here\n", "df = pd.read_excel('default.xls')\n", "df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Gender**\n", "- We can see that male(X=1) tend to have more chance of defualt payment than male." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y
X2
10.241672
20.207763
\n", "
" ], "text/plain": [ " Y\n", "X2 \n", "1 0.241672\n", "2 0.207763" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.groupby('X2')[['Y']].mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Education**\n", "- We can see that person in high school has the chance of default payments. Less education, more chance." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y
X3
00.000000
10.192348
20.237349
30.251576
40.056911
50.064286
60.156863
\n", "
" ], "text/plain": [ " Y\n", "X3 \n", "0 0.000000\n", "1 0.192348\n", "2 0.237349\n", "3 0.251576\n", "4 0.056911\n", "5 0.064286\n", "6 0.156863" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.groupby('X3')[['Y']].mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Marital Status**\n", "- We can see that marital status quite has no effect on defualt payment becuase all of the category has result in the same way." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y
X4
00.092593
10.234717
20.209283
30.260062
\n", "
" ], "text/plain": [ " Y\n", "X4 \n", "0 0.092593\n", "1 0.234717\n", "2 0.209283\n", "3 0.260062" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.groupby('X4')[['Y']].mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, partition the data into training and test sets using 80 $\\%$ to be the train data. The model will be fit to the training data and evaluated on the test set. \n", "\n", "1.2 Use the Logistic regression to train the model with all features. Then, use the test data to conduct the confusion matrix. Interpret the results carefully [10 Points]." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Unnamed: 0',\n", " 'X1',\n", " 'X2',\n", " 'X3',\n", " 'X4',\n", " 'X5',\n", " 'X6',\n", " 'X7',\n", " 'X8',\n", " 'X9',\n", " 'X10',\n", " 'X11',\n", " 'X12',\n", " 'X13',\n", " 'X14',\n", " 'X15',\n", " 'X16',\n", " 'X17',\n", " 'X18',\n", " 'X19',\n", " 'X20',\n", " 'X21',\n", " 'X22',\n", " 'X23',\n", " 'Y']" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(df)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
X1X2X3X4X5X6X7X8X9X10...X14X15X16X17X18X19X20X21X22X23
0200002212422-1-1-2...68900006890000
112000022226-12000...2682327234553261010001000100002000
2900002223400000...13559143311494815549151815001000100010005000
3500002213700000...49291283142895929547200020191200110010691000
45000012157-10-100...35835209401914619131200036681100009000689679
..................................................................
299952200001313900000...2083658800431237159808500200005003304750001000
2999615000013243-1-1-1-10...350289795190018373526899812900
299973000012237432-10...27582087820582193570022000420020003100
2999880000131411-1000...7630452774118554894485900340911781926529641804
29999500001214600000...49764365353242815313207818001430100010001000
\n", "

30000 rows × 23 columns

\n", "
" ], "text/plain": [ " X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 ... X14 X15 X16 \\\n", "0 20000 2 2 1 24 2 2 -1 -1 -2 ... 689 0 0 \n", "1 120000 2 2 2 26 -1 2 0 0 0 ... 2682 3272 3455 \n", "2 90000 2 2 2 34 0 0 0 0 0 ... 13559 14331 14948 \n", "3 50000 2 2 1 37 0 0 0 0 0 ... 49291 28314 28959 \n", "4 50000 1 2 1 57 -1 0 -1 0 0 ... 35835 20940 19146 \n", "... ... .. .. .. .. .. .. .. .. ... ... ... ... ... \n", "29995 220000 1 3 1 39 0 0 0 0 0 ... 208365 88004 31237 \n", "29996 150000 1 3 2 43 -1 -1 -1 -1 0 ... 3502 8979 5190 \n", "29997 30000 1 2 2 37 4 3 2 -1 0 ... 2758 20878 20582 \n", "29998 80000 1 3 1 41 1 -1 0 0 0 ... 76304 52774 11855 \n", "29999 50000 1 2 1 46 0 0 0 0 0 ... 49764 36535 32428 \n", "\n", " X17 X18 X19 X20 X21 X22 X23 \n", "0 0 0 689 0 0 0 0 \n", "1 3261 0 1000 1000 1000 0 2000 \n", "2 15549 1518 1500 1000 1000 1000 5000 \n", "3 29547 2000 2019 1200 1100 1069 1000 \n", "4 19131 2000 36681 10000 9000 689 679 \n", "... ... ... ... ... ... ... ... \n", "29995 15980 8500 20000 5003 3047 5000 1000 \n", "29996 0 1837 3526 8998 129 0 0 \n", "29997 19357 0 0 22000 4200 2000 3100 \n", "29998 48944 85900 3409 1178 1926 52964 1804 \n", "29999 15313 2078 1800 1430 1000 1000 1000 \n", "\n", "[30000 rows x 23 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#put your code here\n", "X=df[[\n", " 'X1',\n", " 'X2',\n", " 'X3',\n", " 'X4',\n", " 'X5',\n", " 'X6',\n", " 'X7',\n", " 'X8',\n", " 'X9',\n", " 'X10',\n", " 'X11',\n", " 'X12',\n", " 'X13',\n", " 'X14',\n", " 'X15',\n", " 'X16',\n", " 'X17',\n", " 'X18',\n", " 'X19',\n", " 'X20',\n", " 'X21',\n", " 'X22',\n", " 'X23'\n", " ]]\n", "X" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 1\n", "1 1\n", "2 0\n", "3 0\n", "4 0\n", " ..\n", "29995 0\n", "29996 0\n", "29997 1\n", "29998 1\n", "29999 1\n", "Name: Y, Length: 30000, dtype: int64" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y=df['Y']\n", "y" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = model_selection.train_test_split(\n", " X, y, test_size=0.2, random_state=100\n", ")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\User\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:814: ConvergenceWarning: lbfgs failed to converge (status=1):\n", "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n", "\n", "Increase the number of iterations (max_iter) or scale the data as shown in:\n", " https://scikit-learn.org/stable/modules/preprocessing.html\n", "Please also refer to the documentation for alternative solver options:\n", " https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n", " n_iter_i = _check_optimize_result(\n" ] }, { "data": { "text/plain": [ "LogisticRegression()" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "logistic_model = linear_model.LogisticRegression(solver=\"lbfgs\")\n", "logistic_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
predictedactualp[0]p[1]
6102000.7224290.277571
2539000.5676280.432372
21576000.5873820.412618
19574000.7223400.277660
12804000.9968360.003164
...............
27648000.8094270.190573
1917000.8027610.197239
20886000.9532740.046726
29931000.6674770.332523
865000.5440140.455986
\n", "

6000 rows × 4 columns

\n", "
" ], "text/plain": [ " predicted actual p[0] p[1]\n", "6102 0 0 0.722429 0.277571\n", "2539 0 0 0.567628 0.432372\n", "21576 0 0 0.587382 0.412618\n", "19574 0 0 0.722340 0.277660\n", "12804 0 0 0.996836 0.003164\n", "... ... ... ... ...\n", "27648 0 0 0.809427 0.190573\n", "1917 0 0 0.802761 0.197239\n", "20886 0 0 0.953274 0.046726\n", "29931 0 0 0.667477 0.332523\n", "865 0 0 0.544014 0.455986\n", "\n", "[6000 rows x 4 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "logistic_pred =logistic_model.predict(X_test)\n", "logistic_proba =logistic_model.predict_proba(X_test)\n", "logistic_result=pd.DataFrame({'predicted': logistic_pred,\n", " 'actual':y_test,\n", " 'p[0]':[p[0] for p in logistic_proba ],\n", " 'p[1]':[p[1] for p in logistic_proba ],\n", " })\n", "logistic_result" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " no default 0.77 1.00 0.87 4625\n", " default 0.00 0.00 0.00 1375\n", "\n", " accuracy 0.77 6000\n", " macro avg 0.39 0.50 0.44 6000\n", "weighted avg 0.59 0.77 0.67 6000\n", "\n" ] } ], "source": [ "report = metrics.classification_report(\n", " y_test, logistic_model.predict(X_test),\n", " target_names=[\"no default\", \"default\"]\n", ")\n", "print(report)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The model only predict that their will be no default." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1.3 Use the KNN to train the model with all features (you should convert the features to be the standarized variables first). Then, use the test data to conduct the confusion matrix. Interpret the results carefully [10 Points]." ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "#put your code here\n", "%matplotlib inline\n", "\n", "from pathlib import Path\n", "\n", "import pandas as pd\n", "from sklearn import preprocessing\n", "from sklearn.metrics import pairwise\n", "from scipy.cluster.hierarchy import dendrogram, linkage, fcluster\n", "from sklearn.cluster import KMeans\n", "import matplotlib.pylab as plt\n", "import seaborn as sns\n", "from pandas.plotting import parallel_coordinates" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [], "source": [ "X1 = X.astype(int)" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "#make it standardized\n", "from sklearn.preprocessing import StandardScaler\n", "sc = StandardScaler()\n", "X_scaled = sc.fit_transform(X)" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "X_scaled_train, X_scaled_test, y_train, y_test = model_selection.train_test_split(\n", " X, y, test_size=0.2, random_state=100\n", ")" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "from sklearn.neighbors import KNeighborsRegressor\n", "knn_model = KNeighborsRegressor(n_neighbors=10)" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "KNeighborsRegressor(n_neighbors=10)" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "knn_model.fit(X_scaled_train, y_train)" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "y_pred = knn_model.predict(X_scaled_test)" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "ename": "ValueError", "evalue": "Classification metrics can't handle a mix of binary and continuous targets", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", "Input \u001b[1;32mIn [60]\u001b[0m, in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmetrics\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m classification_report\n\u001b[1;32m----> 2\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[43mclassification_report\u001b[49m\u001b[43m(\u001b[49m\u001b[43my_test\u001b[49m\u001b[43m,\u001b[49m\u001b[43my_pred\u001b[49m\u001b[43m)\u001b[49m)\n", "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:2110\u001b[0m, in \u001b[0;36mclassification_report\u001b[1;34m(y_true, y_pred, labels, target_names, sample_weight, digits, output_dict, zero_division)\u001b[0m\n\u001b[0;32m 1998\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mclassification_report\u001b[39m(\n\u001b[0;32m 1999\u001b[0m y_true,\n\u001b[0;32m 2000\u001b[0m y_pred,\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 2007\u001b[0m zero_division\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwarn\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 2008\u001b[0m ):\n\u001b[0;32m 2009\u001b[0m \u001b[38;5;124;03m\"\"\"Build a text report showing the main classification metrics.\u001b[39;00m\n\u001b[0;32m 2010\u001b[0m \n\u001b[0;32m 2011\u001b[0m \u001b[38;5;124;03m Read more in the :ref:`User Guide `.\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 2107\u001b[0m \u001b[38;5;124;03m \u001b[39;00m\n\u001b[0;32m 2108\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[1;32m-> 2110\u001b[0m y_type, y_true, y_pred \u001b[38;5;241m=\u001b[39m \u001b[43m_check_targets\u001b[49m\u001b[43m(\u001b[49m\u001b[43my_true\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my_pred\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 2112\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m labels \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m 2113\u001b[0m labels \u001b[38;5;241m=\u001b[39m unique_labels(y_true, y_pred)\n", "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\metrics\\_classification.py:93\u001b[0m, in \u001b[0;36m_check_targets\u001b[1;34m(y_true, y_pred)\u001b[0m\n\u001b[0;32m 90\u001b[0m y_type \u001b[38;5;241m=\u001b[39m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmulticlass\u001b[39m\u001b[38;5;124m\"\u001b[39m}\n\u001b[0;32m 92\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(y_type) \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[1;32m---> 93\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[0;32m 94\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mClassification metrics can\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mt handle a mix of \u001b[39m\u001b[38;5;132;01m{0}\u001b[39;00m\u001b[38;5;124m and \u001b[39m\u001b[38;5;132;01m{1}\u001b[39;00m\u001b[38;5;124m targets\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(\n\u001b[0;32m 95\u001b[0m type_true, type_pred\n\u001b[0;32m 96\u001b[0m )\n\u001b[0;32m 97\u001b[0m )\n\u001b[0;32m 99\u001b[0m \u001b[38;5;66;03m# We can't have more than one value on y_type => The set is no more needed\u001b[39;00m\n\u001b[0;32m 100\u001b[0m y_type \u001b[38;5;241m=\u001b[39m y_type\u001b[38;5;241m.\u001b[39mpop()\n", "\u001b[1;31mValueError\u001b[0m: Classification metrics can't handle a mix of binary and continuous targets" ] } ], "source": [ "from sklearn.metrics import classification_report\n", "print(classification_report(y_test,y_pred))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ " 1.4 Compare the predictive performance of two models. Which one you select? And Why? [10 Points]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#put your code here\n", "Since the logistic regression report that all predict is \"no defualt\" so i can assume that the knn method is better." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" } }, "nbformat": 4, "nbformat_minor": 4 }