Linear Regression ์‹ ๊ฒฝ๋ง

 

 

Keras DNN model(Linear Regression ์‹ ๊ฒฝ๋ง)

Hidden layer : 2๊ฐœ(๋‰ด๋Ÿฐ 2๊ฐœ)

model = Sequential()

#hidden layer1 : shape = [2, 2]

model.add(Dense(2, input_shape=(2,), activation = 'relu')) # 1์ธต(hidden1)

#hidden layer2 : shape = [2, 2]

model.add(Dense(2, activation = 'relu')) # 2์ธต(hidden2)

#output layer : shape = [2, 1]

model.add(Dense(units=1))

#์ถœ๋ ฅ์ธต(output)

 

 

 

 

 

keras model

High Level API : Keras - ์ธ๊ฐ„์–ด ์ˆ˜์ค€ ์–ธ์–ด  
Low Level API : ๊ธฐ๊ณ„์–ด ์ˆ˜์ค€ ์–ธ์–ด 

dataset & ๋ชจ๋ธ ํ‰๊ฐ€

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score


keras model

import tensorflow as tf
from tensorflow.keras import Sequential #keras model
from tensorflow.keras.layers import Dense #DNN layer



1. dataset load

X, y = load_iris(return_X_y = True)
X.shape # (150, 4)
y #0~1

 



2. train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size = 0.3, random_state=123)

* ์กฐ์ ˆ๋ณ€์ˆ˜(w, b) ์ •์˜ ์—†์Œ : model์—์„œ ์ž๋™ ์ƒ์„ฑ



3. DNN model ์ƒ์„ฑ

model = Sequential() #keras model
print(model) #Sequential class์—์„œ ๋งŒ๋“ค์–ด์ง„ object

 

 

 

 

 

model์— layer์ถ”๊ฐ€
hidden layer 2๊ฐœ : unit = 12, unit=6


hidden layer1 : unit = 12 -> w[4, 12], b=12

model.add(Dense(units=12, input_shape=(4,), activation='relu')) #1์ธต


hidden layer2 : unit = 6 -> w[12, 6], b=6

model.add(Dense(units=6, activation='relu')) #2์ธต

* 6๊ฐœ์˜ ๋‰ด๋Ÿฐ์— ์˜ํ•ด 2์ธต ๋ ˆ์ด์–ด ์ƒ์„ฑ. 1์ธต์—์„œ ๋ฐ›์•„ ์˜จ ์ž๋ฃŒ๊ฐ€ input๋˜๋ฏ€๋กœ, input_shape=(4,)์กฐ๊ฑด์€ ํ•„์š”์น˜ ์•Š๋‹ค.

output layer : unit = 1 -> w[6, 1], b=1

model.add(Dense(units=1)) #3์ธต


model layer ๊ตฌ์กฐ ํ™•์ธ

print(model.summary())

_________________________________________________________________
Layer (type)                 Output Shape              Param -> (in*out)+b   
=================================================================
dense (Dense)              (None, 12)                60    -> (4*12)+12
_________________________________________________________________
dense_1 (Dense)              (None, 6)                 78    -> (12*6)+6
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 7     -> (6*1)+1
=================================================================
Total params: 145
Trainable params: 145
Non-trainable params: 0
_________________________________________________________________



4. model compile : ํ•™์Šต๊ณผ์ • ์„ค์ •

model.compile(optimizer='adam', loss='mse', metrics=['mae'])

* ์–ด๋А์ •๋„ ์ •ํ˜•ํ™” ๋˜์–ด์žˆ๋Š” ํ•™์Šต๊ณผ์ •
optimizer : ์ตœ์ ํ™” ์•Œ๊ณ ๋ฆฌ์ฆ˜ ('adam' or 'sgd')
loss : ์†์‹คํ•จ์ˆ˜(mes : ํ‰๊ท ์ œ๊ณฑ์˜ค์ฐจ)
metrics : ํ‰๊ฐ€ ๋ฐฉ๋ฒ• ('mae' or 'mse')



5. model training : model ํ•™์Šต

model.fit(x=X_train, y=y_train, #ํ›ˆ๋ จ์…‹
          epochs=100, #๋ฐ˜๋ณตํ•™์ŠตํšŸ์ˆ˜ (์‚ฌ์šฉ์ž๊ฐ€ ์กฐ์ ˆ๊ฐ€๋Šฅ)
          verbose=1, #ํ•™์Šต๊ฒฐ๊ณผ ์ถœ๋ ฅ์—ฌ๋ถ€ Y
          validation_data=(X_test, y_test)) #๊ฒ€์ฆ์…‹

Epoch 1/100
loss: 16.4911 - mae: 3.7187 - val_loss: 14.8984 - val_mae: 3.4226
:
Epoch 100/100
loss: 0.1761 - mae: 0.3791 - val_loss: 0.1872 - val_mae: 0.3978

* val_ ์€ ๊ฒ€์ฆ๊ฒฐ๊ณผ

 

 

 

 

 

keras_history

keras history : model ํ•™์Šต๊ณผ์ •๊ณผ ๊ฒ€์ฆ๊ณผ์ •์˜ ์†์‹ค๊ฐ’ ๋“ฑ์„ ๊ธฐ์–ตํ•˜๋Š” ๊ธฐ๋Šฅ 

 


dataset & ๋ชจ๋ธ ํ‰๊ฐ€ 

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score

#keras model 
import tensorflow as tf 
from tensorflow.keras import Sequential #keras model 
from tensorflow.keras.layers import Dense #DNN layer 

#keras seed๊ฐ’ ์ง€์ • 
import numpy as np 
import random as rd

 

keras ๋‚ด๋ถ€ ๊ฐ€์ค‘์น˜ seed ์ ์šฉ 

tf.random.set_seed(123) #global seed
np.random.seed(123) #numpy seed
rd.seed(123) #random seed



1. dataset load 

X, y = load_iris(return_X_y=True)
X.shape  #(150, 4)
y #0~2



2. train_test_split : ํ›ˆ๋ จ์…‹ vs ๊ฒ€์ฆ์…‹ 

X_train, X_val, y_train, y_val = train_test_split(
    X, y, test_size=0.3, random_state=123)


์กฐ์ ˆ๋ณ€์ˆ˜(w, b) ์ •์˜ ์—†์Œ : model์—์„œ ์ž๋™ ์ƒ์„ฑ 

 

3. DNN model ์ƒ์„ฑ 

model = Sequential() #keras model 
print(model) #Sequential object




hidden layer 2๊ฐœ : unit=12, unit=6
hidden layer1 : unit = 12 -> w[4, 12], b=12

model.add(Dense(units=12, input_shape=(4,), activation='relu')) #1์ธต


hidden layer2 : unit = 6 -> w[12, 6], b=6 

model.add(Dense(units=6, activation='relu')) #2์ธต


output layer : unit = 1 -> w[6, 1], b=1

model.add(Dense(units=1)) #3์ธต


model layer ํ™•์ธ 

print(model.summary())

_________________________________________________________________
Layer (type)                 Output Shape              Param #(in*out)+b   
=================================================================
dense (Dense)                (None, 12)                60=(4*12)+12        
_________________________________________________________________
dense_1 (Dense)              (None, 6)                 78=(12*6)+6        
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 7=(6*1)+1         
=================================================================
Total params: 145
Trainable params: 145
Non-trainable params: 0
_________________________________________________________________


4. model compile : ํ•™์Šต๊ณผ์ • ์„ค์ • 

model.compile(optimizer='adam', loss='mse', metrics=['mae'])

optimizer : ์ตœ์ ํ™” ์•Œ๊ณ ๋ฆฌ์ฆ˜('adam', 'sgd')
loss : ์†์‹คํ•จ์ˆ˜(mse) 
metrics : ํ‰๊ฐ€๋ฐฉ๋ฒ•('mae', 'mse')

 


[์ˆ˜์ •] 5. model training : model ํ•™์Šต 

model_fit = model.fit(x=X_train, y=y_train, #ํ›ˆ๋ จ์…‹
          epochs=100, #ํ•™์ŠตํšŸ์ˆ˜ 
          verbose=1, #์ถœ๋ ฅ์—ฌ๋ถ€ 
          validation_data=(X_val, y_val)) #๊ฒ€์ฆ์…‹



6. model evaluation : model ํ‰๊ฐ€ 

loss, mae = model.evaluate(X_val, y_val) 
print('loss value =', loss)
print('mae =', mae )

training data : 70% ํ›ˆ๋ จ์…‹ 
validation data : 30% ๊ฒ€์ฆ์…‹ 
test data : ์‹ค์ œ ์—…๋ฌด์šฉ dataset 


[์ถ”๊ฐ€] 7. model history

print(model_fit.history.keys()) #key ํ™•์ธ 
#dict_keys(['loss', 'mae', 'val_loss', 'val_mae'])
model_fit.history['loss'] #ํ›ˆ๋ จ์…‹ ์†์‹ค๊ฐ’ 
model_fit.history['val_loss'] #๊ฒ€์ฆ์…‹ ์†์‹ค๊ฐ’


loss vs val loss 

import matplotlib.pyplot as plt 
plt.plot(model_fit.history['loss'], 'y', label='training loss value')
plt.plot(model_fit.history['val_loss'], 'r', label='val loss value')
plt.xlabel('epochs')
plt.ylabel('loss value')
plt.legend(loc = 'best')
plt.show()


model ํ‰๊ฐ€ : mae vs val mae 

plt.plot(model_fit.history['mae'], 'y', label='training mae value')
plt.plot(model_fit.history['val_mae'], 'r', label='val mae value')
plt.xlabel('epochs')
plt.ylabel('mae')
plt.legend(loc = 'best')
plt.show()

+ Recent posts