karas mnist history

* keras mnist batch์ฐธ๊ณ  
History : ํ›ˆ๋ จ๊ณผ ๊ฒ€์ฆ๊ณผ์ •์—์„œ ๋ฐœ์ƒํ•˜๋Š” ์†์‹ค๊ฐ’/์ •ํ™•๋„ ๊ฒฐ๊ณผ ๊ธฐ์–ต ๊ธฐ๋Šฅ 


from tensorflow.keras.datasets import mnist #mnist load 
from tensorflow.keras.utils import to_categorical #Y๋ณ€์ˆ˜ : encoding 
from tensorflow.keras import Sequential #keras model ์ƒ์„ฑ 
from tensorflow.keras.layers import Dense #DNN layer ๊ตฌ์ถ•
import matplotlib.pyplot as plt #์‹œ๊ฐํ™” ๋„๊ตฌ




keras ๋‚ด๋ถ€ w,b๋ณ€์ˆ˜ seed ์ ์šฉ

import tensorflow as tf
import numpy as np 
import random as rd

tf.random.set_seed(123)
np.random.seed(123)
rd.seed(123)
import time #ํ•™์Šต ์†Œ์š” ์‹œ๊ฐ„ ์ธก์ •



1. mnist dataset load 

(x_train, y_train), (x_val, y_val) = mnist.load_data() #(images, labels)


images : X๋ณ€์ˆ˜ 

x_train.shape #(60000, 28, 28) - (size, h, w) : 2d ์ œ๊ณต 
x_val.shape #(10000, 28, 28)

x_train[0] #0~255
x_train.max() #255

 

labels : y๋ณ€์ˆ˜ 

y_train.shape #(60000,)
y_train[0] #5




2. X,y๋ณ€์ˆ˜ ์ „์ฒ˜๋ฆฌ 
1) X๋ณ€์ˆ˜ : ์ •๊ทœํ™” & reshape(2d -> 1d)

x_train = x_train / 255. #์ •๊ทœํ™” 
x_val = x_val / 255.

x_train[0]


reshape(2d -> 1d)

x_train = x_train.reshape(-1, 784) #(60000, 28*28)
x_val = x_val.reshape(-1, 784) #(10000, 28*28)


2) y๋ณ€์ˆ˜ : class(10์ง„์ˆ˜) -> one-hot encoding(2์ง„์ˆ˜)

y_train = to_categorical(y_train)
y_val = to_categorical(y_val)


์ „์ฒ˜๋ฆฌ ํ™•์ธ 

x_train.shape #(60000, 784)
y_train[0] #[0., 0., 0., 0., 0., 1., 0., 0., 0., 0.] - 5
y_train.shape #(60000, 10)

start_time = time.time() #์†Œ์š” ์‹œ๊ฐ„ ์ฒดํŠธ




3. keras model

model = Sequential()




4. DNN model layer ๊ตฌ์ถ• 
hidden layer1 : w[784, 128]

model.add(Dense(units=128, input_shape=(784,), activation='relu')) #1์ธต


hidden layer2 : w[128, 64]

model.add(Dense(units=64, activation='relu')) #2์ธต


hidden layer3 : w[64, 32]

model.add(Dense(units=32, activation='relu')) #3์ธต


output layer : w[32, 10]

model.add(Dense(units=10, activation='softmax')) #4์ธต


model layer ํ™•์ธ 

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_8 (Dense)              (None, 128)               100480=784x128+128    
_________________________________________________________________
dense_9 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_10 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_11 (Dense)             (None, 10)                330       
=================================================================
Total params: 111,146


 

5. model compile : ํ•™์Šต๊ณผ์ • ์„ค์ •(๋‹คํ•ญ๋ถ„๋ฅ˜๊ธฐ) 

model.compile(optimizer='adam', #default : learning_rate=0.001
              loss='categorical_crossentropy', 
              metrics=['accuracy'])




6. [์ˆ˜์ •] model training : train(70) vs val(30)

model_fit = model.fit(x=x_train, y=y_train, #ํ›ˆ๋ จ์…‹ 
          epochs=15, #๋ฐ˜๋ณตํ•™์Šต ํšŸ์ˆ˜ : 60000 * 10 = 600,000 -> full batch
          batch_size=100, # 1epoch(100 * 600) * 10 = 600,000 -> mini batch 
          verbose=1, #์ถœ๋ ฅ์—ฌ๋ถ€ 
          validation_data= (x_val, y_val)) #๊ฒ€์ฆ์…‹

stop_time = time.time() - start_time 

print('์†Œ์š”์‹œ๊ฐ„ : ', stop_time)

full batch 
accuracy: 0.9923 - val_loss: 0.0873 - val_accuracy: 0.9793
์†Œ์š”์‹œ๊ฐ„ :  25.037985801696777

mini batch 
accuracy: 0.9933 - val_loss: 0.0916 - val_accuracy: 0.9739
์†Œ์š”์‹œ๊ฐ„ :  11.333117961883545



7. model evaluation : val dataset 

print('model evaluation')
model.evaluate(x=x_val, y=y_val)
#loss: 0.0916 - accuracy: 0.9739

 



8. model history 

print(model_fit.history.keys()) #dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])


loss vs val_loss : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['loss'], 'y', label='train loss')
plt.plot(model_fit.history['val_loss'], 'r', label='val loss')
plt.xlabel('epochs')
plt.ylabel('loss')
plt.legend(loc = 'best')
plt.show()


accuracy vs val_accuracy : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['accuracy'], 'y', label='train accuracy')
plt.plot(model_fit.history['val_accuracy'], 'r', label='val accuracy')
plt.xlabel('epochs')
plt.ylabel('accuracy')
plt.legend(loc = 'best')
plt.show()

 

 

 

 

 

 

karas mnist dropout

Dropout : ๋ฌด์ž‘์œ„ ๋„คํŠธ์›Œํฌ ์‚ญ์ œ -> ๊ณผ์ ํ™” ์ตœ์†Œํ™”  
* karas_mnist_history ์ฐธ๊ณ 

from tensorflow.keras.datasets import mnist #mnist load 
from tensorflow.keras.utils import to_categorical #Y๋ณ€์ˆ˜ : encoding 
from tensorflow.keras import Sequential #keras model ์ƒ์„ฑ 
from tensorflow.keras.layers import Dense, Dropout #[์ถ”๊ฐ€] DNN layer ๊ตฌ์ถ•
import matplotlib.pyplot as plt #์‹œ๊ฐํ™” ๋„๊ตฌ




keras ๋‚ด๋ถ€ w,b๋ณ€์ˆ˜ seed ์ ์šฉ 

import tensorflow as tf
import numpy as np 
import random as rd

tf.random.set_seed(123)
np.random.seed(123)
rd.seed(123)
import time #ํ•™์Šต ์†Œ์š” ์‹œ๊ฐ„ ์ธก์ •



1. mnist dataset load 

(x_train, y_train), (x_val, y_val) = mnist.load_data() #(images, labels)


images : X๋ณ€์ˆ˜ 

x_train.shape #(60000, 28, 28) - (size, h, w) : 2d ์ œ๊ณต 
x_val.shape #(10000, 28, 28)

x_train[0] #0~255
x_train.max() #255


labels : y๋ณ€์ˆ˜ 

y_train.shape #(60000,)
y_train[0] #5

 

 


2. X,y๋ณ€์ˆ˜ ์ „์ฒ˜๋ฆฌ 
1) X๋ณ€์ˆ˜ : ์ •๊ทœํ™” & reshape(2d -> 1d)

x_train = x_train / 255. #์ •๊ทœํ™” 
x_val = x_val / 255.

x_train[0]


reshape(2d -> 1d)

x_train = x_train.reshape(-1, 784) #(60000, 28*28)
x_val = x_val.reshape(-1, 784) #(10000, 28*28)


2) y๋ณ€์ˆ˜ : class(10์ง„์ˆ˜) -> one-hot encoding(2์ง„์ˆ˜)

y_train = to_categorical(y_train)
y_val = to_categorical(y_val)


์ „์ฒ˜๋ฆฌ ํ™•์ธ 

x_train.shape # (60000, 784)
y_train[0] #[0., 0., 0., 0., 0., 1., 0., 0., 0., 0.] - 5
y_train.shape #(60000, 10)

start_time = time.time() #์†Œ์š” ์‹œ๊ฐ„ ์ฒดํŠธ




3. keras model

model = Sequential()




4. DNN model layer ๊ตฌ์ถ• 
hidden layer1 : w[784, 128]

model.add(Dense(units=128, input_shape=(784,), activation='relu'))#1์ธต 
model.add(Dropout(rate=0.3)) #[์ถ”๊ฐ€]


hidden layer2 : w[128, 64]

model.add(Dense(units=64, activation='relu')) #2์ธต 
model.add(Dropout(rate=0.1)) #[์ถ”๊ฐ€]


hidden layer3 : w[64, 32]

model.add(Dense(units=32, activation='relu')) #3์ธต
model.add(Dropout(rate=0.1)) #[์ถ”๊ฐ€]


output layer : w[32, 10]

model.add(Dense(units=10, activation='softmax')) #4์ธต


model layer ํ™•์ธ 

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_8 (Dense)              (None, 128)               100480=784x128+128    
_________________________________________________________________
dense_9 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_10 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_11 (Dense)             (None, 10)                330       
=================================================================
Total params: 111,146

 

 


5. model compile : ํ•™์Šต๊ณผ์ • ์„ค์ •(๋‹คํ•ญ๋ถ„๋ฅ˜๊ธฐ) 

model.compile(optimizer='adam', #default : learning_rate=0.001
              loss='categorical_crossentropy', 
              metrics=['accuracy'])




6. model training : train(70) vs val(30)

model_fit = model.fit(x=x_train, y=y_train, #ํ›ˆ๋ จ์…‹ 
          epochs=15, #๋ฐ˜๋ณตํ•™์Šต ํšŸ์ˆ˜ : 60000 * 10 = 600,000 -> full batch
          batch_size=100, #1epoch(100 * 600) * 10 = 600,000 -> mini batch 
          verbose=1, #์ถœ๋ ฅ์—ฌ๋ถ€ 
          validation_data= (x_val, y_val)) #๊ฒ€์ฆ์…‹

stop_time = time.time() - start_time 

print('์†Œ์š”์‹œ๊ฐ„ : ', stop_time)

full batch 
accuracy: 0.9923 - val_loss: 0.0873 - val_accuracy: 0.9793
์†Œ์š”์‹œ๊ฐ„ :  25.037985801696777

mini batch 
accuracy: 0.9933 - val_loss: 0.0916 - val_accuracy: 0.9739
์†Œ์š”์‹œ๊ฐ„ :  11.333117961883545



7. model evaluation : val dataset 

print('model evaluation')
model.evaluate(x=x_val, y=y_val) #loss: 0.0916 - accuracy: 0.9739




8. model history 

print(model_fit.history.keys()) #dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])


loss vs val_loss : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['loss'], 'y', label='train loss')
plt.plot(model_fit.history['val_loss'], 'r', label='val loss')
plt.xlabel('epochs')
plt.ylabel('loss')
plt.legend(loc = 'best')
plt.show()


accuracy vs val_accuracy : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['accuracy'], 'y', label='train accuracy')
plt.plot(model_fit.history['val_accuracy'], 'r', label='val accuracy')
plt.xlabel('epochs')
plt.ylabel('accuracy')
plt.legend(loc = 'best')
plt.show()

 

 

 

 

 

karas mnist earlyStopping

* step02_karas_mnist_dropout ์ฐธ๊ณ  

 

1. Dropout : ๋ฌด์ž‘์œ„ ๋„คํŠธ์›Œํฌ ์‚ญ์ œ -> ๊ณผ์ ํ™” ์ตœ์†Œํ™”  
2. EarlyStopping 
- val loss value์— ๋ณ€ํ™”๊ฐ€ ์—†๋Š” ๊ฒฝ์šฐ ํŠน์ • ์‹œ์ ์—์„œ ํ•™์Šต ์กฐ๊ธฐ ์ข…๋ฃŒ 
 

from tensorflow.keras.datasets import mnist #mnist load 
from tensorflow.keras.utils import to_categorical # Y๋ณ€์ˆ˜ : encoding 
from tensorflow.keras import Sequential #keras model ์ƒ์„ฑ 
from tensorflow.keras.layers import Dense, Dropout #DNN layer ๊ตฌ์ถ•
from tensorflow.keras.callbacks import EarlyStopping #[์ถ”๊ฐ€]

import matplotlib.pyplot as plt #์‹œ๊ฐํ™” ๋„๊ตฌ




keras ๋‚ด๋ถ€ w,b๋ณ€์ˆ˜ seed ์ ์šฉ

import tensorflow as tf
import numpy as np 
import random as rd

tf.random.set_seed(123)
np.random.seed(123)
rd.seed(123)
import time #ํ•™์Šต ์†Œ์š” ์‹œ๊ฐ„ ์ธก์ •



1. mnist dataset load 

(x_train, y_train), (x_val, y_val) = mnist.load_data() #(images, labels)


images : X๋ณ€์ˆ˜ 

x_train.shape #(60000, 28, 28) - (size, h, w) : 2d ์ œ๊ณต 
x_val.shape #(10000, 28, 28)

x_train[0] #0~255
x_train.max() #255


labels : y๋ณ€์ˆ˜ 

y_train.shape #(60000,)
y_train[0] #5




2. X,y๋ณ€์ˆ˜ ์ „์ฒ˜๋ฆฌ 
1) X๋ณ€์ˆ˜ : ์ •๊ทœํ™” & reshape(2d -> 1d)

x_train = x_train / 255. #์ •๊ทœํ™” 
x_val = x_val / 255.

x_train[0]


reshape(2d -> 1d)

x_train = x_train.reshape(-1, 784) #(60000, 28*28)
x_val = x_val.reshape(-1, 784) #(10000, 28*28)


2) y๋ณ€์ˆ˜ : class(10์ง„์ˆ˜) -> one-hot encoding(2์ง„์ˆ˜)

y_train = to_categorical(y_train)
y_val = to_categorical(y_val)


์ „์ฒ˜๋ฆฌ ํ™•์ธ 

x_train.shape #(60000, 784)
y_train[0] #[0., 0., 0., 0., 0., 1., 0., 0., 0., 0.] - 5
y_train.shape #(60000, 10)

start_time = time.time() # ์š” ์‹œ๊ฐ„ ์ฒดํŠธ




3. keras model

model = Sequential()




4. DNN model layer ๊ตฌ์ถ• 
hidden layer1 : w[784, 128]

model.add(Dense(units=128, input_shape=(784,), activation='relu')) #1์ธต 
model.add(Dropout(rate=0.3))


hidden layer2 : w[128, 64]

model.add(Dense(units=64, activation='relu')) #2์ธต 
model.add(Dropout(rate=0.1))


hidden layer3 : w[64, 32]

model.add(Dense(units=32, activation='relu')) #3์ธต
model.add(Dropout(rate=0.1))


output layer : w[32, 10]

model.add(Dense(units=10, activation='softmax')) #4์ธต


model layer ํ™•์ธ 

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_8 (Dense)              (None, 128)               100480=784x128+128    
_________________________________________________________________
dense_9 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_10 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_11 (Dense)             (None, 10)                330       
=================================================================
Total params: 111,146



5. model compile : ํ•™์Šต๊ณผ์ • ์„ค์ •(๋‹คํ•ญ๋ถ„๋ฅ˜๊ธฐ) 

model.compile(optimizer='adam', #default : learning_rate=0.001
              loss='categorical_crossentropy', 
              metrics=['accuracy'])




6. model training : train(70) vs val(30)
[์ถ”๊ฐ€] epoch=5 ์ดํ›„ ๊ฒ€์ฆ ์†์‹ค์ด ๊ฐœ์„ ๋˜์ง€ ์•Š์œผ๋ฉด ์กฐ๊ธฐ์ข…๋ฃŒ 

callback = EarlyStopping(monitor='val_loss', patience=5)

model_fit = model.fit(x=x_train, y=y_train, #ํ›ˆ๋ จ์…‹ 
          epochs=30, #[์ˆ˜์ •] ๋ฐ˜๋ณตํ•™์Šต ํšŸ์ˆ˜ 
          batch_size=100, # 1epoch(100 * 600) * 10 = 600,000 -> mini batch 
          verbose=1, #์ถœ๋ ฅ์—ฌ๋ถ€ 
          validation_data= (x_val, y_val), # ๊ฒ€์ฆ์…‹
          callbacks = callback) #[์ถ”๊ฐ€] ์กฐ๊ธฐ์ข…๋ฃŒ 

stop_time = time.time() - start_time 

print('์†Œ์š”์‹œ๊ฐ„ : ', stop_time)

full batch 
accuracy: 0.9923 - val_loss: 0.0873 - val_accuracy: 0.9793
์†Œ์š”์‹œ๊ฐ„ :  25.037985801696777

mini batch 
accuracy: 0.9933 - val_loss: 0.0916 - val_accuracy: 0.9739
์†Œ์š”์‹œ๊ฐ„ :  11.333117961883545



7. model evaluation : val dataset 

print('model evaluation')
model.evaluate(x=x_val, y=y_val) #loss: 0.0916 - accuracy: 0.9739




8. model history 

print(model_fit.history.keys()) #dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])


loss vs val_loss : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['loss'], 'y', label='train loss')
plt.plot(model_fit.history['val_loss'], 'r', label='val loss')
plt.xlabel('epochs')
plt.ylabel('loss')
plt.legend(loc = 'best')
plt.show()


accuracy vs val_accuracy : overfitting ์‹œ์ž‘์  : epoch=2

plt.plot(model_fit.history['accuracy'], 'y', label='train accuracy')
plt.plot(model_fit.history['val_accuracy'], 'r', label='val accuracy')
plt.xlabel('epochs')
plt.ylabel('accuracy')
plt.legend(loc = 'best')
plt.show()

+ Recent posts