TensorFlow
TensorFlow 기초 15 - 활성화 함수, 학습 조기 종료
코딩탕탕
2022. 12. 2. 11:19
# 활성화 함수, 학습 조기 종료
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import boston_housing
# print(boston_housing.load_data())
(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
print(x_train.shape, y_train.shape, x_test.shape, y_test.shape) # (404, 13) (404,) (102, 13) (102,)
print(x_train[0])
print(y_train[0])
# 데이터 표준화
x_mean = x_train.mean(axis=0)
x_std = x_train.std(axis=0)
x_train -= x_mean
x_train /= x_std
x_test -= x_mean
x_test /= x_std
y_mean = y_train.mean(axis=0)
y_std = y_train.std(axis=0)
y_train -= y_mean
y_train /= y_std
y_test -= y_mean
y_test /= y_std
print(x_train[0])
print(y_train[0])
# model
model = tf.keras.Sequential([
tf.keras.layers.Dense(units=52, activation='relu', input_shape=(13, )),
tf.keras.layers.Dense(units=35, activation='relu'),
tf.keras.layers.Dense(units=24, activation='relu'),
tf.keras.layers.Dense(units=1)
])
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.07), loss='mse', metrics=['mse'])
model.summary()
import math
# 활성화 함수 비교
def sigmoid(x):
return 1 / (1 + math.exp(-x))
x = np.arange(-5, 5, 0.01)
sigmoid_x = [sigmoid(z) for z in x]
tanh_x = [math.tanh(z) for z in x]
relu = [0 if z<0 else z for z in x]
plt.axhline(0, color='gray')
plt.axvline(0, color='gray')
plt.plot(x, sigmoid_x, 'b--', label='sigmoid')
plt.plot(x, tanh_x, 'r--', label='tanh')
plt.plot(x, relu, 'g--', label='relu')
plt.show()
history = model.fit(x_train, y_train, epochs=25, batch_size=32, validation_split=0.025)
plt.plot(history.history['loss'], 'b-',label='loss')
plt.plot(history.history['val_loss'], 'r-', label='loss')
plt.xlabel('epochs')
plt.legend()
plt.show()
print(model.evaluate(x_test, y_test))
# 주택가격(실제 , 예측) 시각화
pred_y = model.predict(x_test)
plt.figure(figsize=(5,5))
plt.plot(y_test, pred_y, 'b.')
plt.xlabel('y_test')
plt.xlabel('pred_y')
plt.show()
print('-----------------------------')
# 학습 조기 종료
# model
model2 = tf.keras.Sequential([
tf.keras.layers.Dense(units=52, activation='relu', input_shape=(13, )),
tf.keras.layers.Dense(units=35, activation='relu'),
tf.keras.layers.Dense(units=24, activation='relu'),
tf.keras.layers.Dense(units=1)
])
model2.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.07), loss='mse', metrics=['mse'])
history = model2.fit(x_train, y_train, epochs=100, batch_size=1, validation_split=0.025,
callbacks=[tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3)])
plt.plot(history.history['loss'], 'b-',label='loss')
plt.plot(history.history['val_loss'], 'r-', label='loss') # # val_loss: 0.6014
plt.xlabel('epochs')
plt.legend()
plt.show()
print(model2.evaluate(x_test, y_test))
<console>
(404, 13) (404,) (102, 13) (102,)
[ 1.23247 0. 8.14 0. 0.538 6.142 91.7
3.9769 4. 307. 21. 396.9 18.72 ]
15.2
[-0.27224633 -0.48361547 -0.43576161 -0.25683275 -0.1652266 -0.1764426
0.81306188 0.1166983 -0.62624905 -0.59517003 1.14850044 0.44807713
0.8252202 ]
-0.7821526033779157
2022-12-02 11:14:50.386347: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 52) 728
dense_1 (Dense) (None, 35) 1855
dense_2 (Dense) (None, 24) 864
dense_3 (Dense) (None, 1) 25
=================================================================
Total params: 3,472
Trainable params: 3,472
Non-trainable params: 0
_________________________________________________________________
Epoch 1/25
1/13 [=>............................] - ETA: 5s - loss: 0.8341 - mse: 0.8341
13/13 [==============================] - 1s 10ms/step - loss: 1.2738 - mse: 1.2738 - val_loss: 0.1571 - val_mse: 0.1571
Epoch 2/25
1/13 [=>............................] - ETA: 0s - loss: 0.7676 - mse: 0.7676
13/13 [==============================] - 0s 2ms/step - loss: 0.5072 - mse: 0.5072 - val_loss: 0.2402 - val_mse: 0.2402
Epoch 3/25
1/13 [=>............................] - ETA: 0s - loss: 0.2254 - mse: 0.2254
13/13 [==============================] - 0s 2ms/step - loss: 0.2911 - mse: 0.2911 - val_loss: 0.1136 - val_mse: 0.1136
Epoch 4/25
1/13 [=>............................] - ETA: 0s - loss: 0.1009 - mse: 0.1009
13/13 [==============================] - 0s 2ms/step - loss: 0.2225 - mse: 0.2225 - val_loss: 0.2866 - val_mse: 0.2866
Epoch 5/25
1/13 [=>............................] - ETA: 0s - loss: 0.2632 - mse: 0.2632
13/13 [==============================] - 0s 2ms/step - loss: 0.2044 - mse: 0.2044 - val_loss: 0.0801 - val_mse: 0.0801
Epoch 6/25
1/13 [=>............................] - ETA: 0s - loss: 0.1355 - mse: 0.1355
13/13 [==============================] - 0s 2ms/step - loss: 0.1432 - mse: 0.1432 - val_loss: 0.1430 - val_mse: 0.1430
Epoch 7/25
1/13 [=>............................] - ETA: 0s - loss: 0.0687 - mse: 0.0687
13/13 [==============================] - 0s 2ms/step - loss: 0.1580 - mse: 0.1580 - val_loss: 0.1139 - val_mse: 0.1139
Epoch 8/25
1/13 [=>............................] - ETA: 0s - loss: 0.0586 - mse: 0.0586
13/13 [==============================] - 0s 2ms/step - loss: 0.1256 - mse: 0.1256 - val_loss: 0.0988 - val_mse: 0.0988
Epoch 9/25
1/13 [=>............................] - ETA: 0s - loss: 0.1528 - mse: 0.1528
13/13 [==============================] - 0s 2ms/step - loss: 0.1257 - mse: 0.1257 - val_loss: 0.0562 - val_mse: 0.0562
Epoch 10/25
1/13 [=>............................] - ETA: 0s - loss: 0.0654 - mse: 0.0654
13/13 [==============================] - 0s 2ms/step - loss: 0.1171 - mse: 0.1171 - val_loss: 0.1176 - val_mse: 0.1176
Epoch 11/25
1/13 [=>............................] - ETA: 0s - loss: 0.0736 - mse: 0.0736
13/13 [==============================] - 0s 2ms/step - loss: 0.1272 - mse: 0.1272 - val_loss: 0.0375 - val_mse: 0.0375
Epoch 12/25
1/13 [=>............................] - ETA: 0s - loss: 0.0920 - mse: 0.0920
13/13 [==============================] - 0s 1ms/step - loss: 0.1483 - mse: 0.1483 - val_loss: 0.0853 - val_mse: 0.0853
Epoch 13/25
1/13 [=>............................] - ETA: 0s - loss: 0.1556 - mse: 0.1556
13/13 [==============================] - 0s 2ms/step - loss: 0.2038 - mse: 0.2038 - val_loss: 0.0445 - val_mse: 0.0445
Epoch 14/25
1/13 [=>............................] - ETA: 0s - loss: 0.1102 - mse: 0.1102
13/13 [==============================] - 0s 1ms/step - loss: 0.2279 - mse: 0.2279 - val_loss: 0.2642 - val_mse: 0.2642
Epoch 15/25
1/13 [=>............................] - ETA: 0s - loss: 0.1071 - mse: 0.1071
13/13 [==============================] - 0s 1ms/step - loss: 0.1664 - mse: 0.1664 - val_loss: 0.0774 - val_mse: 0.0774
Epoch 16/25
1/13 [=>............................] - ETA: 0s - loss: 0.0988 - mse: 0.0988
13/13 [==============================] - 0s 2ms/step - loss: 0.1426 - mse: 0.1426 - val_loss: 0.1932 - val_mse: 0.1932
Epoch 17/25
1/13 [=>............................] - ETA: 0s - loss: 0.1421 - mse: 0.1421
13/13 [==============================] - 0s 2ms/step - loss: 0.1368 - mse: 0.1368 - val_loss: 0.0712 - val_mse: 0.0712
Epoch 18/25
1/13 [=>............................] - ETA: 0s - loss: 0.0987 - mse: 0.0987
13/13 [==============================] - 0s 2ms/step - loss: 0.1091 - mse: 0.1091 - val_loss: 0.1485 - val_mse: 0.1485
Epoch 19/25
1/13 [=>............................] - ETA: 0s - loss: 0.0870 - mse: 0.0870
13/13 [==============================] - 0s 2ms/step - loss: 0.0922 - mse: 0.0922 - val_loss: 0.1834 - val_mse: 0.1834
Epoch 20/25
1/13 [=>............................] - ETA: 0s - loss: 0.1262 - mse: 0.1262
13/13 [==============================] - 0s 1ms/step - loss: 0.1139 - mse: 0.1139 - val_loss: 0.0713 - val_mse: 0.0713
Epoch 21/25
1/13 [=>............................] - ETA: 0s - loss: 0.1287 - mse: 0.1287
13/13 [==============================] - 0s 2ms/step - loss: 0.0926 - mse: 0.0926 - val_loss: 0.0499 - val_mse: 0.0499
Epoch 22/25
1/13 [=>............................] - ETA: 0s - loss: 0.0754 - mse: 0.0754
13/13 [==============================] - 0s 1ms/step - loss: 0.0870 - mse: 0.0870 - val_loss: 0.0787 - val_mse: 0.0787
Epoch 23/25
1/13 [=>............................] - ETA: 0s - loss: 0.0708 - mse: 0.0708
13/13 [==============================] - 0s 2ms/step - loss: 0.0751 - mse: 0.0751 - val_loss: 0.0928 - val_mse: 0.0928
Epoch 24/25
1/13 [=>............................] - ETA: 0s - loss: 0.0429 - mse: 0.0429
13/13 [==============================] - 0s 2ms/step - loss: 0.0817 - mse: 0.0817 - val_loss: 0.0422 - val_mse: 0.0422
Epoch 25/25
1/13 [=>............................] - ETA: 0s - loss: 0.0488 - mse: 0.0488
13/13 [==============================] - 0s 2ms/step - loss: 0.0739 - mse: 0.0739 - val_loss: 0.0539 - val_mse: 0.0539
1/4 [======>.......................] - ETA: 0s - loss: 0.1125 - mse: 0.1125
4/4 [==============================] - 0s 664us/step - loss: 0.2225 - mse: 0.2225
[0.2225160449743271, 0.2225160449743271]
1/4 [======>.......................] - ETA: 0s
4/4 [==============================] - 0s 665us/step
-----------------------------
Epoch 1/100
1/393 [..............................] - ETA: 2:33 - loss: 0.1313 - mse: 0.1313
109/393 [=======>......................] - ETA: 0s - loss: 1.2960 - mse: 1.2960
217/393 [===============>..............] - ETA: 0s - loss: 1.2162 - mse: 1.2162
327/393 [=======================>......] - ETA: 0s - loss: 1.1653 - mse: 1.1653
393/393 [==============================] - 1s 687us/step - loss: 1.1334 - mse: 1.1334 - val_loss: 0.4813 - val_mse: 0.4813
Epoch 2/100
1/393 [..............................] - ETA: 0s - loss: 0.0245 - mse: 0.0245
123/393 [========>.....................] - ETA: 0s - loss: 1.0369 - mse: 1.0369
248/393 [=================>............] - ETA: 0s - loss: 1.0108 - mse: 1.0108
377/393 [===========================>..] - ETA: 0s - loss: 1.0774 - mse: 1.0774
393/393 [==============================] - 0s 442us/step - loss: 1.0535 - mse: 1.0535 - val_loss: 0.4742 - val_mse: 0.4742
Epoch 3/100
1/393 [..............................] - ETA: 0s - loss: 0.0682 - mse: 0.0682
126/393 [========>.....................] - ETA: 0s - loss: 1.0124 - mse: 1.0124
246/393 [=================>............] - ETA: 0s - loss: 0.9941 - mse: 0.9941
363/393 [==========================>...] - ETA: 0s - loss: 1.0575 - mse: 1.0575
393/393 [==============================] - 0s 458us/step - loss: 1.0592 - mse: 1.0592 - val_loss: 0.4379 - val_mse: 0.4379
Epoch 4/100
1/393 [..............................] - ETA: 0s - loss: 0.0988 - mse: 0.0988
114/393 [=======>......................] - ETA: 0s - loss: 1.2445 - mse: 1.2445
241/393 [=================>............] - ETA: 0s - loss: 1.1173 - mse: 1.1173
368/393 [===========================>..] - ETA: 0s - loss: 1.0398 - mse: 1.0398
393/393 [==============================] - 0s 445us/step - loss: 1.0462 - mse: 1.0462 - val_loss: 0.8275 - val_mse: 0.8275
Epoch 5/100
1/393 [..............................] - ETA: 0s - loss: 0.6620 - mse: 0.6620
127/393 [========>.....................] - ETA: 0s - loss: 1.0636 - mse: 1.0636
259/393 [==================>...........] - ETA: 0s - loss: 1.0258 - mse: 1.0258
391/393 [============================>.] - ETA: 0s - loss: 1.0440 - mse: 1.0440
393/393 [==============================] - 0s 426us/step - loss: 1.0418 - mse: 1.0418 - val_loss: 0.6161 - val_mse: 0.6161
Epoch 6/100
1/393 [..............................] - ETA: 0s - loss: 1.5824 - mse: 1.5824
130/393 [========>.....................] - ETA: 0s - loss: 0.9247 - mse: 0.9247
250/393 [==================>...........] - ETA: 0s - loss: 1.0311 - mse: 1.0311
368/393 [===========================>..] - ETA: 0s - loss: 1.0397 - mse: 1.0397
393/393 [==============================] - 0s 455us/step - loss: 1.0301 - mse: 1.0301 - val_loss: 0.4592 - val_mse: 0.4592
1/4 [======>.......................] - ETA: 0s - loss: 0.6706 - mse: 0.6706
4/4 [==============================] - 0s 665us/step - loss: 1.0588 - mse: 1.0588
[1.058756709098816, 1.058756709098816]
blue = sigmoid
red = tanh
gray = relu
