DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD,weight_decay】对Mnist数据集训练来抑制过拟合

DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD,weight_decay】对Mnist数据集训练来抑制过拟合

目录

输出结果

设计思路

核心代码

更多输出

输出结果

设计思路

核心代码

 weight_decay_lambda = 0
weight_decay_lambda = 0.1
 
 
for i in range(1000000):
    batch_mask = np.random.choice(train_size, batch_size)
    x_batch = x_train[batch_mask]
    t_batch = t_train[batch_mask]
 
    grads = network.gradient(x_batch, t_batch)  
    optimizer.update(network.params, grads)   
 
    if i % iter_per_epoch == 0:   
        train_acc = network.accuracy(x_train, t_train)  
        test_acc = network.accuracy(x_test, t_test)     
        train_acc_list.append(train_acc)
        test_acc_list.append(test_acc)
 
        print("epoch:" + str(epoch_cnt) + ", train_acc:" + str(float('%.4f' % train_acc)) + ", test_acc:" + str(float('%.4f' % test_acc))) 
        epoch_cnt += 1
        if epoch_cnt >= max_epochs:  
            break

更多输出

1、MultiLayerNet[6*100+ReLU+SGD]: DIY Overfitting Data Set based on mnist,train_acc VS test_acc

epoch:0, train_acc:0.06, test_acc:0.0834
epoch:1, train_acc:0.1233, test_acc:0.1109
epoch:2, train_acc:0.1467, test_acc:0.1292
epoch:3, train_acc:0.2233, test_acc:0.1717
epoch:4, train_acc:0.2567, test_acc:0.1891
epoch:5, train_acc:0.27, test_acc:0.2181
epoch:6, train_acc:0.31, test_acc:0.229
epoch:7, train_acc:0.32, test_acc:0.24
epoch:8, train_acc:0.3567, test_acc:0.2502
epoch:9, train_acc:0.37, test_acc:0.2651
epoch:10, train_acc:0.3767, test_acc:0.2743
epoch:11, train_acc:0.39, test_acc:0.2833
epoch:12, train_acc:0.3767, test_acc:0.2769
epoch:13, train_acc:0.4067, test_acc:0.295
epoch:14, train_acc:0.4667, test_acc:0.3169
epoch:15, train_acc:0.45, test_acc:0.3213
epoch:16, train_acc:0.5067, test_acc:0.3439
epoch:17, train_acc:0.54, test_acc:0.3593
epoch:18, train_acc:0.5233, test_acc:0.3687
epoch:19, train_acc:0.5367, test_acc:0.3691
epoch:20, train_acc:0.5667, test_acc:0.4051
epoch:21, train_acc:0.5967, test_acc:0.4265
epoch:22, train_acc:0.63, test_acc:0.4477
epoch:23, train_acc:0.6467, test_acc:0.4627
epoch:24, train_acc:0.6567, test_acc:0.4708
epoch:25, train_acc:0.6533, test_acc:0.4896
epoch:26, train_acc:0.66, test_acc:0.5034
epoch:27, train_acc:0.68, test_acc:0.5107
epoch:28, train_acc:0.6833, test_acc:0.5083
epoch:29, train_acc:0.7067, test_acc:0.5244
epoch:30, train_acc:0.7567, test_acc:0.5564
epoch:31, train_acc:0.7333, test_acc:0.5411
epoch:32, train_acc:0.7533, test_acc:0.5698
epoch:33, train_acc:0.7633, test_acc:0.5738
epoch:34, train_acc:0.7833, test_acc:0.5764
epoch:35, train_acc:0.7633, test_acc:0.5863
epoch:36, train_acc:0.7733, test_acc:0.5915
epoch:37, train_acc:0.8067, test_acc:0.608
epoch:38, train_acc:0.81, test_acc:0.6113
epoch:39, train_acc:0.8033, test_acc:0.5922
epoch:40, train_acc:0.8233, test_acc:0.6192
epoch:41, train_acc:0.83, test_acc:0.6203
epoch:42, train_acc:0.8033, test_acc:0.6066
epoch:43, train_acc:0.8333, test_acc:0.6311
epoch:44, train_acc:0.8433, test_acc:0.6273
epoch:45, train_acc:0.85, test_acc:0.6413
epoch:46, train_acc:0.85, test_acc:0.6375
epoch:47, train_acc:0.86, test_acc:0.6352
epoch:48, train_acc:0.8667, test_acc:0.6504
epoch:49, train_acc:0.8767, test_acc:0.6588
epoch:50, train_acc:0.8667, test_acc:0.6592
epoch:51, train_acc:0.89, test_acc:0.6648
epoch:52, train_acc:0.88, test_acc:0.6605
epoch:53, train_acc:0.88, test_acc:0.6654
epoch:54, train_acc:0.8967, test_acc:0.6674
epoch:55, train_acc:0.8967, test_acc:0.6701
epoch:56, train_acc:0.9, test_acc:0.6636
epoch:57, train_acc:0.9, test_acc:0.6755
epoch:58, train_acc:0.9167, test_acc:0.6763
epoch:59, train_acc:0.9133, test_acc:0.6748
epoch:60, train_acc:0.92, test_acc:0.6788
epoch:61, train_acc:0.9033, test_acc:0.6759
epoch:62, train_acc:0.9133, test_acc:0.6747
epoch:63, train_acc:0.9233, test_acc:0.6915
epoch:64, train_acc:0.9267, test_acc:0.687
epoch:65, train_acc:0.92, test_acc:0.6822
epoch:66, train_acc:0.9133, test_acc:0.6827
epoch:67, train_acc:0.92, test_acc:0.6932
epoch:68, train_acc:0.9333, test_acc:0.6976
epoch:69, train_acc:0.94, test_acc:0.6953
epoch:70, train_acc:0.94, test_acc:0.7031
epoch:71, train_acc:0.9367, test_acc:0.6951
epoch:72, train_acc:0.9433, test_acc:0.7036
epoch:73, train_acc:0.9367, test_acc:0.7051
epoch:74, train_acc:0.9433, test_acc:0.706
epoch:75, train_acc:0.95, test_acc:0.707
epoch:76, train_acc:0.9567, test_acc:0.7052
epoch:77, train_acc:0.9433, test_acc:0.6991
epoch:78, train_acc:0.9567, test_acc:0.7121
epoch:79, train_acc:0.9633, test_acc:0.7055
epoch:80, train_acc:0.96, test_acc:0.7088
epoch:81, train_acc:0.9567, test_acc:0.7105
epoch:82, train_acc:0.9633, test_acc:0.7091
epoch:83, train_acc:0.9567, test_acc:0.7159
epoch:84, train_acc:0.9567, test_acc:0.7072
epoch:85, train_acc:0.9633, test_acc:0.7138
epoch:86, train_acc:0.9767, test_acc:0.7127
epoch:87, train_acc:0.9733, test_acc:0.7167
epoch:88, train_acc:0.9733, test_acc:0.7241
epoch:89, train_acc:0.98, test_acc:0.721
epoch:90, train_acc:0.9767, test_acc:0.7202
epoch:91, train_acc:0.9767, test_acc:0.7232
epoch:92, train_acc:0.9833, test_acc:0.717
epoch:93, train_acc:0.9867, test_acc:0.7215
epoch:94, train_acc:0.9867, test_acc:0.7299
epoch:95, train_acc:0.9833, test_acc:0.728
epoch:96, train_acc:0.99, test_acc:0.7223
epoch:97, train_acc:0.9867, test_acc:0.7205
epoch:98, train_acc:0.99, test_acc:0.7287
epoch:99, train_acc:0.9967, test_acc:0.7298
epoch:100, train_acc:0.99, test_acc:0.7288
epoch:101, train_acc:1.0, test_acc:0.7258
epoch:102, train_acc:0.9967, test_acc:0.7274
epoch:103, train_acc:0.9967, test_acc:0.7238
epoch:104, train_acc:1.0, test_acc:0.7275
epoch:105, train_acc:0.9967, test_acc:0.7275
epoch:106, train_acc:1.0, test_acc:0.7209
epoch:107, train_acc:1.0, test_acc:0.7306
epoch:108, train_acc:0.9933, test_acc:0.7267
epoch:109, train_acc:0.9967, test_acc:0.7278
epoch:110, train_acc:1.0, test_acc:0.7306
epoch:111, train_acc:1.0, test_acc:0.7279
epoch:112, train_acc:0.9967, test_acc:0.7326
epoch:113, train_acc:0.9967, test_acc:0.7274
epoch:114, train_acc:0.9967, test_acc:0.7279
epoch:115, train_acc:1.0, test_acc:0.7301
epoch:116, train_acc:1.0, test_acc:0.7296
epoch:117, train_acc:1.0, test_acc:0.7327
epoch:118, train_acc:1.0, test_acc:0.7248
epoch:119, train_acc:1.0, test_acc:0.733
epoch:120, train_acc:1.0, test_acc:0.7286
epoch:121, train_acc:1.0, test_acc:0.7302
epoch:122, train_acc:1.0, test_acc:0.7346
epoch:123, train_acc:1.0, test_acc:0.7309
epoch:124, train_acc:1.0, test_acc:0.7309
epoch:125, train_acc:1.0, test_acc:0.7327
epoch:126, train_acc:1.0, test_acc:0.7353
epoch:127, train_acc:1.0, test_acc:0.7316
epoch:128, train_acc:1.0, test_acc:0.7296
epoch:129, train_acc:1.0, test_acc:0.731
epoch:130, train_acc:1.0, test_acc:0.733
epoch:131, train_acc:1.0, test_acc:0.7331
epoch:132, train_acc:1.0, test_acc:0.732
epoch:133, train_acc:1.0, test_acc:0.7333
epoch:134, train_acc:1.0, test_acc:0.7288
epoch:135, train_acc:1.0, test_acc:0.7347
epoch:136, train_acc:1.0, test_acc:0.7349
epoch:137, train_acc:1.0, test_acc:0.7356
epoch:138, train_acc:1.0, test_acc:0.7308
epoch:139, train_acc:1.0, test_acc:0.7359
epoch:140, train_acc:1.0, test_acc:0.7337
epoch:141, train_acc:1.0, test_acc:0.7355
epoch:142, train_acc:1.0, test_acc:0.7349
epoch:143, train_acc:1.0, test_acc:0.7327
epoch:144, train_acc:1.0, test_acc:0.7344
epoch:145, train_acc:1.0, test_acc:0.7367
epoch:146, train_acc:1.0, test_acc:0.7372
epoch:147, train_acc:1.0, test_acc:0.7353
epoch:148, train_acc:1.0, test_acc:0.7373
epoch:149, train_acc:1.0, test_acc:0.7362
epoch:150, train_acc:1.0, test_acc:0.7366
epoch:151, train_acc:1.0, test_acc:0.7376
epoch:152, train_acc:1.0, test_acc:0.7357
epoch:153, train_acc:1.0, test_acc:0.7341
epoch:154, train_acc:1.0, test_acc:0.7338
epoch:155, train_acc:1.0, test_acc:0.7351
epoch:156, train_acc:1.0, test_acc:0.7339
epoch:157, train_acc:1.0, test_acc:0.7383
epoch:158, train_acc:1.0, test_acc:0.7366
epoch:159, train_acc:1.0, test_acc:0.7376
epoch:160, train_acc:1.0, test_acc:0.7383
epoch:161, train_acc:1.0, test_acc:0.7404
epoch:162, train_acc:1.0, test_acc:0.7373
epoch:163, train_acc:1.0, test_acc:0.7357
epoch:164, train_acc:1.0, test_acc:0.7359
epoch:165, train_acc:1.0, test_acc:0.7392
epoch:166, train_acc:1.0, test_acc:0.7384
epoch:167, train_acc:1.0, test_acc:0.7381
epoch:168, train_acc:1.0, test_acc:0.734
epoch:169, train_acc:1.0, test_acc:0.7352
epoch:170, train_acc:1.0, test_acc:0.7356
epoch:171, train_acc:1.0, test_acc:0.7381
epoch:172, train_acc:1.0, test_acc:0.7384
epoch:173, train_acc:1.0, test_acc:0.7398
epoch:174, train_acc:1.0, test_acc:0.7395
epoch:175, train_acc:1.0, test_acc:0.7413
epoch:176, train_acc:1.0, test_acc:0.7387
epoch:177, train_acc:1.0, test_acc:0.7402
epoch:178, train_acc:1.0, test_acc:0.7378
epoch:179, train_acc:1.0, test_acc:0.7389
epoch:180, train_acc:1.0, test_acc:0.7396
epoch:181, train_acc:1.0, test_acc:0.7375
epoch:182, train_acc:1.0, test_acc:0.7403
epoch:183, train_acc:1.0, test_acc:0.7392