DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD】对Mnist数据集训练来理解过拟合现象

DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD】对Mnist数据集训练来理解过拟合现象

导读
自定义少量的Mnist数据集，利用全连接神经网络MultiLayerNet模型【6*100+ReLU+SGD】进行训练，观察过拟合现象。

目录

输出结果

设计思路

核心代码

更多输出

输出结果

设计思路

核心代码

for i in range(1000000):
    batch_mask = np.random.choice(train_size, batch_size)
    x_batch = x_train[batch_mask]
    t_batch = t_train[batch_mask]
 
    grads = network.gradient(x_batch, t_batch)  
    optimizer.update(network.params, grads)   
 
    if i % iter_per_epoch == 0:   
        train_acc = network.accuracy(x_train, t_train)  
        test_acc = network.accuracy(x_test, t_test)     
        train_acc_list.append(train_acc)
        test_acc_list.append(test_acc)
 
        print("epoch:" + str(epoch_cnt) + ", train_acc:" + str(float('%.4f' % train_acc)) + ", test_acc:" + str(float('%.4f' % test_acc))) 
        epoch_cnt += 1
        if epoch_cnt >= max_epochs:  
            break

更多输出

epoch:0, train_acc:0.0733, test_acc:0.0792
epoch:1, train_acc:0.0767, test_acc:0.0878
epoch:2, train_acc:0.0967, test_acc:0.0966
epoch:3, train_acc:0.1, test_acc:0.1016
epoch:4, train_acc:0.1133, test_acc:0.1065
epoch:5, train_acc:0.1167, test_acc:0.1166
epoch:6, train_acc:0.13, test_acc:0.1249
epoch:7, train_acc:0.1567, test_acc:0.1348
epoch:8, train_acc:0.1867, test_acc:0.1441
epoch:9, train_acc:0.2067, test_acc:0.1602
epoch:10, train_acc:0.2333, test_acc:0.1759
epoch:11, train_acc:0.24, test_acc:0.1812
epoch:12, train_acc:0.2567, test_acc:0.1963
epoch:13, train_acc:0.2867, test_acc:0.2161
epoch:14, train_acc:0.31, test_acc:0.2292
epoch:15, train_acc:0.35, test_acc:0.2452
epoch:16, train_acc:0.3567, test_acc:0.2609
epoch:17, train_acc:0.3867, test_acc:0.2678
epoch:18, train_acc:0.4, test_acc:0.2796
epoch:19, train_acc:0.41, test_acc:0.291
epoch:20, train_acc:0.42, test_acc:0.2978
epoch:21, train_acc:0.4267, test_acc:0.3039
epoch:22, train_acc:0.4433, test_acc:0.3122
epoch:23, train_acc:0.4533, test_acc:0.3199
epoch:24, train_acc:0.4633, test_acc:0.3252
epoch:25, train_acc:0.47, test_acc:0.3326
epoch:26, train_acc:0.4733, test_acc:0.3406
epoch:27, train_acc:0.4733, test_acc:0.3506
epoch:28, train_acc:0.4733, test_acc:0.3537
epoch:29, train_acc:0.4867, test_acc:0.3582
epoch:30, train_acc:0.4933, test_acc:0.3583
epoch:31, train_acc:0.4967, test_acc:0.3655
epoch:32, train_acc:0.4933, test_acc:0.3707
epoch:33, train_acc:0.4967, test_acc:0.3722
epoch:34, train_acc:0.5033, test_acc:0.3806
epoch:35, train_acc:0.5133, test_acc:0.3776
epoch:36, train_acc:0.51, test_acc:0.3804
epoch:37, train_acc:0.5167, test_acc:0.3837
epoch:38, train_acc:0.52, test_acc:0.3838
epoch:39, train_acc:0.5167, test_acc:0.3844
epoch:40, train_acc:0.5167, test_acc:0.3933
epoch:41, train_acc:0.5233, test_acc:0.397
epoch:42, train_acc:0.5267, test_acc:0.3967
epoch:43, train_acc:0.5333, test_acc:0.4021
epoch:44, train_acc:0.5267, test_acc:0.3961
epoch:45, train_acc:0.5367, test_acc:0.3997
epoch:46, train_acc:0.54, test_acc:0.4126
epoch:47, train_acc:0.5533, test_acc:0.421
epoch:48, train_acc:0.5533, test_acc:0.4274
epoch:49, train_acc:0.5533, test_acc:0.4246
epoch:50, train_acc:0.5633, test_acc:0.4322
epoch:51, train_acc:0.5667, test_acc:0.4372
epoch:52, train_acc:0.5867, test_acc:0.4544
epoch:53, train_acc:0.6133, test_acc:0.4631
epoch:54, train_acc:0.6167, test_acc:0.475
epoch:55, train_acc:0.6167, test_acc:0.4756
epoch:56, train_acc:0.6267, test_acc:0.4801
epoch:57, train_acc:0.6333, test_acc:0.4822
epoch:58, train_acc:0.62, test_acc:0.4809
epoch:59, train_acc:0.63, test_acc:0.491
epoch:60, train_acc:0.6233, test_acc:0.4939
epoch:61, train_acc:0.6367, test_acc:0.501
epoch:62, train_acc:0.65, test_acc:0.5156
epoch:63, train_acc:0.65, test_acc:0.5192
epoch:64, train_acc:0.65, test_acc:0.518
epoch:65, train_acc:0.6367, test_acc:0.5204
epoch:66, train_acc:0.6667, test_acc:0.527
epoch:67, train_acc:0.6567, test_acc:0.533
epoch:68, train_acc:0.6633, test_acc:0.5384
epoch:69, train_acc:0.6733, test_acc:0.5374
epoch:70, train_acc:0.67, test_acc:0.5365
epoch:71, train_acc:0.69, test_acc:0.5454
epoch:72, train_acc:0.68, test_acc:0.5479
epoch:73, train_acc:0.6833, test_acc:0.553
epoch:74, train_acc:0.6967, test_acc:0.5568
epoch:75, train_acc:0.68, test_acc:0.55
epoch:76, train_acc:0.7, test_acc:0.5567
epoch:77, train_acc:0.71, test_acc:0.5617
epoch:78, train_acc:0.7167, test_acc:0.5705
epoch:79, train_acc:0.73, test_acc:0.5722
epoch:80, train_acc:0.74, test_acc:0.5831
epoch:81, train_acc:0.73, test_acc:0.5778
epoch:82, train_acc:0.7567, test_acc:0.5845
epoch:83, train_acc:0.7533, test_acc:0.587
epoch:84, train_acc:0.75, test_acc:0.5809
epoch:85, train_acc:0.7433, test_acc:0.5869
epoch:86, train_acc:0.7533, test_acc:0.5996
epoch:87, train_acc:0.75, test_acc:0.5963
epoch:88, train_acc:0.7667, test_acc:0.6079
epoch:89, train_acc:0.7733, test_acc:0.6247
epoch:90, train_acc:0.7633, test_acc:0.6152
epoch:91, train_acc:0.79, test_acc:0.6307
epoch:92, train_acc:0.7967, test_acc:0.637
epoch:93, train_acc:0.8033, test_acc:0.6351
epoch:94, train_acc:0.8, test_acc:0.6464
epoch:95, train_acc:0.7967, test_acc:0.6308
epoch:96, train_acc:0.8067, test_acc:0.6406
epoch:97, train_acc:0.8033, test_acc:0.6432
epoch:98, train_acc:0.81, test_acc:0.657
epoch:99, train_acc:0.81, test_acc:0.6523
epoch:100, train_acc:0.8167, test_acc:0.6487
epoch:101, train_acc:0.8033, test_acc:0.6532
epoch:102, train_acc:0.8133, test_acc:0.672
epoch:103, train_acc:0.8233, test_acc:0.6738
epoch:104, train_acc:0.82, test_acc:0.6588
epoch:105, train_acc:0.8167, test_acc:0.659
epoch:106, train_acc:0.82, test_acc:0.6643
epoch:107, train_acc:0.8233, test_acc:0.6696
epoch:108, train_acc:0.8167, test_acc:0.6665
epoch:109, train_acc:0.8133, test_acc:0.6523
epoch:110, train_acc:0.83, test_acc:0.6744
epoch:111, train_acc:0.8267, test_acc:0.6746
epoch:112, train_acc:0.83, test_acc:0.6757
epoch:113, train_acc:0.8267, test_acc:0.6749
epoch:114, train_acc:0.8167, test_acc:0.668
epoch:115, train_acc:0.8267, test_acc:0.6726
epoch:116, train_acc:0.83, test_acc:0.6794
epoch:117, train_acc:0.8167, test_acc:0.6632
epoch:118, train_acc:0.8233, test_acc:0.6599
epoch:119, train_acc:0.8267, test_acc:0.6692
epoch:120, train_acc:0.83, test_acc:0.6695
epoch:121, train_acc:0.8367, test_acc:0.6781
epoch:122, train_acc:0.8333, test_acc:0.6689
epoch:123, train_acc:0.8367, test_acc:0.6789
epoch:124, train_acc:0.8333, test_acc:0.6821
epoch:125, train_acc:0.8367, test_acc:0.6821
epoch:126, train_acc:0.8267, test_acc:0.6742
epoch:127, train_acc:0.8433, test_acc:0.6823
epoch:128, train_acc:0.8367, test_acc:0.6828
epoch:129, train_acc:0.8367, test_acc:0.6864
epoch:130, train_acc:0.84, test_acc:0.674
epoch:131, train_acc:0.84, test_acc:0.676
epoch:132, train_acc:0.83, test_acc:0.6715
epoch:133, train_acc:0.84, test_acc:0.6938
epoch:134, train_acc:0.8333, test_acc:0.7013
epoch:135, train_acc:0.84, test_acc:0.6979
epoch:136, train_acc:0.84, test_acc:0.6822
epoch:137, train_acc:0.84, test_acc:0.6929
epoch:138, train_acc:0.8433, test_acc:0.6921
epoch:139, train_acc:0.8433, test_acc:0.6963
epoch:140, train_acc:0.83, test_acc:0.6976
epoch:141, train_acc:0.84, test_acc:0.6897
epoch:142, train_acc:0.8433, test_acc:0.6994
epoch:143, train_acc:0.8467, test_acc:0.7042
epoch:144, train_acc:0.8567, test_acc:0.6963
epoch:145, train_acc:0.86, test_acc:0.6966
epoch:146, train_acc:0.8533, test_acc:0.6813
epoch:147, train_acc:0.85, test_acc:0.6891
epoch:148, train_acc:0.8667, test_acc:0.6908
epoch:149, train_acc:0.8467, test_acc:0.6719
epoch:150, train_acc:0.85, test_acc:0.6783
epoch:151, train_acc:0.86, test_acc:0.6969
epoch:152, train_acc:0.86, test_acc:0.7071
epoch:153, train_acc:0.8567, test_acc:0.6974
epoch:154, train_acc:0.86, test_acc:0.7009
epoch:155, train_acc:0.86, test_acc:0.6931
epoch:156, train_acc:0.8567, test_acc:0.6946
epoch:157, train_acc:0.86, test_acc:0.7004
epoch:158, train_acc:0.86, test_acc:0.7023
epoch:159, train_acc:0.85, test_acc:0.7054
epoch:160, train_acc:0.8633, test_acc:0.6933
epoch:161, train_acc:0.8667, test_acc:0.6872
epoch:162, train_acc:0.86, test_acc:0.6844
epoch:163, train_acc:0.8567, test_acc:0.6909
epoch:164, train_acc:0.8633, test_acc:0.6884
epoch:165, train_acc:0.87, test_acc:0.7005
epoch:166, train_acc:0.8667, test_acc:0.6926
epoch:167, train_acc:0.8633, test_acc:0.7131
epoch:168, train_acc:0.86, test_acc:0.7068
epoch:169, train_acc:0.87, test_acc:0.7045
epoch:170, train_acc:0.8633, test_acc:0.7027
epoch:171, train_acc:0.87, test_acc:0.6917
epoch:172, train_acc:0.87, test_acc:0.7046
epoch:173, train_acc:0.87, test_acc:0.71
epoch:174, train_acc:0.8767, test_acc:0.714
epoch:175, train_acc:0.87, test_acc:0.6925
epoch:176, train_acc:0.8633, test_acc:0.7112
epoch:177, train_acc:0.8733, test_acc:0.7149
epoch:178, train_acc:0.8567, test_acc:0.7056
epoch:179, train_acc:0.8633, test_acc:0.7149
epoch:180, train_acc:0.8567, test_acc:0.6962
epoch:181, train_acc:0.87, test_acc:0.7011
epoch:182, train_acc: