.ipynb .pdf

repository

Contents

レッスン2：ディープラーニングの基礎

このセクションでは、ディープラーニングの基本的な概念とPythonでの実装について学びます。具体的には、以下のトピックをカバーします。

• ニューラルネットワーク

• 畳み込みニューラルネットワーク（CNN）

• 再帰型ニューラルネットワーク（RNN）

用語の説明

• ニューロン: ニューロンは、人間の脳の神経細胞を模倣した概念で、ニューラルネットワークの中心的な要素です。それぞれのニューロンは、複数の入力を受け取り、それらを一つの出力に変換します。この出力は、次の層のニューロンへの入力となります。

• 入力層: 入力層は、ニューラルネットワークの最初の層で、データを受け取る部分です。例えば、画像を認識するネットワークでは、各ピクセルの色情報が入力層に供給されます。

• 出力層: 出力層は、ニューラルネットワークの最後の層で、最終的な結果を出力します。例えば、手書き数字の認識では、出力層は0から9までの10個のニューロンから成り、最も高い値を出力するニューロンが認識結果となります。

• 正規化: 正規化は、データを一定の範囲や形状に変換する処理です。例えば、色の強度を0から1の範囲に変換したり、平均が0で標準偏差が1になるようにデータをスケーリングしたりします。これにより、機械学習モデルの学習が容易になります。

• 活性化関数: 活性化関数は、ニューロンの出力を決定する関数です。例えば、シグモイド関数は出力を0から1の間に制限し、ReLU関数は負の入力を0に、正の入力はそのままにします。これにより、ニューラルネットワークは非線形の問題も学習できます。

• 損失関数: 損失関数は、モデルの予測がどれだけ正解から外れているかを測定する関数です。例えば、二乗誤差損失は、予測と正解の差を二乗したものを使います。損失が小さいほど、モデルの予測は正確です。

• 最適化アルゴリズム: 最適化アルゴリズムは、損失関数を最小化するようにモデルのパラメータを更新する方法です。例えば、勾配降下法は、損失関数の勾配（傾き）が最も急な方向にパラメータを少しずつ移動させて、損失を減らします。これにより、モデルはデータから学習して改善します。

基本的なニューラルネットワーク（多層パーセプトロン）

ここでは、Pythonの深層学習ライブラリであるKerasを用いて、ニューラルネットワークの基本的な実装を示します。具体的には、手書き数字の画像を分類するニューラルネットワークを訓練します。

まず、必要なライブラリをインポートします。これらのライブラリは、ニューラルネットワークの構築と訓練に必要な機能を提供します。

# 必要なライブラリのインポート
import tensorflow as tf
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import to_categorical

# 乱数の種を固定
tf.random.set_seed(0)

次に、MNISTという手書き数字のデータセットをロードします。このデータセットは、0から9までの手書き数字の画像（28x28ピクセル）と、それぞれの画像が何の数字であるかを示すラベルから構成されています。データセットは訓練データとテストデータの2つに分けられ、訓練データでモデルを訓練した後、テストデータでモデルの性能を評価します。

# データセットのロード
(X_train, y_train), (X_test, y_test) = mnist.load_data()

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz

   16384/11490434 [..............................] - ETA: 42s


   49152/11490434 [..............................] - ETA: 37s


   81920/11490434 [..............................] - ETA: 36s


  147456/11490434 [..............................] - ETA: 24s


  212992/11490434 [..............................] - ETA: 20s


  294912/11490434 [..............................] - ETA: 16s


  458752/11490434 [>.............................] - ETA: 11s


  622592/11490434 [>.............................] - ETA: 9s 


  811008/11490434 [=>............................] - ETA: 7s


 1376256/11490434 [==>...........................] - ETA: 4s


 2080768/11490434 [====>.........................] - ETA: 3s


 3088384/11490434 [=======>......................] - ETA: 2s


 3907584/11490434 [=========>....................] - ETA: 1s


 5095424/11490434 [============>.................] - ETA: 1s


 5963776/11490434 [==============>...............] - ETA: 0s


 6963200/11490434 [=================>............] - ETA: 0s


 7782400/11490434 [===================>..........] - ETA: 0s


 8830976/11490434 [======================>.......] - ETA: 0s


 9568256/11490434 [=======================>......] - ETA: 0s


10633216/11490434 [==========================>...] - ETA: 0s


11337728/11490434 [============================>.] - ETA: 0s


11493376/11490434 [==============================] - 1s 0us/step


11501568/11490434 [==============================] - 1s 0us/step

データの前処理を行います。画像データは、各ピクセルが0から255の値を持つ2次元配列として表現されます。これを1次元配列に変換し、値を0から1の範囲に正規化します。また、ラベルデータは、各画像がどの数字であるかを示す整数ですが、これをカテゴリ形式（多クラス分類）に変換します。

# データの前処理
X_train = X_train.reshape((X_train.shape[0], 28 * 28))
X_train = X_train.astype('float32') / 255

X_test = X_test.reshape((X_test.shape[0], 28 * 28))
X_test = X_test.astype('float32') / 255

y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

次に、ニューラルネットワークのモデルを作成します。ここでは、全結合層（Dense）のみからなるシンプルなニューラルネットワークを作成します。全結合層とは、前の層の全てのニューロンが次の層の全てのニューロンと接続されている層のことを指します。

# モデルの作成
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(512, activation='relu', input_shape=(28 * 28,)))
model.add(tf.keras.layers.Dense(10, activation='softmax'))

このモデルは、入力層、隠れ層、出力層の3つの層から構成されています。入力層のニューロンの数は、入力データの次元数（ここでは28*28=784）と同じです。隠れ層は512のニューロンを持ち、活性化関数としてReLU（Rectified Linear Unit）を使用します。出力層は10のニューロンを持ち、活性化関数としてソフトマックスを使用します。ソフトマックス関数は、各出力ニューロンの出力を0から1の間の確率に変換します。

また、ReLU関数は以下のような形状になっています。

参考：ReLU関数　https://www.wolframalpha.com/input?i=ReLU+function

次に、モデルをコンパイルします。これは、モデルの学習方法を設定するステップです。損失関数、最適化アルゴリズム、評価指標を指定します。

# モデルのコンパイル
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

最後に、モデルを訓練します。訓練データを用いてモデルのパラメータを更新し、最適なパラメータを見つけ出します。エポック数は訓練データを何回繰り返して学習するかを指定します。バッチサイズは一度に学習するデータの数を指定します。

# モデルの訓練
history = model.fit(X_train, y_train,
                    epochs=5,
                    batch_size=128,
                    validation_data=(X_test, y_test))

Epoch 1/5

  1/469 [..............................] - ETA: 3:13 - loss: 2.4631 - accuracy: 0.1016


  6/469 [..............................] - ETA: 4s - loss: 1.5708 - accuracy: 0.5013  


 15/469 [..............................] - ETA: 3s - loss: 1.0866 - accuracy: 0.6771


 29/469 [>.............................] - ETA: 2s - loss: 0.8118 - accuracy: 0.7651


 47/469 [==>...........................] - ETA: 1s - loss: 0.6677 - accuracy: 0.8067


 66/469 [===>..........................] - ETA: 1s - loss: 0.5811 - accuracy: 0.8325


 82/469 [====>.........................] - ETA: 1s - loss: 0.5370 - accuracy: 0.8446


 97/469 [=====>........................] - ETA: 1s - loss: 0.5030 - accuracy: 0.8545


113/469 [======>.......................] - ETA: 1s - loss: 0.4688 - accuracy: 0.8640


131/469 [=======>......................] - ETA: 1s - loss: 0.4384 - accuracy: 0.8718


148/469 [========>.....................] - ETA: 1s - loss: 0.4175 - accuracy: 0.8789


163/469 [=========>....................] - ETA: 1s - loss: 0.4009 - accuracy: 0.8837


179/469 [==========>...................] - ETA: 1s - loss: 0.3852 - accuracy: 0.8883


196/469 [===========>..................] - ETA: 0s - loss: 0.3714 - accuracy: 0.8923


214/469 [============>.................] - ETA: 0s - loss: 0.3575 - accuracy: 0.8962


230/469 [=============>................] - ETA: 0s - loss: 0.3495 - accuracy: 0.8984


248/469 [==============>...............] - ETA: 0s - loss: 0.3397 - accuracy: 0.9014


266/469 [================>.............] - ETA: 0s - loss: 0.3292 - accuracy: 0.9044


285/469 [=================>............] - ETA: 0s - loss: 0.3200 - accuracy: 0.9069


304/469 [==================>...........] - ETA: 0s - loss: 0.3112 - accuracy: 0.9095


323/469 [===================>..........] - ETA: 0s - loss: 0.3034 - accuracy: 0.9120


342/469 [====================>.........] - ETA: 0s - loss: 0.2961 - accuracy: 0.9145


360/469 [======================>.......] - ETA: 0s - loss: 0.2894 - accuracy: 0.9163


378/469 [=======================>......] - ETA: 0s - loss: 0.2826 - accuracy: 0.9184


394/469 [========================>.....] - ETA: 0s - loss: 0.2791 - accuracy: 0.9194


411/469 [=========================>....] - ETA: 0s - loss: 0.2744 - accuracy: 0.9206


428/469 [==========================>...] - ETA: 0s - loss: 0.2681 - accuracy: 0.9222


442/469 [===========================>..] - ETA: 0s - loss: 0.2642 - accuracy: 0.9233


458/469 [============================>.] - ETA: 0s - loss: 0.2604 - accuracy: 0.9245


469/469 [==============================] - 2s 4ms/step - loss: 0.2573 - accuracy: 0.9253 - val_loss: 0.1260 - val_accuracy: 0.9623

Epoch 2/5

  1/469 [..............................] - ETA: 1s - loss: 0.1235 - accuracy: 0.9609


 16/469 [>.............................] - ETA: 1s - loss: 0.0897 - accuracy: 0.9707


 32/469 [=>............................] - ETA: 1s - loss: 0.0931 - accuracy: 0.9707


 50/469 [==>...........................] - ETA: 1s - loss: 0.1018 - accuracy: 0.9683


 68/469 [===>..........................] - ETA: 1s - loss: 0.1069 - accuracy: 0.9661


 86/469 [====>.........................] - ETA: 1s - loss: 0.1139 - accuracy: 0.9648


104/469 [=====>........................] - ETA: 1s - loss: 0.1126 - accuracy: 0.9656


123/469 [======>.......................] - ETA: 1s - loss: 0.1129 - accuracy: 0.9665


142/469 [========>.....................] - ETA: 0s - loss: 0.1140 - accuracy: 0.9658


160/469 [=========>....................] - ETA: 0s - loss: 0.1119 - accuracy: 0.9666


179/469 [==========>...................] - ETA: 0s - loss: 0.1128 - accuracy: 0.9660


198/469 [===========>..................] - ETA: 0s - loss: 0.1129 - accuracy: 0.9659


216/469 [============>.................] - ETA: 0s - loss: 0.1105 - accuracy: 0.9666


233/469 [=============>................] - ETA: 0s - loss: 0.1114 - accuracy: 0.9663


251/469 [===============>..............] - ETA: 0s - loss: 0.1094 - accuracy: 0.9669


269/469 [================>.............] - ETA: 0s - loss: 0.1099 - accuracy: 0.9671


287/469 [=================>............] - ETA: 0s - loss: 0.1094 - accuracy: 0.9673


303/469 [==================>...........] - ETA: 0s - loss: 0.1087 - accuracy: 0.9674


320/469 [===================>..........] - ETA: 0s - loss: 0.1074 - accuracy: 0.9678


337/469 [====================>.........] - ETA: 0s - loss: 0.1068 - accuracy: 0.9682


355/469 [=====================>........] - ETA: 0s - loss: 0.1063 - accuracy: 0.9684


372/469 [======================>.......] - ETA: 0s - loss: 0.1058 - accuracy: 0.9686


391/469 [========================>.....] - ETA: 0s - loss: 0.1051 - accuracy: 0.9689


409/469 [=========================>....] - ETA: 0s - loss: 0.1042 - accuracy: 0.9692


427/469 [==========================>...] - ETA: 0s - loss: 0.1048 - accuracy: 0.9690


446/469 [===========================>..] - ETA: 0s - loss: 0.1050 - accuracy: 0.9689


464/469 [============================>.] - ETA: 0s - loss: 0.1045 - accuracy: 0.9691


469/469 [==============================] - 1s 3ms/step - loss: 0.1043 - accuracy: 0.9691 - val_loss: 0.1072 - val_accuracy: 0.9657

Epoch 3/5

  1/469 [..............................] - ETA: 1s - loss: 0.0571 - accuracy: 0.9844


 17/469 [>.............................] - ETA: 1s - loss: 0.0761 - accuracy: 0.9784


 33/469 [=>............................] - ETA: 1s - loss: 0.0760 - accuracy: 0.9777


 51/469 [==>...........................] - ETA: 1s - loss: 0.0796 - accuracy: 0.9755


 68/469 [===>..........................] - ETA: 1s - loss: 0.0773 - accuracy: 0.9769


 86/469 [====>.........................] - ETA: 1s - loss: 0.0768 - accuracy: 0.9777


104/469 [=====>........................] - ETA: 1s - loss: 0.0760 - accuracy: 0.9778


123/469 [======>.......................] - ETA: 1s - loss: 0.0744 - accuracy: 0.9782


141/469 [========>.....................] - ETA: 0s - loss: 0.0729 - accuracy: 0.9782


160/469 [=========>....................] - ETA: 0s - loss: 0.0732 - accuracy: 0.9783


179/469 [==========>...................] - ETA: 0s - loss: 0.0732 - accuracy: 0.9785


197/469 [===========>..................] - ETA: 0s - loss: 0.0718 - accuracy: 0.9787


216/469 [============>.................] - ETA: 0s - loss: 0.0719 - accuracy: 0.9786


233/469 [=============>................] - ETA: 0s - loss: 0.0714 - accuracy: 0.9786


251/469 [===============>..............] - ETA: 0s - loss: 0.0710 - accuracy: 0.9787


269/469 [================>.............] - ETA: 0s - loss: 0.0709 - accuracy: 0.9789


287/469 [=================>............] - ETA: 0s - loss: 0.0715 - accuracy: 0.9788


305/469 [==================>...........] - ETA: 0s - loss: 0.0714 - accuracy: 0.9788


323/469 [===================>..........] - ETA: 0s - loss: 0.0708 - accuracy: 0.9790


341/469 [====================>.........] - ETA: 0s - loss: 0.0702 - accuracy: 0.9791


358/469 [=====================>........] - ETA: 0s - loss: 0.0700 - accuracy: 0.9791


376/469 [=======================>......] - ETA: 0s - loss: 0.0698 - accuracy: 0.9792


395/469 [========================>.....] - ETA: 0s - loss: 0.0694 - accuracy: 0.9793


413/469 [=========================>....] - ETA: 0s - loss: 0.0697 - accuracy: 0.9792


431/469 [==========================>...] - ETA: 0s - loss: 0.0693 - accuracy: 0.9792


450/469 [===========================>..] - ETA: 0s - loss: 0.0688 - accuracy: 0.9795


468/469 [============================>.] - ETA: 0s - loss: 0.0686 - accuracy: 0.9795


469/469 [==============================] - 1s 3ms/step - loss: 0.0687 - accuracy: 0.9794 - val_loss: 0.0822 - val_accuracy: 0.9746

Epoch 4/5

  1/469 [..............................] - ETA: 1s - loss: 0.0564 - accuracy: 0.9844


 19/469 [>.............................] - ETA: 1s - loss: 0.0474 - accuracy: 0.9881


 37/469 [=>............................] - ETA: 1s - loss: 0.0471 - accuracy: 0.9871


 55/469 [==>...........................] - ETA: 1s - loss: 0.0481 - accuracy: 0.9864


 73/469 [===>..........................] - ETA: 1s - loss: 0.0485 - accuracy: 0.9861


 91/469 [====>.........................] - ETA: 1s - loss: 0.0491 - accuracy: 0.9857


109/469 [=====>........................] - ETA: 1s - loss: 0.0489 - accuracy: 0.9855


127/469 [=======>......................] - ETA: 0s - loss: 0.0488 - accuracy: 0.9855


145/469 [========>.....................] - ETA: 0s - loss: 0.0496 - accuracy: 0.9852


159/469 [=========>....................] - ETA: 0s - loss: 0.0500 - accuracy: 0.9847


177/469 [==========>...................] - ETA: 0s - loss: 0.0517 - accuracy: 0.9843


195/469 [===========>..................] - ETA: 0s - loss: 0.0525 - accuracy: 0.9840


212/469 [============>.................] - ETA: 0s - loss: 0.0526 - accuracy: 0.9837


229/469 [=============>................] - ETA: 0s - loss: 0.0523 - accuracy: 0.9838


245/469 [==============>...............] - ETA: 0s - loss: 0.0523 - accuracy: 0.9837


260/469 [===============>..............] - ETA: 0s - loss: 0.0523 - accuracy: 0.9839


278/469 [================>.............] - ETA: 0s - loss: 0.0513 - accuracy: 0.9842


296/469 [=================>............] - ETA: 0s - loss: 0.0511 - accuracy: 0.9842


314/469 [===================>..........] - ETA: 0s - loss: 0.0511 - accuracy: 0.9843


332/469 [====================>.........] - ETA: 0s - loss: 0.0507 - accuracy: 0.9842


350/469 [=====================>........] - ETA: 0s - loss: 0.0512 - accuracy: 0.9841


368/469 [======================>.......] - ETA: 0s - loss: 0.0515 - accuracy: 0.9841


386/469 [=======================>......] - ETA: 0s - loss: 0.0509 - accuracy: 0.9843


404/469 [========================>.....] - ETA: 0s - loss: 0.0506 - accuracy: 0.9845


422/469 [=========================>....] - ETA: 0s - loss: 0.0503 - accuracy: 0.9845


440/469 [===========================>..] - ETA: 0s - loss: 0.0499 - accuracy: 0.9847


458/469 [============================>.] - ETA: 0s - loss: 0.0505 - accuracy: 0.9846


469/469 [==============================] - 1s 3ms/step - loss: 0.0505 - accuracy: 0.9845 - val_loss: 0.0718 - val_accuracy: 0.9790

Epoch 5/5

  1/469 [..............................] - ETA: 1s - loss: 0.0134 - accuracy: 1.0000


 19/469 [>.............................] - ETA: 1s - loss: 0.0244 - accuracy: 0.9926


 37/469 [=>............................] - ETA: 1s - loss: 0.0300 - accuracy: 0.9911


 52/469 [==>...........................] - ETA: 1s - loss: 0.0303 - accuracy: 0.9911


 70/469 [===>..........................] - ETA: 1s - loss: 0.0304 - accuracy: 0.9911


 87/469 [====>.........................] - ETA: 1s - loss: 0.0307 - accuracy: 0.9911


104/469 [=====>........................] - ETA: 1s - loss: 0.0316 - accuracy: 0.9907


122/469 [======>.......................] - ETA: 1s - loss: 0.0329 - accuracy: 0.9905


141/469 [========>.....................] - ETA: 0s - loss: 0.0333 - accuracy: 0.9900


160/469 [=========>....................] - ETA: 0s - loss: 0.0323 - accuracy: 0.9902


178/469 [==========>...................] - ETA: 0s - loss: 0.0327 - accuracy: 0.9901


196/469 [===========>..................] - ETA: 0s - loss: 0.0342 - accuracy: 0.9896


215/469 [============>.................] - ETA: 0s - loss: 0.0337 - accuracy: 0.9897


231/469 [=============>................] - ETA: 0s - loss: 0.0334 - accuracy: 0.9898


248/469 [==============>...............] - ETA: 0s - loss: 0.0341 - accuracy: 0.9897


267/469 [================>.............] - ETA: 0s - loss: 0.0348 - accuracy: 0.9895


286/469 [=================>............] - ETA: 0s - loss: 0.0352 - accuracy: 0.9893


305/469 [==================>...........] - ETA: 0s - loss: 0.0352 - accuracy: 0.9894


323/469 [===================>..........] - ETA: 0s - loss: 0.0356 - accuracy: 0.9892


342/469 [====================>.........] - ETA: 0s - loss: 0.0358 - accuracy: 0.9892


361/469 [======================>.......] - ETA: 0s - loss: 0.0360 - accuracy: 0.9892


380/469 [=======================>......] - ETA: 0s - loss: 0.0367 - accuracy: 0.9889


399/469 [========================>.....] - ETA: 0s - loss: 0.0374 - accuracy: 0.9887


416/469 [=========================>....] - ETA: 0s - loss: 0.0375 - accuracy: 0.9886


434/469 [==========================>...] - ETA: 0s - loss: 0.0376 - accuracy: 0.9886


450/469 [===========================>..] - ETA: 0s - loss: 0.0376 - accuracy: 0.9885


468/469 [============================>.] - ETA: 0s - loss: 0.0379 - accuracy: 0.9885


469/469 [==============================] - 1s 3ms/step - loss: 0.0378 - accuracy: 0.9885 - val_loss: 0.0677 - val_accuracy: 0.9793

以下のコードは、訓練データと検証データ（ここではテストデータ）の精度と損失をそれぞれプロットします。これにより、モデルが訓練データにどれだけ適合しているか、また、未見のデータに対するモデルの性能を視覚的に理解することができます。

なお、’bo’は青色の丸を、’b’は青色の線を意味します。また、’Training acc’や’Validation acc’などは、グラフの凡例に表示されるラベルです。

import matplotlib.pyplot as plt

# 訓練の精度と損失値を取得
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))

# 訓練データと検証データの精度をプロット
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

# 訓練データと検証データの損失値をプロット
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-7-9b46599f798f> in <module>
----> 1 import matplotlib.pyplot as plt
      2 
      3 # 訓練の精度と損失値を取得
      4 acc = history.history['accuracy']
      5 val_acc = history.history['val_accuracy']

ModuleNotFoundError: No module named 'matplotlib'

以上が、ニューラルネットワークを用いた手書き数字の分類の基本的な流れです。一連のコードによって、手書き数字の画像を認識するニューラルネットワークのモデルを訓練し、新しい手書き文字に対してその数字がなんであるかを分類することができます。

畳み込みニューラルネットワーク（CNN）

畳み込みニューラルネットワーク（CNN）とは、画像や音声を認識するための特殊な種類の人工知能です。これは、画像の一部分の特徴を見つける「畳み込み層」、特徴の情報を簡単にする「プーリング層」、そして最終的にどのカテゴリに分類するかを決める「全結合層」の三つの部分から成り立っています。

以下に、PythonとKeras（人工知能を作るためのツール）を使って、CNNを作る基本的な手順を説明します。

まず、必要なツールを読み込みます。

# 必要なライブラリのインポート
import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from tensorflow.keras.utils import to_categorical
import numpy as np
import matplotlib.pyplot as plt

# 乱数の種を固定
np.random.seed(0)
tf.random.set_seed(0)

次に、学習に使うデータを読み込みます。今回はCIFAR-10を使います。CIFAR-10は、10種類のラベル（自動車、鳥、猫など）が付けられた32x32ピクセルのカラー画像60000枚から成るデータセットです。

# データセットのロード
(X_train, y_train), (X_test, y_test) = cifar10.load_data()

その後、データを適切な形に整形します。画像データは0から255の範囲の整数で表されますが、これを0から1の範囲に変換します。 また、ラベルデータ（y_trainとy_test）をカテゴリカル表現に変換します。これは、各ラベルを10次元のベクトルで表現する方法で、モデルの出力と一致させるために必要です。

# データの前処理
X_train = X_train.astype('float32') / 255
X_test = X_test.astype('float32') / 255

y_train = to_categorical(y_train, 10)
y_test = to_categorical(y_test, 10)

そして、CNNのモデルを作ります。これは、畳み込み層(Conv2D)、プーリング層(MaxPooling2D)、全結合層(Dense)を順番に追加していきます。

# モデルの作成開始
model = Sequential()

model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

モデルを作ったら、それをどのように学習させるかを設定します。

# モデルのコンパイル
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

最後に、モデルを実際に学習させます。 今回は少し時間がかかるので、このウィンドウは開いたままにしておいてください。

# モデルの訓練
history = model.fit(X_train, y_train,
                    epochs=10,
                    batch_size=64,
                    validation_data=(X_test, y_test))

Epoch 1/10
782/782 [==============================] - 81s 103ms/step - loss: 1.6790 - accuracy: 0.3951 - val_loss: 1.6580 - val_accuracy: 0.4282
Epoch 2/10
782/782 [==============================] - 77s 99ms/step - loss: 1.2715 - accuracy: 0.5502 - val_loss: 1.2968 - val_accuracy: 0.5437
Epoch 3/10
782/782 [==============================] - 75s 96ms/step - loss: 1.0806 - accuracy: 0.6216 - val_loss: 1.1639 - val_accuracy: 0.6026
Epoch 4/10
782/782 [==============================] - 79s 101ms/step - loss: 0.9567 - accuracy: 0.6668 - val_loss: 1.0006 - val_accuracy: 0.6499
Epoch 5/10
782/782 [==============================] - 77s 98ms/step - loss: 0.8634 - accuracy: 0.6999 - val_loss: 1.0632 - val_accuracy: 0.6449
Epoch 6/10
782/782 [==============================] - 79s 101ms/step - loss: 0.7899 - accuracy: 0.7246 - val_loss: 1.0297 - val_accuracy: 0.6461
Epoch 7/10
782/782 [==============================] - 77s 98ms/step - loss: 0.7276 - accuracy: 0.7464 - val_loss: 0.9319 - val_accuracy: 0.6842
Epoch 8/10
782/782 [==============================] - 77s 98ms/step - loss: 0.6738 - accuracy: 0.7653 - val_loss: 1.0721 - val_accuracy: 0.6444
Epoch 9/10
782/782 [==============================] - 77s 98ms/step - loss: 0.6217 - accuracy: 0.7839 - val_loss: 0.9912 - val_accuracy: 0.6725
Epoch 10/10
782/782 [==============================] - 78s 99ms/step - loss: 0.5731 - accuracy: 0.8005 - val_loss: 0.9337 - val_accuracy: 0.6961

訓練済みのモデルの出力を確認してみましょう。

# 予測
predictions = model.predict(X_test)

# # 予測結果を表示するための準備。CIFAR-10の各クラスに対応する名前をリストとして作成。
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

# 最初の25枚の画像と予測結果を表示
plt.figure(figsize=(10,10))
for i in range(25):
    plt.subplot(5,5,i+1)
    plt.xticks([])
    plt.yticks([])
    plt.grid(False)
    plt.imshow(X_test[i], cmap=plt.cm.binary)
    plt.xlabel(class_names[np.argmax(predictions[i])])
plt.show()

313/313 [==============================] - 4s 14ms/step

以上が、CNNを作る基本的な手順です。

再帰型ニューラルネットワーク（RNN）

再帰型ニューラルネットワーク（RNN）は、時間の流れに沿ったデータ（例えば、文章や音声などのシーケンスデータ）を処理するのに適した人工知能の一種です。RNNは特別な「隠れ層」を持っており、これが過去の情報を覚えておく役割を果たします。そのため、時刻が進むごとにこの隠れ層の情報が更新され、新しい出力を生み出します。

以下に、PythonとKerasを使ったRNNの作り方を示します。以下のコードは映画のレビューが肯定的か否定的かを判定する人工知能を作ります。人工知能はレビューの文章を読んで、その内容が肯定的か否定的かを学習し、新しいレビューに対する予測を行います。

まずはデータの準備から始めます。

# 必要なライブラリのインポート
from tensorflow.keras.layers import SimpleRNN, Embedding
from tensorflow.keras.datasets import imdb

# 映画のレビューデータセットの読み込み
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=10000)

# データが長さの異なる文章からなるため、全ての文章が同じ長さになるように補正
X_train = tf.keras.preprocessing.sequence.pad_sequences(X_train, maxlen=500)
X_test = tf.keras.preprocessing.sequence.pad_sequences(X_test, maxlen=500)

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz
17464789/17464789 [==============================] - 0s 0us/step

次に、ニューラルネットワークのモデルを作ります。

# モデルの作成開始
model = Sequential()
# Embedding層は単語をベクトル（数値の並び）に変換する
model.add(Embedding(10000, 32))
# SimpleRNN層がRNNの中心的な部分
model.add(SimpleRNN(32))
# Dense層は最終的な出力を生成する
model.add(Dense(1, activation='sigmoid'))

そして、このモデルをどのように学習させるかを設定し、実際にモデルを学習させます。

# モデルの学習方法を決める
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['acc'])

# モデルの訓練
history = model.fit(X_train, y_train,
                    epochs=10,
                    batch_size=128,
                    validation_split=0.2)

Epoch 1/10
157/157 [==============================] - 31s 192ms/step - loss: 0.6713 - acc: 0.5814 - val_loss: 0.6302 - val_acc: 0.6338
Epoch 2/10
157/157 [==============================] - 30s 190ms/step - loss: 0.4620 - acc: 0.7892 - val_loss: 0.3804 - val_acc: 0.8418
Epoch 3/10
157/157 [==============================] - 30s 192ms/step - loss: 0.3347 - acc: 0.8638 - val_loss: 0.5417 - val_acc: 0.7556
Epoch 4/10
157/157 [==============================] - 28s 181ms/step - loss: 0.2522 - acc: 0.9003 - val_loss: 0.3802 - val_acc: 0.8430
Epoch 5/10
157/157 [==============================] - 28s 182ms/step - loss: 0.1913 - acc: 0.9301 - val_loss: 0.5282 - val_acc: 0.7550
Epoch 6/10
157/157 [==============================] - 29s 182ms/step - loss: 0.1317 - acc: 0.9548 - val_loss: 0.4642 - val_acc: 0.8084
Epoch 7/10
157/157 [==============================] - 30s 191ms/step - loss: 0.0880 - acc: 0.9712 - val_loss: 0.5166 - val_acc: 0.8428
Epoch 8/10
157/157 [==============================] - 29s 183ms/step - loss: 0.0691 - acc: 0.9782 - val_loss: 0.5721 - val_acc: 0.7882
Epoch 9/10
157/157 [==============================] - 28s 182ms/step - loss: 0.0437 - acc: 0.9867 - val_loss: 0.5681 - val_acc: 0.8450
Epoch 10/10
157/157 [==============================] - 28s 181ms/step - loss: 0.0314 - acc: 0.9919 - val_loss: 0.6087 - val_acc: 0.8320

historyという変数には学習過程の情報が保存されています。これをプロットすることで学習の経過を視覚的に確認することができます。

import matplotlib.pyplot as plt

# 精度と損失の履歴を取得します
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)

# 精度のプロット
plt.figure(figsize=(12,6))
plt.subplot(1, 2, 1)
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

# 損失のプロット
plt.subplot(1, 2, 2)
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()

以下のコードはテストデータセットの最初の10個のレビューに対する予測を表示します。予測値（prediction）は0から1の間の値で、値が0.5以上ならば肯定的なレビュー（’Positive’）、0.5未満ならば否定的なレビュー（’Negative’）と判断しています。 また、imdb.load_data()関数でロードしたデータは既に単語を数値に変換した形で提供されているため、実際のレビューのテキストとともに結果を表示するために、数値を再び単語に戻す処理が追加されています。

# IMDBデータセットの単語とインデックスの対応関係を取得
word_index = imdb.get_word_index()

# インデックスと単語の対応関係（逆引き）を作成
reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])

# テストデータから10個のサンプルを取り出し、予測と共に表示
samples = X_test[:10]
predictions = model.predict(samples)

for i, (sample, prediction) in enumerate(zip(samples, predictions)):
    # 数値を単語に戻す
    decoded_review = ' '.join([reverse_word_index.get(index - 3, '?') for index in sample if index > 3])
    print(f"Review {i+1}: {'Positive' if prediction > 0.5 else 'Negative'}")
    print(decoded_review, "\n")

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json
1641221/1641221 [==============================] - 0s 0us/step
1/1 [==============================] - 0s 182ms/step
Review 1: Negative
please give this one a miss br br and the rest of the cast rendered terrible performances the show is flat flat flat br br i don't know how michael madison could have allowed this one on his plate he almost seemed to know this wasn't going to work out and his performance was quite so all you madison fans give this a miss 

Review 2: Positive
this film requires a lot of patience because it focuses on mood and character development the plot is very simple and many of the scenes take place on the same set in frances the sandy dennis character apartment but the film builds to a disturbing climax br br the characters create an atmosphere with sexual tension and psychological it's very interesting that robert altman directed this considering the style and structure of his other films still the trademark altman audio style is evident here and there i think what really makes this film work is the brilliant performance by sandy dennis it's definitely one of her darker characters but she plays it so perfectly and convincingly that it's scary michael burns does a good job as the mute young man regular altman player michael murphy has a small part the moody set fits the content of the story very well in short this movie is a powerful study of loneliness sexual and desperation be patient up the atmosphere and pay attention to the wonderfully written script br br i praise robert altman this is one of his many films that deals with unconventional fascinating subject matter this film is disturbing but it's sincere and it's sure to a strong emotional response from the viewer if you want to see an unusual film some might even say bizarre this is worth the time br br unfortunately it's very difficult to find in video stores you may have to buy it off the internet 

Review 3: Positive
at a time when motion picture animation of all sorts was in its br br the political of the russian revolution caused to move to paris where one of his first productions was a dark political satire known as or the who wanted a king a strain of black comedy can be found in almost all of films but here it is very dark indeed aimed more at grown ups who can appreciate the satirical aspects than children who would most likely find the climax i'm middle aged and found it pretty myself and indeed of the film intended for english speaking viewers of the 1920s were given title cards filled with and in order to help the sharp of the finale br br our tale is set in a swamp the where the citizens are unhappy with their government and have called a special session to see what they can do to improve matters they decide to for a king the crowds are animated in this opening sequence it couldn't have been easy to make so many frog puppets look alive simultaneously while for his part is depicted as a white guy in the clouds who looks like he'd rather be taking a when sends them a tree like god who regards them the decide that this is no improvement and demand a different king irritated sends them a br br delighted with this looking new king who towers above them the welcome him with a of dressed the mayor steps forward to hand him the key to the as cameras record the event to everyone's horror the promptly eats the mayor and then goes on a merry rampage citizens at random a title card reads news of the king's throughout the kingdom when the now terrified once more for help he loses his temper and their community with lightning the moral of our story delivered by a hapless frog just before he is eaten is let well enough alone br br considering the time period when this startling little film was made and considering the fact that it was made by a russian at the height of that country's civil war it would be easy to see this as a about those events may or may not have had turmoil in mind when he made but whatever his choice of material the film stands as a tale of universal could be the soviet union italy germany or japan in the 1930s or any country of any era that lets its guard down and is overwhelmed by it's a fascinating film even a charming one in its macabre way but its message is no joke 

Review 4: Positive
i generally love this type of movie however this time i found myself wanting to kick the screen since i can't do that i will just complain about it this was absolutely idiotic the things that happen with the dead kids are very cool but the alive people are absolute idiots i am a grown man pretty big and i can defend myself well however i would not do half the stuff the little girl does in this movie also the mother in this movie is reckless with her children to the point of neglect i wish i wasn't so angry about her and her actions because i would have otherwise enjoyed the flick what a number she was take my advise and fast forward through everything you see her do until the end also is anyone else getting sick of watching movies that are filmed so dark anymore one can hardly see what is being filmed as an audience we are involved with the actions on the screen so then why the hell can't we have night vision 

Review 5: Positive
like some other people wrote i'm a die hard mario fan and i loved this game br br this game starts slightly boring but trust me it's worth it as soon as you start your hooked the levels are fun and they will hook you your mind turns to i'm not kidding this game is also and is beautifully done br br to keep this spoiler free i have to keep my mouth shut about details but please try this game it'll be worth it br br story 9 9 action 10 1 it's that good 10 attention 10 average 10 

Review 6: Positive
i'm absolutely disgusted this movie isn't being sold all who love this movie should email disney and increase the demand for it they'd eventually have to sell it then i'd buy copies for everybody i know everything and everybody in this movie did a good job and i haven't figured out why disney hasn't put this movie on dvd or on vhs in rental stores at least i haven't seen any copies this is a wicked good movie and should be seen by all the kids in the new generation don't get to see it and i think they should it should at least be put back on the channel this movie doesn't deserve a cheap it deserves the real thing i'm them now this movie will be on dvd 

Review 7: Positive
later used by frank in mr deeds goes to town and meet john but in no one individual is cast as a hero or heroine the story is told through a series of scenes that are combined in a special effect known as montage the editing and selection of short segments to produce a desired effect on the viewer d w griffith also used the montage but no one it so well as br br the artistic filming of the crew sleeping in their is by the swinging of tables suspended from chains in the in contrast the confrontation between the crew and their officers is charged with electricity and the of the masses demonstrate their rage with injustice br br introduced the technique of showing an action and repeating it again but from a slightly different angle to demonstrate intensity the breaking of a plate bearing the words give us this day our daily bread the beginning of the end this technique is used in last year at also when the surgeon is tossed over the side his from the it was these glasses that the officer used to and pass the meat this sequence ties the punishment to the corruption of the era br br the most noted sequence in the film and perhaps in all of film history is the steps the broad of the steps are filled with hundreds of extras rapid and dramatic violence is always suggested and not explicit yet the visual images of the deaths of a few will last in the minds of the viewer forever br br the shots of boots and legs the steps are cleverly with long menacing shadows from a sun at the top of the steps the pace of the sequence is deliberately varied between the soldiers and a few civilians who up courage to beg them to stop a close up of a woman's face frozen in horror after being struck by a sword is the direct of the bank in bonnie in clyde and gives a lasting impression of the horror of the regime br br the death of a young mother leads to a baby down the steps in a sequence that has been copied by hitchcock in foreign by terry gilliam in brazil and brian in the this sequence is shown repeatedly from various angles thus drawing out what probably was only a five second event br br is a film that the revolutionary spirit it for those already committed and it for the it of fire and with the senseless of the regime its greatest impact has been on film students who have borrowed and only slightly improved on techniques invented in russia several generations ago 

Review 8: Negative
the richard dog is to joan fontaine dog however when bing crosby arrives in town to sell a record player to the emperor his dog is attacked by dog after a revenge attack where is from town a insists that dog must confront dog so that she can overcome her fears this is arranged and the dogs fall in love so do and the rest of the film passes by with romance and at the end dog gives birth but who is the father br br the dog story is the very weak vehicle that is used to try and create a story between humans its a terrible storyline there are 3 main musical pieces all of which are rubbish bad songs and dreadful choreography its just an extremely boring film bing has too many words in each sentence and delivers them in an almost irritating manner its not funny ever but its meant to be bing and joan have done much better than this 

Review 9: Positive
hollywood had a long love affair with bogus nights tales but few of these products have stood the test of time the most memorable were the jon hall maria films which have long since become camp this one is filled with dubbed songs and slapstick it's a truly crop of corn and pretty near today it was nominated for its imaginative special effects which are almost in this day and age mainly of trick photography the only outstanding positive feature which survives is its beautiful color and clarity sad to say of the many films made in this genre few of them come up to alexander original thief of almost any other nights film is superior to this one though it's a loser 

Review 10: Positive
this film is where the batman franchise ought to have stopped though i will that the ideas behind batman forever were excellent and could have been easily realised by a competent director as it turned out this was not to be the case br br apparently warner brothers executives were disappointed with how dark this second batman film from tim burton turned out apart from the idiocy of expecting anything else from burton and the conservative of their subsequent decision to turn the franchise into an homage to the sixties tv series i fail to understand how batman returns can be considered at all disappointing br br true it is not quite the equal of the first film though it all the minor of style found in batman a weaker script that the between not just two but three characters invites comparisons to the masterful pairing of keaton and jack nicholson as the joker in the first film yet for all this it remains a dark film true to the way the batman was always meant to be and highly satisfying br br michael keaton returns as the batman and his alter ego bruce wayne with max christopher walken named in honour of the 1920s german silent actor his partner in crime the penguin danny in brilliant makeup reminiscent of laurence richard iii and kyle the michelle pfeiffer whom wayne romances both as himself and as the batman the four principals turn in excellent performances especially walken and while together keaton and pfeiffer explore the darker side of double identities br br there are some intriguing concepts in this film about the only weakness i can really point out is a certain to the script in some places which i think is due mostly to the way this film is a four fight there simply isn't enough time to properly explore what's going on br br nevertheless this is a damn good film i highly recommend watching this in with the first and then for how good the series could have been had it continued under burton and keaton 

以上がRNNの作り方の一例です。

「epochs=10」は全データを10回繰り返し学習することを意味し、「batch_size=128」は128個のデータごとに学習を進めることを意味します。また、「validation_split=0.2」は、全データのうち20%を検証データ（モデルが学習に使わないで途中結果のチェックに使うデータ）として使うことを意味します。

このコードを実行すると、人工知能はレビューを読み、肯定的か否定的かを学習し、その結果を「history」という名前の変数に保存します。これにより、学習がうまく進んだかどうかを後から確認することが可能になります。

以上が、再帰型ニューラルネットワーク（RNN）の説明と基本的な実装例です。特に時系列データを扱う際には非常に役立つツールですので、ぜひ理解しておくと良いでしょう。

previous

レッスン1：機械学習の基礎