Deep Learning To Mobile ⚡️

Curated way to convert deep learning model to mobile.

This repository will show you how to put your own model directly into mobile(iOS/Android) with basic example. First part is about deep learning model to mobile machine learning framework, and second part is about deep learning framework to mobile machine learning framework

Intro

Part 1. Deep learning model to mobile machine learning framework

Neural Network	CoreML	TensorFlow Mobile	Tensorflow Lite
Feedforward NN	✔️	✔️	✔️
Convolutional NN	✔️	✔️	✔️
Recurrent NN	✔️	✔️	❗️

Part 2. Deep learning framework to mobile machine learning framework

Framework	CoreML	TensorFlow Mobile	Tensorflow Lite
Tensorflow	tf-coreml	tensorflow	tensorflow
Pytorch	onnx	←	←
Keras	coremltools	tensorflow backend	←
Caffe	coremltools	caffe-tensorflow	←

Part 0.

Basic FFNN example

I'll use Golbin code in this TensorFlow-Tutorials, and simple Keras code to convert. I use two examples because there are different limits.

TensorFlow

import tensorflow as tf
import numpy as np

x_data = np.array(
    [[0, 0], [1, 0], [1, 1], [0, 0], [0, 0], [0, 1]])

y_data = np.array([
    [1, 0, 0], 
    [0, 1, 0],  
    [0, 0, 1],  
    [1, 0, 0],
    [1, 0, 0],
    [0, 0, 1]
])

global_step = tf.Variable(0, trainable=False, name='global_step')
X = tf.placeholder(tf.float32, name='Input')
Y = tf.placeholder(tf.float32, name='Output')

with tf.name_scope('layer1'):
    W1 = tf.Variable(tf.random_uniform([2, 10], -1., 1.), name='W1')
    b1 = tf.Variable(tf.zeros([10]), name='b1')
    L1 = tf.add(tf.matmul(X, W1), b1, name='L1')
    L1 = tf.nn.relu(L1)
    
with tf.name_scope('layer2'):
    W2 = tf.Variable(tf.random_uniform([10, 3], -1., 1.), name='W2')
    b2 = tf.Variable(tf.zeros([3]), name='b2')
    model = tf.add(tf.matmul(L1, W2), b2, name='model')
    prediction = tf.argmax(model, 1, name='prediction')

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=Y, logits=model), name='cost')
optimizer = tf.train.AdamOptimizer(learning_rate=0.01, name='optimizer')
train_op = optimizer.minimize(cost, global_step=global_step)

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

saver = tf.train.Saver(tf.global_variables())
merged = tf.summary.merge_all()
writer = tf.summary.FileWriter('./logs', sess.graph)

for step in range(30):
    sess.run(train_op, feed_dict={X: x_data, Y: y_data})

    if (step + 1) % 30 == 0:
        print(step + 1, sess.run(cost, feed_dict={X: x_data, Y: y_data}))
        tf.train.write_graph(sess.graph_def, '.', './model/FFNN.pbtxt')  
        saver.save(sess, './model/FFNN.ckpt', global_step=global_step)
        break
        
target = tf.argmax(Y, 1)
is_correct = tf.equal(prediction, target)
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))
print('Accuracy: %.2f' % sess.run(accuracy * 100, feed_dict={X: x_data, Y: y_data}))

Keras

import numpy as np
import tensorflow as tf

x_data = np.array(
    [[0, 0], [1, 0], [1, 1], [0, 0], [0, 0], [0, 1]])

y_data = np.array([
    [1, 0, 0], 
    [0, 1, 0],  
    [0, 0, 1],  
    [1, 0, 0],
    [1, 0, 0],
    [0, 0, 1]
])

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(10, input_dim=2,activation="relu"))
model.add(tf.keras.layers.Dense(2, activation="relu", kernel_initializer="uniform"))
model.add(tf.keras.layers.Dense(3))
model.add(tf.keras.layers.Activation("softmax"))

adam = tf.keras.optimizers.Adam(lr=0.01)

model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
model.summary()

reult = model.fit(x_data, y_data, shuffle=True, epochs=10, batch_size=2, validation_data=(x_data, y_data))

Part 1.

Deep learning model to mobile machine learning framework

CoreML

• ML Framework supported by Apple, using .mlmodel extension
• Automatically generated wrapper for iOS(Swift or Objective-C)

• Neural Network	CoreML
  Feedforward NN	✔️
  Convolutional NN	✔️
  Recurrent NN	✔️

REFERENCE

• Core ML
• Converting Trained Models to Core ML

TensorFlow Mobile🔒

TensorFlow Mobile is now deprecated

• ML Framework supported by Google, using .pb extension
• Support Java for Android, Objective-C++ for iOS

• Neural Network	TensorFlow Mobile
  Feedforward NN	✔️
  Convolutional NN	✔️
  Recurrent NN	✔️

Reference

• TensorFlow Mobile
• TensorFlow on Mobile: Tutorial

TensorFlow Lite

• ML Framework supported by Google, using .tflite extension
• Support Java for Android, Objective-C++ for iOS
• Recommand way by Google to use tensorflow in Mobile

• Neural Network	TensorFlow Mobile
  Feedforward NN	✔️
  Convolutional NN	✔️
  Recurrent NN	RNN is not supported see more information in this link

Reference

• TensorFlow Lite
• TensorFlow Lite & TensorFlow Compatibility Guide

Part 2.

Deep learning framework to mobile machine learning framework

TensorFlow to Tensorflow Mobile

We can get FFNN.pbtxtand FFNN.ckpt-90 in Part 0 code.

Freeze graph using freeze_graph from tensorflow.python.tools

from tensorflow.python.tools import freeze_graph

freeze_graph.freeze_graph("model/FFNN.pbtxt", "",
                          "", "model/FFNN.ckpt-90", "Output",
                          "", "",
                          "FFNN_frozen_graph.pb", True, "")

Now you can use FFNN_frozen_graph.pb in TensorFlow Mobile!

Neural Network	freeze_graph
Feedforward NN	✔️
Convolutional NN	✔️
Recurrent NN	✔️

Check your Tensor graph

You have to check frozen tensor graph

def load_graph(frozen_graph_filename):
    with tf.gfile.GFile(frozen_graph_filename, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())

    with tf.Graph().as_default() as graph:
        tf.import_graph_def(graph_def, name="")

    return graph
    
graph = load_graph('FFNN_frozen_graph.pb')

for op in graph.get_operations():
    print(op.name)

Reference

• Graphs and Sessions

TensorFlow to CoreML (iOS)

tf-coreml is the recommended way from Apple to convert tensorflow to CoreML

tf-coreml currently could not convert cycled graph like RNN... etc #124

import tfcoreml

mlmodel = tfcoreml.convert(
        tf_model_path = 'FFNN_frozen_graph.pb',
        mlmodel_path = 'FFNN.mlmodel',
        output_feature_names = ['layer2/prediction:0'],
        input_name_shape_dict = {'Input:0': [1, 2]})

Now you can use FFNN.mlmodel in iOS project!

Neural Network	tf-coreml
Feedforward NN	✔️
Convolutional NN	✔️
Recurrent NN	✖️

Reference

• tf-coreml

TensorFlow to TensorFlow Lite (Android)

toco is the recommended way from Google to convert TensorFlow to TensorFlow Lite

import tensorflow as tf

graph_def_file = "FFNN_frozen_graph.pb"
input_arrays = ["Input"]
output_arrays = ["layer2/prediction"]

converter = tf.contrib.lite.TocoConverter.from_frozen_graph(
  graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("FFNN.tflite", "wb").write(tflite_model)

Now you can use FFNN.tflite in Android project!

Neural Network	toco
Feedforward NN	✔️
Convolutional NN	✔️
Recurrent NN	✖️

Reference

• Toco
• Intro to Machine Learning on Android — How to convert a custom model to TensorFlow Lite

Keras to CoreML (iOS)

coremltools is the recommended way from Apple to convert Keras to CoreML

import coremltools

coreml_model = coremltools.converters.keras.convert(model)
coreml_model.save('FFNN.mlmodel')

Now you can use FFNN.mlmodel in Android project!

Neural Network	coremltools
Feedforward NN	✔️
Convolutional NN	✔️
Recurrent NN	✔️

Reference

• coremltools

Keras to TensorFlow Lite (Android)

toco is the recommended way from Google to convert Keras to TensorFlow Lite

Make .h5 Keras extension and then convert it to .tflie extension

keras_file = "FFNN.h5"
tf.keras.models.save_model(model, keras_file)

converter = tf.contrib.lite.TocoConverter.from_keras_model_file(keras_file)
tflite_model = converter.convert()
open("FFNN.tflite", "wb").write(tflite_model)

Now you can use FFNN.tflite in Android project!

Neural Network	toco
Feedforward NN	✔️
Convolutional NN	✔️
Recurrent NN	✖️

Reference

• TensorFlow Lite Optimizing Converter & Interpreter Python API reference

Author

younatics

License

DeepLearningToMobile is available under the MIT license. See the LICENSE file for more info.