# Tensorflow Summary API V2

Following the announcement in Tensorflow Dev Summit 2018 in Eager Execution part: https://www.youtube.com/watch?v=T8AW0fKP0Hs&vl=en

It suggests that we should use `tf.contrib.summary`

. Here is the link to the documentation https://www.tensorflow.org/api_docs/python/tf/contrib/summary

To me, it is not a very good documentation and hard to follow. I have here some example codes to guide you to the new summary API.

I will try to cover both modes, eager and graph. I shall stick with the graph mode and then add some notes for the eager mode.

Overall code template will be put at the end of this article.

### Creating the summary writer

This is the same with both graph mode and eager mode.

Since the new summary API is very context based, I think it is a good practice to separate graph for each summary writer.

Initializing the summary:

```
import tensorflow as tf
save_path = 'logs'
graph = tf.Graph()
with graph.as_default():
global_step = tf.train.create_global_step()
writer = tf.contrib.summary.create_file_writer(save_path)
with writer.as_default():
tf.contrib.summary.always_record_summaries()
```

Now, we have `graph`

and `writer`

.

Note: I usually also create a `global_step`

with the creation of graph. Which is very convenient because the summary writer will automatically utilize it. You don’t really need to keep the `global_step`

though, you can always get it from `tf.train.get_global_step()`

given that you properly set the default graph.

### Preparing the dataset and creating the model

This is not the main topic of this article though, but for the sake of expressiveness I found it useful for explanation.

First we prepare the dataset, I will demonstrate with `tf.data.Dataset`

here.

```
with graph.as_default():
# dataset
data = ... some tf.data.Dataset ...
data_itr = data.make_initializable_iterator()
x, y = data_itr.get_next() # x is feature, y is label
```

Note that in eager mode you don’t need the `make_initializable_iterator()`

because the dataset is itself iterable. That means in eager you don’t need `get_next()`

as well.

#### At the time we define model, we add something to summary

The following code is in graph mode:

```
with graph.as_default():
with writer.as_default():
# let's say we have "net" as our model
prediction_op = net(x)
loss_op = tf.losses.sparse_softmax_cross_entropy(y, prediction_op)
opt_op = tf.train.AdamOptimizer(0.001).minimize(
loss_op,
global_step=tf.train.get_global_step())
with tf.contrib.summary.record_summaries_every_n_global_steps(1):
tf.contrib.summary.scalar('loss', loss_op)
summary_op = tf.contrib.summary.all_summary_ops()
```

You see that we use `tf.contrib.summary.record_summaries_every_n_global_steps`

(https://www.tensorflow.org/api_docs/python/tf/contrib/summary/record_summaries_every_n_global_steps) to log anything inside the context manager every “n” steps.

In eager you might consider creating a function for running in each iteration like so:

```
def train(net, optimizer, x, y):
with tf.contrib.eager.GradientTape() as tape:
prediction = net(x)
loss = tf.losses.sparse_softmax_cross_entropy(y, prediction)
grads = tape.gradient(loss, net.variables)
grads_vars = zip(grads, net.variables)
optimizer.apply_gradients(
grads_vars,
global_step=tf.train.get_global_step()
)
# here is how you log every step (n=1)
with tf.contrib.summary.record_summaries_every_n_global_steps(1):
tf.contrib.summary.scalar('loss', loss)
return loss
```

### At the time of running

Here is the code for graph mode:

```
with graph.as_default():
with writer.as_default():
with tf.Session() as sess:
# initialize the summary
tf.contrib.summary.initialize(
graph=tf.get_default_graph()
)
# init vars
sess.run(tf.global_variables_initializer())
# init iterator
sess.run(data_itr.initializer)
# run
while True:
try:
_, _, loss = sess.run([
summary_op, opt_op, loss_op
])
...
except tf.errors.OutOfRangeError:
break
```

Two things to keep in mind:

`tf.contrib.summary.initialize(graph=...)`

— You can supply “None” to the graph (leave it blank), it won’t save the graph to the Tensorboard. However, this option won’t have any effect in the eager mode anyway (since it doesn’t really construct a graph)- You need to run
`tf.contrib.summary.all_summary_ops()`

which will actually write the summary. However, you don’t need this for eager mode since it’s executed in real time anyway

#### To summarize

- Create a summary writer under a graph, and use that graph throughout
- Define what you want to include in the summary and how frequent
- Initialize the summary (you might supply the graph here if you want to get the “Graph” page in Tensorboard, won’t work in eager)
- In graph mode, also run
`tf.contrib.summary.all_summary_ops()`

to actually write the summary

#### An example of graph mode

```
import tensorflow as tf
import numpy as np
save_path = 'summary_path'
graph = tf.Graph()
with graph.as_default():
global_step = tf.train.create_global_step()
writer = tf.contrib.summary.create_file_writer(save_path)
with writer.as_default():
tf.contrib.summary.always_record_summaries()
# simulate dataset
fake_dataset = np.random.randn(1000, 100).astype(np.float32)
fake_label = np.random.randint(low=0, high=9, size=1000)
# preparing a fake dataset
with graph.as_default():
x = tf.data.Dataset.from_tensor_slices(fake_dataset)
y = tf.data.Dataset.from_tensor_slices(fake_label)
data = tf.data.Dataset.zip((x, y))
data = data.shuffle(10000)
data = data.batch(32)
data_itr = data.make_initializable_iterator()
x, y = data_itr.get_next()
# define the computing graph
with graph.as_default():
with writer.as_default():
# construct a simple classifier
net = tf.keras.Sequential([
tf.keras.layers.Dense(300, activation=tf.nn.relu),
tf.keras.layers.Dense(10)
])
prediction_op = net(x)
loss_op = tf.losses.sparse_softmax_cross_entropy(y, prediction_op)
opt_op = tf.train.AdamOptimizer(0.001).minimize(
loss_op,
global_step=tf.train.get_global_step())
# here is how you log every step (n=1)
with tf.contrib.summary.record_summaries_every_n_global_steps(1):
tf.contrib.summary.scalar('loss', loss_op)
summary_op = tf.contrib.summary.all_summary_ops()
# compute the graph
with graph.as_default():
with writer.as_default():
with tf.Session() as sess:
# initialize the summary writer
tf.contrib.summary.initialize(
graph=tf.get_default_graph()
)
# init vars
sess.run(tf.global_variables_initializer())
# init iterator
sess.run(data_itr.initializer)
# run until the dataset is exhausted
while True:
try:
_, _, loss = sess.run([
summary_op, opt_op, loss_op
])
except tf.errors.OutOfRangeError:
break
```

#### An example of eager mode

```
import tensorflow as tf
import numpy as np
tf.enable_eager_execution()
save_path = 'logs/test10'
graph = tf.Graph()
with graph.as_default():
global_step = tf.train.create_global_step()
writer = tf.contrib.summary.create_file_writer(save_path)
with writer.as_default():
tf.contrib.summary.always_record_summaries()
# simulate dataset
fake_dataset = np.random.randn(1000, 100).astype(np.float32)
fake_label = np.random.randint(low=0, high=9, size=1000)
# preparing a fake dataset
with graph.as_default():
x = tf.data.Dataset.from_tensor_slices(fake_dataset)
y = tf.data.Dataset.from_tensor_slices(fake_label)
data = tf.data.Dataset.zip((x, y))
data = data.shuffle(10000)
data = data.batch(32)
# define the model
with graph.as_default():
with writer.as_default():
# construct a simple classifier
net = tf.keras.Sequential([
tf.keras.layers.Dense(300, activation=tf.nn.relu),
tf.keras.layers.Dense(10)
])
optimizer = tf.train.AdamOptimizer(0.001)
def train(net, optimizer, x, y):
with tf.contrib.eager.GradientTape() as tape:
prediction = net(x)
loss = tf.losses.sparse_softmax_cross_entropy(y, prediction)
grads = tape.gradient(loss, net.variables)
grads_vars = zip(grads, net.variables)
optimizer.apply_gradients(
grads_vars,
global_step=tf.train.get_global_step()
)
# here is how you log every step (n=1)
with tf.contrib.summary.record_summaries_every_n_global_steps(1):
tf.contrib.summary.scalar('loss', loss)
return loss
# start the training process
with graph.as_default():
with writer.as_default():
# initialize the summary writer
tf.contrib.summary.initialize()
# run until the dataset is exhausted
for x, y in data:
loss = train(net, optimizer, x, y)
print(float(loss))
```