You might need to use this magic:
Every time you want to force the inline output for a graph figure generated from matplotlib, which is actually most of the times.
There is a way to make this "automatic" by configuring the file
c.InteractiveShellApp.matplotlib = "inline"
And just reload the notebook in your browser, this should work automatically without the explicit magic anymore.