Skip to content

Instantly share code, notes, and snippets.

@HarvsG
Last active July 15, 2021 02:52
Show Gist options
  • Save HarvsG/9463bbe06301e8fb58498ba1ec835893 to your computer and use it in GitHub Desktop.
Save HarvsG/9463bbe06301e8fb58498ba1ec835893 to your computer and use it in GitHub Desktop.
draft for gitea blog post
date author title tags draft
2020-07-22T20:00:00+00:00
HarvsG
How to render Jupyter Notebooks on Gitea
rendering
jupyter
ui
true

How to render Jupyter Notebooks on Gitea


This guide will show you how to configure an external renderer to display Jupyter Notebooks. However this guide will also work for other types of files and even binary files! The sky is the limit.

How Gitea displays .ipynb files natively

Lets create a new repository on our Gitea instance and push an example notebook to it:

20200721 before ext renderer

So as we can see Gitea just renders the raw content of the file - efficient but hard to read.

How to generate HTML to display

In order to display something more attractive to the user we need some HTML, luckily Jupyter has a module called nbconvert:

Install our converter software of choice on the Gitea machine:

sudo apt install python3-pip
pip3 install nbconvert

If we wanted we could test this by running a test command:

jupyter nbconvert --to html --template full path/to/some/test/notebook.ipynb

If we open the resulting .html file in a browser we get something that looks like: image

This looks promising....

Configuring Gitea to use the converter

Like most options we can configure the Gitea instance using only app.ini.

add this it custom/conf/app.ini:

; Gitea looks for markup.xxxxx and will apply both "markup" and "xxxxx" as a class to the parent <div>
[markup.jupyter]
ENABLED = true
; all the file extensions we want to convert, comma separated.
FILE_EXTENSIONS = .ipynb
; Lets use out nbconvert command from earlier - making to sure to convert to HTML and to output to stdout
RENDER_COMMAND = "jupyter nbconvert --stdout --to html --template full "
; nbconvert accepts a path to a file and not stdin
IS_INPUT_FILE = true

; the name after sanitizer doesn't really matter
[markup.sanitizer.jupyter0]
; Jupyter chiefly uses divs
ELEMENT = div
; we will need access to html classes later
ALLOW_ATTR = class
; we don't mind which classes we keep, so let's keep all of them
REGEXP =

Now lets see what we get when we restart Gitea.

20200721 after install of nb convert

As you can see this is better, but not the same as when we opened our HTML file earlier. This is becasue Gitea removes the inline style sheets for security needs. What styling there is is inherited from the markup class in the <div> that wraps the code.

Getting back our styling

If we look at the contents of the HTML file we created earlier we can see several inline stylesheets

<style type="text/css">
    /*!
*
* Twitter Bootstrap
*
*/
/*!
 * Bootstrap v3.3.7 (http://getbootstrap.com)
 * Copyright 2011-2016 Twitter, Inc.
 * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
 */
/*! normalize.css v3.0.3 | MIT License | github.com/necolas/normalize.css */
html {
  font-family: sans-serif;
  -ms-text-size-adjust: 100%;
  -webkit-text-size-adjust: 100%;
}
body {
  margin: 0;
}
  
...
  
  </stlye>

So lets strip out all of style sheets and combine them into one jupyter.less making sure to delete all the HTML tags like <style></style>. We now have a stylesheet that contains all our beautiful styling. However it has lots of generic selectors like:

body {
  margin: 0;
}

This is certainly going to conflict with the default styling of Gitea. So we need to make sure that we limit the scope of the styling. Luckily, Gitea and less can help us with that.

In less

.someclass {
  body {
    margin: 0;
  }
  summary {
    display: block;
  }
}

Is equivalent to the css:

.someclass body {
	margin: 0;
}

.someclass summary {
	display: block;
}

And Gitea has given the classes markup and jupyter from app.ini to the <div> that surrounds the code. image

So lets wrap the whole contents of jupyter.less in .markup.jupyter {} to generate something that looks like this.

NB: if you are using Gitea before the merging of #12261 then you will need to use .markdown.jupyter {} instead.

Now lets put that file in our 'custom' directory , in my case that is /root/custom/public/css/jupyter.less.

I then added to /root/custom/templates/header.tmpl

<!-- lets import the less stylesheet {{AppSubUrl}} is a variable that gitea will autofill -->
<link rel="stylesheet/less" type="text/css" href="{{AppSubUrl}}/css/jupyter.less" />
<!-- we need the javascript to compile the less into css -->
<script src="//cdn.jsdelivr.net/npm/less" ></script>

Now let's restart gitea and see what we get.

Final appearance with the .markup.jupyer custom styling 20200721 after css

Looks good but we are getting some problems with text over-flowing the edge of the border. This is caused by some pesky CSS at-rules, so lets delete the below from the jupyter.less

@media (min-width: 768px) {
  .container {
    width: 768px;
  }
}
@media (min-width: 992px) {
  .container {
    width: 940px;
  }
}
@media (min-width: 1200px) {
  .container {
    width: 1140px;
  }
}

image

Ta-da! Perfectly styled Jupyter Notebooks.

@fengxiong001
Copy link

How to configure styles, can you give an example

@HarvsG
Copy link
Author

HarvsG commented Apr 17, 2021

This process won't work until go-gitea/gitea#12261 is merged.

can you give an example

The whole post above is an example

@matthewlootens
Copy link

Thanks, @HarvsG, for this excellent guide!

I've got all the css-styling working, but I'm still having trouble getting output plots (i.e., the embedded png images) to get through the sanitizer. I'm using the latest release 1.14.2.

nbconvert embeds the images as base64-encoded data uri's in a src attribute of an img element. I've added the following to my app.ini file, but when the html-rendered .ipynb file displays in gitea the img elements have been stripped of their src attribute and no image displays.

[markup.sanitizer.rule1]
ELEMENT = img
ALLOW_ATTR = src
REGEXP =

[markdown]
CUSTOM_URL_SCHEMES = data

Are you able to display Jupyter notebook images using the procedure described in your guide? Or any help as to what might be causing this issue? Thanks

@fengxiong001
Copy link

Thanks, @HarvsG, for this excellent guide!

I've got all the css-styling working, but I'm still having trouble getting output plots (i.e., the embedded png images) to get through the sanitizer. I'm using the latest release 1.14.2.

nbconvert embeds the images as base64-encoded data uri's in a src attribute of an img element. I've added the following to my app.ini file, but when the html-rendered .ipynb file displays in gitea the img elements have been stripped of their src attribute and no image displays.

[markup.sanitizer.rule1]
ELEMENT = img
ALLOW_ATTR = src
REGEXP =

[markdown]
CUSTOM_URL_SCHEMES = data

Are you able to display Jupyter notebook images using the procedure described in your guide? Or any help as to what might be causing this issue? Thanks

i have met the same question,have you solved this challenge?

@HarvsG
Copy link
Author

HarvsG commented Jul 13, 2021

@ziongfen
@matthewlootens

Hmm, might be one for the gitea devs.
go-gitea/gitea#16020

@matthewlootens
Copy link

@ziongfen, the issue has been fixed as of version 1.14.3, and I've been able to get all styling and images to display properly.

In addition to following @HarvsG's guide for getting the CSS styling, you also need to add the following to the app.ini file:

[markdown]
CUSTOM_URL_SCHEMES = data

[markup.sanitizer.jupyter_rule1]
ELEMENT = img
ALLOW_ATTR = src
REGEXP = 

You might also need to change one line in the nbconvert 'classic' template file, base.html.j2, from

<img src="data:image/png;base64,{{ output.data['image/png'] }}"

to

<img src="data:image/png;base64,{{ output.data['image/png'][:-1] }}"

The template adds an extra line-feed character at the end of the base-64 image that does not seem to get stripped by the sanitizer in Gitea.

@fengxiong001
Copy link

@ziongfen, the issue has been fixed as of version 1.14.3, and I've been able to get all styling and images to display properly.

In addition to following @HarvsG's guide for getting the CSS styling, you also need to add the following to the app.ini file:

[markdown]
CUSTOM_URL_SCHEMES = data

[markup.sanitizer.jupyter_rule1]
ELEMENT = img
ALLOW_ATTR = src
REGEXP = 

You might also need to change one line in the nbconvert 'classic' template file, base.html.j2, from

<img src="data:image/png;base64,{{ output.data['image/png'] }}"

to

<img src="data:image/png;base64,{{ output.data['image/png'][:-1] }}"

The template adds an extra line-feed character at the end of the base-64 image that does not seem to get stripped by the sanitizer in Gitea.

Thank you very much for your reply, You are wonderful.

@fengxiong001
Copy link

@ziongfen, the issue has been fixed as of version 1.14.3, and I've been able to get all styling and images to display properly.

In addition to following @HarvsG's guide for getting the CSS styling, you also need to add the following to the app.ini file:

[markdown]
CUSTOM_URL_SCHEMES = data

[markup.sanitizer.jupyter_rule1]
ELEMENT = img
ALLOW_ATTR = src
REGEXP = 

You might also need to change one line in the nbconvert 'classic' template file, base.html.j2, from

<img src="data:image/png;base64,{{ output.data['image/png'] }}"

to

<img src="data:image/png;base64,{{ output.data['image/png'][:-1] }}"

The template adds an extra line-feed character at the end of the base-64 image that does not seem to get stripped by the sanitizer in Gitea.

can you give me the nbconvert 'classic' template file source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment