Skip to content

Instantly share code, notes, and snippets.

@MattDMo
Last active April 3, 2022 23:16
Show Gist options
  • Star 35 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save MattDMo/6cb1dfbe8a124e1ca5af to your computer and use it in GitHub Desktop.
Save MattDMo/6cb1dfbe8a124e1ca5af to your computer and use it in GitHub Desktop.
SublimeREPL ipy_repl.py for running IPython/Jupyter in Sublime Text
# from https://gist.github.com/MattDMo/6cb1dfbe8a124e1ca5af
import os
import json
import socket
import threading
activate_this = os.environ.get("SUBLIMEREPL_ACTIVATE_THIS", None)
# turn off pager
os.environ["TERM"] = "emacs"
if activate_this:
with open(activate_this, "r") as f:
exec(f.read(), {"__file__": activate_this})
try:
import jupyter_console.app
JUPYTER = True
except ImportError:
JUPYTER = False
if not JUPYTER:
try:
import IPython
IPYTHON = True
version = IPython.version_info[0]
except ImportError:
# for virtualenvs w/o IPython
import code
code.InteractiveConsole().interact()
# Jupyter and IPython 4+
if JUPYTER or (IPYTHON and version > 3):
from traitlets.config.loader import Config
# all other versions
else:
from IPython.config.loader import Config
editor = "subl -w"
cfg = Config()
cfg.ZMQTerminalInteractiveShell.simple_prompt = True
# cfg.InteractiveShell.readline_use = False # not needed in recent IPython versions
cfg.InteractiveShell.autoindent = True
cfg.InteractiveShell.colors = "NoColor"
cfg.InteractiveShell.editor = os.environ.get("SUBLIMEREPL_EDITOR", editor)
if JUPYTER:
app = jupyter_console.app.ZMQTerminalIPythonApp(config=cfg, user_ns={})
elif IPYTHON:
app = IPython.terminal.ipapp.TerminalIPythonApp(config=cfg, user_ns={})
app.initialize()
ac_port = int(os.environ.get("SUBLIMEREPL_AC_PORT", "0"))
ac_ip = os.environ.get("SUBLIMEREPL_AC_IP", "127.0.0.1")
if ac_port:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ac_ip, ac_port))
def read_netstring(s):
size = 0
while True:
ch = s.recv(1)
if ch == b":":
break
size = size * 10 + int(ch)
msg = b""
while size != 0:
msg += s.recv(size)
size -= len(msg)
ch = s.recv(1)
assert ch == b','
return msg
def send_netstring(sock, msg):
payload = b"".join([str(len(msg)).encode("ascii"), b":", msg.encode("utf-8"), b","])
sock.sendall(payload)
def complete(shell, req):
# Jupyter
if JUPYTER:
result = shell.Completer.complete_request(req["line"], req["cursor_pos"])
return (req["line"], result["matches"])
# IPython 4
elif IPYTHON and version > 3:
return shell.complete(**req)
# IPython 1-3
else:
return []
def handle():
while True:
msg = read_netstring(s).decode("utf-8")
try:
req = json.loads(msg)
result = complete(app.shell, req)
res = json.dumps(result)
send_netstring(s, res)
except Exception:
send_netstring(s, b"[]")
if ac_port:
t = threading.Thread(target=handle)
t.start()
app.start()
if ac_port:
s.close()
@MattDMo
Copy link
Author

MattDMo commented Aug 20, 2016

@sth4nth @gwerbin @samkrishna please see my updated instructions above. It seems that I can only get it working with ipython==4.1.1 and jupyter_console==4.1.0. I didn't talk about it up there, but with the introduction of the prompt_toolkit interface, IPython now only works on TTYs, which Sublime is not. I don't know if there are any plans to change this. It would be worthwhile to either file or vote up (react to) an issue asking for this feature.

@samkrishna
Copy link

samkrishna commented Oct 16, 2016

@MattDMo -- Yeah I don't have a good idea on this one. I do get IPython / Jupiter 4.1.1 working here but the vt100-sourced error messages related to TTY are just gross. (Again, macOS 10.11.6 final)

Everything seems to be working relatively fine apart from the gross exception throw, but that's about all I've got right now.

@Tokin256
Copy link

Thanks! With the downgrade I could help a few people to get the IPython repl working again.
Any idea if/when it would be possible to have a repl which is compatible with a higher version of IPython? Or is there a fundamental 'show stopper'?
I'm afraid my Python skills are limited in these matters, but still: anything that I can do to help?

@oocarain
Copy link

Thanks!The version is key,if you does not work and you can try check out your Jupyter_console and Ipython version.In my case,ipython==4.1.1 and jupyter_console==4.1.0 it works well.

@pe224
Copy link

pe224 commented Jul 19, 2017

If anybody is still interested, you might want to try the following:

You only need to insert the additional config
cfg.ZMQTerminalInteractiveShell.simple_prompt = True
somewhere between line 38 and 41.

The hack at line 68-73 starting with
if os.name == "nt":
is not necessary for me on Win 7/10. It only displays the banner twice if you don't comment it out.

The simple-prompt option is necessary to bypass the usage of prompt_toolkit which is the part failing when not being called from a proper shell. This option, however, disables some convenience features, like multi-line commands.
Also, I couldn't figure out how to properly exit the Jupyter shell when closing the shell window in Sublime Text. Without issuing a quit command first, the connection file of the jupyter kernel is not deleted - who knows what else keeps running...

@Tokin256
Copy link

Thanks Pe224! You're a life saver!!!

I was afraid that I wouldn't be able to move to Python 3.6 because of this issue!

But with the fix I can use Ipython 5.3.0 in combination with Jupyter console 5.2.0 (which is the current version in the Anaconda distribution, Sept 2017). Super!

Differences (minor inconveniences) I noticed are:

  • Use "help( numpy.std)", since "?numpy.std" causes the REPL to freeze
  • Also, do not use "??numpy.std", since that also causes the REPL to freeze
    The workaround is to open a 'qtconsole' by the command %qtconsole, and issue these commands in there. I regularly open such a console anyway since the post mortem debugger (which can be accessed by %debug) in the "%qtconsole" is much more responsive than the REPL shell in SublimeText.

@bagua092883
Copy link

For some reason I still can't get this to work. I'm getting this error

Traceback (most recent call last):
File "/Users/jl/Library/Application Support/Sublime Text 3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 66, in
embedded_shell.initialize()
File "", line 2, in initialize
File "/Users/jl/anaconda2/lib/python2.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/Users/jl/anaconda2/lib/python2.7/site-packages/jupyter_console/app.py", line 141, in initialize
self.init_shell()
File "/Users/jl/anaconda2/lib/python2.7/site-packages/jupyter_console/app.py", line 114, in init_shell
client=self.kernel_client,
File "/Users/jl/anaconda2/lib/python2.7/site-packages/traitlets/config/configurable.py", line 412, in instance
inst = cls(*args, **kwargs)
File "/Users/jl/anaconda2/lib/python2.7/site-packages/jupyter_console/ptshell.py", line 271, in init
self.init_prompt_toolkit_cli()
File "/Users/jl/anaconda2/lib/python2.7/site-packages/jupyter_console/ptshell.py", line 425, in init_prompt_toolkit_cli
output=create_output(true_color=self.true_color),
File "/Users/jl/anaconda2/lib/python2.7/site-packages/prompt_toolkit/shortcuts.py", line 126, in create_output
ansi_colors_only=ansi_colors_only, term=term)
File "/Users/jl/anaconda2/lib/python2.7/site-packages/prompt_toolkit/terminal/vt100_output.py", line 424, in from_pty
assert stdout.isatty()
AssertionError
[IPKernelApp] WARNING | Parent appears to have exited, shutting down.

Please help.

Thanks

@tadeo
Copy link

tadeo commented Apr 10, 2018

I'm getting this same error @bagua092883 reports

@ipstone
Copy link

ipstone commented Jun 1, 2018

I have similar problem with using sublime REPL ipython on mac, together with this gist, and the additional line noted by Pybe at https://stackoverflow.com/questions/39058283/how-do-i-get-ipython-5-to-work-with-sublimerepl

"You only need to insert the additional config
cfg.ZMQTerminalInteractiveShell.simple_prompt = True
somewhere between line 38 and 41."

Then it works for my ipython 6 setup on mac os x.

Thank you everybody!

@Auerilas
Copy link

Auerilas commented Jul 2, 2018

Hi,
Thanks for writing this! It works great. I'm having an issue with the matplotlib backend. By default, when I use a regular Terminal, the backend is Qt5:

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'Qt5Agg'

In SublimeREPL, it seems to default to the inline version:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'module://ipykernel.pylab.backend_inline'

In [3]: 

This isn't a big deal, but it makes my figures when I use plt.show() open up in a tiny ImageMagick window instead of the matplotlib Qt GUI. I've tried modifying the ipy_repl.py file by adding:

cfg.InteractiveShellApp.matplotlib = 'qt5'

but that doesn't seem to work:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'module://ipykernel.pylab.backend_inline'

In [3]: [IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
    handler(stream, idents, msg)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 434, in complete_request
    matches = self.do_complete(code, cursor_pos)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 262, in do_complete
    return self._experimental_do_complete(code, cursor_pos)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 287, in _experimental_do_complete
    completions = list(_rectify_completions(code, raw_completions))
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 485, in rectify_completions
    completions = list(completions)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 1808, in completions
    for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 1849, in _completions
    cursor_line, cursor_column = position_to_cursor(full_text, offset)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 872, in position_to_cursor
    assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
AssertionError: 0 <= -350 <= 329
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 98, in handle
    result = complete(app.shell, req)
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 84, in complete
    result = shell.Completer.complete_request(req['line'], req['cursor_pos'])
  File "/home/nate/anaconda3/lib/python3.6/site-packages/jupyter_console/completer.py", line 39, in complete_request
    msg = self.client.shell_channel.get_msg(timeout=self.timeout)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nate/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/nate/anaconda3/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 102, in handle
    send_netstring(s, b"[]")
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 78, in send_netstring
    payload = b"".join([str(len(msg)).encode("ascii"), b':', msg.encode("utf-8"), b','])
AttributeError: 'bytes' object has no attribute 'encode'

Am I missing something? I'd appreciate any help.

Note, it's not just qt that throws an error: cfg.InteractiveShellApp.matplotlib = 'tk' doesn't work, but cfg.InteractiveShellApp.matplotlib = 'auto' works fine (but still uses the inline backend).

If I don't change anything and do:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: %matplotlib
Using matplotlib backend: Qt5Agg

In [2]: import matplotlib

In [4]: matplotlib.get_backend()
Out[4]: 'Qt5Agg'

then everything works fine. I just don't want to have to type %matplotlib every time.

@cgjosephlee
Copy link

https://gist.github.com/pe224/ec5d7effa26e8f736fa2f6bb8f515606
I found this and it worked with ipython 6.

@OpenWaygate
Copy link

OpenWaygate commented Nov 25, 2018

All solution above are not perfect and will crash sometimes, e.g.github issues
I have a more detail answer in stack overflow,

For short, just modified \Packages\SublimeREPL\config\Python\ipy_repl.py to 3 lines

import os
# change dir to where the ipython.exe located
os.chdir(r'C:\Python\envs\py37\Scripts')
os.system('ipython')

@ste-goldstein
Copy link

ste-goldstein commented Jan 23, 2020

Hi. Thanks for this file.
I'm trying to pass additional IPython configuration to the REPL, but it has no effect, e.g:
cfg.InteractiveShellApp.exec_lines = [ 'print("\\nimporting some things\\n")', 'import math', "math" ]
Any ideas why ?

@MattDMo
Copy link
Author

MattDMo commented Mar 26, 2020

@skualos that needs to go into ipython_config.py. Follow my instructions just above the crossed-out section for removing an error message to create the file if you don't already have it in ~/.ipython.

@wtfzambo
Copy link

wtfzambo commented Jun 23, 2020

Hi, thanks for this.

I wish I could make it work but my problem is that I don't have an activate_this.py file in my Scripts folder.
I believe the reason for this is due to the fact that I use python venv module to create the virtual environment in my project folder, like this:
`python -m venv .venv'

So when I want to activate my virtual environment, I open the shell, cd into my project folder and run > .venv\Scripts\activate, and that does the trick.

I tried adapting your script, using the subprocess module:

if activate_this:
    with open(activate_this, "r") as f:
        # exec(f.read(), {"__file__": activate_this})  # instead of this
        subprocess.run(f.read(), shell=True)        # I use this

I tried several possibilities of the above (such as, using subprocess.run(activate_this), not using shell=True), but to no avail. Honestly, at this point I don't even know what I'm doing anymore :(.

Any help?

Forgot to mention:
I'm on windows 10 running python 3.7

@MattDMo
Copy link
Author

MattDMo commented Jun 27, 2020

@wtfzambo Thanks for your comment. activate_this.py is part of the virtualenv module, not the builtin venv, so that's why you're not seeing it. I actually haven't played with virtual environments in SublimeREPL, so all of this is speculation, but I would assume that @wuub knew there were multiple tools for creating venvs and is only calling activate_this.py due to some particular issue with virtualenv. I monitor the SublimeREPL tag on Stack Overflow, and I can't recall seeing any questions where users were using venv and couldn't get SublimeREPL's virtual environment functionality to work, although that doesn't mean such issues haven't happened.

I guess I should ask, were you having issues before? Would it be possible to just run a separate REPL instance using your venv's python.exe? If you don't have a ton of venvs, this might be the best way to go. Here's how:

  1. Create a Packages/User/SublimeREPL folder and in it create a new file called Main.sublime-menu (make sure to capitalize Main).
  2. Add the following JSON to Main.sublime-menu:
[
    {
        "id": "tools",
        "children":
        [{
            "command": "repl_open",
            "caption": "IPython - Python 3.7 venv",
            "id": "repl_python_ipython",
            "args": {
                "type": "subprocess",
                "encoding": "utf8",
                "autocomplete_server": true,
                "cmd": {
                    "windows": ["c:/path/to/venv/Scripts/python.exe", "-u", "${packages}/SublimeREPL/config/Python/ipy_repl.py"]
                },
                "cwd": "c:/path/to/desired/working/dir",
                "syntax": "Packages/Python/Python.sublime-syntax",
                "external_id": "python",
                "extend_env": {
                    "PYTHONIOENCODING": "utf-8",
                    "SUBLIMEREPL_EDITOR": "$editor"
                }
            }
        }]
    }
]
  1. Make sure to customize the paths for your system. Save the file.

This will create a menu item Tools → IPython - Python 3.7 venv that should work as expected. Let me know how it goes!

@wtfzambo
Copy link

This will create a menu item Tools → IPython - Python 3.7 venv that should work as expected. Let me know how it goes!

@MattDMo

Yeah, I had actually came to this solution on my own before asking here and actually managed to make it work. The problem I have with it is that it hardcodes a specific env, whereas I would like to keep it dynamic.

The reason for this is that I have several different projects (mostly simple web-apps or AWS lambda functions) and each one of them has its own venv. In normal situations, I just use my conda base environment, which is in PATH so it's everywhere, but in those specific projects I want to be able to access their respective virtual env.

That's it basically. Do you know if there's a way to make python compile a .bat or .ps1 script?

@samwega
Copy link

samwega commented Dec 12, 2020

Works as described, even on newer versions than the instructions. Now I can run bash commands from the REPL in sublime.. awesome! Just don't try to launch kakoune inside sublime, it will crash the REPL haha
Thank you!

@researchdatascience
Copy link

hi there - I get python but my tabnine intelligence doesn't work while using this. Would you happened to know if I need to do something to get the auto completion to work?
thanks for your help with this.

@MattDMo
Copy link
Author

MattDMo commented Oct 30, 2021

@pushpaghimire I can get Anaconda's code completion to work, but not using any of the LSP plugins like pyright or pylsp, so it's not surprising that TabNine doesn't work. Unfortunately, I don't have any suggestions on what to change, as I haven't figured out what the underlying problem is in the first place. Perhaps someone else can chime in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment