Skip to content

Instantly share code, notes, and snippets.

@MattDMo
Last active April 3, 2022 23:16
Show Gist options
  • Star 35 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save MattDMo/6cb1dfbe8a124e1ca5af to your computer and use it in GitHub Desktop.
Save MattDMo/6cb1dfbe8a124e1ca5af to your computer and use it in GitHub Desktop.
SublimeREPL ipy_repl.py for running IPython/Jupyter in Sublime Text
# from https://gist.github.com/MattDMo/6cb1dfbe8a124e1ca5af
import os
import json
import socket
import threading
activate_this = os.environ.get("SUBLIMEREPL_ACTIVATE_THIS", None)
# turn off pager
os.environ["TERM"] = "emacs"
if activate_this:
with open(activate_this, "r") as f:
exec(f.read(), {"__file__": activate_this})
try:
import jupyter_console.app
JUPYTER = True
except ImportError:
JUPYTER = False
if not JUPYTER:
try:
import IPython
IPYTHON = True
version = IPython.version_info[0]
except ImportError:
# for virtualenvs w/o IPython
import code
code.InteractiveConsole().interact()
# Jupyter and IPython 4+
if JUPYTER or (IPYTHON and version > 3):
from traitlets.config.loader import Config
# all other versions
else:
from IPython.config.loader import Config
editor = "subl -w"
cfg = Config()
cfg.ZMQTerminalInteractiveShell.simple_prompt = True
# cfg.InteractiveShell.readline_use = False # not needed in recent IPython versions
cfg.InteractiveShell.autoindent = True
cfg.InteractiveShell.colors = "NoColor"
cfg.InteractiveShell.editor = os.environ.get("SUBLIMEREPL_EDITOR", editor)
if JUPYTER:
app = jupyter_console.app.ZMQTerminalIPythonApp(config=cfg, user_ns={})
elif IPYTHON:
app = IPython.terminal.ipapp.TerminalIPythonApp(config=cfg, user_ns={})
app.initialize()
ac_port = int(os.environ.get("SUBLIMEREPL_AC_PORT", "0"))
ac_ip = os.environ.get("SUBLIMEREPL_AC_IP", "127.0.0.1")
if ac_port:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ac_ip, ac_port))
def read_netstring(s):
size = 0
while True:
ch = s.recv(1)
if ch == b":":
break
size = size * 10 + int(ch)
msg = b""
while size != 0:
msg += s.recv(size)
size -= len(msg)
ch = s.recv(1)
assert ch == b','
return msg
def send_netstring(sock, msg):
payload = b"".join([str(len(msg)).encode("ascii"), b":", msg.encode("utf-8"), b","])
sock.sendall(payload)
def complete(shell, req):
# Jupyter
if JUPYTER:
result = shell.Completer.complete_request(req["line"], req["cursor_pos"])
return (req["line"], result["matches"])
# IPython 4
elif IPYTHON and version > 3:
return shell.complete(**req)
# IPython 1-3
else:
return []
def handle():
while True:
msg = read_netstring(s).decode("utf-8")
try:
req = json.loads(msg)
result = complete(app.shell, req)
res = json.dumps(result)
send_netstring(s, res)
except Exception:
send_netstring(s, b"[]")
if ac_port:
t = threading.Thread(target=handle)
t.start()
app.start()
if ac_port:
s.close()
@ipstone
Copy link

ipstone commented Jun 1, 2018

I have similar problem with using sublime REPL ipython on mac, together with this gist, and the additional line noted by Pybe at https://stackoverflow.com/questions/39058283/how-do-i-get-ipython-5-to-work-with-sublimerepl

"You only need to insert the additional config
cfg.ZMQTerminalInteractiveShell.simple_prompt = True
somewhere between line 38 and 41."

Then it works for my ipython 6 setup on mac os x.

Thank you everybody!

@Auerilas
Copy link

Auerilas commented Jul 2, 2018

Hi,
Thanks for writing this! It works great. I'm having an issue with the matplotlib backend. By default, when I use a regular Terminal, the backend is Qt5:

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'Qt5Agg'

In SublimeREPL, it seems to default to the inline version:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'module://ipykernel.pylab.backend_inline'

In [3]: 

This isn't a big deal, but it makes my figures when I use plt.show() open up in a tiny ImageMagick window instead of the matplotlib Qt GUI. I've tried modifying the ipy_repl.py file by adding:

cfg.InteractiveShellApp.matplotlib = 'qt5'

but that doesn't seem to work:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: import matplotlib

In [2]: matplotlib.get_backend()
Out[2]: 'module://ipykernel.pylab.backend_inline'

In [3]: [IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
    handler(stream, idents, msg)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 434, in complete_request
    matches = self.do_complete(code, cursor_pos)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 262, in do_complete
    return self._experimental_do_complete(code, cursor_pos)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 287, in _experimental_do_complete
    completions = list(_rectify_completions(code, raw_completions))
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 485, in rectify_completions
    completions = list(completions)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 1808, in completions
    for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 1849, in _completions
    cursor_line, cursor_column = position_to_cursor(full_text, offset)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/IPython/core/completer.py", line 872, in position_to_cursor
    assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
AssertionError: 0 <= -350 <= 329
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 98, in handle
    result = complete(app.shell, req)
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 84, in complete
    result = shell.Completer.complete_request(req['line'], req['cursor_pos'])
  File "/home/nate/anaconda3/lib/python3.6/site-packages/jupyter_console/completer.py", line 39, in complete_request
    msg = self.client.shell_channel.get_msg(timeout=self.timeout)
  File "/home/nate/anaconda3/lib/python3.6/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nate/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/nate/anaconda3/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 102, in handle
    send_netstring(s, b"[]")
  File "/home/nate/.config/sublime-text-3/Packages/SublimeREPL/config/Python/ipy_repl.py", line 78, in send_netstring
    payload = b"".join([str(len(msg)).encode("ascii"), b':', msg.encode("utf-8"), b','])
AttributeError: 'bytes' object has no attribute 'encode'

Am I missing something? I'd appreciate any help.

Note, it's not just qt that throws an error: cfg.InteractiveShellApp.matplotlib = 'tk' doesn't work, but cfg.InteractiveShellApp.matplotlib = 'auto' works fine (but still uses the inline backend).

If I don't change anything and do:

Jupyter console 5.2.0

Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: %matplotlib
Using matplotlib backend: Qt5Agg

In [2]: import matplotlib

In [4]: matplotlib.get_backend()
Out[4]: 'Qt5Agg'

then everything works fine. I just don't want to have to type %matplotlib every time.

@cgjosephlee
Copy link

https://gist.github.com/pe224/ec5d7effa26e8f736fa2f6bb8f515606
I found this and it worked with ipython 6.

@OpenWaygate
Copy link

OpenWaygate commented Nov 25, 2018

All solution above are not perfect and will crash sometimes, e.g.github issues
I have a more detail answer in stack overflow,

For short, just modified \Packages\SublimeREPL\config\Python\ipy_repl.py to 3 lines

import os
# change dir to where the ipython.exe located
os.chdir(r'C:\Python\envs\py37\Scripts')
os.system('ipython')

@ste-goldstein
Copy link

ste-goldstein commented Jan 23, 2020

Hi. Thanks for this file.
I'm trying to pass additional IPython configuration to the REPL, but it has no effect, e.g:
cfg.InteractiveShellApp.exec_lines = [ 'print("\\nimporting some things\\n")', 'import math', "math" ]
Any ideas why ?

@MattDMo
Copy link
Author

MattDMo commented Mar 26, 2020

@skualos that needs to go into ipython_config.py. Follow my instructions just above the crossed-out section for removing an error message to create the file if you don't already have it in ~/.ipython.

@wtfzambo
Copy link

wtfzambo commented Jun 23, 2020

Hi, thanks for this.

I wish I could make it work but my problem is that I don't have an activate_this.py file in my Scripts folder.
I believe the reason for this is due to the fact that I use python venv module to create the virtual environment in my project folder, like this:
`python -m venv .venv'

So when I want to activate my virtual environment, I open the shell, cd into my project folder and run > .venv\Scripts\activate, and that does the trick.

I tried adapting your script, using the subprocess module:

if activate_this:
    with open(activate_this, "r") as f:
        # exec(f.read(), {"__file__": activate_this})  # instead of this
        subprocess.run(f.read(), shell=True)        # I use this

I tried several possibilities of the above (such as, using subprocess.run(activate_this), not using shell=True), but to no avail. Honestly, at this point I don't even know what I'm doing anymore :(.

Any help?

Forgot to mention:
I'm on windows 10 running python 3.7

@MattDMo
Copy link
Author

MattDMo commented Jun 27, 2020

@wtfzambo Thanks for your comment. activate_this.py is part of the virtualenv module, not the builtin venv, so that's why you're not seeing it. I actually haven't played with virtual environments in SublimeREPL, so all of this is speculation, but I would assume that @wuub knew there were multiple tools for creating venvs and is only calling activate_this.py due to some particular issue with virtualenv. I monitor the SublimeREPL tag on Stack Overflow, and I can't recall seeing any questions where users were using venv and couldn't get SublimeREPL's virtual environment functionality to work, although that doesn't mean such issues haven't happened.

I guess I should ask, were you having issues before? Would it be possible to just run a separate REPL instance using your venv's python.exe? If you don't have a ton of venvs, this might be the best way to go. Here's how:

  1. Create a Packages/User/SublimeREPL folder and in it create a new file called Main.sublime-menu (make sure to capitalize Main).
  2. Add the following JSON to Main.sublime-menu:
[
    {
        "id": "tools",
        "children":
        [{
            "command": "repl_open",
            "caption": "IPython - Python 3.7 venv",
            "id": "repl_python_ipython",
            "args": {
                "type": "subprocess",
                "encoding": "utf8",
                "autocomplete_server": true,
                "cmd": {
                    "windows": ["c:/path/to/venv/Scripts/python.exe", "-u", "${packages}/SublimeREPL/config/Python/ipy_repl.py"]
                },
                "cwd": "c:/path/to/desired/working/dir",
                "syntax": "Packages/Python/Python.sublime-syntax",
                "external_id": "python",
                "extend_env": {
                    "PYTHONIOENCODING": "utf-8",
                    "SUBLIMEREPL_EDITOR": "$editor"
                }
            }
        }]
    }
]
  1. Make sure to customize the paths for your system. Save the file.

This will create a menu item Tools → IPython - Python 3.7 venv that should work as expected. Let me know how it goes!

@wtfzambo
Copy link

This will create a menu item Tools → IPython - Python 3.7 venv that should work as expected. Let me know how it goes!

@MattDMo

Yeah, I had actually came to this solution on my own before asking here and actually managed to make it work. The problem I have with it is that it hardcodes a specific env, whereas I would like to keep it dynamic.

The reason for this is that I have several different projects (mostly simple web-apps or AWS lambda functions) and each one of them has its own venv. In normal situations, I just use my conda base environment, which is in PATH so it's everywhere, but in those specific projects I want to be able to access their respective virtual env.

That's it basically. Do you know if there's a way to make python compile a .bat or .ps1 script?

@samwega
Copy link

samwega commented Dec 12, 2020

Works as described, even on newer versions than the instructions. Now I can run bash commands from the REPL in sublime.. awesome! Just don't try to launch kakoune inside sublime, it will crash the REPL haha
Thank you!

@researchdatascience
Copy link

hi there - I get python but my tabnine intelligence doesn't work while using this. Would you happened to know if I need to do something to get the auto completion to work?
thanks for your help with this.

@MattDMo
Copy link
Author

MattDMo commented Oct 30, 2021

@pushpaghimire I can get Anaconda's code completion to work, but not using any of the LSP plugins like pyright or pylsp, so it's not surprising that TabNine doesn't work. Unfortunately, I don't have any suggestions on what to change, as I haven't figured out what the underlying problem is in the first place. Perhaps someone else can chime in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment