J.DepP usage and output example

README.md

Example output

➜ echo あの壁にかかっている絵はきれいですね | mecab -d /usr/local/lib/mecab/dic/unidic | jdepp 2> /dev/null | to_tree.py
# S-ID: 1; J.DepP
  0:　　　　　　　　　あの━━┓
  1:　　　壁に━━┓　　　　　┃
  2:　　かかっている━━┓　　┃
  3:　　　　　　　　　絵は━━┫
  4:　　　　　　　　きれいですねEOS
➜ echo あの壁にかかっている絵はきれいですね | mecab -d /usr/local/lib/mecab/dic/unidic | jdepp 2> /dev/null | to_chunk.py
# S-ID: 1; J.DepP
あの │ 壁 に │ かかっ て いる │ 絵 は │ きれい です ね EOS

Download and install instructions on Naoki Yoshinaga's website: http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/jdepp/#dl

Edit: Dr Yoshinaga has fixed this URL issue so this comment is only for historical purposes only!

Original comment:

@MeikyuuTrader ah, I see the problem, the KNBC URL has moved—you can confirm this because if you open the URL from the Makefile

KNBC_URL = http://nlp.ist.i.kyoto-u.ac.jp/kuntt/KNBC_v1.0_090925.tar.bz2

in your browser (also available via their website), your browser will be able to download it. To make the Makefile work, you can either

1. download http://nlp.ist.i.kyoto-u.ac.jp/kuntt/KNBC_v1.0_090925.tar.bz2 from your browser, place it in the same directory you uncompressed jdepp, and rerun make model,
2. edit the Makefile so on line 926 (in the section, knbc-tarball:), you have

$(CURL) -LO $(KNBC_URL); \

(note the extra "L" argument that tells curl to follow 301 Moved Permanently.)

I'll email Dr Yoshinaga and ask if he can update the download.

A quick note, version jdepp-2015-10-05 that you download has a bug where it doesn't work with UniDic. Dr Yoshinaga kindly fixed the bug and has provided an alternative download link which might be useful to you too: http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/jdepp/jdepp-2018-08-16.tar.gz

@MeikyuuTrader also make sure you have Python 2.x available because jdepp needs it even to build—Python 3 didn't work. I love pyenv, it lets me run pyenv local 2.7.14 in the jdepp directory so that any invocation of python in that directory will automatically run 2.7.14. With the Makefile fix and setting up Python 2 for the directory, I got jdepp to build.

I deleted my comment because I thought a reply was impossible, after all, this is a 4 year old post haha.

I've downloaded the KNBC and then just renamed the extracted folder the same as the makefile requested, that's fine. Just trying to fix the no command issue now. Might be due to not having python 2.4.

I'll try that, thanks

Yes, I am surprised at my own longevity on this platform 😅 sometimes the internet delivers!

Tragic but the python install doesn't seem fix the no: command not found part and thus the make model fails.

find /home/jdepp-2018-08-16/KNBC_v1.0_090925/corpus1 -type f -name "KN*" | LC_ALL=C sort | xargs cat | no /home/jdepp-2018-08-16/tools/knbc2kyoto.py KNP | no /home/jdepp-2018-08-16/tools/replace_pos.py /home/linuxbrew/.linuxbrew/bin/mecab -d /home/linuxbrew/.linuxbrew/lib/mecab/dic/unidic > model/knbc/train.JDP.euc
/bin/bash: no: command not found
/bin/bash: no: command not found

I wonder if it's because my gcc is version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04) whilst the site requires has only tested Compiler: tested with GNU gcc (≥ 4.0) or clang (≥ 2.9). Either way, thank you for the assistance. I'll need to test different stuff to get it working.

Ooh I think I see what's happening. When I run make model, that same line for me is:

find /snip/jdepp-2018-08-16/KNBC_v1.0_090925/corpus1 -type f -name "KN*" | LC_ALL=C sort | xargs cat | /Users/fasih/.pyenv/shims/python /snip/jdepp-2018-08-16/tools/knbc2kyoto.py KNP | /Users/fasih/.pyenv/shims/python /snip/jdepp-2018-08-16/tools/replace_pos.py /usr/local/bin/mecab -d /usr/local/lib/mecab/dic/jumandic > model/knbc/train.JDP.euc

If you look at this carefully, where you have no, I have /Users/fasih/.pyenv/shims/python. I suspect the ./configure step couldn't find a valid Python in the path, and perhaps it encountered "no … such file: python" or something like that? And stuck that "no" into the Makefile (instead of sensibly failing)?

Can you confirm that there's a python available when you're in that directory, and if so, try rerunning make clean && ./configure && make model?

I think you're right. When I ran make clean && ./configure | grep python it spits out checking for python... no

I tried installing via pyenv like yourself and python local returns 2.7.14, however, that checking for python...no exists. I'll try to install python normally and see what happens. Perhaps because I can't use python in the command line, it acts as if it doesn't exist?

Hmm, pyenv does do a lot of path shenanigans so you might need to close your terminal and reopen it to ascertain that when you run python it loads Python 2 from pyenv (which you can confirm with which python)? I'm sure I've always had Python available before installing jdepp so never encountered this issue before 😢 best of luck.

@fasiha THANK YOU for all your help, I don't think I would get bloody J.DepP without you noticing ./configure assigned python to no when it didn't exist. The woes of thinking a critical dependency would fail the script :). I was trying to install and get older gcc versions to run makefiles xD

Despite having pyenv, it always returned no python. Which python never returned anything. I think you could get it working with pyenv, you'd need to make sure which python returns python 2.7.

I'm using Ubuntu so I outright installed python with apt-get and then set python2.7 to the be main version. After that, running make clean && ./configure showed checking for python... /usr/bin/python

Also got file permission error when running make install due to /usr/bin/install: cannot create regular file '/usr/local/bin/jdepp': Permission denied ran as sudo to fix that.

Actually was so damned hard to get J.DepP installed.

Notes for anyone else

• Install python 2.7 (run which python and if there's no ouput, it won't show). Check this in the ./configure script, if checking for python = no, the model training won't work.
• You may get permission errors when trying to create a folder in /usr/local/bin when running make install so run it as sudo make install
• Homebrew is the easiest way to install MeCab and the mecab-unidic
• Use the linked KNBC in the first post, it's the EUC_JP version, the UTF-8 version will fail in training the model due to the script trying to convert to UTF-8.

Glad it's working, what an ordeal 😭!

Just because even the thought of using a system-wide python gives me anxiety, I did want to risk annoying you by asking if you noticed the stupidly complicated installation instructions for pyenv: in the macOS section it says "Then follow the rest of the post-installation steps under Basic GitHub Checkout, starting with #2 ("Configure your shell's environment for Pyenv")", and in that section it tells you how to futz with your .bashrc or .bash_profile to make it work.

I use oh-my-zsh and I just have the following line in ~/.zshrc: eval "$(pyenv init -)".

Again, totally feel free to ignore me! I'm just going on because I can't imagine life without pyenv/conda to keep my python sanity 🙃.

jdepp is also pretty interesting and unusual. I really like it but had to think about what exactly it was doing, what bunsetsu were, how to take advantage of bunsetsu and their inter-dependencies. Feel free to reach out if you want to check notes on it!

I was actually only using jdepp to create "words" and then I'd search the definition. More or less copying your method from the stackoverlfow post. So more advance stuff isn't going to be touched.

It's unfortunate that you can't get rid of the extra dependencies, like particles, to get the real word.

@MeikyuuTrader have you seen Ichiran? https://ichi.moe and https://github.com/tshatrov/ichiran its a very nice way (and an alternative to MeCab + JDepP) to find particles, etc. but via a totally different technique.