Skip to content

Instantly share code, notes, and snippets.

@shashishailaj

shashishailaj/gist:091a2cd5146c36604ecac9e70ce6d50a

Created November 5, 2017 09:54
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shashishailaj/091a2cd5146c36604ecac9e70ce6d50a to your computer and use it in GitHub Desktop.
Save shashishailaj/091a2cd5146c36604ecac9e70ce6d50a to your computer and use it in GitHub Desktop.
Download ZIP
How to Setup USB Disk, NFS Server and Bittorrent on OpenWrt Attitude Adjustment and connect from OSX
Raw
gistfile1.md

Background:

  • I have a TP-Link MR3020 router. I live in Bangalore and am using gBroadband.
  • I frequently travel from home and feel I am not making 100% use of my broadband bandwidth.
  • I also don't like keeping my USB HDD attached to my computer because if I move physically even the USB HDD gets disconnected.

So best option was to have NAS (Network Attached Device), set up NFS (Network File Server) on my router and access my files wirelessly from my computer running OSX Yosemite. Hence the title of this article! After many failed attempts, looking through incomplete documentation, I finaly have the perfect setup. Hope this helps you in some way :)

Jargon/terminology

  • USB Disk means a physical thing like a flash drive, a pendrive, a thumbdrive.
  • USB HDD is a USB Disk albiet with more space like 250GB/1TB etc, basically a USB disk with big amount of space.
  • Small USB Disk is are those with less than 32GB of space. You know those small pendrives we get as freebies at offices/events? :)
  • Disk and Drive are the same thing referring to a storage medium. So we will use these terms interchangably.

I am assuming you generally know your way around command line and basic commands of vim text editor.

Let's do this!

  1. Install OpenWrt Attitude Adjustment. Why such this particular release? I havent been able to get my router recognize USB on the stable release (Breaking Barrier) or Trunk release (Chaos Calmer). Download the firmware for TP-LINK MR3020 from here. Go to your existing router page's "Firmware upgrade" UI and upload the Openwrt firmware. I am assuming here that you are connected to your existing router either by wire or wifi.

  2. Once installed, you have to access the router initially using the USB to Ethernet adapter (should you like me have a computer e.g. macbook air which doesnt have a ethernet port). I got one of those cheap USB to Ethernet adapter and OSX wasnt recognizing it. So I followed instructions from How To Use A Generic Usb 10/100M Ethernet Adaptor On Mac Os.

  3. Once OSX started recognizing the USB to Ethernet interface, I set its TCP/IP.

IPv4 address: 192.168.1.2 
Subnet: 255.255.255.0 
Gateway/Router: 192.168.1.1.

Now visit http://192.168.1.1 in your browser (or alternatively telnet 192.168.1.1 23 and follow instructions on screen).

  1. In the web UI: Change the root password and also put in your Public key found in ~/.ssh/id_rsa.pub file because you will be logging in and out of the router through ssh a couple of times in this setup and you dont want to have to type the password all the time.

  2. In the web UI: Enable the default wifi. Then edit the wifi to add your security to it or rename it to your liking.

  3. Disconnect the USB to Ethernet cable from computer and router. Attach your ISP's ethernet cable to your router.

  4. I am running Gatik Broaband (gBroadband) here in Bangalore, India and they run their Ethernet as Hotpost, which is pretty messed up setup to have but oh well ;/ So basically gBroaband asks me to login to their network before they give me access to the Internet. So for this to work, you have edit the default interface br-lan to uncheck from its physical settings 'eth0' but leave 'wan' checked.

http://i.imgur.com/x4GpnTMh.png

To be able to make our NFS setup easy later on, I have limited the DHCP addresses given out by the router to start from 192.168.1.2 up to 191.168.1.24. This means that any device that connects to your router will have local IP addresses no greater than 192.168.1.24.

  1. Now create a new interface called 'eth_wan', set its Protocol to be 'dhcpclient', in its physical settings check 'eth0' and in it's firewall settings check 'wan'.

  2. You should now have access to the Internets through gBroadband ISP, yay! 50% work done :)

USB Support and Extending router memory with pivot-overlay

Now, we have to make Openwrt recognize USB. Since router's main memory is Flash and is limited, we need to install USB support and then extend its main memory by putting in a small USB drive that will act as its new main memory.

This means physically you have to attach a USB hub to your router's USB port, add a USB drive which has like 1gb memory or however much you like. Then attach your USB HDD (which you plan to use for keeping media files/downloading etc) to another port in the USB hub.

My setup is like this because sometimes I like to take my external HDD away with me, so no point in having my router's main memory also as a partition on a external HDD, thus rendering the router (when bare without the HDD) useless.

http://i.imgur.com/2BMyiC2m.jpg

  1. Opkg is Openwrt's package manager. It installs and helps manage software on your Openwrt router. Install USB support by running opkg update and opkg install kmod-usb-storage kmod-fs-vfat kmod-fs-ext4 kmod-nls-cp437 kmod-nls-iso8859-1 block-mount. Am installing vfat, cp437, iso8859 and ext4 kernel modules because my USB HDD is FAT formatted and the USB drive which I will use to extend my router's memory is ext4. block-mount is to automount these USB disks when my router boots.

  2. Optional: install 'hd-idle' and 'luci-app-hd-idl' to spin down my USB HDD when its idle.

  3. Go to "Mount Points" in the router page UI. You should be able to mount your USB disks to any mount points you want. For e.g. I have mounted my smaller USB disk (which will extend my router's memory at /openwrt) and my USB HDD at /mnt/hdd. http://i.imgur.com/33tBeHqh.png

  4. Now lets prepare the smaller USB disk we have to become our router's new main memory. SSH into your router with ssh 192.168.1.1 and run: tar -C /overlay -cvf - . | tar -C /<your-mount-point-of-small-usb-disk> -xf -
    e.g, tar -C /overlay -cvf - . | tar -C /openwrt -xf - .

What does this do? Remember the firmware you flashed on to the router itself? Well that's of a fixed size, so it takes up a finite amount of space in the flash memory of the router and you cannot delete any 'uncessary' packages to clear space, should you be short of space. BUT we need more space to add more functionality to our router. Do you see the /overlay on your router? that's where the software you install from opkg goes. So TL;DR is the '/rom' on the router is mapped to '/overlay'. We are copying the entire /overlay to our smaller USB disk with that tar command. This is called 'pivot-overlay' method.Wew!

  1. Once you've successfully replicated the router's /overlay onto the USB disk, in the router page "Mount Points" UI again, edit the small USB disk's mount points, check the option 'Configures this mount as overlay storage for block-extroot'. https://i.imgur.com/AusgeM5h.png

  2. Reboot the router. You can do this via the web UI itself or in the terminal's SSH session of 192.168.1.1, run reboot.

  3. SSH into your router and run df -h and you should see more space for your router, yay!

e.g output:

root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    3.6G    126.6M      3.3G   4% /
/dev/root                 2.0M      2.0M         0 100% /rom
tmpfs                    14.3M    972.0K     13.3M   7% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda1                 3.6G    126.6M      3.3G   4% /overlay
overlayfs:/overlay        3.6G    126.6M      3.3G   4% /
/dev/sdb2               465.3G    310.3G    154.9G  67% /mnt/hdd

  1. Let's prepare our USB HDD's contents to be served to us. Since mine is a FAT formatted disk, it has to be mounted with special options: 'umask=000,dmask=000,fmask=000,uid=<your_uid>,gid=<your_gid>'. This is because vfat doesnt have linux like content permissions so when you mount vfat you have to establish a base file permission on which all vfat mounted will work. To find out your uid and gid in OSX, run 'id' and note down number against uid and gid. Mine is 501 and 20: http://i.imgur.com/SNSQqNlh.png.

  2. Install NFS program so that the router can serve the files from your USB HDD. SSH into your router and run opkg update, opkg install unfs3 portmap.

  3. Now you need to expose the directory on your USB HDD, that is the directory you want your NFS client (such as your computer) to get access to. Edit the configuration file with vim /etc/exports. For e.g, mine looks like this:/mnt/hdd 192.168.1.2/24(rw,fsid=root,no_subtree_check,all_squash,async,insecure). This means:

  • USB HDD is mounted on the router at /mnt/hdd
  • This disk will be read-write enabled ('rw')
  • TODO: What does fsid=root mean and do?
  • This disk won't be checking if accessed tree of dirs is in the NFS filesystem (no_subtree_check
  • This disk treats all users who access this disk's contents (create, read, modify, delete) as anonymous. (all_squash.
  • This disk replies to requests before any changes made by that requests have been committed to the storage (async.
  • This disk allows access from clients that dont use a reserved port for NFS (insecure.
  1. 'iptables' is the firewall of Openwrt. Let's add rules that allow the NFS port 2049 and portmap port 111 to be allowed to accept TCP/UDP data from 192.168.1.2/24. Edit /etc/firewall.user file with vim /etc/firewall.user to add:
IPT=iptables
NET_LAN=192.168.1.2/24
IF_LAN=eth0

# portmap
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p tcp --dport 111
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p udp --dport 111

# nfsd
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p tcp --dport 32777:32780
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p udp --dport 32777:32780
  1. Alright! time to fire up the NFS program! Run:
  • /etc/init.d/portmap enable # Enables the service to start everytime your router boots
  • etc/init.d/portmap start # Starts the service
  • /etc/init.d/unfs3 enable
  • /etc/init.d/unfs3 start
  1. If all went well you should be well to see the folders you have exposed from the router on your computer. In this case on OSX, I ran:
  • rpcinfo -p 192.168.1.1

e.g output:

➜  ~  rpcinfo -p 192.168.1.1
  program vers proto   port
   100000    2   tcp    111  portmapper
   100003    3   udp   2049  nfs
   100003    3   tcp   2049  nfs
   100005    1   udp   2049  mountd
   100005    3   udp   2049  mountd
   100005    1   tcp   2049  mountd
   100005    3   tcp   2049  mountd
  • showmount -e 192.168.1.1

e.g output:

➜  ~  showmount -e 192.168.1.1
Exports list on 192.168.1.1:
/mnt/hdd                            192.168.1.2/24

  1. Woo! Almost there. Now try mounting the NFS share manually with: sudo mount -t nfs 192.168.1.1:/mnt/hdd/ /Volumes/hdd. In 'Finder' program, press Command + Shift + G, type '/Volumes'. You should see a 'hdd' directory, see if you can view its contents, edit, read, write, update, delete modify the contents. If you can, success! http://i.imgur.com/HyUrtcRl.png

  2. If you want OSX to automount your NFS share every time you boot, then:

  • Mount manually as above
  • In OSX, go to Settings > Users & Groups > Login Items. Click + and select the NFS share you mounted.
  • Logout and/or reboot and repeat the check on mounted dir ('/Volumes/hdd') to see if you can read,write,update,modify the contents correctly!

  1. If you have come this far, then awesome! buckle up cause we are about to get elite by making our router run a bittorrent daemon! This is cool becuase your router can chug away on the downloads even as you leave home with your laptop/machine! (your HDD is attached to your router, remember? ;)). I followed instructions from here.

  2. On your router: opkg update, opkg install transmission-daemon transmission-web. Turn it on with /etc/init.d/transmission enable, /etc/init.d/transmission start. Add the following to your /etc/firewall.user file:

#bittorrent
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p tcp --dport 9091 
$IPT -I INPUT -j ACCEPT -i $IF_LAN -s $NET_LAN -p udp --dport 9091

Restart firewall with /etc/init.d/firewall restart.

  1. Configure transmission by editing /etc/config/transmission file. Mostly you need to set

option download_dir '/path/to/your/usb/disk'
option incomplete_dir '/path/to/your/usb/disk'

  1. Now access transmission on http://192.168.1.1:9091.

  2. In order to download faster, open the 51413 port number by adding the following to your /etc/config/firewall file:

config rule
        option target 'ACCEPT'
        option src 'wan'
        option proto 'tcpudp'
        option dest_port '51413'
        option name 'Transmission'

Restart with /etc/init.d/transmission restart, /etc/init.d/firewall restat. Check in the web UI of Transmission in the Settings, under Network tab should say 'Port Open'.

Running youtube-dl as server

I want more!

What would be even more cool? Being able to use your ISP's Internet on demand remotely :O In layman terms this translates to say you are away from home and you want to put some file for download on your router...

Now we know that when we connect to the Internet, our ISP gives us a IP address. This identifies your activity on the web, it's your computer's address. Some ISP's give their user's a static IP that is, you always have the same IP address every time you connect to the Internet, some assign dynamic IPs, that is you have a different IP address every time you connect to the internet. Most ISP's give out dynamic IP addresses. So it's hard to really tell at which address to reach my computer from the Internet.

I hear that the way to access your computer with a dynamic IP is 'reverse DNS'. The way to access your ISP's network from the internet is to use this revese DNS'ed address and a program called 'OpenVPN', where VPN is Virtual Private Network' to tunnel your bandwidth needs from another network to yours.

I haven't yet poked around these options but when I do, I will document it here :) Until then please comment to fix my glaring mistakes in this document.

Who are you??

If you liked this, holler me at sindhu@alum.hackerschool.com. If live in Bangalore and want to give me a high-five then you'll find me at upcoming HasGeek conferences :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment