Skip to content

Instantly share code, notes, and snippets.

@798886844
Created October 1, 2018 14:08
Show Gist options
  • Save 798886844/49366e52a7dac4e8f014ba3ecc2ce2b9 to your computer and use it in GitHub Desktop.
Save 798886844/49366e52a7dac4e8f014ba3ecc2ce2b9 to your computer and use it in GitHub Desktop.
VPS+Rclone实现网盘对拷
网盘的稳定性一直是网盘用户的痛点。
360倒闭之后许多文件来不及下载,而后又没有合适的地方存储,导致很多人咬牙掏钱买硬盘。然而更多没钱的人选择了其他网盘,比如我,最后选择了OnedriveForBusiness。
但是无论数据存储在哪里,都有丢失的时候,存硬盘里面硬盘会坏,存Onedrive里面……额,订阅哪来的你们也清楚,所以做好数据备份很重要。
所以我一直在寻找这样一个方法,能够帮我完成Onedrive For Business网盘对拷功能。一开始尝试了Flow,然而不知道怎么回事,最开始还能用,到后面就不行了,估计是用的人太多了吧。然后又选择了MultCloud,然而这货有2T流量限制,速度还死慢,作为没钱的我,只好选择了第三条路,利用手头的资源自己搭建。
经过不懈的努力摸索,今晚洗澡的时候我终于萌发出灵感:利用Rclone和VPS进行网盘对拷。下面我具体来说说吧:
搭建环境:Docker环境,镜像:[rastasheep/ubuntu-sshd](https://hub.docker.com/r/rastasheep/ubuntu-sshd/)
其中,该镜像的环境是
```c#
ubuntu-sshd-910292714-kpnx7 4.4.0-72-generic
用户名、密码均为:root
```
需要开放的端口包括:22、80、6800,但不限于以上端口。
好了,接下来开始操作:
### Reclone安装与挂载
关于Reclone挂在网盘的操作方法,我主要参考了以下两篇文章:
* [可乐博客-通过Rclone及Oneindex实现Onedrive离线下载教程](https://www.cokemine.com/rclone-onedrive-oneindex.html)
*
* [Rat'sBlog-在Debian/Ubuntu上使用rclone挂载OneDrive网盘](https://www.moerats.com/archives/491/)
有问题的可以去原帖看一下。
下面开始说我的操作:
首先,需要一台Windows电脑:
打开[https://rclone.org/downloads/](https://rclone.org/downloads/),下载Windows-AMD64 - 64 Bit压缩包文件,并解压至`D:\rclone`。
然后打开管理员命令提示符,输入
```C#
cd /d d:\rclone
rclone authorize "onedrive"
```
登陆你的账户,获取code,例如我的:
```language
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayIsImtpZCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayJ9.eyJhdWQiOiJodHRwczovL2FwaS5vZmZpY2UuY29tL2Rpc2NvdmVyeS8iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9kNWEzODc3YS02ZjBkLTQ4NzUtOWUyMS03MjZhMWJkOTRjMTUvIiwiaWF0IjoxNTI3MzIwOTczLCJuYmYiOjE1MjczMjA5NzMsImV4cCI6MTUyNzMyNDg3MywiYWNyIjoiMSIsImFpbyI6IlkyZGdZRmgrdjYxRFRIcm5xcm1OOVpIbFNxWDNsRU92ekZrMjd3ZXIzTXprMUdXK0JxOEEiLCJhbXIiOlsicHdkIl0sImFwcGlkIjoiNTI4NTdmZWMtNGJjMi00ODNmLTlmMWItNWZlMjhlOTc1MzJjIiwiYXBwaWRhY3IiOiIxIiwiaXBhZGRyIjoiMTEwLjE4NC4xNzkuMTMwIiwibmFtZSI6InRlc3QiLCJvaWQiOiI2YTQzMGIzOC01N2VhLTQxZTQtYWI0My1iZDhiZTZmNzUwZmIiLCJwdWlkIjoiMTAwMzNGRkZBQjlDMEQxRSIsInNjcCI6Ik15RmlsZXMuUmVhZCBNeUZpbGVzLldyaXRlIFVzZXIuUmVhZCIsInN1YiI6IjlseWt4dlBNUGNmT2FOWVlqQUNkbDNyaUs1YkY1anFUS0RmY003SDg5QjAiLCJ0aWQiOiJkNWEzODc3YS02ZjBkLTQ4NzUtOWUyMS03MjZhMWJkOTRjMTUiLCJ1bmlxdWVfbmFtZSI6InRlc3RAZ2hkZ2pkZi5vbm1pY3Jvc29mdC5jb20iLCJ1cG4iOiJ0ZXN0QGdoZGdqZGYub25taWNyb3NvZnQuY29tIiwidXRpIjoiV28yOE1pWGpRa1doTGQ0RGdKd0VBQSIsInZlciI6IjEuMCJ9.QtuYv9vpSYZSNNlRDLYlf0MjLo399aWz26PIVX7lz9FWceeCkydGjMtrWhb7S5qgy2Z-yqnJnN-XpZAdlztM7x5CSqbPNSX1n95YNxnW6Bl423rdosIw6xjDkBQf7jFvPutlGgVkNEmzk-znmJwSDNtsEY8j5LUzLrtWpNweAWK-I35TT5hb6ejXP66NsJc3yDP7-Qy3fhUX402vZl4NZCMkb1G4DVm6dNoWihxcZWhrP91eYqfMy_XlaZhexyD0NUqV2O28y3Rs_aGn2MMJonTBOdj18vV31DBiTooD-ImA3W69gDqThH85Ql8eRoMZ1j76hwELLPKPA7-UjCmkBg","token_type":"Bearer","refresh_token":"AQABAAAAAADX8GCi6Js6SK82TsD2Pb7rnd9hYhHoPmWiPciFQK-L_jFPvodZQ9cadNF6AYwGPH73A-rW3fUWBZ6ZIRNzWeqXotbtGERnsbsY9oSv2dESZf0PAZ0SmlhliTB55p-UHMBhhA02RZzqRO-VYQxC6PYMP20P8PBSvqS7BvkurelSog8L4EpncgyQWpof16aHz6mVgJ3CD-OA7m9zmF3k41TSNhoYxeDznfe4eUYe0vHwbz7Ubfv7xpNwiXKw2JuqkL_UikS-Nmq3_ls-suRZ0VDQxRT7uEc_Z-0H4PeIVH2Cvai7EoQ2WVfZXFZzV8qS_ryjbWspnO6u2_WyReEDUIukHR7BILUOVi141i1L6VXEpw6S-x4TGqKf5q88-BPtbmNz9IwO7nwIVkGe5vY111yVYbQg3j3JzbW0eReByKnSETyypovMEEk_08TYJdD-Ks7AByAUolDrZhFuEUDZlh4VGNNG5DGTf4opoaSNYNt4Rkf9R2b1rL46x66DOBGD2cRMtSUAuy1pNSt1YdddYwV4tiiY15XzCdGiY8Hm39gw9T3eVP1ja_SaQRCiKpZfr63BUO3Hzt1DITgmYqUC_NtDzcyG3c_lbnUrCl4QB2h5f37U8cdRrHJ0uzWwezFL80xmLCdA1Xl-V-OEKjqvWvBNuitNuVbX_20V7wTQgQWSXcwOU1I_BYOwI1aCUmjQBax2GGJQDSVAPu4jJVQOG-7FV8S2__5-ushbSsCqVmqi14ECKQPsm_4pwuDx9W8Tphpbw68fVgSHn3eJqAunovydIAA","expiry":"2018-05-26T16:54:32.8494162+08:00"}
```
这一段保存好,可以反复使用。
然后打开shell工具,连接你的VPS,开始挂载:
安装相关软件:
```language
apt-get update
apt-get install -y unzip fuse vim
```
安装Reclone:
```c#
wget https://downloads.rclone.org/rclone-current-linux-amd64.zip
unzip rclone-current-linux-amd64.zip
chmod 0777 ./rclone-*/rclone
cp ./rclone-*/rclone /usr/bin/
rm -rf ./rclone-*
```
配置Reclone:
`rclone config`
然后出现以下内容:
```c#
2018/05/26 08:09:11 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n #选择"New remote"
name> OneDrive #自定义网盘名称
Type of storage to configure.
Choose a number from below, or type in your own value
1 / Alias for a existing remote
\ "alias"
2 / Amazon Drive
\ "amazon cloud drive"
3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
\ "s3"
4 / Backblaze B2
\ "b2"
5 / Box
\ "box"
6 / Cache a remote
\ "cache"
7 / Dropbox
\ "dropbox"
8 / Encrypt/Decrypt a remote
\ "crypt"
9 / FTP Connection
\ "ftp"
10 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
11 / Google Drive
\ "drive"
12 / Hubic
\ "hubic"
13 / Local Disk
\ "local"
14 / Mega
\ "mega"
15 / Microsoft Azure Blob Storage
\ "azureblob"
16 / Microsoft OneDrive
\ "onedrive"
17 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
18 / Pcloud
\ "pcloud"
19 / QingCloud Object Storage
\ "qingstor"
20 / SSH/SFTP Connection
\ "sftp"
21 / Webdav
\ "webdav"
22 / Yandex Disk
\ "yandex"
23 / http Connection
\ "http"
Storage> 16 #选择网盘类型,这里选择16:Onedrive
Microsoft App Client Id - leave blank normally.
client_id> #留空
Microsoft App Client Secret - leave blank normally.
client_secret> #留空
Remote config
Choose OneDrive account type?
* Say b for a OneDrive business account
* Say p for a personal OneDrive account
b) Business
p) Personal
b/p> b #我是教育版,属于Onedrive For Business,故选择“b) Business”
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n #因为是远程操作,所以选择“n) No”
For this to work, you will need rclone available on a machine that has a web browser available.
Execute the following on your machine:
rclone authorize "onedrive"
Then paste the result below:
result> #粘贴上面那一大串code,然后会提示下面的数据请你确认
--------------------
[OneDrive]
type = onedrive
client_id =
client_secret =
token =
resource_url =
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y 确认后选择 "y"
Current remotes:
Name Type
==== ====
OneDrive onedrive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q #如果需要连接第二个网盘,选择“n”,如果不需要添加其他网盘了,选择“q”退出。
```
重复上述步骤,再连接一个网盘。
到这里,已经完成了网盘在该VPS上的登录认证,并已经映射为一个叫做OneDrive的网盘。
完成上面对两个OFB网盘的连接后,开始对拷:
键入命令`rclone copy gdrive1: gdrive2:`
表示从gdrive1的根目录开始将所有文件拷贝到gdrive2,同时gdrive2里面的数据可能会被完全清空。
给大家看一下效果:
首先是我在对拷的过程中,用宝塔面板进行了后台服务性能的监控,大概峰值就在这个样子:
![峰值](https://ws1.sinaimg.cn/large/0068Q8Uggy1frp8jyswepj312k08iwfa.jpg)
可以看到其实对于CPU的使用并没有像病毒一样的疯狂占满,只在有的时候可能会稍微大一点,所以建议用空的小鸡对拷。另外可以看到,对本机的空间占用几乎没有,所以无论大鸡小鸡都能做到。
在看一下接口流量:
![接口流量](https://ws1.sinaimg.cn/large/0068Q8Uggy1frp8r5qpsqj30n10ie3zt.jpg)
可以看出,速度受下载和上传两个因素同时限制。
再看一下十分钟的拷贝速度和效果:
![gdrive1](https://ws1.sinaimg.cn/large/0068Q8Uggy1frp8n3ylsjj31hb0jagoc.jpg)
![gdrive2](https://ws1.sinaimg.cn/large/0068Q8Uggy1frp8ni9ay5j31hc0mf41t.jpg)
需要提醒的是,因为即使是对拷,用的也是服务器本机的流量,包括下载和上传的流量。所以如果你想拷50G的文件,就要花费至少100G的流量,这是需要计费的。
@zengyu24
Copy link

zengyu24 commented Nov 1, 2020

看到你的方法解决了我的freenas上jail 无法验证token的问题,我也分享个小技巧同样类型的网盘可以实现 sever-side copy 不走本地流量rclone copy "remote1:소스" "remote2:타겟" --drive-server-side-across-configs -P -v,不过跨网盘还是要本地流量哈,这是视频https://www.youtube.com/watch?v=3MRWENXyEb4&list=PLTanR3CoBkXphIx1vA-OHPrnHwqd3ZKvP&index=2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment