Ma/mscp
1
0
Fork 0
mirror of https://github.com/upa/mscp.git synced 2026-02-04 03:24:58 +08:00
Code Issues Packages Projects Releases 1 Wiki Activity

update README

Browse Source
This commit is contained in:
Ryo Nakamura
2022-11-13 18:14:06 +09:00
parent 613961b71d
commit b8e204ae41
1 changed files with 59 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
README.md
View File

@@ -3,22 +3,23 @@
[![build on ubuntu](https://github.com/upa/mscp/actions/workflows/build-ubuntu.yml/badge.svg)](https://github.com/upa/mscp/actions/workflows/build-ubuntu.yml) [![build on macOS](https://github.com/upa/mscp/actions/workflows/build-macos.yml/badge.svg)](https://github.com/upa/mscp/actions/workflows/build-macos.yml) [![test](https://github.com/upa/mscp/actions/workflows/test.yml/badge.svg)](https://github.com/upa/mscp/actions/workflows/test.yml)
`mscp`, a variant of `scp`, copies files over multiple ssh (sftp)
`mscp`, a variant of `scp`, copies files over multiple ssh (SFTP)
sessions. Multiple threads in mscp transfer (1) multiple files
simultaneously and (2) a large file in parallel. It may shorten the
waiting time for transferring a lot of/large files over networks.
You can use `mscp` like `scp`, for example, `mscp example.com:srcfile
/tmp/dstfile`. Remote hosts only need to run `sshd` supporting the
SFTP subsystem, and you need to be able to ssh to the hosts (as
usual).
You can use `mscp` like `scp`, for example, `mscp
user@example.com:srcfile /tmp/dstfile`. Remote hosts only need to run
normal `sshd` supporting the SFTP subsystem, and you need to be able
to ssh to the hosts (as usual). `mscp` does not require anything else.
Differences from `scp` are:
- remote glob on remote shell expansion is not supported.
- remote to remote copy is not supported.
- `-r` option is not needed.
- and any other differences I have not noticed and implemented...
- and any other differences I have not implemented and noticed...
## Install
@@ -29,8 +30,10 @@ Differences from `scp` are:
brew install upa/tap/mscp
```
- Linux: Download a package for your environment from
[Releases page](https://github.com/upa/mscp/releases).
- Linux
Download a package for your environment from [Releases
page](https://github.com/upa/mscp/releases).
## Build from source
@@ -72,50 +75,30 @@ make install
- Usage
```shell-session
```console
$ mscp
mscp v0.0.0: copy files over multiple ssh connections
Usage: mscp [vqDCHdh] [-n nr_conns]
[-s min_chunk_sz] [-S max_chunk_sz]
[-b sftp_buf_sz] [-B io_buf_sz]
[-b sftp_buf_sz] [-B io_buf_sz] [-a nr_ahead]
[-l login_name] [-p port] [-i identity_file]
[-c cipher_spec] source ... target
-n NR_CONNECTIONS number of connections (default: half of # of cpu cores)
-s MIN_CHUNK_SIZE min chunk size (default: 64MB)
-S MAX_CHUNK_SIZE max chunk size (default: filesize / nr_conn)
-b SFTP_BUF_SIZE buf size for sftp_read/write (default 131072B)
-B IO_BUF_SIZE buf size for read/write (default 131072B)
Note that this value is derived from
qemu/block/ssh.c. need investigation...
-v increment verbose output level
-q disable output
-D dry run
-l LOGIN_NAME login name
-p PORT port number
-i IDENTITY identity file for publickey authentication
-c CIPHER cipher spec, see `ssh -Q cipher`
-C enable compression on libssh
-H disable hostkey check
-d increment ssh debug output level
-h print this help
```
- Example: copy an 8GB file on tmpfs over a 100Gbps link
- Two Intel Xeon Gold 6130 machines directly connected with Intel E810 100Gbps NICs.
```shell-session
```console
$ mscp /tmp/test.img 10.0.0.1:/tmp/
[===============================================================] 100% 8GB/8GB 3.02GB/s
[=====================================================] 100% 8GB/8GB 3.02GB/s
```
- `-v` options increment verbose output level.
- `-v` option increments verbose output level.
```shell-session
```console
$ mscp test 10.0.0.1:
[===============================================================] 100% 13B/13B 2.41KB/s
[=====================================================] 100% 13B/13B 2.41KB/s
$ mscp -v test 10.0.0.1:
file test/test.txt (local) -> ./test/test.txt (remote) 9B
@@ -127,7 +110,7 @@ copy start: test/test2/2.txt
copy done: test/1.txt
copy done: test/test2/2.txt
copy done: test/test.txt
[===============================================================] 100% 13B/13B 2.51KB/s
[=====================================================] 100% 13B/13B 2.51KB/s
$ mscp -vv -n 4 test 10.0.0.1:
connecting to 10.0.0.1 for checking destinations...
@@ -144,8 +127,46 @@ copy start: test/test2/2.txt
copy done: test/test.txt
copy done: test/test2/2.txt
copy done: test/1.txt
[===============================================================] 100% 13B/13B 3.27KB/s
[=====================================================] 100% 13B/13B 3.27KB/s
```
- Full usage
```console
$ mscp -h
mscp v0.0.0: copy files over multiple ssh connections
Usage: mscp [vqDCHdh] [-n nr_conns]
[-s min_chunk_sz] [-S max_chunk_sz]
[-b sftp_buf_sz] [-B io_buf_sz] [-a nr_ahead]
[-l login_name] [-p port] [-i identity_file]
[-c cipher_spec] source ... target
-n NR_CONNECTIONS number of connections (default: half of # of cpu cores)
-s MIN_CHUNK_SIZE min chunk size (default: 64MB)
-S MAX_CHUNK_SIZE max chunk size (default: filesize / nr_conn)
-b SFTP_BUF_SIZE buf size for sftp_read/write (default 131072B)
-B IO_BUF_SIZE buf size for read/write (default 131072B)
Note that the default value is derived from
qemu/block/ssh.c. need investigation...
-b and -B affect only local to remote copy
-a NR_AHEAD number of inflight SFTP read commands (default 16)
-v increment verbose output level
-q disable output
-D dry run
-l LOGIN_NAME login name
-p PORT port number
-i IDENTITY identity file for publickey authentication
-c CIPHER cipher spec, see `ssh -Q cipher`
-C enable compression on libssh
-H disable hostkey check
-d increment ssh debug output level
-h print this help
```
Note: mscp is still under development, and the author is not
responsible for any accidents on mscp.
responsible for any accidents due to mscp.