Ma/mscp
1
0
Fork 0
mirror of https://github.com/upa/mscp.git synced 2026-02-11 15:34:53 +08:00
Code Issues Packages Projects Releases 1 Wiki Activity
293 Commits 5 Branches 22 Tags
eb5a9e30352f7b3aec6dcfced736192b6d12cdae
Commit Graph

158 Commits

Author SHA1 Message Date
Ryo Nakamura
2b71c4bf8c add -P, equivalent to -p for just compatibility 
and add a test case for -p and -P
 2023-11-25 15:49:09 +09:00
Ryo Nakamura
0cf3acee20 add -I interval option 
-I INTERVAL option inserts sleep for interval (seconds) between
SSH connection attempts (issue #7).
 2023-11-25 15:17:33 +09:00
Ryo Nakamura
8e590da322 fix parsing user@host:path. 
This commit fixes issue #6. Now mscp command correctly parses
[x::x] IPv6 address notation in hostname.
 2023-11-01 19:54:18 +09:00
Ryo Nakamura
b298b2ec35 main: adopt rolling average of recent eight bps values to calculate ETA 2023-11-01 19:54:18 +09:00
Ryo Nakamura
05a7e96759 main: call only mscp_stop() when receives sigint 2023-11-01 19:54:18 +09:00
Ryo Nakamura
139ba12f1a write total transferred bytes and number of files 
at the end of output when serverity is notice.
 2023-11-01 19:54:18 +09:00
Ryo Nakamura
cfbadebe6d change msg: thread[%d] to thread:%d 2023-11-01 19:54:18 +09:00
Ryo Nakamura
d7365683a9 print 1st decimal point in the progress bar 2023-11-01 19:54:18 +09:00
Ryo Nakamura
bf74aa095a add -g option to specify TCP cc algorithm 
This commit introduce SSH_OPTIONS_CCALGO option to the libssh patch
and add -g CONGESTION option to mscp.
 2023-11-01 19:54:18 +09:00
Ryo Nakamura
72841ec12d fix: use off_t for lseek 2023-09-08 17:19:13 +09:00
Ryo Nakamura
11e024c1da fix libmscp python bindings. 
- fix libmscp install path by setup.py with data_files
- fix return values of mscp_get_stats()
- add examples directory for mscp python binding
 2023-08-30 21:24:00 +09:00
Ryo Nakamura
13ec652195 fix mscp_opendir, do not use tls_sftp, use sftp isntead. 
The fixed issue causes mscp_opendir wrongly calls opendir() for
local when tls_sftp is NULL although sftp is not NULL.
 2023-08-30 19:09:29 +09:00
Ryo Nakamura
58026790d9 fix usage: "none" is not supported for -F 2023-08-04 16:11:29 +09:00
Ryo Nakamura
23d9577bde introduce git-based versioning 
MSCP_BUILD_VERSION (`git describe --tags --dirty --match "v*"`) is
passed through include/mscp_version.h.in and cmake.

When git is failed, use VERSION file instead (for building from
source tar balls that excludes .git).
 2023-08-04 16:07:37 +09:00
Ryo Nakamura
24c1bc9149 do not set O_TRUNC when opening destination file. 
It prevents `mscp localhost:hoge ~/hoge` from truncating the source
file. See https://bugzilla.mindrot.org/show_bug.cgi?id=3431.

https://github.com/upa/mscp/issues/1
 2023-08-04 15:06:14 +09:00
Ryo Nakamura
518aa42208 add -F ssh_config option 2023-08-04 13:31:10 +09:00
Ryo Nakamura
fbc817213b use pseudo glob/globfree for remote-glob when musl 
musllibc does not implement GLOB_ALTDIRFUNC, so do not call
glob for remote sides when libc is musl.

test_e2e.py skips test_glob_src_path when running on alpine.
 2023-08-03 21:59:54 +09:00
Ryo Nakamura
ba6f53d253 add glob for source paths 
https://github.com/upa/mscp/issues/3
 2023-08-03 20:26:13 +09:00
Ryo Nakamura
9f7c135b15 cleanup wrappers for file operations 
Previously wrapper functions for open(), opendir(), and stat(), etc,
are implemneted in path.h, and now they are in fileops.h and fileops.c.
This commit is a reparation for remote glob.
 2023-08-03 17:07:39 +09:00
Ryo Nakamura
24e86f58d8 mscp: maintain mscp_thread structs in list 
Instead of m->threads array, struct mscp_thread instanes are
maintained in m->thread_list. This enables stable counter access
via mscp_get_stats().
 2023-05-07 21:05:05 +09:00
Ryo Nakamura
1d3b3a2261 main: add a white space to the elapsed time output 
It adjusts the position of XX:XX in elapsed timeou output.
 2023-04-05 19:07:10 +09:00
Ryo Nakamura
575c920b6e main: print elapsed time instead ETA at the end 2023-04-05 19:00:29 +09:00
Ryo Nakamura
1bd832a135 Merge branch 'main' of github.com:upa/mscp 2023-03-26 01:50:00 +09:00
Ryo Nakamura
834407379d fix error handling when scan thread failed. 
set chunk pool to fill to invoke copy threads when scan failed.
 2023-03-25 22:29:09 +09:00
Ryo Nakamura
8192151154 fix invalid return sem 2023-03-22 18:06:19 +09:00
Ryo Nakamura
e0e6fae296 do not sem_close() for unnamed semaphore 2023-03-16 01:01:46 +09:00
Ryo Nakamura
6305f02770 fix semaphore handling for macOS 2023-03-16 00:03:22 +09:00
Ryo Nakamura
ae4b848ba0 add sem_create(), wrappign sem_init() for linux and sem_open() for macOS 2023-03-15 23:54:57 +09:00
Ryo Nakamura
3902fb584a linux also needs stdlib.h for random() 2023-03-15 23:26:11 +09:00
Ryo Nakamura
f0c70a148b macOS does not support sem_init. use sem_open instead 2023-03-15 23:18:33 +09:00
Ryo Nakamura
f5d0f526f2 add comment to mscp_scan(), why usleep(100) 2023-03-15 22:19:09 +09:00
Ryo Nakamura
a086e6a154 rename mscp_prepare to mscp_scan 2023-03-15 22:03:14 +09:00
Ryo Nakamura
3bce4ec277 set m->tid_prepare 0 to avoid duble join 2023-03-15 21:56:46 +09:00
Ryo Nakamura
a923d40ada mscp: add -u max_startups option. 
pymscp also accepts keyword 'max_startups' (int).
 2023-03-15 21:53:34 +09:00
Ryo Nakamura
24fef5f539 fix: when msg_fd is 0, use STDOUT_FILENO 2023-03-15 01:35:55 +09:00
Ryo Nakamura
4e80b05da7 do not fdopen(msg_fd) if msg_fd < 0 2023-03-15 00:39:56 +09:00
Ryo Nakamura
98eca409af introduce semaphore for concurrent connecting ssh 
instead of ssh_estab_queue (delay-based approach). MaxStartups in
sshd_config limits number of conccurent incoming ssh connections.
mscp_opts->max_startups adjusts this value.
 2023-03-15 00:35:48 +09:00
Ryo Nakamura
cf99a439cb cleanup message print functions 2023-03-15 00:00:23 +09:00
Ryo Nakamura
3077bb0856 rename ssh_connect_flag to ssh_estab_queue 2023-03-14 01:20:55 +09:00
Ryo Nakamura
72c27f16d6 implement ssh_connect_flag 
Each copy thread establishes SSH/SFTP connection to remote host.
A delay is inserted between SSH connecting to the remote.
 2023-03-14 00:43:53 +09:00
Ryo Nakamura
9b0eb668f9 cleanup mscp_prepare-related code 2023-03-14 00:11:13 +09:00
Ryo Nakamura
5f9f20f150 mscp_prepare() scans source paths in a thread. 
This commit runs mscp_prepare() in a pthread. mscp copy threads
run aysnchronously with mscp_prepare(). So, when mscp_prepare()
has not finished yet (due to too many source files), we can start
to copy files.
 2023-03-13 22:35:51 +09:00
Ryo Nakamura
ceb9ebd5a8 revise walk_src_path. 
In new walk_src_path, resolve dst path and resolve chunks are
invoked when adding a path.
 2023-03-13 21:02:26 +09:00
Ryo Nakamura
cafbefe48c fix mscp.py 2023-03-13 14:53:54 +09:00
Ryo Nakamura
597a7a8cba little cleanup for python bindings 2023-03-12 17:39:51 +09:00
Ryo Nakamura
2416b5f182 fix cleanup 2023-03-12 17:06:02 +09:00
Ryo Nakamura
10812071aa mscp copy via python bindings works. 
ToDo: memory for struct instance in pymscp.c is not released until
mscp.free() is called. It is memory leak in a typical pyhotn usage.
Use python extension refcnt instead.
 2023-03-12 00:05:39 +09:00
Ryo Nakamura
8ea05729c2 add mscp and pymscp python modules. 
pymscp is a C Python wrapper for libmscp functions. mscp module
provides simple (?) Python API.
 2023-03-11 22:49:19 +09:00
Ryo Nakamura
74d58e986a move direction from mscp_opts to mscp_init argument 2023-03-11 21:48:07 +09:00
Ryo Nakamura
7e7bc61ff2 start to implement pymscp.c 2023-03-11 20:54:45 +09:00