Ma/mscp
1
0
Fork 0
mirror of https://github.com/upa/mscp.git synced 2026-02-14 00:54:43 +08:00
Code Issues Packages Projects Releases 1 Wiki Activity
306 Commits 5 Branches 22 Tags
020dadfbafed638ad3674325acbd5db4d136ac53
Commit Graph

44 Commits

Author SHA1 Message Date
Ryo Nakamura
9bdbfe8487 add SPDX-License-Identifier to source files 2024-01-18 12:59:49 +09:00
Ryo Nakamura
bba53fab03 don't allocate char[PATH_MAX] for each file 
This commit makes struct path allocation use strndup().
It reduices the memory footprint for struct path per file (issue #8).
 2024-01-18 12:59:49 +09:00
Ryo Nakamura
0cf3acee20 add -I interval option 
-I INTERVAL option inserts sleep for interval (seconds) between
SSH connection attempts (issue #7).
 2023-11-25 15:17:33 +09:00
Ryo Nakamura
139ba12f1a write total transferred bytes and number of files 
at the end of output when serverity is notice.
 2023-11-01 19:54:18 +09:00
Ryo Nakamura
cfbadebe6d change msg: thread[%d] to thread:%d 2023-11-01 19:54:18 +09:00
Ryo Nakamura
fbc817213b use pseudo glob/globfree for remote-glob when musl 
musllibc does not implement GLOB_ALTDIRFUNC, so do not call
glob for remote sides when libc is musl.

test_e2e.py skips test_glob_src_path when running on alpine.
 2023-08-03 21:59:54 +09:00
Ryo Nakamura
ba6f53d253 add glob for source paths 
https://github.com/upa/mscp/issues/3
 2023-08-03 20:26:13 +09:00
Ryo Nakamura
9f7c135b15 cleanup wrappers for file operations 
Previously wrapper functions for open(), opendir(), and stat(), etc,
are implemneted in path.h, and now they are in fileops.h and fileops.c.
This commit is a reparation for remote glob.
 2023-08-03 17:07:39 +09:00
Ryo Nakamura
24e86f58d8 mscp: maintain mscp_thread structs in list 
Instead of m->threads array, struct mscp_thread instanes are
maintained in m->thread_list. This enables stable counter access
via mscp_get_stats().
 2023-05-07 21:05:05 +09:00
Ryo Nakamura
834407379d fix error handling when scan thread failed. 
set chunk pool to fill to invoke copy threads when scan failed.
 2023-03-25 22:29:09 +09:00
Ryo Nakamura
ae4b848ba0 add sem_create(), wrappign sem_init() for linux and sem_open() for macOS 2023-03-15 23:54:57 +09:00
Ryo Nakamura
f0c70a148b macOS does not support sem_init. use sem_open instead 2023-03-15 23:18:33 +09:00
Ryo Nakamura
f5d0f526f2 add comment to mscp_scan(), why usleep(100) 2023-03-15 22:19:09 +09:00
Ryo Nakamura
a086e6a154 rename mscp_prepare to mscp_scan 2023-03-15 22:03:14 +09:00
Ryo Nakamura
3bce4ec277 set m->tid_prepare 0 to avoid duble join 2023-03-15 21:56:46 +09:00
Ryo Nakamura
a923d40ada mscp: add -u max_startups option. 
pymscp also accepts keyword 'max_startups' (int).
 2023-03-15 21:53:34 +09:00
Ryo Nakamura
24fef5f539 fix: when msg_fd is 0, use STDOUT_FILENO 2023-03-15 01:35:55 +09:00
Ryo Nakamura
4e80b05da7 do not fdopen(msg_fd) if msg_fd < 0 2023-03-15 00:39:56 +09:00
Ryo Nakamura
98eca409af introduce semaphore for concurrent connecting ssh 
instead of ssh_estab_queue (delay-based approach). MaxStartups in
sshd_config limits number of conccurent incoming ssh connections.
mscp_opts->max_startups adjusts this value.
 2023-03-15 00:35:48 +09:00
Ryo Nakamura
cf99a439cb cleanup message print functions 2023-03-15 00:00:23 +09:00
Ryo Nakamura
3077bb0856 rename ssh_connect_flag to ssh_estab_queue 2023-03-14 01:20:55 +09:00
Ryo Nakamura
72c27f16d6 implement ssh_connect_flag 
Each copy thread establishes SSH/SFTP connection to remote host.
A delay is inserted between SSH connecting to the remote.
 2023-03-14 00:43:53 +09:00
Ryo Nakamura
9b0eb668f9 cleanup mscp_prepare-related code 2023-03-14 00:11:13 +09:00
Ryo Nakamura
5f9f20f150 mscp_prepare() scans source paths in a thread. 
This commit runs mscp_prepare() in a pthread. mscp copy threads
run aysnchronously with mscp_prepare(). So, when mscp_prepare()
has not finished yet (due to too many source files), we can start
to copy files.
 2023-03-13 22:35:51 +09:00
Ryo Nakamura
ceb9ebd5a8 revise walk_src_path. 
In new walk_src_path, resolve dst path and resolve chunks are
invoked when adding a path.
 2023-03-13 21:02:26 +09:00
Ryo Nakamura
2416b5f182 fix cleanup 2023-03-12 17:06:02 +09:00
Ryo Nakamura
74d58e986a move direction from mscp_opts to mscp_init argument 2023-03-11 21:48:07 +09:00
Ryo Nakamura
2477647a3b fix uninitialized dst_path_is_dir bool 2023-03-10 21:42:00 +09:00
Ryo Nakamura
d5a86292b7 add doxygen for mscp.h 2023-03-04 18:47:44 +09:00
Ryo Nakamura
e67b7468e5 use setitimer instead of alarm, and print message. 
print_stat now prints messages per interval.
ToDo:
- realtime message printing
- use timer_create instead of setitimer (mscOS has different one)
 2023-03-04 16:48:26 +09:00
Ryo Nakamura
9342c18f0e accidentaly swap min_chunk_sz and max_chunk_sz... 2023-03-04 15:53:42 +09:00
Ryo Nakamura
df2f922b0d remove pprint, use message (mpr_*) instead. 
ToDo:
main should use pipe to receive messages from libmscp.
 2023-03-04 15:44:10 +09:00
Ryo Nakamura
1e57e8fb2f implementing messaging. 
ToDo: remove pprint.
mscp should use mpr_* functions, and main.c should use
just fprintf(stdout, "\r\033[K" fmt, ...) for printing progress bar.
 2023-03-03 22:14:54 +09:00
Ryo Nakamura
1b9ae51974 add message.h and message.c, mscp_set|get_error() 
Instead of pr_err(), libmscp uses mscp_set_error() and
applications use mscp_get_errror() to get error message.
 2023-03-03 21:29:43 +09:00
Ryo Nakamura
c5aa70d9c9 tiny cleanup 2023-03-03 18:30:34 +09:00
Ryo Nakamura
a0b7482f66 add mscp_get_stats 
move progress bar-related functions from mscp.c to main.c.
 2023-03-03 18:27:14 +09:00
Ryo Nakamura
363296f499 add mscp_ssh_opts and change -C optarg 2023-03-03 16:50:06 +09:00
Ryo Nakamura
fc45fa2532 add comments to mscp.h 2023-02-26 23:56:57 +09:00
Ryo Nakamura
c649742b3e fix dst path resolve 2023-02-26 23:42:25 +09:00
Ryo Nakamura
700d64b375 now mscp links libmscp 2023-02-26 23:18:39 +09:00
Ryo Nakamura
2bad21bdc2 set default params in mscp_init 2023-02-26 18:43:24 +09:00
Ryo Nakamura
89777032cd have written mscp.c 2023-02-26 18:17:58 +09:00
Ryo Nakamura
3d26cc2c18 add copy-related functions to path 2023-02-25 23:39:20 +09:00
Ryo Nakamura
1be9b70808 start to impliment mscp as a library 
this commit starts to refactor file.h|c to path.h|c and
add mscp.c|h. not completed yet.
 2023-02-25 22:17:29 +09:00