23 Commits

Author SHA1 Message Date
k4yt3x
ebd5e8eb3d fix(realesrgan): fixed incorrect type for ncnn::Mat elemsize
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-26 00:00:00 +00:00
k4yt3x
50d5cb16d1 docs(video2x): updated command line help messages
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-26 00:00:00 +00:00
k4yt3x
8eac1a7393 feat(libplacebo): added more modes for Anime4K v4
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-26 00:00:00 +00:00
k4yt3x
f0f3166d92 chore(models): moved model files into subdirectories
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-26 00:00:00 +00:00
k4yt3x
482e82f9c4 style(*): reduced implicit casting and other styling issues
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-25 00:00:00 +00:00
k4yt3x
58ea9e4b35 feat(cmake): removed CMake policy CMP0146 for compatibility
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-22 00:00:00 +00:00
k4yt3x
e077849a2c docs(readme): updated links for 6.0.0-beta.4
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-22 00:00:00 +00:00
k4yt3x
0fb547589d feat(*): updated Dockerfile and PKGBUILD for 6.0.0-beta.4
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-22 00:00:00 +00:00
k4yt3x
1f18b8507d feat(cmake): added installation rules for FFmpeg and OpenCV DLLs 2024-10-21 21:43:11 -04:00
k4yt3x
eb7d411f49 perf(libvideo2x): removed unnecessary read of nb_frames
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-22 00:00:00 +00:00
k4yt3x
640d9cd52b fix(encoder): fixed memory leak in the encoder
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-21 00:00:00 +00:00
K4YT3X
bc168d11ab feat(libvideo2x): use OpenCV to retrieve total frame count (#1194) 2024-10-21 23:54:22 +00:00
Owen Quinlan
e09f348890 Fix missing spdlog in docker build [Fix for #1185] (#1186)
* Fix missing spdlog in docker build
* fix: added the spdlog dependency for the Dockerfile building stage

---------

Co-authored-by: K4YT3X <i@k4yt3x.com>
2024-10-14 16:51:40 +00:00
k4yt3x
f3caf22765 docs(readme): updated links for 6.0.0-beta.3
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-14 00:00:00 +00:00
K4YT3X
cd2006b4d9 feat(*): switched to spdlog for logging and organized headers (#1183)
* feat: updated PKGBUILD description
* feat: updated workflow syntax and dependencies
* feat: switched logging to spdlog
* chore: adjusted library defaults
* ci: fixed spdlog format string issues
* docs: fixed docs for libvideo2x functions
* feat: organized header files
* fix: fixed header installation directory
* feat: link spdlog statically if compiled from source
* feat: adjusted libvideo2x log level enum names
* feat: added version.h header

Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-14 02:46:59 +00:00
k4yt3x
ecbc512711 ci(build): fixed librealesrgan library name
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-11 00:00:00 +00:00
k4yt3x
3fe1f6d544 docs(readme): updated Windows download link
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-11 00:00:00 +00:00
k4yt3x
ee4d6f360e chore(librealesrgan): updated librealesrgan submodule
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-10 00:00:00 +00:00
k4yt3x
7ae1943754 docs(readme): added EOL notice for versions 4 and 5
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-10 00:00:00 +00:00
k4yt3x
c2a4b25290 ci(build): fixed missing include in conversions.h
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-10 00:00:00 +00:00
k4yt3x
f79d4893db docs(readme): updated README for 6.0.0-beta.2
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-10 00:00:00 +00:00
K4YT3X
37c2c4c647 feat(*): added support for copying audio/subtitle streams and pause/abort (#1179)
* feat: added Makefile target for debian
* fix: fixed Dockerfile installing the wrong package
* feat: added hwaccel for encoder and decoder
* feat: added benchmark mode
* feat: removed hard-coded keyframe info
* chore: cleaned up headers and organized code
* style: cleaned up headers and includes
* feat: added a progress bar for CLI
* feat: removed atomicity requirements on processed frames
* feat: added pause and abort for CLI
* chore: updated default preset and crf settings
* feat: added support for copying audio and subtitle streams
* fix: fixed syntax issues for MSVC
* fix: fixed audio/subtitle timestamp rescaling

Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-10 07:23:13 +00:00
k4yt3x
c7fa9c10e6 docs(readme): fixed logo for GitHub light theme
Signed-off-by: k4yt3x <i@k4yt3x.com>
2024-10-08 00:00:00 +00:00
50 changed files with 12071 additions and 657 deletions

10
.github/FUNDING.yml vendored
View File

@@ -1,12 +1,2 @@
# These are supported funding model platforms
github: k4yt3x
patreon: k4yt3x
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

View File

@@ -6,6 +6,7 @@ on:
- dev
pull_request: {}
workflow_dispatch: {}
jobs:
ubuntu:
runs-on: ubuntu-latest
@@ -29,11 +30,12 @@ jobs:
libswscale-dev \
libvulkan-dev \
glslang-tools \
libomp-dev
libomp-dev \
libopencv-dev
- name: Build Video2X
run: |
mkdir -p /tmp/build /tmp/install
cmake -B /tmp/build -S . -DUSE_SYSTEM_NCNN=OFF \
cmake -B /tmp/build -S . -DUSE_SYSTEM_NCNN=OFF -DUSE_SYSTEM_SPDLOG=OFF \
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/install \
-DINSTALL_BIN_DESTINATION=. -DINSTALL_INCLUDE_DESTINATION=include \
@@ -44,6 +46,7 @@ jobs:
with:
name: video2x-nightly-linux-amd64
path: /tmp/install
windows:
runs-on: windows-latest
steps:
@@ -61,6 +64,7 @@ jobs:
run: |
$ffmpegVersion = "7.1"
$ncnnVersion = "20240820"
$opencvVersion = "4.10.0"
git submodule update --init --recursive
@@ -71,9 +75,16 @@ jobs:
curl -Lo ncnn-shared.zip "https://github.com/Tencent/ncnn/releases/download/$ncnnVersion/ncnn-$ncnnVersion-windows-vs2022-shared.zip"
Expand-Archive -Path ncnn-shared.zip -DestinationPath third_party
Rename-Item -Path "third_party/ncnn-$ncnnVersion-windows-vs2022-shared" -NewName ncnn-shared
curl -Lo opencv-shared.exe "https://github.com/opencv/opencv/releases/download/$opencvVersion/opencv-$opencvVersion-windows.exe"
7z x opencv-shared.exe -y
Move-Item -Path opencv -Destination third_party\opencv-shared
- name: Build Video2X
shell: pwsh
run: |
cmake -S . -B build -DUSE_SYSTEM_NCNN=OFF -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=build/video2x_install
cmake -S . -B build `
-DUSE_SYSTEM_NCNN=OFF -DUSE_SYSTEM_SPDLOG=OFF -DUSE_SYSTEM_OPENCV=OFF `
-DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=build/video2x_install
cmake --build build --config Debug --parallel --target install
- name: Upload artifacts
uses: actions/upload-artifact@v4

View File

@@ -4,17 +4,20 @@ on:
tags:
- "*"
permissions:
contents: write
jobs:
setup:
if: github.event.base_ref == 'refs/heads/master'
name: Setup
runs-on: ubuntu-latest
outputs:
tag: ${{ steps.get_tag.outputs.tag }}
version: ${{ steps.get_version.outputs.version }}
steps:
- name: Get tag
id: get_tag
run: echo ::set-output name=tag::${GITHUB_REF/refs\/tags\//}
- name: Get version
id: get_version
run: echo version=${GITHUB_REF/refs\/tags\//} >> $GITHUB_OUTPUT
container:
name: Build and upload container
@@ -32,7 +35,7 @@ jobs:
password: ${{ secrets.GHCR_TOKEN }}
dockerfile: Dockerfile
image: video2x
tags: latest, ${{ needs.setup.outputs.tag }}
tags: latest, ${{ needs.setup.outputs.version }}
create-release:
name: Create release
@@ -45,11 +48,10 @@ jobs:
steps:
- name: Create release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ needs.setup.outputs.tag }}
release_name: Video2X ${{ needs.setup.outputs.tag }}
token: ${{ secrets.GITHUB_TOKEN }}
tag_name: ${{ needs.setup.outputs.version }}
release_name: Video2X ${{ needs.setup.outputs.version }}
draft: true
prerelease: false

6
.gitmodules vendored
View File

@@ -4,3 +4,9 @@
[submodule "third_party/ncnn"]
path = third_party/ncnn
url = https://github.com/Tencent/ncnn.git
[submodule "third_party/spdlog"]
path = third_party/spdlog
url = https://github.com/gabime/spdlog.git
[submodule "third_party/opencv"]
path = third_party/opencv
url = https://github.com/opencv/opencv.git

View File

@@ -1,8 +1,8 @@
cmake_minimum_required(VERSION 3.10)
project(video2x VERSION 6.0.0 LANGUAGES CXX C)
project(video2x VERSION 6.0.0 LANGUAGES C CXX)
# Set the C standard
set(CMAKE_C_STANDARD 99)
set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
# Set the C++ standard
@@ -28,18 +28,47 @@ if(CMAKE_BUILD_TYPE STREQUAL "Release")
endif()
endif()
# Set global compile options for all targets
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
add_compile_options(/W4 /permissive-)
elseif (CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic -Wconversion -Wshadow)
endif()
# Build options
option(BUILD_SHARED_LIBS "Build libvideo2x as a shared library" ON)
option(BUILD_VIDEO2X "Build the video2x executable" ON)
option(BUILD_VIDEO2X_CLI "Build the video2x executable" ON)
option(USE_SYSTEM_SPDLOG "Use system spdlog library" ON)
option(USE_SYSTEM_OPENCV "Use system OpenCV library" ON)
option(USE_SYSTEM_NCNN "Use system ncnn library" ON)
# Generate the version header file
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/include/libvideo2x/version.h.in"
"${CMAKE_CURRENT_BINARY_DIR}/libvideo2x/version.h"
@ONLY
)
# Find the required packages
set(ALL_INCLUDE_DIRS)
set(ALL_LIBRARIES)
# spdlog
if (USE_SYSTEM_SPDLOG)
find_package(spdlog REQUIRED)
list(APPEND ALL_INCLUDE_DIRS ${spdlog_INCLUDE_DIRS})
list(APPEND ALL_LIBRARIES spdlog::spdlog)
else()
add_subdirectory(third_party/spdlog)
list(APPEND ALL_LIBRARIES spdlog::spdlog_header_only)
endif()
# Platform-specific dependencies
if(WIN32)
# Define base paths for FFmpeg and ncnn
set(FFMPEG_BASE_PATH ${PROJECT_SOURCE_DIR}/third_party/ffmpeg-shared)
set(NCNN_BASE_PATH ${PROJECT_SOURCE_DIR}/third_party/ncnn-shared/x64)
set(OPENCV_BASE_PATH ${PROJECT_SOURCE_DIR}/third_party/opencv-shared)
# FFmpeg
list(APPEND ALL_LIBRARIES
@@ -54,7 +83,9 @@ if(WIN32)
# ncnn
# TODO: Figure out why this file is not being copied to the install directory
set(SPIRV_BUILD_PATH ${CMAKE_BINARY_DIR}/realesrgan-prefix/src/realesrgan-build/ncnn/glslang/SPIRV)
set(SPIRV_BUILD_PATH
${CMAKE_BINARY_DIR}/realesrgan-prefix/src/realesrgan-build/ncnn/glslang/SPIRV
)
if (CMAKE_BUILD_TYPE STREQUAL "Release")
set(SPIRV_LIB ${SPIRV_BUILD_PATH}/Release/SPIRV.lib)
else()
@@ -66,8 +97,12 @@ if(WIN32)
${SPIRV_LIB}
)
list(APPEND ALL_INCLUDE_DIRS ${NCNN_BASE_PATH}/include/ncnn)
# OpenCV
list(APPEND ALL_LIBRARIES ${OPENCV_BASE_PATH}/build/x64/vc16/lib/opencv_world4100.lib)
list(APPEND ALL_INCLUDE_DIRS ${OPENCV_BASE_PATH}/build/include)
else()
# Find the required packages using pkg-config
# FFmpeg
find_package(PkgConfig REQUIRED)
set(REQUIRED_PKGS
libavcodec
@@ -84,11 +119,55 @@ else()
list(APPEND ALL_INCLUDE_DIRS ${${PKG}_INCLUDE_DIRS})
list(APPEND ALL_LIBRARIES ${${PKG}_LIBRARIES})
endforeach()
endif()
# Remove duplicate entries
list(REMOVE_DUPLICATES ALL_INCLUDE_DIRS)
list(REMOVE_DUPLICATES ALL_LIBRARIES)
# OpenCV
if (USE_SYSTEM_OPENCV)
find_package(OpenCV REQUIRED)
list(APPEND ALL_INCLUDE_DIRS ${OpenCV_INCLUDE_DIRS}/opencv2)
list(APPEND ALL_LIBRARIES opencv_core opencv_videoio)
else()
option(BUILD_opencv_calib3d "" OFF)
option(BUILD_opencv_core "" ON)
option(BUILD_opencv_dnn "" OFF)
option(BUILD_opencv_features2d "" OFF)
option(BUILD_opencv_flann "" OFF)
option(BUILD_opencv_gapi "" OFF)
option(BUILD_opencv_highgui "" OFF)
option(BUILD_opencv_imgcodecs "" OFF)
option(BUILD_opencv_imgproc "" OFF)
option(BUILD_opencv_java "" OFF)
option(BUILD_opencv_js "" OFF)
option(BUILD_opencv_ml "" OFF)
option(BUILD_opencv_objc "" OFF)
option(BUILD_opencv_objdetect "" OFF)
option(BUILD_opencv_photo "" OFF)
option(BUILD_opencv_python "" OFF)
option(BUILD_opencv_python2 "" OFF)
option(BUILD_opencv_python3 "" OFF)
option(BUILD_opencv_stitching "" OFF)
option(BUILD_opencv_ts "" OFF)
option(BUILD_opencv_video "" OFF)
option(BUILD_opencv_videoio "" ON)
option(BUILD_opencv_world "" OFF)
option(WITH_PNG "" OFF)
option(WITH_JPEG "" OFF)
option(WITH_TIFF "" OFF)
option(WITH_WEBP "" OFF)
option(WITH_OPENJPEG "" OFF)
option(WITH_JASPER "" OFF)
option(WITH_OPENEXR "" OFF)
option(WITH_IMGCODEC "" OFF)
add_subdirectory(third_party/opencv)
list(APPEND ALL_INCLUDE_DIRS
${PROJECT_SOURCE_DIR}/third_party/opencv/include
${PROJECT_SOURCE_DIR}/third_party/opencv/modules/core/include
${PROJECT_SOURCE_DIR}/third_party/opencv/modules/videoio/include
)
list(APPEND ALL_LIBRARIES opencv_core opencv_videoio)
endif() # USE_SYSTEM_OPENCV
endif() # WIN32
# Find ncnn package
if(USE_SYSTEM_NCNN)
@@ -205,6 +284,10 @@ ExternalProject_Add(
INSTALL_COMMAND ${CMAKE_COMMAND} --build . --target install --config ${CMAKE_BUILD_TYPE}
)
# Remove duplicate entries
list(REMOVE_DUPLICATES ALL_INCLUDE_DIRS)
list(REMOVE_DUPLICATES ALL_LIBRARIES)
# Add all source files for libvideo2x
file(GLOB LIBVIDEO2X_SOURCES src/*.cpp)
@@ -223,13 +306,14 @@ add_dependencies(libvideo2x realesrgan)
# Include directories for the shared library
target_include_directories(libvideo2x PRIVATE
${ALL_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
${PROJECT_SOURCE_DIR}/include
${PROJECT_SOURCE_DIR}/include/libvideo2x
${PROJECT_SOURCE_DIR}/third_party/libreal_esrgan_ncnn_vulkan/src
)
# Compile options for the shared library
target_compile_options(libvideo2x PRIVATE
-Wall
-fPIC
$<$<CONFIG:Release>:-Ofast>
$<$<CONFIG:Debug>:-g -DDEBUG>
@@ -237,7 +321,7 @@ target_compile_options(libvideo2x PRIVATE
# Define the path to the built libresrgan-ncnn-vulkan library
if(WIN32)
set(REALESRGAN_LIB ${CMAKE_BINARY_DIR}/realesrgan_install/lib/realesrgan-ncnn-vulkan.lib)
set(REALESRGAN_LIB ${CMAKE_BINARY_DIR}/realesrgan_install/lib/librealesrgan-ncnn-vulkan.lib)
else()
set(REALESRGAN_LIB ${CMAKE_BINARY_DIR}/realesrgan_install/lib/librealesrgan-ncnn-vulkan.so)
endif()
@@ -254,21 +338,19 @@ if(NOT WIN32)
endif()
# Create the executable 'video2x'
if (BUILD_VIDEO2X)
if (BUILD_VIDEO2X_CLI)
add_executable(video2x src/video2x.c src/getopt.c)
set_target_properties(video2x PROPERTIES OUTPUT_NAME video2x)
# Include directories for the executable
target_include_directories(video2x PRIVATE
${ALL_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
${PROJECT_SOURCE_DIR}/include
)
# Compile options for the executable
target_compile_options(video2x PRIVATE
-Wall
$<$<CONFIG:Debug>:-g -DDEBUG>
)
target_compile_options(video2x PRIVATE $<$<CONFIG:Debug>:-g -DDEBUG>)
# Link the executable with the shared library
target_link_libraries(video2x PRIVATE ${ALL_LIBRARIES} libvideo2x)
@@ -277,16 +359,20 @@ endif()
# Define the default installation directories
if(WIN32)
set(BIN_DESTINATION_DEFAULT ".")
set(INCLUDE_DESTINATION_DEFAULT "include")
set(INCLUDE_DESTINATION_DEFAULT "include/libvideo2x")
set(LIB_DESTINATION_DEFAULT ".")
set(MODEL_DESTINATION_DEFAULT ".")
else()
set(BIN_DESTINATION_DEFAULT "bin")
set(INCLUDE_DESTINATION_DEFAULT "include")
set(INCLUDE_DESTINATION_DEFAULT "include/libvideo2x")
set(LIB_DESTINATION_DEFAULT "lib")
set(MODEL_DESTINATION_DEFAULT "share/video2x")
endif()
# Explicitly set the output paths since OpenCV changes these variables
set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR})
set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR})
# Set the installation directories
set(INSTALL_BIN_DESTINATION ${BIN_DESTINATION_DEFAULT} CACHE STRING "")
set(INSTALL_INCLUDE_DESTINATION ${INCLUDE_DESTINATION_DEFAULT} CACHE STRING "")
@@ -306,31 +392,58 @@ install(TARGETS libvideo2x
# Install model files
install(DIRECTORY ${CMAKE_SOURCE_DIR}/models DESTINATION ${INSTALL_MODEL_DESTINATION})
# Install the executable if BUILD_VIDEO2X is enabled
if(BUILD_VIDEO2X)
# Install the executable if BUILD_VIDEO2X_CLI is enabled
if(BUILD_VIDEO2X_CLI)
install(TARGETS video2x RUNTIME DESTINATION ${INSTALL_BIN_DESTINATION})
endif()
# Install the header file
install(FILES ${PROJECT_SOURCE_DIR}/include/libvideo2x.h DESTINATION ${INSTALL_INCLUDE_DESTINATION})
install(FILES ${PROJECT_SOURCE_DIR}/include/libvideo2x/libvideo2x.h
DESTINATION ${INSTALL_INCLUDE_DESTINATION}
)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/libvideo2x/version.h
DESTINATION ${INSTALL_INCLUDE_DESTINATION}
)
# Platform-specific installation rules
if(WIN32)
# Install Windows-specific dependencies
install(FILES ${CMAKE_BINARY_DIR}/realesrgan_install/bin/realesrgan-ncnn-vulkan.dll
install(FILES ${CMAKE_BINARY_DIR}/realesrgan_install/bin/librealesrgan-ncnn-vulkan.dll
DESTINATION ${INSTALL_BIN_DESTINATION}
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE
GROUP_READ GROUP_EXECUTE
WORLD_READ WORLD_EXECUTE
)
install(FILES ${NCNN_BASE_PATH}/bin/ncnn.dll DESTINATION ${INSTALL_BIN_DESTINATION}
install(FILES ${FFMPEG_BASE_PATH}/bin/swscale-8.dll
${FFMPEG_BASE_PATH}/bin/avcodec-61.dll
${FFMPEG_BASE_PATH}/bin/avdevice-61.dll
${FFMPEG_BASE_PATH}/bin/avfilter-10.dll
${FFMPEG_BASE_PATH}/bin/avformat-61.dll
${FFMPEG_BASE_PATH}/bin/avutil-59.dll
${FFMPEG_BASE_PATH}/bin/postproc-58.dll
${FFMPEG_BASE_PATH}/bin/swresample-5.dll
DESTINATION ${INSTALL_BIN_DESTINATION}
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE
GROUP_READ GROUP_EXECUTE
WORLD_READ WORLD_EXECUTE
)
install(FILES ${OPENCV_BASE_PATH}/build/x64/vc16/bin/opencv_world4100.dll
${OPENCV_BASE_PATH}/build/x64/vc16/bin/opencv_videoio_msmf4100_64.dll
DESTINATION ${INSTALL_BIN_DESTINATION}
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE
GROUP_READ GROUP_EXECUTE
WORLD_READ WORLD_EXECUTE
)
install(FILES ${NCNN_BASE_PATH}/bin/ncnn.dll
DESTINATION ${INSTALL_BIN_DESTINATION}
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE
GROUP_READ GROUP_EXECUTE
WORLD_READ WORLD_EXECUTE
)
else()
# Install Unix-specific dependencies
install(FILES ${REALESRGAN_LIB} DESTINATION ${INSTALL_LIB_DESTINATION}
install(FILES ${REALESRGAN_LIB}
DESTINATION ${INSTALL_LIB_DESTINATION}
PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE
GROUP_READ GROUP_EXECUTE
WORLD_READ WORLD_EXECUTE

View File

@@ -1,14 +1,15 @@
# Name: Video2X Dockerfile
# Creator: K4YT3X
# Date Created: February 3, 2022
# Last Modified: October 7, 2024
# Last Modified: October 21, 2024
# stage 1: build the python components into wheels
FROM docker.io/archlinux:latest AS builder
# Install dependencies and create a non-root user
RUN pacman -Syy --noconfirm \
base-devel ffmpeg ncnn git cmake make clang pkgconf vulkan-headers openmp sudo \
base-devel git cmake make clang pkgconf sudo \
ffmpeg ncnn vulkan-headers openmp spdlog opencv \
nvidia-utils vulkan-radeon vulkan-intel vulkan-swrast \
&& useradd -m builder \
&& echo 'builder ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/builder
@@ -20,7 +21,7 @@ WORKDIR /video2x
# Build the package
RUN makepkg -s --noconfirm \
&& find /video2x -maxdepth 1 -name '*.pkg.tar.zst' | head -n 1 | \
&& find /video2x -maxdepth 1 -name 'video2x-*.pkg.tar.zst' ! -name '*-debug-*' | head -n 1 | \
xargs -I {} cp {} /tmp/video2x.pkg.tar.zst
# stage 2: install wheels into the final image
@@ -31,13 +32,12 @@ LABEL maintainer="K4YT3X <i@k4yt3x.com>" \
ENV VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.json\
:/usr/share/vulkan/icd.d/radeon_icd.x86_64.json\
:/usr/share/vulkan/icd.d/intel_icd.x86_64.json
:/usr/share/vulkan/icd.d/intel_icd.x86_64.json\
:/usr/share/vulkan/icd.d/lvp_icd.x86_64.json
COPY --from=builder /tmp/video2x.pkg.tar.zst /video2x.pkg.tar.zst
COPY . /video2x
WORKDIR /video2x
RUN pacman -Sy --noconfirm ffmpeg ncnn \
nvidia-utils vulkan-radeon vulkan-intel vulkan-swrast \
RUN pacman -Sy --noconfirm nvidia-utils vulkan-radeon vulkan-intel vulkan-swrast \
ffmpeg ncnn spdlog opencv \
&& pacman -U --noconfirm /video2x.pkg.tar.zst \
&& rm -rf /video2x.pkg.tar.zst /var/cache/pacman/pkg/*

113
Makefile
View File

@@ -1,9 +1,15 @@
.PHONY: build static debug windows test-realesrgan test-libplacebo leakcheck clean
.PHONY: build static debug clean \
test-realesrgan test-libplacebo \
memcheck-realesrgan memcheck-libplacebo \
heaptrack-realesrgan heaptrack-libplacebo
BINDIR=build
CC=clang
CXX=clang++
TEST_VIDEO=data/standard-test.mp4
TEST_OUTPUT=data/output.mp4
build:
cmake -S . -B $(BINDIR) \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
@@ -19,8 +25,7 @@ static:
-DCMAKE_C_COMPILER=$(CC) \
-DCMAKE_CXX_COMPILER=$(CXX) \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=OFF \
-DUSE_SYSTEM_NCNN=OFF
-DBUILD_SHARED_LIBS=OFF
cmake --build $(BINDIR) --config Release --parallel
cp $(BINDIR)/compile_commands.json .
@@ -33,39 +38,75 @@ debug:
cmake --build $(BINDIR) --config Debug --parallel
cp $(BINDIR)/compile_commands.json .
test-realesrgan:
LD_LIBRARY_PATH=$(BINDIR) $(BINDIR)/video2x -i data/standard-test.mp4 -o data/output.mp4 \
-f realesrgan -r 4 --model realesr-animevideov3
test-libplacebo:
LD_LIBRARY_PATH=$(BINDIR) $(BINDIR)/video2x -i data/standard-test.mp4 -o data/output.mp4 \
-f libplacebo -w 1920 -h 1080 -s anime4k-mode-a
leakcheck-realesrgan:
LD_LIBRARY_PATH=$(BINDIR) valgrind \
--tool=memcheck \
--leak-check=full \
--show-leak-kinds=all \
--track-origins=yes \
--show-reachable=yes \
--verbose --log-file="valgrind.log" \
$(BINDIR)/video2x \
-i data/standard-test.mp4 -o data/output.mp4 \
-f realesrgan -r 2 --model realesr-animevideov3 \
-p veryfast -b 1000000 -q 30
leakcheck-libplacebo:
LD_LIBRARY_PATH=$(BINDIR) valgrind \
--tool=memcheck \
--leak-check=full \
--show-leak-kinds=all \
--track-origins=yes \
--show-reachable=yes \
--verbose --log-file="valgrind.log" \
$(BINDIR)/video2x \
-i data/standard-test.mp4 -o data/output.mp4 \
-f libplacebo -w 1920 -h 1080 -s anime4k-mode-a \
-p veryfast -b 1000000 -q 30
debian:
apt-get update
apt-get install -y --no-install-recommends \
build-essential cmake clang pkg-config \
libavcodec-dev \
libavdevice-dev \
libavfilter-dev \
libavformat-dev \
libavutil-dev \
libswscale-dev \
libvulkan-dev \
glslang-tools \
libomp-dev \
libspdlog-dev \
libopencv-dev
cmake -B /tmp/build -S . -DUSE_SYSTEM_NCNN=OFF \
-DCMAKE_C_COMPILER=$(CC) -DCMAKE_CXX_COMPILER=$(CXX) \
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp/install \
-DINSTALL_BIN_DESTINATION=. -DINSTALL_INCLUDE_DESTINATION=include \
-DINSTALL_LIB_DESTINATION=. -DINSTALL_MODEL_DESTINATION=.
cmake --build /tmp/build --config Release --target install --parallel
clean:
rm -rf $(BINDIR)
test-realesrgan:
LD_LIBRARY_PATH=$(BINDIR) $(BINDIR)/video2x -i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f realesrgan -r 4 -m realesr-animevideov3
test-libplacebo:
LD_LIBRARY_PATH=$(BINDIR) $(BINDIR)/video2x -i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f libplacebo -w 1920 -h 1080 -s anime4k-mode-a
memcheck-realesrgan:
LD_LIBRARY_PATH=$(BINDIR) valgrind \
--tool=memcheck \
--leak-check=full \
--show-leak-kinds=all \
--track-origins=yes \
--show-reachable=yes \
--verbose --log-file="valgrind.log" \
$(BINDIR)/video2x \
-i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f realesrgan -r 2 -m realesr-animevideov3 \
-p veryfast -b 1000000 -q 30
memcheck-libplacebo:
LD_LIBRARY_PATH=$(BINDIR) valgrind \
--tool=memcheck \
--leak-check=full \
--show-leak-kinds=all \
--track-origins=yes \
--show-reachable=yes \
--verbose --log-file="valgrind.log" \
$(BINDIR)/video2x \
-i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f libplacebo -w 1920 -h 1080 -s anime4k-mode-a \
-p veryfast -b 1000000 -q 30
heaptrack-realesrgan:
LD_LIBRARY_PATH=$(BINDIR) HEAPTRACK_ENABLE_DEBUGINFOD=1 heaptrack \
$(BINDIR)/video2x \
-i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f realesrgan -r 4 -m realesr-animevideov3 \
-p veryfast -b 1000000 -q 30
heaptrack-libplacebo:
LD_LIBRARY_PATH=$(BINDIR) HEAPTRACK_ENABLE_DEBUGINFOD=1 heaptrack \
$(BINDIR)/video2x \
-i $(TEST_VIDEO) -o $(TEST_OUTPUT) \
-f libplacebo -w 1920 -h 1080 -s anime4k-mode-a \
-p veryfast -b 1000000 -q 30

View File

@@ -1,11 +1,11 @@
pkgname=video2x
pkgver=r862.f590ead
pkgver=r843.e09f348
pkgrel=1
pkgdesc="A lossless video super resolution framework"
pkgdesc="A machine learning-based lossless video super resolution framework"
arch=('x86_64')
url="https://github.com/k4yt3x/video2x"
license=('AGPL3')
depends=('ffmpeg' 'ncnn' 'vulkan-driver')
depends=('ffmpeg' 'ncnn' 'vulkan-driver' 'opencv' 'spdlog')
makedepends=('git' 'cmake' 'make' 'clang' 'pkgconf' 'vulkan-headers' 'openmp')
pkgver() {

View File

@@ -1,5 +1,5 @@
<p align="center">
<img src="https://github.com/user-attachments/assets/0c6c5d31-7fd5-4e50-b41e-91a58aae995c"/>
<img src="https://github.com/user-attachments/assets/5cd63373-e806-474f-94ec-6e04963bf90f"/>
</br>
<img src="https://img.shields.io/github/v/release/k4yt3x/video2x?style=flat-square"/>
<img src="https://img.shields.io/github/actions/workflow/status/k4yt3x/video2x/build.yml?label=Build&style=flat-square"/>
@@ -8,13 +8,21 @@
<img src="https://img.shields.io/badge/dynamic/json?color=%23e85b46&label=Patreon&query=data.attributes.patron_count&suffix=%20patrons&url=https%3A%2F%2Fwww.patreon.com%2Fapi%2Fcampaigns%2F4507807&style=flat-square"/>
</p>
> [!IMPORTANT]
> Versions 4 and 5 have reached end-of-life (EOL) status. Due to limited development resources, issues related to any version earlier than 6 will no longer be addressed.
## 🌟 Version 6.0.0 Preview
**[Direct download link for Windows (Installer)](https://github.com/k4yt3x/video2x/releases/download/6.0.0-beta.4/video2x-qt6-windows-amd64-installer.exe)**
![6.0.0-beta-screenshot](https://github.com/user-attachments/assets/bd9e0b82-67ee-4b62-a830-a694db6f5de3)
Version 6.0.0 is a complete rewrite of this project in C/C++. It:
- actually works, with less pain;
- actually works this time, with less pain (in comparison to 5.0.0 beta);
- is blazing fast, thanks to the redesigned efficient pipeline and the speed of C/C++;
- is cross-platform, available right now for both Windows and Linux;
- provides much better output quality with Anime4K v4 and RealESRGAN;
- supports Anime4K v4 and all other custom MPV-compatible GLSL shaders;
- supports RealESRGAN (all three models) via ncnn and Vulkan;
- requires 0 disk space for processing the video, just space for storing the final output; and
@@ -22,16 +30,17 @@ Version 6.0.0 is a complete rewrite of this project in C/C++. It:
These are available for download now:
- 6.0.0 beta CLI preview builds for Windows and Linux are on the [releases page](https://github.com/k4yt3x/video2x/releases).
- **6.0.0 beta Qt6-based GUI for Windows** is on the [releases page](https://github.com/k4yt3x/video2x/releases).
- **6.0.0 beta CLI preview builds for Windows and Linux** are on the [releases page](https://github.com/k4yt3x/video2x/releases).
- You will need to install the dependencies and set `LD_LIBRARY_PATH` for the Linux build to work. Refer to the [PKGBUILD](PKGBUILD) file to see what needs to be installed.
- Alternatively, you can build it from source. Take a look at the [Makefile](Makefile).
- 6.0.0 beta AUR package for Arch Linux (`video2x-git`).
- 6.0.0 beta [container image](https://github.com/k4yt3x/video2x/pkgs/container/video2x).
- A GUI for 6.0.0 is in the making. More information about it will be available soon.
- A new Colab will be made for 6.0.0 at a later time.
There is still much to be done and optimize. Stay tuned for more updates. As for why the 5.0.0 branch was abandoned, here are some of the reasons:
- Wrapped C++ libraries for Python is too painful to build for cross-platform distribution.
- Wrapped C++ libraries for Python are too painful to build for cross-platform distribution.
- Some wrapped C++ libraires exhibited unexpected behaviors.
- Running FFmpeg via commands and piping data through stdin/stdout are inefficient.
- C/C++ native binaries are much smaller and much more efficient.
@@ -42,7 +51,7 @@ Join our Telegram discussion group to ask any questions you have about Video2X,
## [🪟 Download Windows Releases](https://github.com/k4yt3x/video2x/releases/tag/4.8.1)
The latest Windows update is built based on version 4.8.1. GUI is not available for 6.0.0 yet, but is already under development. Go to the [GUI](https://github.com/k4yt3x/video2x/wiki/GUI) page to see the basic usages of the GUI. Try the [mirror](https://files.k4yt3x.com/Projects/Video2X/latest) if you can't download releases directly from GitHub.
The latest Windows release build based on version 4.8.1. Go to the [GUI](https://github.com/k4yt3x/video2x/wiki/GUI) page to see the basic usages of the GUI. Try the [mirror](https://files.k4yt3x.com/Projects/Video2X/latest) if you can't download releases directly from GitHub.
## [📔 Google Colab](https://colab.research.google.com/drive/1gWEwcA9y57EsxwOjmLNmNMXPsafw0kGo)
@@ -89,11 +98,11 @@ Copyright of this clip belongs to 株式会社アニプレックス.
## License
This project is licensed under the [GNU Affero General Public License Version 3 (GNU AGPL v3)](https://www.gnu.org/licenses/agpl-3.0.txt)\
Copyright (C) 2018-2024 K4YT3X and contributors.
Copyright (C) 2018-2024 K4YT3X and [contributors](https://github.com/k4yt3x/video2x/graphs/contributors).
![AGPLv3](https://www.gnu.org/graphics/agplv3-155x51.png)
This project includes or depends on these following projects:
This project (`libvideo2x`) includes or depends on these following projects:
| Project | License |
| ----------------------------------------------------------------------------- | --------------- |

View File

@@ -22,7 +22,7 @@ struct option {
#define required_argument 1
#define optional_argument 2
int getopt(int, char **, const char *);
// int getopt(int, char **, const char *);
int getopt_long(int, char **, const char *, const struct option *, int *);
#ifdef __cplusplus

View File

@@ -1,7 +1,11 @@
#ifndef CONVERSIONS_H
#define CONVERSIONS_H
extern "C" {
#include <libavutil/frame.h>
#include <libswscale/swscale.h>
}
#include <mat.h>
// Convert AVFrame to another pixel format

View File

@@ -1,15 +1,18 @@
#ifndef DECODER_H
#define DECODER_H
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/avfilter.h>
#include <libavformat/avformat.h>
}
int init_decoder(
const char *input_filename,
AVHWDeviceType hw_type,
AVBufferRef *hw_ctx,
const char *in_fname,
AVFormatContext **fmt_ctx,
AVCodecContext **dec_ctx,
int *video_stream_index
int *vstream_idx
);
#endif // DECODER_H

View File

@@ -1,20 +1,32 @@
#ifndef ENCODER_H
#define ENCODER_H
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/opt.h>
}
#include "libvideo2x.h"
int init_encoder(
const char *output_filename,
AVBufferRef *hw_ctx,
const char *out_fname,
AVFormatContext *ifmt_ctx,
AVFormatContext **ofmt_ctx,
AVCodecContext **enc_ctx,
AVCodecContext *dec_ctx,
EncoderConfig *encoder_config
EncoderConfig *encoder_config,
int vstream_idx,
int **stream_map
);
int encode_and_write_frame(AVFrame *frame, AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx);
int encode_and_write_frame(
AVFrame *frame,
AVCodecContext *enc_ctx,
AVFormatContext *ofmt_ctx,
int vstream_idx
);
int flush_encoder(AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx);

View File

@@ -6,15 +6,16 @@
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/avfilter.h>
#include <libavutil/buffer.h>
}
// Abstract base class for filters
class Filter {
public:
virtual ~Filter() {}
virtual int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) = 0;
virtual AVFrame *process_frame(AVFrame *input_frame) = 0;
virtual int flush(std::vector<AVFrame *> &processed_frames) = 0;
virtual ~Filter() = default;
virtual int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx, AVBufferRef *hw_ctx) = 0;
virtual int process_frame(AVFrame *in_frame, AVFrame **out_frame) = 0;
virtual int flush(std::vector<AVFrame *> &flushed_frames) { return 0; }
};
#endif // FILTER_H

View File

@@ -3,18 +3,19 @@
#include <filesystem>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/avfilter.h>
#include <libavutil/buffer.h>
}
int init_libplacebo(
AVBufferRef *hw_ctx,
AVFilterGraph **filter_graph,
AVFilterContext **buffersrc_ctx,
AVFilterContext **buffersink_ctx,
AVBufferRef **device_ctx,
AVCodecContext *dec_ctx,
int output_width,
int output_height,
int out_width,
int out_height,
const std::filesystem::path &shader_path
);

View File

@@ -3,7 +3,11 @@
#include <filesystem>
#include <libavutil/buffer.h>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
}
#include "filter.h"
@@ -13,27 +17,27 @@ class LibplaceboFilter : public Filter {
AVFilterGraph *filter_graph;
AVFilterContext *buffersrc_ctx;
AVFilterContext *buffersink_ctx;
AVBufferRef *device_ctx;
int output_width;
int output_height;
int out_width;
int out_height;
const std::filesystem::path shader_path;
AVRational output_time_base;
AVRational in_time_base;
AVRational out_time_base;
public:
// Constructor
LibplaceboFilter(int width, int height, const std::filesystem::path &shader_path);
// Destructor
virtual ~LibplaceboFilter();
virtual ~LibplaceboFilter() override;
// Initializes the filter with decoder and encoder contexts
int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) override;
int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx, AVBufferRef *hw_ctx) override;
// Processes an input frame and returns the processed frame
AVFrame *process_frame(AVFrame *input_frame) override;
int process_frame(AVFrame *in_frame, AVFrame **out_frame) override;
// Flushes any remaining frames
int flush(std::vector<AVFrame *> &processed_frames) override;
int flush(std::vector<AVFrame *> &flushed_frames) override;
};
#endif // LIBPLACEBO_FILTER_H

View File

@@ -1,13 +1,10 @@
#ifndef LIBVIDEO2X_H
#define LIBVIDEO2X_H
#include <libavutil/pixfmt.h>
#include <stdbool.h>
#include <stdint.h>
#include <time.h>
#include <libavcodec/avcodec.h>
#include <libavcodec/codec_id.h>
#ifdef _WIN32
#ifdef LIBVIDEO2X_EXPORTS
#define LIBVIDEO2X_API __declspec(dllexport)
@@ -22,23 +19,37 @@
extern "C" {
#endif
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
// Enum to specify filter type
enum FilterType {
FILTER_LIBPLACEBO,
FILTER_REALESRGAN
};
// Enum to specify log level
enum Libvideo2xLogLevel {
LIBVIDEO2X_LOG_LEVEL_TRACE,
LIBVIDEO2X_LOG_LEVEL_DEBUG,
LIBVIDEO2X_LOG_LEVEL_INFO,
LIBVIDEO2X_LOG_LEVEL_WARNING,
LIBVIDEO2X_LOG_LEVEL_ERROR,
LIBVIDEO2X_LOG_LEVEL_CRITICAL,
LIBVIDEO2X_LOG_LEVEL_OFF
};
// Configuration for Libplacebo filter
struct LibplaceboConfig {
int output_width;
int output_height;
int out_width;
int out_height;
const char *shader_path;
};
// Configuration for RealESRGAN filter
struct RealESRGANConfig {
int gpuid;
int tta_mode;
bool tta_mode;
int scaling_factor;
const char *model;
};
@@ -54,8 +65,9 @@ struct FilterConfig {
// Encoder configuration
struct EncoderConfig {
int output_width;
int output_height;
int out_width;
int out_height;
bool copy_streams;
enum AVCodecID codec;
enum AVPixelFormat pix_fmt;
const char *preset;
@@ -63,20 +75,26 @@ struct EncoderConfig {
float crf;
};
// Processing status
struct ProcessingStatus {
// Video processing context
struct VideoProcessingContext {
int64_t processed_frames;
int64_t total_frames;
time_t start_time;
bool pause;
bool abort;
bool completed;
};
// C-compatible process_video function
LIBVIDEO2X_API int process_video(
const char *input_filename,
const char *output_filename,
const char *in_fname,
const char *out_fname,
enum Libvideo2xLogLevel log_level,
bool benchmark,
enum AVHWDeviceType hw_device_type,
const struct FilterConfig *filter_config,
struct EncoderConfig *encoder_config,
struct ProcessingStatus *status
struct VideoProcessingContext *proc_ctx
);
#ifdef __cplusplus

View File

@@ -3,6 +3,10 @@
#include <filesystem>
extern "C" {
#include <libavcodec/avcodec.h>
}
#include "filter.h"
#include "realesrgan.h"
@@ -16,9 +20,9 @@ class RealesrganFilter : public Filter {
const char *model;
const std::filesystem::path custom_model_param_path;
const std::filesystem::path custom_model_bin_path;
AVRational input_time_base;
AVRational output_time_base;
AVPixelFormat output_pix_fmt;
AVRational in_time_base;
AVRational out_time_base;
AVPixelFormat out_pix_fmt;
public:
// Constructor
@@ -27,21 +31,18 @@ class RealesrganFilter : public Filter {
bool tta_mode = false,
int scaling_factor = 4,
const char *model = "realesr-animevideov3",
const std::filesystem::path custom_model_bin_pathmodel_param_path = std::filesystem::path(),
const std::filesystem::path custom_model_bin_pathmodel_bin_path = std::filesystem::path()
const std::filesystem::path custom_model_param_path = std::filesystem::path(),
const std::filesystem::path custom_model_bin_path = std::filesystem::path()
);
// Destructor
virtual ~RealesrganFilter();
virtual ~RealesrganFilter() override;
// Initializes the filter with decoder and encoder contexts
int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) override;
int init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx, AVBufferRef *hw_ctx) override;
// Processes an input frame and returns the processed frame
AVFrame *process_frame(AVFrame *input_frame) override;
// Flushes any remaining frames (if necessary)
int flush(std::vector<AVFrame *> &processed_frames) override;
int process_frame(AVFrame *in_frame, AVFrame **out_frame) override;
};
#endif

View File

@@ -0,0 +1,6 @@
#ifndef VERSION_H
#define VERSION_H
#define LIBVIDEO2X_VERSION_STRING "@PROJECT_VERSION@"
#endif // VERSION_H

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,46 +0,0 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
import shutil
from pathlib import Path
import requests
ANIME4K_COMMIT = "master"
GITHUB_GLSL_ROOT = (
f"https://raw.githubusercontent.com/bloc97/Anime4K/{ANIME4K_COMMIT}/glsl"
)
SHADERS_DIR = Path(__file__).parent.parent / "data"
def download_and_combine_files():
modes = {
"ModeA": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
]
}
for mode in modes:
file_contents = ""
for file in modes[mode]:
response = requests.get(file, timeout=5)
response.raise_for_status()
file_contents += response.text + "\n"
with (SHADERS_DIR / Path(f"Anime4K_{mode}.glsl")).open("w") as output_file:
output_file.write(file_contents)
if __name__ == "__main__":
# clear shaders directory
if SHADERS_DIR.exists():
shutil.rmtree(SHADERS_DIR)
SHADERS_DIR.mkdir(exist_ok=True)
# download and combine shaders
download_and_combine_files()

View File

@@ -0,0 +1,86 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
import shutil
from pathlib import Path
import requests
ANIME4K_COMMIT = "master"
GITHUB_GLSL_ROOT = (
f"https://raw.githubusercontent.com/bloc97/Anime4K/{ANIME4K_COMMIT}/glsl"
)
SHADERS_DIR = Path(__file__).parent.parent / "models" / "libplacebo"
def download_and_combine_files():
modes = {
"a": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
"b": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_Soft_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
"c": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale+Denoise/Anime4K_Upscale_Denoise_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
"a+a": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_M.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
"b+b": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_Soft_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_Soft_M.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
"c+a": [
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Clamp_Highlights.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale+Denoise/Anime4K_Upscale_Denoise_CNN_x2_VL.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x2.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_AutoDownscalePre_x4.glsl",
f"{GITHUB_GLSL_ROOT}/Restore/Anime4K_Restore_CNN_M.glsl",
f"{GITHUB_GLSL_ROOT}/Upscale/Anime4K_Upscale_CNN_x2_M.glsl",
],
}
for mode in modes:
file_contents = ""
for file in modes[mode]:
response = requests.get(file, timeout=5)
response.raise_for_status()
file_contents += response.text + "\n"
with (SHADERS_DIR / Path(f"anime4k-{mode}.glsl")).open("w") as output_file:
output_file.write(file_contents)
if __name__ == "__main__":
# clear shaders directory
if SHADERS_DIR.exists():
shutil.rmtree(SHADERS_DIR)
SHADERS_DIR.mkdir(exist_ok=True)
# download and combine shaders
download_and_combine_files()

View File

@@ -1,22 +1,15 @@
#include "conversions.h"
#include <cstddef>
#include <cstdio>
// FFmpeg includes
extern "C" {
#include <libavutil/frame.h>
#include <libavutil/imgutils.h>
#include <libswscale/swscale.h>
}
// ncnn includes
#include <mat.h>
#include "conversions.h"
#include <spdlog/spdlog.h>
// Convert AVFrame format
AVFrame *convert_avframe_pix_fmt(AVFrame *src_frame, AVPixelFormat pix_fmt) {
AVFrame *dst_frame = av_frame_alloc();
if (dst_frame == nullptr) {
fprintf(stderr, "Failed to allocate destination AVFrame.\n");
spdlog::error("Failed to allocate destination AVFrame.");
return nullptr;
}
@@ -26,7 +19,7 @@ AVFrame *convert_avframe_pix_fmt(AVFrame *src_frame, AVPixelFormat pix_fmt) {
// Allocate memory for the converted frame
if (av_frame_get_buffer(dst_frame, 32) < 0) {
fprintf(stderr, "Failed to allocate memory for AVFrame.\n");
spdlog::error("Failed to allocate memory for AVFrame.");
av_frame_free(&dst_frame);
return nullptr;
}
@@ -46,7 +39,7 @@ AVFrame *convert_avframe_pix_fmt(AVFrame *src_frame, AVPixelFormat pix_fmt) {
);
if (sws_ctx == nullptr) {
fprintf(stderr, "Failed to initialize swscale context.\n");
spdlog::error("Failed to initialize swscale context.");
av_frame_free(&dst_frame);
return nullptr;
}
@@ -76,24 +69,27 @@ ncnn::Mat avframe_to_ncnn_mat(AVFrame *frame) {
if (frame->format != AV_PIX_FMT_BGR24) {
converted_frame = convert_avframe_pix_fmt(frame, AV_PIX_FMT_BGR24);
if (!converted_frame) {
fprintf(stderr, "Failed to convert AVFrame to BGR24.\n");
return ncnn::Mat(); // Return an empty ncnn::Mat on failure
spdlog::error("Failed to convert AVFrame to BGR24.");
return ncnn::Mat();
}
} else {
converted_frame = frame; // If the frame is already in BGR24, use it directly
// If the frame is already in BGR24, use it directly
converted_frame = frame;
}
// Allocate a new ncnn::Mat and copy the data
int width = converted_frame->width;
int height = converted_frame->height;
ncnn::Mat ncnn_image = ncnn::Mat(width, height, (size_t)3, 3); // BGR has 3 channels
ncnn::Mat ncnn_image = ncnn::Mat(width, height, static_cast<size_t>(3), 3);
// Manually copy the pixel data from AVFrame to the new ncnn::Mat
const uint8_t *src_data = converted_frame->data[0];
for (int y = 0; y < height; y++) {
uint8_t *dst_row = ncnn_image.row<uint8_t>(y);
const uint8_t *src_row = src_data + y * converted_frame->linesize[0];
memcpy(dst_row, src_row, width * 3); // Copy 3 channels (BGR) per pixel
// Copy 3 channels (BGR) per pixel
memcpy(dst_row, src_row, static_cast<size_t>(width) * 3);
}
// If we allocated a converted frame, free it
@@ -111,7 +107,7 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
// Step 1: Allocate a destination AVFrame for the specified pixel format
AVFrame *dst_frame = av_frame_alloc();
if (!dst_frame) {
fprintf(stderr, "Failed to allocate destination AVFrame.\n");
spdlog::error("Failed to allocate destination AVFrame.");
return nullptr;
}
@@ -121,7 +117,7 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
// Allocate memory for the frame buffer
if (av_frame_get_buffer(dst_frame, 32) < 0) {
fprintf(stderr, "Failed to allocate memory for destination AVFrame.\n");
spdlog::error("Failed to allocate memory for destination AVFrame.");
av_frame_free(&dst_frame);
return nullptr;
}
@@ -129,7 +125,7 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
// Step 2: Convert ncnn::Mat to BGR AVFrame
AVFrame *bgr_frame = av_frame_alloc();
if (!bgr_frame) {
fprintf(stderr, "Failed to allocate intermediate BGR AVFrame.\n");
spdlog::error("Failed to allocate intermediate BGR AVFrame.");
av_frame_free(&dst_frame);
return nullptr;
}
@@ -140,20 +136,19 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
// Allocate memory for the intermediate BGR frame
if (av_frame_get_buffer(bgr_frame, 32) < 0) {
fprintf(stderr, "Failed to allocate memory for BGR AVFrame.\n");
spdlog::error("Failed to allocate memory for BGR AVFrame.");
av_frame_free(&dst_frame);
av_frame_free(&bgr_frame);
return nullptr;
}
// Copy data from ncnn::Mat to the BGR AVFrame
// mat.to_pixels(bgr_frame->data[0], ncnn::Mat::PIXEL_BGR);
// Manually copy the pixel data from ncnn::Mat to the BGR AVFrame
// Copy the pixel data from ncnn::Mat to the BGR AVFrame
for (int y = 0; y < mat.h; y++) {
uint8_t *dst_row = bgr_frame->data[0] + y * bgr_frame->linesize[0];
const uint8_t *src_row = mat.row<const uint8_t>(y);
memcpy(dst_row, src_row, mat.w * 3); // Copy 3 channels (BGR) per pixel
// Copy 3 channels (BGR) per pixel
memcpy(dst_row, src_row, static_cast<size_t>(mat.w) * 3);
}
// Step 3: Convert the BGR frame to the desired pixel format
@@ -171,7 +166,7 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
);
if (sws_ctx == nullptr) {
fprintf(stderr, "Failed to initialize swscale context.\n");
spdlog::error("Failed to initialize swscale context.");
av_frame_free(&bgr_frame);
av_frame_free(&dst_frame);
return nullptr;
@@ -193,7 +188,7 @@ AVFrame *ncnn_mat_to_avframe(const ncnn::Mat &mat, AVPixelFormat pix_fmt) {
av_frame_free(&bgr_frame);
if (ret != dst_frame->height) {
fprintf(stderr, "Failed to convert BGR AVFrame to destination pixel format.\n");
spdlog::error("Failed to convert BGR AVFrame to destination pixel format.");
av_frame_free(&dst_frame);
return nullptr;
}

View File

@@ -1,42 +1,50 @@
#include "decoder.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/avfilter.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavformat/avformat.h>
#include <libavutil/opt.h>
#include <libavutil/pixdesc.h>
#include <libavutil/rational.h>
#include <spdlog/spdlog.h>
static enum AVPixelFormat hw_pix_fmt = AV_PIX_FMT_NONE;
// Callback function to choose the hardware-accelerated pixel format
static enum AVPixelFormat get_hw_format(AVCodecContext *_, const enum AVPixelFormat *pix_fmts) {
for (const enum AVPixelFormat *p = pix_fmts; *p != AV_PIX_FMT_NONE; p++) {
if (*p == hw_pix_fmt) {
return *p;
}
}
spdlog::error("Failed to get HW surface format.");
return AV_PIX_FMT_NONE;
}
int init_decoder(
const char *input_filename,
AVHWDeviceType hw_type,
AVBufferRef *hw_ctx,
const char *in_fname,
AVFormatContext **fmt_ctx,
AVCodecContext **dec_ctx,
int *video_stream_index
int *vstream_idx
) {
AVFormatContext *ifmt_ctx = NULL;
AVCodecContext *codec_ctx = NULL;
int ret;
if ((ret = avformat_open_input(&ifmt_ctx, input_filename, NULL, NULL)) < 0) {
fprintf(stderr, "Could not open input file '%s'\n", input_filename);
if ((ret = avformat_open_input(&ifmt_ctx, in_fname, NULL, NULL)) < 0) {
spdlog::error("Could not open input file '{}'", in_fname);
return ret;
}
if ((ret = avformat_find_stream_info(ifmt_ctx, NULL)) < 0) {
fprintf(stderr, "Failed to retrieve input stream information\n");
spdlog::error("Failed to retrieve input stream information");
return ret;
}
// Find the first video stream
ret = av_find_best_stream(ifmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
if (ret < 0) {
fprintf(stderr, "Could not find video stream in the input, aborting\n");
spdlog::error("Could not find video stream in the input file");
return ret;
}
@@ -44,20 +52,49 @@ int init_decoder(
AVStream *video_stream = ifmt_ctx->streams[stream_index];
// Set up the decoder
const AVCodec *dec = avcodec_find_decoder(video_stream->codecpar->codec_id);
if (!dec) {
fprintf(stderr, "Failed to find decoder for stream #%u\n", stream_index);
const AVCodec *decoder = avcodec_find_decoder(video_stream->codecpar->codec_id);
if (!decoder) {
spdlog::error(
"Failed to find decoder for codec ID {}",
static_cast<int>(video_stream->codecpar->codec_id)
);
return AVERROR_DECODER_NOT_FOUND;
}
codec_ctx = avcodec_alloc_context3(dec);
codec_ctx = avcodec_alloc_context3(decoder);
if (!codec_ctx) {
fprintf(stderr, "Failed to allocate the decoder context\n");
spdlog::error("Failed to allocate the decoder context");
return AVERROR(ENOMEM);
}
// Set hardware device context
if (hw_ctx != nullptr) {
codec_ctx->hw_device_ctx = av_buffer_ref(hw_ctx);
codec_ctx->get_format = get_hw_format;
// Automatically determine the hardware pixel format
for (int i = 0;; i++) {
const AVCodecHWConfig *config = avcodec_get_hw_config(decoder, i);
if (config == nullptr) {
spdlog::error(
"Decoder {} does not support device type {}.",
decoder->name,
av_hwdevice_get_type_name(hw_type)
);
avcodec_free_context(&codec_ctx);
avformat_close_input(&ifmt_ctx);
return AVERROR(ENOSYS);
}
if (config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
config->device_type == hw_type) {
hw_pix_fmt = config->pix_fmt;
break;
}
}
}
if ((ret = avcodec_parameters_to_context(codec_ctx, video_stream->codecpar)) < 0) {
fprintf(stderr, "Failed to copy decoder parameters to input decoder context\n");
spdlog::error("Failed to copy decoder parameters to input decoder context");
return ret;
}
@@ -66,14 +103,14 @@ int init_decoder(
codec_ctx->pkt_timebase = video_stream->time_base;
codec_ctx->framerate = av_guess_frame_rate(ifmt_ctx, video_stream, NULL);
if ((ret = avcodec_open2(codec_ctx, dec, NULL)) < 0) {
fprintf(stderr, "Failed to open decoder for stream #%u\n", stream_index);
if ((ret = avcodec_open2(codec_ctx, decoder, NULL)) < 0) {
spdlog::error("Failed to open decoder for stream #{}", stream_index);
return ret;
}
*fmt_ctx = ifmt_ctx;
*dec_ctx = codec_ctx;
*video_stream_index = stream_index;
*vstream_idx = stream_index;
return 0;
}

View File

@@ -1,105 +1,173 @@
#include "encoder.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavcodec/codec.h>
#include <libavcodec/codec_id.h>
#include <libavfilter/avfilter.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavformat/avformat.h>
#include <libavutil/opt.h>
#include <libavutil/pixdesc.h>
#include <libavutil/rational.h>
}
#include <spdlog/spdlog.h>
#include "conversions.h"
#include "libvideo2x.h"
static enum AVPixelFormat get_encoder_default_pix_fmt(const AVCodec *encoder) {
const enum AVPixelFormat *p = encoder->pix_fmts;
if (!p) {
spdlog::error("No pixel formats supported by encoder");
return AV_PIX_FMT_NONE;
}
return *p;
}
int init_encoder(
const char *output_filename,
AVBufferRef *hw_ctx,
const char *out_fname,
AVFormatContext *ifmt_ctx,
AVFormatContext **ofmt_ctx,
AVCodecContext **enc_ctx,
AVCodecContext *dec_ctx,
EncoderConfig *encoder_config
EncoderConfig *encoder_config,
int vstream_idx,
int **stream_map
) {
AVFormatContext *fmt_ctx = NULL;
AVCodecContext *codec_ctx = NULL;
int stream_index = 0;
int ret;
avformat_alloc_output_context2(&fmt_ctx, NULL, NULL, output_filename);
avformat_alloc_output_context2(&fmt_ctx, NULL, NULL, out_fname);
if (!fmt_ctx) {
fprintf(stderr, "Could not create output context\n");
spdlog::error("Could not create output context");
return AVERROR_UNKNOWN;
}
// Create a new video stream
const AVCodec *enc = avcodec_find_encoder(encoder_config->codec);
if (!enc) {
fprintf(stderr, "Necessary encoder not found\n");
const AVCodec *encoder = avcodec_find_encoder(encoder_config->codec);
if (!encoder) {
spdlog::error(
"Required video encoder not found for vcodec {}",
avcodec_get_name(encoder_config->codec)
);
return AVERROR_ENCODER_NOT_FOUND;
}
// Create a new video stream in the output file
AVStream *out_stream = avformat_new_stream(fmt_ctx, NULL);
if (!out_stream) {
fprintf(stderr, "Failed allocating output stream\n");
spdlog::error("Failed to allocate the output video stream");
return AVERROR_UNKNOWN;
}
codec_ctx = avcodec_alloc_context3(enc);
codec_ctx = avcodec_alloc_context3(encoder);
if (!codec_ctx) {
fprintf(stderr, "Failed to allocate the encoder context\n");
spdlog::error("Failed to allocate the encoder context");
return AVERROR(ENOMEM);
}
// Set hardware device context
if (hw_ctx != nullptr) {
codec_ctx->hw_device_ctx = av_buffer_ref(hw_ctx);
}
// Set encoding parameters
codec_ctx->height = encoder_config->output_height;
codec_ctx->width = encoder_config->output_width;
codec_ctx->height = encoder_config->out_height;
codec_ctx->width = encoder_config->out_width;
codec_ctx->sample_aspect_ratio = dec_ctx->sample_aspect_ratio;
codec_ctx->pix_fmt = encoder_config->pix_fmt;
codec_ctx->time_base = av_inv_q(dec_ctx->framerate);
if (codec_ctx->time_base.num == 0 || codec_ctx->time_base.den == 0) {
codec_ctx->time_base = av_inv_q(av_guess_frame_rate(fmt_ctx, out_stream, NULL));
}
// Set the bit rate and other encoder parameters if needed
codec_ctx->bit_rate = encoder_config->bit_rate;
codec_ctx->gop_size = 60; // Keyframe interval
codec_ctx->max_b_frames = 3; // B-frames
codec_ctx->keyint_min = 60; // Maximum GOP size
char crf_str[16];
snprintf(crf_str, sizeof(crf_str), "%.f", encoder_config->crf);
if (encoder_config->codec == AV_CODEC_ID_H264 || encoder_config->codec == AV_CODEC_ID_HEVC) {
av_opt_set(codec_ctx->priv_data, "crf", crf_str, 0);
av_opt_set(codec_ctx->priv_data, "preset", encoder_config->preset, 0);
// Set the pixel format
if (encoder_config->pix_fmt != AV_PIX_FMT_NONE) {
// Use the specified pixel format
codec_ctx->pix_fmt = encoder_config->pix_fmt;
} else {
// Fall back to the default pixel format
codec_ctx->pix_fmt = get_encoder_default_pix_fmt(encoder);
if (codec_ctx->pix_fmt == AV_PIX_FMT_NONE) {
spdlog::error("Could not get the default pixel format for the encoder");
return AVERROR(EINVAL);
}
}
// Set the time base
codec_ctx->time_base = av_inv_q(dec_ctx->framerate);
if (codec_ctx->time_base.num == 0 || codec_ctx->time_base.den == 0) {
codec_ctx->time_base = av_inv_q(av_guess_frame_rate(ifmt_ctx, out_stream, NULL));
}
// Set the CRF and preset for any codecs that support it
char crf_str[16];
snprintf(crf_str, sizeof(crf_str), "%.f", static_cast<double>(encoder_config->crf));
av_opt_set(codec_ctx->priv_data, "crf", crf_str, 0);
av_opt_set(codec_ctx->priv_data, "preset", encoder_config->preset, 0);
if (fmt_ctx->oformat->flags & AVFMT_GLOBALHEADER) {
codec_ctx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}
if ((ret = avcodec_open2(codec_ctx, enc, NULL)) < 0) {
fprintf(stderr, "Cannot open video encoder\n");
if ((ret = avcodec_open2(codec_ctx, encoder, NULL)) < 0) {
spdlog::error("Cannot open video encoder");
return ret;
}
ret = avcodec_parameters_from_context(out_stream->codecpar, codec_ctx);
if (ret < 0) {
fprintf(stderr, "Failed to copy encoder parameters to output stream\n");
spdlog::error("Failed to copy encoder parameters to output video stream");
return ret;
}
out_stream->time_base = codec_ctx->time_base;
if (encoder_config->copy_streams) {
// Allocate the stream map
*stream_map =
reinterpret_cast<int *>(av_malloc_array(ifmt_ctx->nb_streams, sizeof(**stream_map)));
if (!*stream_map) {
spdlog::error("Could not allocate stream mapping");
return AVERROR(ENOMEM);
}
// Map the video stream
(*stream_map)[vstream_idx] = stream_index++;
// Loop through each stream in the input file
for (int i = 0; i < static_cast<int>(ifmt_ctx->nb_streams); i++) {
AVStream *in_stream = ifmt_ctx->streams[i];
AVCodecParameters *in_codecpar = in_stream->codecpar;
if (i == vstream_idx) {
// Video stream is already handled
continue;
}
if (in_codecpar->codec_type != AVMEDIA_TYPE_AUDIO &&
in_codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE) {
(*stream_map)[i] = -1;
continue;
}
// Create corresponding output stream
AVStream *out_copied_stream = avformat_new_stream(fmt_ctx, NULL);
if (!out_copied_stream) {
spdlog::error("Failed allocating output stream");
return AVERROR_UNKNOWN;
}
ret = avcodec_parameters_copy(out_copied_stream->codecpar, in_codecpar);
if (ret < 0) {
spdlog::error("Failed to copy codec parameters");
return ret;
}
out_copied_stream->codecpar->codec_tag = 0;
// Copy time base
out_copied_stream->time_base = in_stream->time_base;
(*stream_map)[i] = stream_index++;
}
}
// Open the output file
if (!(fmt_ctx->oformat->flags & AVFMT_NOFILE)) {
ret = avio_open(&fmt_ctx->pb, output_filename, AVIO_FLAG_WRITE);
ret = avio_open(&fmt_ctx->pb, out_fname, AVIO_FLAG_WRITE);
if (ret < 0) {
fprintf(stderr, "Could not open output file '%s'\n", output_filename);
spdlog::error("Could not open output file '{}'", out_fname);
return ret;
}
}
@@ -110,30 +178,40 @@ int init_encoder(
return 0;
}
int encode_and_write_frame(AVFrame *frame, AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx) {
int encode_and_write_frame(
AVFrame *frame,
AVCodecContext *enc_ctx,
AVFormatContext *ofmt_ctx,
int vstream_idx
) {
AVFrame *converted_frame = nullptr;
int ret;
// Convert the frame to the encoder's pixel format if needed
if (frame->format != enc_ctx->pix_fmt) {
AVFrame *converted_frame = convert_avframe_pix_fmt(frame, enc_ctx->pix_fmt);
converted_frame = convert_avframe_pix_fmt(frame, enc_ctx->pix_fmt);
if (!converted_frame) {
fprintf(stderr, "Error converting frame to encoder's pixel format\n");
spdlog::error("Error converting frame to encoder's pixel format");
return AVERROR_EXTERNAL;
}
converted_frame->pts = frame->pts;
frame = converted_frame;
}
AVPacket *enc_pkt = av_packet_alloc();
if (!enc_pkt) {
fprintf(stderr, "Could not allocate AVPacket\n");
spdlog::error("Could not allocate AVPacket");
return AVERROR(ENOMEM);
}
ret = avcodec_send_frame(enc_ctx, frame);
if (converted_frame != nullptr) {
ret = avcodec_send_frame(enc_ctx, converted_frame);
av_frame_free(&converted_frame);
} else {
ret = avcodec_send_frame(enc_ctx, frame);
}
if (ret < 0) {
fprintf(stderr, "Error sending frame to encoder\n");
spdlog::error("Error sending frame to encoder");
av_packet_free(&enc_pkt);
return ret;
}
@@ -144,20 +222,22 @@ int encode_and_write_frame(AVFrame *frame, AVCodecContext *enc_ctx, AVFormatCont
av_packet_unref(enc_pkt);
break;
} else if (ret < 0) {
fprintf(stderr, "Error during encoding\n");
spdlog::error("Error encoding frame");
av_packet_free(&enc_pkt);
return ret;
}
// Rescale packet timestamps
av_packet_rescale_ts(enc_pkt, enc_ctx->time_base, ofmt_ctx->streams[0]->time_base);
enc_pkt->stream_index = ofmt_ctx->streams[0]->index;
av_packet_rescale_ts(
enc_pkt, enc_ctx->time_base, ofmt_ctx->streams[vstream_idx]->time_base
);
enc_pkt->stream_index = vstream_idx;
// Write the packet
ret = av_interleaved_write_frame(ofmt_ctx, enc_pkt);
av_packet_unref(enc_pkt);
if (ret < 0) {
fprintf(stderr, "Error muxing packet\n");
spdlog::error("Error muxing packet");
av_packet_free(&enc_pkt);
return ret;
}
@@ -171,7 +251,7 @@ int flush_encoder(AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx) {
int ret;
AVPacket *enc_pkt = av_packet_alloc();
if (!enc_pkt) {
fprintf(stderr, "Could not allocate AVPacket\n");
spdlog::error("Could not allocate AVPacket");
return AVERROR(ENOMEM);
}
@@ -182,7 +262,7 @@ int flush_encoder(AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx) {
av_packet_unref(enc_pkt);
break;
} else if (ret < 0) {
fprintf(stderr, "Error during encoding\n");
spdlog::error("Error encoding frame");
av_packet_free(&enc_pkt);
return ret;
}
@@ -195,7 +275,7 @@ int flush_encoder(AVCodecContext *enc_ctx, AVFormatContext *ofmt_ctx) {
ret = av_interleaved_write_frame(ofmt_ctx, enc_pkt);
av_packet_unref(enc_pkt);
if (ret < 0) {
fprintf(stderr, "Error muxing packet\n");
spdlog::error("Error muxing packet");
av_packet_free(&enc_pkt);
return ret;
}

View File

@@ -1,4 +1,4 @@
#include <filesystem>
#include "fsutils.h"
#if _WIN32
#include <windows.h>
@@ -8,16 +8,16 @@
#include <cstring>
#endif
#include "fsutils.h"
#include <spdlog/spdlog.h>
#if _WIN32
std::filesystem::path get_executable_directory() {
static std::filesystem::path get_executable_directory() {
std::vector<wchar_t> filepath(MAX_PATH);
// Get the executable path, expanding the buffer if necessary
DWORD size = GetModuleFileNameW(NULL, filepath.data(), static_cast<DWORD>(filepath.size()));
if (size == 0) {
fprintf(stderr, "Error getting executable path: %lu\n", GetLastError());
spdlog::error("Error getting executable path: {}", GetLastError());
return std::filesystem::path();
}
@@ -26,7 +26,7 @@ std::filesystem::path get_executable_directory() {
filepath.resize(filepath.size() * 2);
size = GetModuleFileNameW(NULL, filepath.data(), static_cast<DWORD>(filepath.size()));
if (size == 0) {
fprintf(stderr, "Error getting executable path: %lu\n", GetLastError());
spdlog::error("Error getting executable path: {}", GetLastError());
return std::filesystem::path();
}
}
@@ -36,12 +36,12 @@ std::filesystem::path get_executable_directory() {
return execpath.parent_path();
}
#else // _WIN32
std::filesystem::path get_executable_directory() {
static std::filesystem::path get_executable_directory() {
std::error_code ec;
std::filesystem::path filepath = std::filesystem::read_symlink("/proc/self/exe", ec);
if (ec) {
fprintf(stderr, "Error reading /proc/self/exe: %s\n", ec.message().c_str());
spdlog::error("Error reading /proc/self/exe: {}", ec.message());
return std::filesystem::path();
}

View File

@@ -1,47 +1,28 @@
#include "libplacebo.h"
#include <stdio.h>
#include <stdlib.h>
#include <filesystem>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/avfilter.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavformat/avformat.h>
#include <libavutil/buffer.h>
#include <libavutil/hwcontext.h>
#include <libavutil/opt.h>
#include <libavutil/pixdesc.h>
#include <libavutil/rational.h>
#include <libswscale/swscale.h>
}
#include <spdlog/spdlog.h>
#include "fsutils.h"
int init_libplacebo(
AVBufferRef *hw_ctx,
AVFilterGraph **filter_graph,
AVFilterContext **buffersrc_ctx,
AVFilterContext **buffersink_ctx,
AVBufferRef **device_ctx,
AVCodecContext *dec_ctx,
int output_width,
int output_height,
int out_width,
int out_height,
const std::filesystem::path &shader_path
) {
char args[512];
int ret;
// Initialize the Vulkan hardware device
AVBufferRef *hw_device_ctx = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_VULKAN);
ret = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_VULKAN, NULL, NULL, 0);
if (ret < 0) {
fprintf(stderr, "Unable to initialize Vulkan device\n");
return ret;
}
AVFilterGraph *graph = avfilter_graph_alloc();
if (!graph) {
fprintf(stderr, "Unable to create filter graph.\n");
spdlog::error("Unable to create filter graph.");
return AVERROR(ENOMEM);
}
@@ -51,7 +32,7 @@ int init_libplacebo(
args,
sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:frame_rate=%d/%d:"
"pixel_aspect=%d/%d:colorspace=%d",
"pixel_aspect=%d/%d:colorspace=%d:range=%d",
dec_ctx->width,
dec_ctx->height,
dec_ctx->pix_fmt,
@@ -61,13 +42,13 @@ int init_libplacebo(
dec_ctx->framerate.den,
dec_ctx->sample_aspect_ratio.num,
dec_ctx->sample_aspect_ratio.den,
dec_ctx->colorspace
dec_ctx->colorspace,
dec_ctx->color_range
);
ret = avfilter_graph_create_filter(buffersrc_ctx, buffersrc, "in", args, NULL, graph);
if (ret < 0) {
fprintf(stderr, "Cannot create buffer source\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Cannot create buffer source.");
avfilter_graph_free(&graph);
return ret;
}
@@ -77,8 +58,7 @@ int init_libplacebo(
// Create the libplacebo filter
const AVFilter *libplacebo_filter = avfilter_get_by_name("libplacebo");
if (!libplacebo_filter) {
fprintf(stderr, "Filter 'libplacebo' not found\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Filter 'libplacebo' not found.");
avfilter_graph_free(&graph);
return AVERROR_FILTER_NOT_FOUND;
}
@@ -97,8 +77,8 @@ int init_libplacebo(
filter_args,
sizeof(filter_args),
"w=%d:h=%d:upscaler=ewa_lanczos:custom_shader_path=%s",
output_width,
output_height,
out_width,
out_height,
shader_path_string.c_str()
);
@@ -107,20 +87,20 @@ int init_libplacebo(
&libplacebo_ctx, libplacebo_filter, "libplacebo", filter_args, NULL, graph
);
if (ret < 0) {
fprintf(stderr, "Cannot create libplacebo filter\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Cannot create libplacebo filter.");
avfilter_graph_free(&graph);
return ret;
}
// Set the hardware device context to Vulkan
libplacebo_ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);
if (hw_ctx != nullptr) {
libplacebo_ctx->hw_device_ctx = av_buffer_ref(hw_ctx);
}
// Link buffersrc to libplacebo
ret = avfilter_link(last_filter, 0, libplacebo_ctx, 0);
if (ret < 0) {
fprintf(stderr, "Error connecting buffersrc to libplacebo filter\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Error connecting buffersrc to libplacebo filter.");
avfilter_graph_free(&graph);
return ret;
}
@@ -131,8 +111,7 @@ int init_libplacebo(
const AVFilter *buffersink = avfilter_get_by_name("buffersink");
ret = avfilter_graph_create_filter(buffersink_ctx, buffersink, "out", NULL, NULL, graph);
if (ret < 0) {
fprintf(stderr, "Cannot create buffer sink\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Cannot create buffer sink.");
avfilter_graph_free(&graph);
return ret;
}
@@ -140,8 +119,7 @@ int init_libplacebo(
// Link libplacebo to buffersink
ret = avfilter_link(last_filter, 0, *buffersink_ctx, 0);
if (ret < 0) {
fprintf(stderr, "Error connecting libplacebo filter to buffersink\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Error connecting libplacebo filter to buffersink.");
avfilter_graph_free(&graph);
return ret;
}
@@ -149,13 +127,11 @@ int init_libplacebo(
// Configure the filter graph
ret = avfilter_graph_config(graph, NULL);
if (ret < 0) {
fprintf(stderr, "Error configuring the filter graph\n");
av_buffer_unref(&hw_device_ctx);
spdlog::error("Error configuring the filter graph.");
avfilter_graph_free(&graph);
return ret;
}
*filter_graph = graph;
*device_ctx = hw_device_ctx;
return 0;
}

View File

@@ -1,23 +1,22 @@
#include "libplacebo_filter.h"
#include <cstdio>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavutil/buffer.h>
}
#include <spdlog/spdlog.h>
#include "fsutils.h"
#include "libplacebo.h"
#include "libplacebo_filter.h"
LibplaceboFilter::LibplaceboFilter(int width, int height, const std::filesystem::path &shader_path)
LibplaceboFilter::LibplaceboFilter(
int out_width,
int out_height,
const std::filesystem::path &shader_path
)
: filter_graph(nullptr),
buffersrc_ctx(nullptr),
buffersink_ctx(nullptr),
device_ctx(nullptr),
output_width(width),
output_height(height),
out_width(out_width),
out_height(out_height),
shader_path(std::move(shader_path)) {}
LibplaceboFilter::~LibplaceboFilter() {
@@ -29,17 +28,13 @@ LibplaceboFilter::~LibplaceboFilter() {
avfilter_free(buffersink_ctx);
buffersink_ctx = nullptr;
}
if (device_ctx) {
av_buffer_unref(&device_ctx);
device_ctx = nullptr;
}
if (filter_graph) {
avfilter_graph_free(&filter_graph);
filter_graph = nullptr;
}
}
int LibplaceboFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
int LibplaceboFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx, AVBufferRef *hw_ctx) {
// Construct the shader path
std::filesystem::path shader_full_path;
if (filepath_is_readable(shader_path)) {
@@ -47,66 +42,76 @@ int LibplaceboFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
shader_full_path = shader_path;
} else {
// Construct the fallback path using std::filesystem
shader_full_path =
find_resource_file(std::filesystem::path("models") / (shader_path.string() + ".glsl"));
shader_full_path = find_resource_file(
std::filesystem::path("models") / "libplacebo" / (shader_path.string() + ".glsl")
);
}
// Check if the shader file exists
if (!std::filesystem::exists(shader_full_path)) {
spdlog::error("libplacebo shader file not found: {}", shader_full_path.string());
return -1;
}
// Save the output time base
output_time_base = enc_ctx->time_base;
in_time_base = dec_ctx->time_base;
out_time_base = enc_ctx->time_base;
return init_libplacebo(
// Initialize the libplacebo filter
int ret = init_libplacebo(
hw_ctx,
&filter_graph,
&buffersrc_ctx,
&buffersink_ctx,
&device_ctx,
dec_ctx,
output_width,
output_height,
out_width,
out_height,
shader_full_path
);
// Set these resources to nullptr since they are already freed by `avfilter_graph_free`
if (ret < 0) {
buffersrc_ctx = nullptr;
buffersink_ctx = nullptr;
filter_graph = nullptr;
}
return ret;
}
AVFrame *LibplaceboFilter::process_frame(AVFrame *input_frame) {
int LibplaceboFilter::process_frame(AVFrame *in_frame, AVFrame **out_frame) {
int ret;
// Get the filtered frame
AVFrame *output_frame = av_frame_alloc();
if (output_frame == nullptr) {
fprintf(stderr, "Failed to allocate output frame\n");
return nullptr;
*out_frame = av_frame_alloc();
if (*out_frame == nullptr) {
spdlog::error("Failed to allocate output frame");
return -1;
}
// Feed the frame to the filter graph
ret = av_buffersrc_add_frame(buffersrc_ctx, input_frame);
ret = av_buffersrc_add_frame(buffersrc_ctx, in_frame);
if (ret < 0) {
fprintf(stderr, "Error while feeding the filter graph\n");
return nullptr;
spdlog::error("Error while feeding the filter graph");
return ret;
}
ret = av_buffersink_get_frame(buffersink_ctx, output_frame);
ret = av_buffersink_get_frame(buffersink_ctx, *out_frame);
if (ret < 0) {
av_frame_free(&output_frame);
if (ret != AVERROR(EAGAIN) && ret != AVERROR_EOF) {
char errbuf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error getting frame from filter graph: %s\n", errbuf);
return nullptr;
}
return (AVFrame *)-1;
av_frame_free(out_frame);
return ret;
}
// Rescale PTS to encoder's time base
output_frame->pts =
av_rescale_q(output_frame->pts, buffersink_ctx->inputs[0]->time_base, output_time_base);
(*out_frame)->pts = av_rescale_q((*out_frame)->pts, in_time_base, out_time_base);
// Return the processed frame to the caller
return output_frame;
return 0;
}
int LibplaceboFilter::flush(std::vector<AVFrame *> &processed_frames) {
int ret = av_buffersrc_add_frame(buffersrc_ctx, nullptr); // Signal EOF to the filter graph
int LibplaceboFilter::flush(std::vector<AVFrame *> &flushed_frames) {
int ret = av_buffersrc_add_frame(buffersrc_ctx, nullptr);
if (ret < 0) {
fprintf(stderr, "Error while flushing filter graph\n");
spdlog::error("Error while flushing filter graph");
return ret;
}
@@ -128,11 +133,10 @@ int LibplaceboFilter::flush(std::vector<AVFrame *> &processed_frames) {
}
// Rescale PTS to encoder's time base
filt_frame->pts =
av_rescale_q(filt_frame->pts, buffersink_ctx->inputs[0]->time_base, output_time_base);
filt_frame->pts = av_rescale_q(filt_frame->pts, in_time_base, out_time_base);
// Add to processed frames
processed_frames.push_back(filt_frame);
flushed_frames.push_back(filt_frame);
}
return 0;

View File

@@ -1,52 +1,71 @@
#include "libvideo2x.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <cstdint>
#include <thread>
// FFmpeg headers
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
}
#include <spdlog/spdlog.h>
#include <opencv2/videoio.hpp>
#include "decoder.h"
#include "encoder.h"
#include "filter.h"
#include "libplacebo_filter.h"
#include "libvideo2x.h"
#include "realesrgan_filter.h"
// Function to process frames using the selected filter (same as before)
int process_frames(
ProcessingStatus *status,
AVFormatContext *fmt_ctx,
/**
* @brief Process frames using the selected filter.
*
* @param[in] encoder_config Encoder configurations
* @param[in,out] proc_ctx Struct containing the processing context
* @param[in] ifmt_ctx Input format context
* @param[in] ofmt_ctx Output format context
* @param[in] dec_ctx Decoder context
* @param[in] enc_ctx Encoder context
* @param[in] filter Filter instance
* @param[in] vstream_idx Index of the video stream in the input format context
* @param[in] stream_map Array mapping input stream indexes to output stream indexes
* @param[in] benchmark Flag to enable benchmarking mode
* @return int 0 on success, negative value on error
*/
static int process_frames(
EncoderConfig *encoder_config,
VideoProcessingContext *proc_ctx,
AVFormatContext *ifmt_ctx,
AVFormatContext *ofmt_ctx,
AVCodecContext *dec_ctx,
AVCodecContext *enc_ctx,
Filter *filter,
int video_stream_index
int vstream_idx,
int *stream_map,
bool benchmark = false
) {
int ret;
AVPacket packet;
std::vector<AVFrame *> flushed_frames;
char errbuf[AV_ERROR_MAX_STRING_SIZE];
// Get the total number of frames in the video
AVStream *video_stream = fmt_ctx->streams[video_stream_index];
status->total_frames = video_stream->nb_frames;
// Get the total number of frames in the video with OpenCV
spdlog::debug("Unable to estimate total number of frames; reading with OpenCV");
cv::VideoCapture cap(ifmt_ctx->url);
if (!cap.isOpened()) {
spdlog::error("Failed to open video file with OpenCV");
return -1;
}
proc_ctx->total_frames = static_cast<int64_t>(cap.get(cv::CAP_PROP_FRAME_COUNT));
cap.release();
// If nb_frames is not set, calculate total frames using duration and frame rate
if (status->total_frames == 0) {
int64_t duration = video_stream->duration;
AVRational frame_rate = video_stream->avg_frame_rate;
if (duration != AV_NOPTS_VALUE && frame_rate.num != 0 && frame_rate.den != 0) {
status->total_frames = duration * frame_rate.num / frame_rate.den;
}
// Check if the total number of frames is still 0
if (proc_ctx->total_frames == 0) {
spdlog::warn("Unable to determine total number of frames");
} else {
spdlog::debug("{} frames to process", proc_ctx->total_frames);
}
// Get start time
status->start_time = time(NULL);
if (status->start_time == -1) {
proc_ctx->start_time = time(NULL);
if (proc_ctx->start_time == -1) {
perror("time");
}
@@ -57,85 +76,109 @@ int process_frames(
}
// Read frames from the input file
while (1) {
ret = av_read_frame(fmt_ctx, &packet);
while (!proc_ctx->abort) {
ret = av_read_frame(ifmt_ctx, &packet);
if (ret < 0) {
break; // End of file or error
if (ret == AVERROR_EOF) {
spdlog::debug("Reached end of file");
break;
}
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error reading packet: {}", errbuf);
goto end;
}
if (packet.stream_index == video_stream_index) {
if (packet.stream_index == vstream_idx) {
// Send the packet to the decoder
ret = avcodec_send_packet(dec_ctx, &packet);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error sending packet to decoder: %s\n", errbuf);
spdlog::error("Error sending packet to decoder: {}", errbuf);
av_packet_unref(&packet);
goto end;
}
// Receive and process frames from the decoder
while (1) {
while (!proc_ctx->abort) {
// Check if the processing is paused
if (proc_ctx->pause) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
continue;
}
ret = avcodec_receive_frame(dec_ctx, frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
spdlog::debug("Frame not ready");
break;
} else if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error decoding video frame: %s\n", errbuf);
spdlog::error("Error decoding video frame: {}", errbuf);
goto end;
}
// Process the frame using the selected filter
AVFrame *processed_frame = filter->process_frame(frame);
if (processed_frame != nullptr && processed_frame != (AVFrame *)-1) {
AVFrame *processed_frame = nullptr;
ret = filter->process_frame(frame, &processed_frame);
if (ret == 0 && processed_frame != nullptr) {
// Encode and write the processed frame
ret = encode_and_write_frame(processed_frame, enc_ctx, ofmt_ctx);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error encoding/writing frame: %s\n", errbuf);
av_frame_free(&processed_frame);
goto end;
if (!benchmark) {
ret =
encode_and_write_frame(processed_frame, enc_ctx, ofmt_ctx, vstream_idx);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error encoding/writing frame: {}", errbuf);
av_frame_free(&processed_frame);
goto end;
}
}
av_frame_free(&processed_frame);
status->processed_frames++;
} else if (processed_frame != (AVFrame *)-1) {
fprintf(stderr, "Error processing frame\n");
proc_ctx->processed_frames++;
} else if (ret != AVERROR(EAGAIN) && ret != AVERROR_EOF) {
spdlog::error("Filter returned an error");
goto end;
}
av_frame_unref(frame);
// Print the processing status
printf(
"\r[Video2X] Processing frame %ld/%ld (%.2f%%); time elapsed: %lds",
status->processed_frames,
status->total_frames,
status->processed_frames * 100.0 / status->total_frames,
time(NULL) - status->start_time
spdlog::debug(
"Processed frame {}/{}", proc_ctx->processed_frames, proc_ctx->total_frames
);
fflush(stdout);
}
} else if (encoder_config->copy_streams && stream_map[packet.stream_index] >= 0) {
AVStream *in_stream = ifmt_ctx->streams[packet.stream_index];
int out_stream_index = stream_map[packet.stream_index];
AVStream *out_stream = ofmt_ctx->streams[out_stream_index];
// Rescale packet timestamps
av_packet_rescale_ts(&packet, in_stream->time_base, out_stream->time_base);
packet.stream_index = out_stream_index;
// If copy streams is enabled, copy the packet to the output
ret = av_interleaved_write_frame(ofmt_ctx, &packet);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error muxing packet: {}", errbuf);
av_packet_unref(&packet);
return ret;
}
}
av_packet_unref(&packet);
}
// Print a newline after processing all frames
printf("\n");
// Flush the filter
ret = filter->flush(flushed_frames);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error flushing filter: %s\n", errbuf);
spdlog::error("Error flushing filter: {}", errbuf);
goto end;
}
// Encode and write all flushed frames
for (AVFrame *&flushed_frame : flushed_frames) {
ret = encode_and_write_frame(flushed_frame, enc_ctx, ofmt_ctx);
ret = encode_and_write_frame(flushed_frame, enc_ctx, ofmt_ctx, vstream_idx);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error encoding/writing flushed frame: %s\n", errbuf);
spdlog::error("Error encoding/writing flushed frame: {}", errbuf);
av_frame_free(&flushed_frame);
flushed_frame = nullptr;
goto end;
@@ -148,7 +191,7 @@ int process_frames(
ret = flush_encoder(enc_ctx, ofmt_ctx);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error flushing encoder: %s\n", errbuf);
spdlog::error("Error flushing encoder: {}", errbuf);
goto end;
}
@@ -163,84 +206,186 @@ end:
return ret;
}
// Cleanup helper function
void cleanup(
AVFormatContext *fmt_ctx,
// Cleanup resources after processing the video
static void cleanup(
AVFormatContext *ifmt_ctx,
AVFormatContext *ofmt_ctx,
AVCodecContext *dec_ctx,
AVCodecContext *enc_ctx,
AVBufferRef *hw_ctx,
int *stream_map,
Filter *filter
) {
if (filter) {
delete filter;
}
if (dec_ctx) {
avcodec_free_context(&dec_ctx);
}
if (enc_ctx) {
avcodec_free_context(&enc_ctx);
}
if (fmt_ctx) {
avformat_close_input(&fmt_ctx);
if (ifmt_ctx) {
avformat_close_input(&ifmt_ctx);
ifmt_ctx = nullptr;
}
if (ofmt_ctx && !(ofmt_ctx->oformat->flags & AVFMT_NOFILE)) {
avio_closep(&ofmt_ctx->pb);
ofmt_ctx->pb = nullptr;
}
if (ofmt_ctx) {
avformat_free_context(ofmt_ctx);
ofmt_ctx = nullptr;
}
if (dec_ctx) {
avcodec_free_context(&dec_ctx);
dec_ctx = nullptr;
}
if (enc_ctx) {
avcodec_free_context(&enc_ctx);
enc_ctx = nullptr;
}
if (hw_ctx) {
av_buffer_unref(&hw_ctx);
hw_ctx = nullptr;
}
if (stream_map) {
av_free(stream_map);
stream_map = nullptr;
}
if (filter) {
delete filter;
filter = nullptr;
}
}
// Main function to process the video
/**
* @brief Process a video file using the selected filter and encoder settings.
*
* @param[in] in_fname Path to the input video file
* @param[in] out_fname Path to the output video file
* @param[in] log_level Log level
* @param[in] benchmark Flag to enable benchmarking mode
* @param[in] hw_type Hardware device type
* @param[in] filter_config Filter configurations
* @param[in] encoder_config Encoder configurations
* @param[in,out] proc_ctx Video processing context
* @return int 0 on success, non-zero value on error
*/
extern "C" int process_video(
const char *input_filename,
const char *output_filename,
const char *in_fname,
const char *out_fname,
Libvideo2xLogLevel log_level,
bool benchmark,
AVHWDeviceType hw_type,
const FilterConfig *filter_config,
EncoderConfig *encoder_config,
ProcessingStatus *status
VideoProcessingContext *proc_ctx
) {
AVFormatContext *fmt_ctx = nullptr;
AVFormatContext *ifmt_ctx = nullptr;
AVFormatContext *ofmt_ctx = nullptr;
AVCodecContext *dec_ctx = nullptr;
AVCodecContext *enc_ctx = nullptr;
AVBufferRef *hw_ctx = nullptr;
int *stream_map = nullptr;
Filter *filter = nullptr;
int video_stream_index = -1;
int ret = 0; // Initialize ret with 0 to assume success
int vstream_idx = -1;
char errbuf[AV_ERROR_MAX_STRING_SIZE];
int ret = 0;
// Set the log level for FFmpeg and spdlog (libvideo2x)
switch (log_level) {
case LIBVIDEO2X_LOG_LEVEL_TRACE:
av_log_set_level(AV_LOG_TRACE);
spdlog::set_level(spdlog::level::trace);
break;
case LIBVIDEO2X_LOG_LEVEL_DEBUG:
av_log_set_level(AV_LOG_DEBUG);
spdlog::set_level(spdlog::level::debug);
break;
case LIBVIDEO2X_LOG_LEVEL_INFO:
av_log_set_level(AV_LOG_INFO);
spdlog::set_level(spdlog::level::info);
break;
case LIBVIDEO2X_LOG_LEVEL_WARNING:
av_log_set_level(AV_LOG_WARNING);
spdlog::set_level(spdlog::level::warn);
break;
case LIBVIDEO2X_LOG_LEVEL_ERROR:
av_log_set_level(AV_LOG_ERROR);
spdlog::set_level(spdlog::level::err);
break;
case LIBVIDEO2X_LOG_LEVEL_CRITICAL:
av_log_set_level(AV_LOG_FATAL);
spdlog::set_level(spdlog::level::critical);
break;
case LIBVIDEO2X_LOG_LEVEL_OFF:
av_log_set_level(AV_LOG_QUIET);
spdlog::set_level(spdlog::level::off);
break;
default:
av_log_set_level(AV_LOG_INFO);
spdlog::set_level(spdlog::level::info);
break;
}
// Initialize hardware device context
if (hw_type != AV_HWDEVICE_TYPE_NONE) {
ret = av_hwdevice_ctx_create(&hw_ctx, hw_type, NULL, NULL, 0);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error initializing hardware device context: {}", errbuf);
return ret;
}
}
// Initialize input
if (init_decoder(input_filename, &fmt_ctx, &dec_ctx, &video_stream_index) < 0) {
fprintf(stderr, "Failed to initialize decoder\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
ret = init_decoder(hw_type, hw_ctx, in_fname, &ifmt_ctx, &dec_ctx, &vstream_idx);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Failed to initialize decoder: {}", errbuf);
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return ret;
}
// Initialize output based on Libplacebo or RealESRGAN configuration
int output_width = 0, output_height = 0;
switch (filter_config->filter_type) {
case FILTER_LIBPLACEBO:
output_width = filter_config->config.libplacebo.output_width;
output_height = filter_config->config.libplacebo.output_height;
output_width = filter_config->config.libplacebo.out_width;
output_height = filter_config->config.libplacebo.out_height;
break;
case FILTER_REALESRGAN:
// Calculate the output dimensions based on the scaling factor
output_width = dec_ctx->width * filter_config->config.realesrgan.scaling_factor;
output_height = dec_ctx->height * filter_config->config.realesrgan.scaling_factor;
break;
default:
spdlog::error("Unknown filter type");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
spdlog::info("Output video dimensions: {}x{}", output_width, output_height);
// Initialize output encoder
encoder_config->output_width = output_width;
encoder_config->output_height = output_height;
if (init_encoder(output_filename, &ofmt_ctx, &enc_ctx, dec_ctx, encoder_config) < 0) {
fprintf(stderr, "Failed to initialize encoder\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
encoder_config->out_width = output_width;
encoder_config->out_height = output_height;
ret = init_encoder(
hw_ctx,
out_fname,
ifmt_ctx,
&ofmt_ctx,
&enc_ctx,
dec_ctx,
encoder_config,
vstream_idx,
&stream_map
);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Failed to initialize encoder: {}", errbuf);
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return ret;
}
// Write the output file header
if (avformat_write_header(ofmt_ctx, NULL) < 0) {
fprintf(stderr, "Error occurred when opening output file\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
ret = avformat_write_header(ofmt_ctx, NULL);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error occurred when opening output file: {}", errbuf);
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return ret;
}
// Create and initialize the appropriate filter
@@ -250,21 +395,21 @@ extern "C" int process_video(
// Validate shader path
if (!config.shader_path) {
fprintf(stderr, "Shader path must be provided for the libplacebo filter\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
spdlog::error("Shader path must be provided for the libplacebo filter");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
// Validate output dimensions
if (config.output_width <= 0 || config.output_height <= 0) {
fprintf(stderr, "Output dimensions must be provided for the libplacebo filter\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
if (config.out_width <= 0 || config.out_height <= 0) {
spdlog::error("Output dimensions must be provided for the libplacebo filter");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
filter = new LibplaceboFilter(
config.output_width, config.output_height, std::filesystem::path(config.shader_path)
);
filter = new LibplaceboFilter{
config.out_width, config.out_height, std::filesystem::path(config.shader_path)
};
break;
}
case FILTER_REALESRGAN: {
@@ -272,56 +417,67 @@ extern "C" int process_video(
// Validate model name
if (!config.model) {
fprintf(stderr, "Model name must be provided for the RealESRGAN filter\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
spdlog::error("Model name must be provided for the RealESRGAN filter");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
// Validate scaling factor
if (config.scaling_factor <= 0) {
fprintf(stderr, "Scaling factor must be provided for the RealESRGAN filter\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
spdlog::error("Scaling factor must be provided for the RealESRGAN filter");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
filter = new RealesrganFilter(
filter = new RealesrganFilter{
config.gpuid, config.tta_mode, config.scaling_factor, config.model
);
};
break;
}
default:
fprintf(stderr, "Unknown filter type\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
spdlog::error("Unknown filter type");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return -1;
}
// Initialize the filter
if (filter->init(dec_ctx, enc_ctx) < 0) {
fprintf(stderr, "Failed to initialize filter\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
ret = filter->init(dec_ctx, enc_ctx, hw_ctx);
if (ret < 0) {
spdlog::error("Failed to initialize filter");
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return ret;
}
// Process frames
if ((ret =
process_frames(status, fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter, video_stream_index)
) < 0) {
fprintf(stderr, "Error processing frames\n");
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
return 1;
ret = process_frames(
encoder_config,
proc_ctx,
ifmt_ctx,
ofmt_ctx,
dec_ctx,
enc_ctx,
filter,
vstream_idx,
stream_map,
benchmark
);
if (ret < 0) {
av_strerror(ret, errbuf, sizeof(errbuf));
spdlog::error("Error processing frames: {}", errbuf);
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
return ret;
}
// Write the output file trailer
av_write_trailer(ofmt_ctx);
// Cleanup before returning
cleanup(fmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, filter);
cleanup(ifmt_ctx, ofmt_ctx, dec_ctx, enc_ctx, hw_ctx, stream_map, filter);
if (ret < 0 && ret != AVERROR_EOF) {
char errbuf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, errbuf, sizeof(errbuf));
fprintf(stderr, "Error occurred: %s\n", errbuf);
return 1;
spdlog::error("Error occurred: {}", errbuf);
return ret;
}
return 0;
}

View File

@@ -1,18 +1,13 @@
#include "realesrgan_filter.h"
#include <cstdint>
#include <cstdio>
#include <filesystem>
#include <string>
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavutil/avutil.h>
#include <libavutil/imgutils.h>
}
#include <spdlog/spdlog.h>
#include "conversions.h"
#include "fsutils.h"
#include "realesrgan.h"
#include "realesrgan_filter.h"
RealesrganFilter::RealesrganFilter(
int gpuid,
@@ -37,16 +32,16 @@ RealesrganFilter::~RealesrganFilter() {
}
}
int RealesrganFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
int RealesrganFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx, AVBufferRef *_) {
// Construct the model paths using std::filesystem
std::filesystem::path model_param_path;
std::filesystem::path model_bin_path;
if (model) {
// Find the model paths by model name if provided
model_param_path = std::filesystem::path("models") /
model_param_path = std::filesystem::path("models") / "realesrgan" /
(std::string(model) + "-x" + std::to_string(scaling_factor) + ".param");
model_bin_path = std::filesystem::path("models") /
model_bin_path = std::filesystem::path("models") / "realesrgan" /
(std::string(model) + "-x" + std::to_string(scaling_factor) + ".bin");
} else if (!custom_model_param_path.empty() && !custom_model_bin_path.empty()) {
// Use the custom model paths if provided
@@ -54,7 +49,7 @@ int RealesrganFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
model_bin_path = custom_model_bin_path;
} else {
// Neither model name nor custom model paths provided
fprintf(stderr, "Model or model paths must be provided for RealESRGAN filter\n");
spdlog::error("Model or model paths must be provided for RealESRGAN filter");
return -1;
}
@@ -62,17 +57,27 @@ int RealesrganFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
std::filesystem::path model_param_full_path = find_resource_file(model_param_path);
std::filesystem::path model_bin_full_path = find_resource_file(model_bin_path);
// Check if the model files exist
if (!std::filesystem::exists(model_param_full_path)) {
spdlog::error("RealESRGAN model param file not found: {}", model_param_full_path.string());
return -1;
}
if (!std::filesystem::exists(model_bin_full_path)) {
spdlog::error("RealESRGAN model bin file not found: {}", model_bin_full_path.string());
return -1;
}
// Create a new RealESRGAN instance
realesrgan = new RealESRGAN(gpuid, tta_mode);
// Store the time bases
input_time_base = dec_ctx->time_base;
output_time_base = enc_ctx->time_base;
output_pix_fmt = enc_ctx->pix_fmt;
in_time_base = dec_ctx->time_base;
out_time_base = enc_ctx->time_base;
out_pix_fmt = enc_ctx->pix_fmt;
// Load the model
if (realesrgan->load(model_param_full_path, model_bin_full_path) != 0) {
fprintf(stderr, "Failed to load RealESRGAN model\n");
spdlog::error("Failed to load RealESRGAN model");
return -1;
}
@@ -95,35 +100,33 @@ int RealesrganFilter::init(AVCodecContext *dec_ctx, AVCodecContext *enc_ctx) {
return 0;
}
AVFrame *RealesrganFilter::process_frame(AVFrame *input_frame) {
int RealesrganFilter::process_frame(AVFrame *in_frame, AVFrame **out_frame) {
int ret;
// Convert the input frame to RGB24
ncnn::Mat input_mat = avframe_to_ncnn_mat(input_frame);
if (input_mat.empty()) {
fprintf(stderr, "Failed to convert AVFrame to ncnn::Mat\n");
return nullptr;
ncnn::Mat in_mat = avframe_to_ncnn_mat(in_frame);
if (in_mat.empty()) {
spdlog::error("Failed to convert AVFrame to ncnn::Mat");
return -1;
}
// Allocate space for ouptut ncnn::Mat
int output_width = input_mat.w * realesrgan->scale;
int output_height = input_mat.h * realesrgan->scale;
ncnn::Mat output_mat = ncnn::Mat(output_width, output_height, (size_t)3, 3);
int output_width = in_mat.w * realesrgan->scale;
int output_height = in_mat.h * realesrgan->scale;
ncnn::Mat out_mat = ncnn::Mat(output_width, output_height, static_cast<size_t>(3), 3);
if (realesrgan->process(input_mat, output_mat) != 0) {
fprintf(stderr, "RealESRGAN processing failed\n");
return nullptr;
ret = realesrgan->process(in_mat, out_mat);
if (ret != 0) {
spdlog::error("RealESRGAN processing failed");
return ret;
}
// Convert ncnn::Mat to AVFrame
AVFrame *output_frame = ncnn_mat_to_avframe(output_mat, output_pix_fmt);
*out_frame = ncnn_mat_to_avframe(out_mat, out_pix_fmt);
// Rescale PTS to encoder's time base
output_frame->pts = av_rescale_q(input_frame->pts, input_time_base, output_time_base);
(*out_frame)->pts = av_rescale_q(in_frame->pts, in_time_base, out_time_base);
// Return the processed frame to the caller
return output_frame;
}
int RealesrganFilter::flush(std::vector<AVFrame *> &processed_frames) {
// No special flushing needed for RealESRGAN
return 0;
return ret;
}

View File

@@ -1,24 +1,41 @@
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <threads.h>
#include <time.h>
#ifdef _WIN32
#include <conio.h>
#else
#include <fcntl.h>
#include <termios.h>
#include <unistd.h>
#endif
#include <libavutil/hwcontext.h>
#include <libavutil/pixdesc.h>
#include <libavutil/pixfmt.h>
#include <libvideo2x.h>
#include <libvideo2x/libvideo2x.h>
#include <libvideo2x/version.h>
#include "getopt.h"
const char *VIDEO2X_VERSION = "6.0.0";
// Define command line options
static struct option long_options[] = {
{"loglevel", required_argument, NULL, 0},
{"noprogress", no_argument, NULL, 0},
{"version", no_argument, NULL, 'v'},
{"help", no_argument, NULL, 0},
// General options
{"input", required_argument, NULL, 'i'},
{"output", required_argument, NULL, 'o'},
{"filter", required_argument, NULL, 'f'},
{"version", no_argument, NULL, 'v'},
{"help", no_argument, NULL, 0},
{"hwaccel", required_argument, NULL, 'a'},
{"nocopystreams", no_argument, NULL, 0},
{"benchmark", no_argument, NULL, 0},
// Encoder options
{"codec", required_argument, NULL, 'c'},
@@ -27,7 +44,7 @@ static struct option long_options[] = {
{"bitrate", required_argument, NULL, 'b'},
{"crf", required_argument, NULL, 'q'},
// Libplacebo options
// libplacebo options
{"shader", required_argument, NULL, 's'},
{"width", required_argument, NULL, 'w'},
{"height", required_argument, NULL, 'h'},
@@ -39,12 +56,27 @@ static struct option long_options[] = {
{0, 0, 0, 0}
};
// List of valid RealESRGAN models
const char *valid_realesrgan_models[] = {
"realesrgan-plus",
"realesrgan-plus-anime",
"realesr-animevideov3",
};
// Indicate if a newline needs to be printed before the next output
bool newline_required = false;
// Structure to hold parsed arguments
struct arguments {
// General options
const char *input_filename;
const char *output_filename;
const char *loglevel;
bool noprogress;
const char *in_fname;
const char *out_fname;
const char *filter_type;
const char *hwaccel;
bool nocopystreams;
bool benchmark;
// Encoder options
const char *codec;
@@ -55,8 +87,8 @@ struct arguments {
// libplacebo options
const char *shader_path;
int output_width;
int output_height;
int out_width;
int out_height;
// RealESRGAN options
int gpuid;
@@ -64,42 +96,81 @@ struct arguments {
int scaling_factor;
};
const char *valid_models[] = {
"realesrgan-plus",
"realesrgan-plus-anime",
"realesr-animevideov3",
struct ProcessVideoThreadArguments {
struct arguments *arguments;
enum AVHWDeviceType hw_device_type;
struct FilterConfig *filter_config;
struct EncoderConfig *encoder_config;
struct VideoProcessingContext *proc_ctx;
};
// Set UNIX terminal input to non-blocking mode
#ifndef _WIN32
void set_nonblocking_input(bool enable) {
static struct termios oldt, newt;
if (enable) {
tcgetattr(STDIN_FILENO, &oldt);
newt = oldt;
newt.c_lflag &= ~(tcflag_t)(ICANON | ECHO);
tcsetattr(STDIN_FILENO, TCSANOW, &newt);
fcntl(STDIN_FILENO, F_SETFL, O_NONBLOCK);
} else {
tcsetattr(STDIN_FILENO, TCSANOW, &oldt);
fcntl(STDIN_FILENO, F_SETFL, 0);
}
}
#endif
// Newline-safe log callback for FFmpeg
void newline_safe_ffmpeg_log_callback(void *ptr, int level, const char *fmt, va_list vl) {
if (level <= av_log_get_level() && newline_required) {
putchar('\n');
newline_required = false;
}
av_log_default_callback(ptr, level, fmt, vl);
}
int is_valid_realesrgan_model(const char *model) {
if (!model) {
return 0;
}
for (int i = 0; i < sizeof(valid_models) / sizeof(valid_models[0]); i++) {
if (strcmp(model, valid_models[i]) == 0) {
for (unsigned long i = 0;
i < sizeof(valid_realesrgan_models) / sizeof(valid_realesrgan_models[0]);
i++) {
if (strcmp(model, valid_realesrgan_models[i]) == 0) {
return 1;
}
}
return 0;
}
void print_help() {
void print_help(void) {
printf("Usage: video2x [OPTIONS]\n");
printf("\nGeneral Options:\n");
printf("\nOptions:\n");
printf(" --loglevel Set log level \n");
printf(" (trace, debug, info, warn, error, critical, none)\n");
printf(" --noprogress Do not display the progress bar\n");
printf(" -v, --version Print program version\n");
printf(" -?, --help Display this help page\n");
printf("\nGeneral Processing Options:\n");
printf(" -i, --input Input video file path\n");
printf(" -o, --output Output video file path\n");
printf(" -f, --filter Filter to use: 'libplacebo' or 'realesrgan'\n");
printf(" -v, --version Print program version\n");
printf(" --help Display this help page\n");
printf(" -a, --hwaccel Hardware acceleration method (default: none)\n");
printf(" --nocopystreams Do not copy audio and subtitle streams\n");
printf(" --benchmark Discard processed frames and calculate average FPS\n");
printf("\nEncoder Options (Optional):\n");
printf(" -c, --codec Output codec (default: libx264)\n");
printf(" -p, --preset Encoder preset (default: veryslow)\n");
printf(" -x, --pixfmt Output pixel format (default: yuv420p)\n");
printf(" -b, --bitrate Bitrate in bits per second (default: 2000000)\n");
printf(" -q, --crf Constant Rate Factor (default: 17.0)\n");
printf(" -p, --preset Encoder preset (default: slow)\n");
printf(" -x, --pixfmt Output pixel format (default: auto)\n");
printf(" -b, --bitrate Bitrate in bits per second (default: 0 (VBR))\n");
printf(" -q, --crf Constant Rate Factor (default: 20.0)\n");
printf("\nlibplacebo Options:\n");
printf(" -s, --shader Name or path to custom GLSL shader file\n");
printf(" -s, --shader Name or path of the GLSL shader file to use \n");
printf(" (built-in: 'anime4k-a', 'anime4k-b', 'anime4k-c',\n");
printf(" 'anime4k-a+a', 'anime4k-b+b', 'anime4k-c+a')\n");
printf(" -w, --width Output width\n");
printf(" -h, --height Output height\n");
@@ -107,6 +178,10 @@ void print_help() {
printf(" -g, --gpuid Vulkan GPU ID (default: 0)\n");
printf(" -m, --model Name of the model to use\n");
printf(" -r, --scale Scaling factor (2, 3, or 4)\n");
printf("\nExamples Usage:\n");
printf(" video2x -i in.mp4 -o out.mp4 -f libplacebo -s anime4k-a+a -w 3840 -h 2160\n");
printf(" video2x -i in.mp4 -o out.mp4 -f realesrgan -m realesr-animevideov3 -r 4\n");
}
void parse_arguments(int argc, char **argv, struct arguments *arguments) {
@@ -114,39 +189,48 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
int c;
// Default argument values
arguments->input_filename = NULL;
arguments->output_filename = NULL;
arguments->loglevel = "info";
arguments->noprogress = false;
arguments->in_fname = NULL;
arguments->out_fname = NULL;
arguments->filter_type = NULL;
arguments->hwaccel = "none";
arguments->nocopystreams = false;
arguments->benchmark = false;
// Encoder options
arguments->codec = "libx264";
arguments->preset = "veryslow";
arguments->pix_fmt = "yuv420p";
arguments->bitrate = 2 * 1000 * 1000;
arguments->crf = 17.0;
arguments->preset = "slow";
arguments->pix_fmt = NULL;
arguments->bitrate = 0;
arguments->crf = 20.0;
// libplacebo options
arguments->shader_path = NULL;
arguments->output_width = 0;
arguments->output_height = 0;
arguments->out_width = 0;
arguments->out_height = 0;
// RealESRGAN options
arguments->gpuid = 0;
arguments->model = NULL;
arguments->scaling_factor = 0;
while ((c = getopt_long(argc, argv, "i:o:f:c:x:p:b:q:s:w:h:r:m:v", long_options, &option_index)
) != -1) {
while ((c = getopt_long(
argc, argv, "i:o:f:a:c:x:p:b:q:s:w:h:r:m:v", long_options, &option_index
)) != -1) {
switch (c) {
case 'i':
arguments->input_filename = optarg;
arguments->in_fname = optarg;
break;
case 'o':
arguments->output_filename = optarg;
arguments->out_fname = optarg;
break;
case 'f':
arguments->filter_type = optarg;
break;
case 'a':
arguments->hwaccel = optarg;
break;
case 'c':
arguments->codec = optarg;
break;
@@ -164,7 +248,7 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
}
break;
case 'q':
arguments->crf = atof(optarg);
arguments->crf = (float)atof(optarg);
if (arguments->crf < 0.0 || arguments->crf > 51.0) {
fprintf(stderr, "Error: CRF must be between 0 and 51.\n");
exit(1);
@@ -174,15 +258,15 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
arguments->shader_path = optarg;
break;
case 'w':
arguments->output_width = atoi(optarg);
if (arguments->output_width <= 0) {
arguments->out_width = atoi(optarg);
if (arguments->out_width <= 0) {
fprintf(stderr, "Error: Output width must be greater than 0.\n");
exit(1);
}
break;
case 'h':
arguments->output_height = atoi(optarg);
if (arguments->output_height <= 0) {
arguments->out_height = atoi(optarg);
if (arguments->out_height <= 0) {
fprintf(stderr, "Error: Output height must be greater than 0.\n");
exit(1);
}
@@ -195,7 +279,8 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
if (!is_valid_realesrgan_model(arguments->model)) {
fprintf(
stderr,
"Error: Invalid model specified. Must be 'realesrgan-plus', 'realesrgan-plus-anime', or 'realesr-animevideov3'.\n"
"Error: Invalid model specified. Must be 'realesrgan-plus', "
"'realesrgan-plus-anime', or 'realesr-animevideov3'.\n"
);
exit(1);
}
@@ -209,12 +294,20 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
}
break;
case 'v':
printf("video2x %s\n", VIDEO2X_VERSION);
printf("Video2X version %s\n", LIBVIDEO2X_VERSION_STRING);
exit(0);
case 0: // Long-only options without short equivalents (e.g., help)
if (strcmp(long_options[option_index].name, "help") == 0) {
case 0: // Long-only options without short equivalents
if (strcmp(long_options[option_index].name, "loglevel") == 0) {
arguments->loglevel = optarg;
} else if (strcmp(long_options[option_index].name, "noprogress") == 0) {
arguments->noprogress = true;
} else if (strcmp(long_options[option_index].name, "help") == 0) {
print_help();
exit(0);
} else if (strcmp(long_options[option_index].name, "nocopystreams") == 0) {
arguments->nocopystreams = true;
} else if (strcmp(long_options[option_index].name, "benchmark") == 0) {
arguments->benchmark = true;
}
break;
default:
@@ -224,8 +317,13 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
}
// Check for required arguments
if (!arguments->input_filename || !arguments->output_filename) {
fprintf(stderr, "Error: Input and output files are required.\n");
if (!arguments->in_fname) {
fprintf(stderr, "Error: Input file path is required.\n");
exit(1);
}
if (!arguments->out_fname && !arguments->benchmark) {
fprintf(stderr, "Error: Output file path is required.\n");
exit(1);
}
@@ -235,11 +333,11 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
}
if (strcmp(arguments->filter_type, "libplacebo") == 0) {
if (!arguments->shader_path || arguments->output_width == 0 ||
arguments->output_height == 0) {
if (!arguments->shader_path || arguments->out_width == 0 || arguments->out_height == 0) {
fprintf(
stderr,
"Error: For libplacebo, shader name/path (-s), width (-w), and height (-e) are required.\n"
"Error: For libplacebo, shader name/path (-s), width (-w), "
"and height (-e) are required.\n"
);
exit(1);
}
@@ -253,7 +351,63 @@ void parse_arguments(int argc, char **argv, struct arguments *arguments) {
}
}
enum Libvideo2xLogLevel parse_log_level(const char *level_name) {
if (strcmp(level_name, "trace") == 0) {
return LIBVIDEO2X_LOG_LEVEL_TRACE;
} else if (strcmp(level_name, "debug") == 0) {
return LIBVIDEO2X_LOG_LEVEL_DEBUG;
} else if (strcmp(level_name, "info") == 0) {
return LIBVIDEO2X_LOG_LEVEL_INFO;
} else if (strcmp(level_name, "warning") == 0) {
return LIBVIDEO2X_LOG_LEVEL_WARNING;
} else if (strcmp(level_name, "error") == 0) {
return LIBVIDEO2X_LOG_LEVEL_ERROR;
} else if (strcmp(level_name, "critical") == 0) {
return LIBVIDEO2X_LOG_LEVEL_CRITICAL;
} else if (strcmp(level_name, "off") == 0) {
return LIBVIDEO2X_LOG_LEVEL_OFF;
} else {
fprintf(stderr, "Warning: Invalid log level specified. Defaulting to 'info'.\n");
return LIBVIDEO2X_LOG_LEVEL_INFO;
}
}
// Wrapper function for video processing thread
int process_video_thread(void *arg) {
struct ProcessVideoThreadArguments *thread_args = (struct ProcessVideoThreadArguments *)arg;
// Extract individual arguments
enum Libvideo2xLogLevel log_level = parse_log_level(thread_args->arguments->loglevel);
struct arguments *arguments = thread_args->arguments;
enum AVHWDeviceType hw_device_type = thread_args->hw_device_type;
struct FilterConfig *filter_config = thread_args->filter_config;
struct EncoderConfig *encoder_config = thread_args->encoder_config;
struct VideoProcessingContext *proc_ctx = thread_args->proc_ctx;
// Call the process_video function
int result = process_video(
arguments->in_fname,
arguments->out_fname,
log_level,
arguments->benchmark,
hw_device_type,
filter_config,
encoder_config,
proc_ctx
);
proc_ctx->completed = true;
return result;
}
int main(int argc, char **argv) {
// Print help if no arguments are provided
if (argc < 2) {
print_help();
return 1;
}
// Parse command line arguments
struct arguments arguments;
parse_arguments(argc, argv, &arguments);
@@ -261,8 +415,8 @@ int main(int argc, char **argv) {
struct FilterConfig filter_config;
if (strcmp(arguments.filter_type, "libplacebo") == 0) {
filter_config.filter_type = FILTER_LIBPLACEBO;
filter_config.config.libplacebo.output_width = arguments.output_width;
filter_config.config.libplacebo.output_height = arguments.output_height;
filter_config.config.libplacebo.out_width = arguments.out_width;
filter_config.config.libplacebo.out_height = arguments.out_height;
filter_config.config.libplacebo.shader_path = arguments.shader_path;
} else if (strcmp(arguments.filter_type, "realesrgan") == 0) {
filter_config.filter_type = FILTER_REALESRGAN;
@@ -283,16 +437,20 @@ int main(int argc, char **argv) {
}
// Parse pixel format to AVPixelFormat
enum AVPixelFormat pix_fmt = av_get_pix_fmt(arguments.pix_fmt);
if (pix_fmt == AV_PIX_FMT_NONE) {
fprintf(stderr, "Error: Invalid pixel format '%s'.\n", arguments.pix_fmt);
return 1;
enum AVPixelFormat pix_fmt = AV_PIX_FMT_NONE;
if (arguments.pix_fmt) {
pix_fmt = av_get_pix_fmt(arguments.pix_fmt);
if (pix_fmt == AV_PIX_FMT_NONE) {
fprintf(stderr, "Error: Invalid pixel format '%s'.\n", arguments.pix_fmt);
return 1;
}
}
// Setup encoder configuration
struct EncoderConfig encoder_config = {
.output_width = 0, // To be filled by libvideo2x
.output_height = 0, // To be filled by libvideo2x
.out_width = 0, // To be filled by libvideo2x
.out_height = 0, // To be filled by libvideo2x
.copy_streams = !arguments.nocopystreams,
.codec = codec->id,
.pix_fmt = pix_fmt,
.preset = arguments.preset,
@@ -300,26 +458,142 @@ int main(int argc, char **argv) {
.crf = arguments.crf,
};
// Setup struct to store processing status
struct ProcessingStatus status = {0};
// Process the video
if (process_video(
arguments.input_filename,
arguments.output_filename,
&filter_config,
&encoder_config,
&status
)) {
fprintf(stderr, "Video processing failed.\n");
return 1;
// Parse hardware acceleration method
enum AVHWDeviceType hw_device_type = AV_HWDEVICE_TYPE_NONE;
if (strcmp(arguments.hwaccel, "none") != 0) {
hw_device_type = av_hwdevice_find_type_by_name(arguments.hwaccel);
if (hw_device_type == AV_HWDEVICE_TYPE_NONE) {
fprintf(stderr, "Error: Invalid hardware device type '%s'.\n", arguments.hwaccel);
return 1;
}
}
// Setup struct to store processing context
struct VideoProcessingContext proc_ctx = {
.processed_frames = 0,
.total_frames = 0,
.start_time = time(NULL),
.pause = false,
.abort = false,
.completed = false
};
// Create a ThreadArguments struct to hold all the arguments for the thread
struct ProcessVideoThreadArguments thread_args = {
.arguments = &arguments,
.hw_device_type = hw_device_type,
.filter_config = &filter_config,
.encoder_config = &encoder_config,
.proc_ctx = &proc_ctx
};
// Register a newline-safe log callback for FFmpeg
// This will ensure that log messages are printed on a new line after the progress bar
av_log_set_callback(newline_safe_ffmpeg_log_callback);
// Create a thread for video processing
thrd_t processing_thread;
if (thrd_create(&processing_thread, process_video_thread, &thread_args) != thrd_success) {
fprintf(stderr, "Failed to create processing thread\n");
return 1;
}
printf("Video processing started; press SPACE to pause/resume, 'q' to abort.\n");
// Enable non-blocking input
#ifndef _WIN32
set_nonblocking_input(true);
#endif
// Main thread loop to display progress and handle input
while (!proc_ctx.completed) {
// Check for key presses
int ch = -1;
// Check for key press
#ifdef _WIN32
if (_kbhit()) {
ch = _getch();
}
#else
ch = getchar();
#endif
if (ch == ' ' || ch == '\n') {
// Toggle pause state
proc_ctx.pause = !proc_ctx.pause;
if (proc_ctx.pause) {
printf("\nProcessing paused. Press SPACE to resume, 'q' to abort.\n");
} else {
printf("Resuming processing...\n");
}
} else if (ch == 'q' || ch == 'Q') {
// Abort processing
printf("\nAborting processing...\n");
proc_ctx.abort = true;
newline_required = false;
break;
}
// Display progress
if (!arguments.noprogress && !proc_ctx.pause && proc_ctx.total_frames > 0) {
printf(
"\rProcessing frame %ld/%ld (%.2f%%); time elapsed: %lds",
proc_ctx.processed_frames,
proc_ctx.total_frames,
proc_ctx.total_frames > 0
? (double)proc_ctx.processed_frames * 100.0 / (double)proc_ctx.total_frames
: 0.0,
time(NULL) - proc_ctx.start_time
);
fflush(stdout);
newline_required = true;
}
// Sleep for 50ms
thrd_sleep(&(struct timespec){.tv_sec = 0, .tv_nsec = 100000000}, NULL);
}
// Restore terminal to blocking mode
#ifndef _WIN32
set_nonblocking_input(false);
#endif
// Join the processing thread to ensure it completes before exiting
int process_result;
thrd_join(processing_thread, &process_result);
// Print a newline if progress bar was displayed
if (newline_required) {
putchar('\n');
}
// Print final message based on processing result
if (proc_ctx.abort) {
fprintf(stderr, "Video processing aborted\n");
return 2;
} else if (process_result != 0) {
fprintf(stderr, "Video processing failed\n");
return process_result;
} else {
printf("Video processing completed successfully\n");
}
// Calculate statistics
time_t time_elapsed = time(NULL) - proc_ctx.start_time;
float average_speed_fps =
(float)proc_ctx.processed_frames / (time_elapsed > 0 ? (float)time_elapsed : 1);
// Print processing summary
printf("====== Video2X Processing summary ======\n");
printf("Video file processed: %s\n", arguments.input_filename);
printf("Total frames processed: %ld\n", status.processed_frames);
printf("Total time taken: %lds\n", time(NULL) - status.start_time);
printf("Output written to: %s\n", arguments.output_filename);
printf("====== Video2X %s summary ======\n", arguments.benchmark ? "Benchmark" : "Processing");
printf("Video file processed: %s\n", arguments.in_fname);
printf("Total frames processed: %ld\n", proc_ctx.processed_frames);
printf("Total time taken: %lds\n", time_elapsed);
printf("Average processing speed: %.2f FPS\n", average_speed_fps);
// Print additional information if not in benchmark mode
if (!arguments.benchmark) {
printf("Output written to: %s\n", arguments.out_fname);
}
return 0;
}

1
third_party/opencv vendored Submodule

Submodule third_party/opencv added at 71d3237a09

1
third_party/spdlog vendored Submodule

Submodule third_party/spdlog added at e593f6695c