Compare commits

..

28 Commits

Author SHA1 Message Date
Kaleb S. KEITHLEY 3c5598efe8 rebuild with libarrow-23.0.1, cmake-4-isms, lua-5.5 2026-02-17 12:52:35 -05:00
Kaleb S. KEITHLEY 8fa2f6e626 cmake-4-isms, use system zstd 2026-02-17 09:26:54 -05:00
Kaleb S. KEITHLEY 3a3173ee12 again 2026-02-16 19:59:06 -05:00
Kaleb S. KEITHLEY 0ceeb07f51 ceph-20.2.0, rebuild with libarrow-23.0.0, side tag f44-build-side-127546 2026-01-28 17:45:31 -05:00
Kaleb S. KEITHLEY df9f29a42f ceph-20.2.0, rhbz#2433903 2026-01-28 10:22:29 -05:00
Fedora Release Engineering 7a652c7616 Rebuilt for https://fedoraproject.org/wiki/Fedora_44_Mass_Rebuild 2026-01-16 08:52:11 +00:00
Fedora Release Engineering d26f7d1015 Rebuilt for https://fedoraproject.org/wiki/Fedora_44_Mass_Rebuild 2026-01-16 04:54:19 +00:00
Kaleb S. KEITHLEY 48feed4a4d disable crimson, core team says it's not ready for prime time 2025-12-10 10:58:08 -05:00
Kaleb S. KEITHLEY cf2bdd5f8e rhbz 2419150 2025-12-05 15:03:14 -05:00
Kaleb S. KEITHLEY 39ab64deb9 enable crimson.
unclear why smp_helpers.h inclusion of <crimson/common/coroutine.h is
not including <coroutine.h>, thus necessitating the patch
2025-12-04 12:50:11 -05:00
Kaleb S. KEITHLEY 73999fa1c0 Ceph 20.2.0 GA 2025-11-18 16:20:47 -05:00
Tom spot Callaway 20f3025d76 rebuild for new fuse3 2025-11-04 11:57:53 -05:00
Kaleb S. KEITHLEY ca9114cab0 rebuild with libarrow-22.0.0, side tag f44-build-side-121464 2025-10-24 15:01:36 -04:00
Kaleb S. KEITHLEY 3fddb94cca rhbz#2403112 2025-10-17 07:41:15 -04:00
Gordon Messmer b65611de9e Use rpm's native resource tunable to limit parallelism.
rpm 4.19 and newer have built-in tunable settings to limit
parallel tasks. This change adopts a common configuration to ease
collaboration with SUSE.
2025-10-16 18:10:57 -07:00
Kaleb S. KEITHLEY 059dc66b73 sync w/ upstream ceph.spec(.in) 2025-10-09 13:54:09 -04:00
Kaleb S. KEITHLEY 65f3783e6b ceph-20.1.1 RC 2025-10-08 06:52:36 -04:00
Kaleb S. KEITHLEY b7390cc379 - ceph-20.1.0 RC, rhbz#2396036 (gmock, gtest again) 2025-09-23 13:25:25 -04:00
Python Maint 12e10018d3 Rebuilt for Python 3.14.0rc3 bytecode 2025-09-19 12:10:54 +02:00
Kaleb S. KEITHLEY 3816c2c84c restore 0051-src-googletest-nosharedlibs.patch
and 0053-src-test-neorados-common_tests.h.patch
    rhbz#2395703
2025-09-16 08:19:30 -04:00
Kaleb S. KEITHLEY d412b12357 restore 0051-src-googletest-nosharedlibs.patch
and 0053-src-test-neorados-common_tests.h.patch
rhbz#2395703
2025-09-16 08:16:31 -04:00
Kaleb S. KEITHLEY 0e004b5c4d ceph-20.1.0 RC, rhbz#2394758 2025-09-12 08:42:08 -04:00
Kaleb S. KEITHLEY e734c59205 ceph-20.1.0 RC, rhbz#2394363 2025-09-10 21:06:19 -04:00
Kaleb S. KEITHLEY eb50428bb2 ceph-20.1.0 RC0, rhbz#2394363 2025-09-10 09:30:08 -04:00
Kaleb S. KEITHLEY ff96b156de ceph-20.1.0 RC0, rhbz#2394363 2025-09-10 09:12:45 -04:00
Kaleb S. KEITHLEY a71490735f ceph-20.1.0 RC
WITH_UADK aarch64 conditional in top-level CMakeLists.txt doesn't
appear to work as expected to disable UADK on aarch64
2025-09-09 16:25:39 -04:00
Kaleb S. KEITHLEY bae6b119a3 ceph-20.1.0 RC 2025-09-09 15:13:25 -04:00
Kaleb S. KEITHLEY 3d257c9866 f44-build-side-116633 2025-08-21 19:32:54 -04:00
28 changed files with 1360 additions and 1162 deletions
+28 -27
View File
@@ -1,6 +1,27 @@
--- ceph-17.2.6/src/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S.orig 2023-04-30 14:25:35.009605033 -0400
+++ ceph-17.2.6/src/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S 2023-04-30 14:28:32.239465067 -0400
@@ -80,3 +80,18 @@
--- ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/jump_x86_64_sysv_elf_gas.S.orig 2025-05-19 18:28:37.137194693 -0400
+++ ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/jump_x86_64_sysv_elf_gas.S 2025-05-19 18:30:54.292857720 -0400
@@ -144,4 +144,18 @@
/* Mark that we don't need executable stack. */
.section .note.GNU-stack,"",%progbits
+
+.section .note.gnu.property
+.align=8
+
+ .byte 0x04, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00
+ .byte 0x05, 0x00, 0x00, 0x00, 0x47, 0x4E, 0x55, 0x00
+ .byte 0x00, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ .byte 0x01, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ .byte 0x04, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00
+ .byte 0x05, 0x00, 0x00, 0x00, 0x47, 0x4E, 0x55, 0x00
+ .byte 0x02, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
# endif
--- ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S.orig 2025-05-19 18:28:22.611442193 -0400
+++ ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S 2025-05-19 18:31:01.037742795 -0400
@@ -159,4 +159,18 @@
/* Mark that we don't need executable stack. */
.section .note.GNU-stack,"",%progbits
@@ -18,30 +39,10 @@
+ .byte 0x05, 0x00, 0x00, 0x00, 0x47, 0x4E, 0x55, 0x00
+ .byte 0x02, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+
--- ceph-17.2.6/src/boost/libs/context/src/asm/jump_x86_64_sysv_elf_gas.S.orig 2023-04-30 14:25:35.008605050 -0400
+++ ceph-17.2.6/src/boost/libs/context/src/asm/jump_x86_64_sysv_elf_gas.S 2023-04-30 14:27:50.145210847 -0400
@@ -89,3 +89,17 @@
/* Mark that we don't need executable stack. */
.section .note.GNU-stack,"",%progbits
+
+.section .note.gnu.property
+.align=8
+
+ .byte 0x04, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00
+ .byte 0x05, 0x00, 0x00, 0x00, 0x47, 0x4E, 0x55, 0x00
+ .byte 0x00, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ .byte 0x01, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ .byte 0x04, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00
+ .byte 0x05, 0x00, 0x00, 0x00, 0x47, 0x4E, 0x55, 0x00
+ .byte 0x02, 0x00, 0x00, 0xC0, 0x04, 0x00, 0x00, 0x00
+ .byte 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
--- ceph-17.2.6/src/boost/libs/context/src/asm/ontop_x86_64_sysv_elf_gas.S.orig 2023-04-30 14:25:35.009605033 -0400
+++ ceph-17.2.6/src/boost/libs/context/src/asm/ontop_x86_64_sysv_elf_gas.S 2023-04-30 14:29:30.402434597 -0400
@@ -92,3 +92,17 @@
# endif
--- ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/ontop_x86_64_sysv_elf_gas.S.orig 2025-05-19 18:29:35.836194529 -0400
+++ ceph-20.0.0-1954-g77a7c83d/src/boost/libs/context/src/asm/ontop_x86_64_sysv_elf_gas.S 2025-05-19 18:31:15.172501956 -0400
@@ -137,3 +137,17 @@
/* Mark that we don't need executable stack. */
.section .note.GNU-stack,"",%progbits
-18
View File
@@ -1,18 +0,0 @@
--- ceph/src/CMakeLists.txt.orig 2023-11-01 11:53:53.618167190 -0400
+++ ceph/src/CMakeLists.txt 2023-11-01 13:52:51.292643490 -0400
@@ -625,6 +625,7 @@
add_subdirectory(perfglue)
add_library(rados_snap_set_diff_obj OBJECT librados/snap_set_diff.cc)
+add_dependencies(rados_snap_set_diff_obj legacy-option-headers)
option(WITH_LIBRADOSSTRIPER "build with libradosstriper support" ON)
@@ -881,6 +882,7 @@
add_library(krbd STATIC krbd.cc
$<TARGET_OBJECTS:parse_secret_objs>)
target_link_libraries(krbd keyutils::keyutils)
+ add_dependencies(krbd legacy-option-headers)
endif()
add_subdirectory(librbd)
if(WITH_FUSE)
-41
View File
@@ -1,41 +0,0 @@
From f5d3e9146d9ceb12858bc94d60090bd851b1c088 Mon Sep 17 00:00:00 2001
From: Adam Emerson <aemerson@redhat.com>
Date: Wed, 20 Dec 2023 13:51:21 -0500
Subject: [PATCH] test/neorados: Use two fewer GTest internals
To get coroutine tests working, I used some internals. Two of these
are no longer available in newer versions. Since they can be
implemented with regular old C++ features, use those.
This fixes an FTBFS on Fedora Rawhide introduced in
`35231f7251ed70d8d817ee7e727d9763669d101f`.
Signed-off-by: Adam Emerson <aemerson@redhat.com>
---
src/test/neorados/common_tests.h | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/src/test/neorados/common_tests.h b/src/test/neorados/common_tests.h
index 9610f08f643f3..396432e417e84 100644
--- a/src/test/neorados/common_tests.h
+++ b/src/test/neorados/common_tests.h
@@ -391,10 +391,14 @@ class NeoRadosECTest : public NeoRadosTestBase {
public: \
GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)() = default; \
~GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)() override = default; \
- GTEST_DISALLOW_COPY_AND_ASSIGN_(GTEST_TEST_CLASS_NAME_(test_suite_name, \
- test_name)); \
- GTEST_DISALLOW_MOVE_AND_ASSIGN_(GTEST_TEST_CLASS_NAME_(test_suite_name, \
- test_name)); \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)( \
+ const GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)&) = delete; \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)& operator =( \
+ const GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)&) = delete; \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)( \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)&&) = delete; \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)& operator =( \
+ GTEST_TEST_CLASS_NAME_(test_suite_name, test_name)&&) = delete; \
\
private: \
boost::asio::awaitable<void> CoTestBody() override; \
@@ -0,0 +1,11 @@
--- ceph-18.2.3/src/boost/boost/python/call_method.hpp.orig 2024-06-07 11:14:58.579734749 -0400
+++ ceph-18.2.3/src/boost/boost/python/call_method.hpp 2024-06-07 11:19:06.754520504 -0400
@@ -59,7 +59,7 @@
)
{
PyObject* const result =
- PyEval_CallMethod(
+ PyObject_CallMethod(
self
, const_cast<char*>(name)
, const_cast<char*>("(" BOOST_PP_REPEAT_1ST(N, BOOST_PYTHON_FIXED, "O") ")")
-21
View File
@@ -1,21 +0,0 @@
--- ceph-19.1.0/src/cpp_redis/CMakeLists.txt.orig 2024-07-11 15:16:22.487082035 -0400
+++ ceph-19.1.0/src/cpp_redis/CMakeLists.txt 2024-07-11 15:17:57.226534456 -0400
@@ -174,6 +174,7 @@
set_target_properties(${PROJECT} PROPERTIES COMPILE_DEFINITIONS "__CPP_REDIS_USE_CUSTOM_TCP_CLIENT=${USE_CUSTOM_TCP_CLIENT}")
endif(USE_CUSTOM_TCP_CLIENT)
+install(TARGETS ${PROJECT} DESTINATION ${CMAKE_INSTALL_LIBDIR}/ceph)
###
# examples
--- ceph-19.1.0/src/cpp_redis/tacopie/CMakeLists.txt.orig 2024-01-30 21:47:59.000000000 -0500
+++ ceph-19.1.0/src/cpp_redis/tacopie/CMakeLists.txt 2024-07-12 07:37:23.584654640 -0400
@@ -162,6 +162,8 @@
ENDIF(SELECT_TIMEOUT)
+install(TARGETS ${PROJECT} DESTINATION ${CMAKE_INSTALL_LIBDIR}/ceph)
+
###
# examples
###
@@ -1,28 +0,0 @@
--- ceph-19.1.0/src/common/crc32c_ppc_fast_zero_asm.S.orig 2024-07-26 07:18:50.733484175 -0400
+++ ceph-19.1.0/src/common/crc32c_ppc_fast_zero_asm.S 2024-07-26 07:17:48.449531183 -0400
@@ -45,8 +45,10 @@
/* unsigned int barrett_reduction(unsigned long val) */
FUNC_START(barrett_reduction)
- lis r4,.constants@ha
- la r4,.constants@l(r4)
+/* lis r4,.constants@ha */
+/* la r4,.constants@l(r4) */
+ addis r4,r4,.constants@toc@ha
+ addi r4,r4,.constants@toc@l
li r5,16
vxor v1,v1,v1 /* zero v1 */
@@ -83,8 +85,10 @@
/* unsigned int barrett_reduction_reflected(unsigned long val) */
FUNC_START(barrett_reduction_reflected)
- lis r4,.bit_reflected_constants@ha
- la r4,.bit_reflected_constants@l(r4)
+/* lis r4,.bit_reflected_constants@ha */
+/* la r4,.bit_reflected_constants@l(r4) */
+ addis r4,r4,.bit_reflected_constants@toc@ha
+ addi r4,r4,.bit_reflected_constants@toc@l
li r5,16
vxor v1,v1,v1 /* zero v1 */
-41
View File
@@ -1,44 +1,3 @@
diff --git a/src/common/openssl_opts_handler.cc b/src/common/openssl_opts_handler.cc
index 81d0c4786..d9866f197 100644
--- a/src/common/openssl_opts_handler.cc
+++ b/src/common/openssl_opts_handler.cc
@@ -16,7 +16,10 @@
#include <openssl/bio.h>
#include <openssl/conf.h>
+#include <openssl/err.h>
+#ifndef OPENSSL_NO_ENGINE
#include <openssl/engine.h>
+#endif
#include <mutex>
#include <vector>
#include <algorithm>
@@ -116,11 +119,13 @@ void load_module(const string &engine_conf)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#ifndef OPENSSL_NO_ENGINE
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
ENGINE_load_builtin_engines();
#pragma clang diagnostic pop
#pragma GCC diagnostic pop
+#endif
if (CONF_modules_load(
conf, nullptr,
diff --git a/src/crypto/openssl/openssl_crypto_accel.cc b/src/crypto/openssl/openssl_crypto_accel.cc
index f99844a38..4944205dd 100644
--- a/src/crypto/openssl/openssl_crypto_accel.cc
+++ b/src/crypto/openssl/openssl_crypto_accel.cc
@@ -14,7 +14,6 @@
#include "crypto/openssl/openssl_crypto_accel.h"
#include <openssl/evp.h>
-#include <openssl/engine.h>
#include "common/debug.h"
// -----------------------------------------------------------------------------
diff --git a/src/jaegertracing/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb/src/civetweb.c b/src/jaegertracing/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb/src/civetweb.c
index fea9e6f98..c7eddb20b 100644
--- a/src/jaegertracing/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb/src/civetweb.c
-16
View File
@@ -1,16 +0,0 @@
--- ceph-19.2.0/src/mds/CMakeLists.txt.orig 2024-12-20 17:15:47.632569537 -0500
+++ ceph-19.2.0/src/mds/CMakeLists.txt 2024-12-20 17:16:48.945481957 -0500
@@ -45,12 +45,11 @@
QuiesceDbManager.cc
QuiesceAgent.cc
MDSRankQuiesce.cc
- BoostUrlImpl.cc
${CMAKE_SOURCE_DIR}/src/common/TrackedOp.cc
${CMAKE_SOURCE_DIR}/src/common/MemoryModel.cc
${CMAKE_SOURCE_DIR}/src/osdc/Journaler.cc
${CMAKE_SOURCE_DIR}/src/mgr/MDSPerfMetricTypes.cc)
add_library(mds STATIC ${mds_srcs})
target_link_libraries(mds PRIVATE
- heap_profiler cpu_profiler osdc ${LUA_LIBRARIES})
+ boost_url heap_profiler cpu_profiler osdc ${LUA_LIBRARIES})
target_include_directories(mds PRIVATE "${LUA_INCLUDE_DIR}")
-257
View File
@@ -1,257 +0,0 @@
--- ceph-19.2.0/src/rgw/driver/posix/zpp_bits.h.orig 2024-09-18 12:27:51.000000000 -0400
+++ ceph-19.2.0/src/rgw/driver/posix/zpp_bits.h 2025-01-16 14:58:48.471438398 -0500
@@ -2687,8 +2687,12 @@
{
using type = std::remove_cvref_t<decltype(container)>;
using value_type = typename type::value_type;
- constexpr auto is_const = std::is_const_v<
- std::remove_reference_t<decltype(container[0])>>;
+ constexpr auto is_const =
+ std::is_const_v<std::remove_reference_t<value_type>> ||
+ requires {
+ requires std::is_const_v<
+ std::remove_reference_t<decltype(container[0])>>;
+ };
if constexpr (!std::is_void_v<SizeType> &&
(requires(type container) { container.resize(1); } ||
@@ -2827,9 +2831,7 @@
if constexpr (requires { typename type::mapped_type; }) {
using value_type = std::pair<typename type::key_type,
typename type::mapped_type>;
- std::aligned_storage_t<sizeof(value_type),
- alignof(value_type)>
- storage;
+ alignas(value_type) std::byte storage[sizeof(value_type)];
auto object = access::placement_new<value_type>(
std::addressof(storage));
@@ -2843,9 +2845,7 @@
} else {
using value_type = typename type::value_type;
- std::aligned_storage_t<sizeof(value_type),
- alignof(value_type)>
- storage;
+ alignas(value_type) std::byte storage[sizeof(value_type)];
auto object = access::placement_new<value_type>(
std::addressof(storage));
@@ -2903,8 +2903,7 @@
return result;
}
} else {
- std::aligned_storage_t<sizeof(value_type), alignof(value_type)>
- storage;
+ alignas(value_type) std::byte storage[sizeof(value_type)];
auto object =
access::placement_new<value_type>(std::addressof(storage));
@@ -2947,9 +2946,7 @@
}
return serialize_one(*std::get_if<element_type>(&variant));
} else {
- std::aligned_storage_t<sizeof(element_type),
- alignof(element_type)>
- storage;
+ alignas(element_type) std::byte storage[sizeof(element_type)];
auto object = access::placement_new<element_type>(
std::addressof(storage));
@@ -2996,8 +2993,7 @@
return self.serialize_one(
*std::get_if<Types>(&variant));
} else {
- std::aligned_storage_t<sizeof(Types), alignof(Types)>
- storage;
+ alignas(Types) std::byte storage[sizeof(Types)];
auto object = access::placement_new<Types>(
std::addressof(storage));
@@ -3008,6 +3004,7 @@
return result;
}
variant = std::move(*object);
+ return errc{};
}
}...};
@@ -3307,7 +3304,7 @@
ZPP_BITS_INLINE constexpr static auto serialize(auto & serializer,
auto & self)
{
- return serializer.template serialize_one(self.variant, self.id);
+ return serializer.serialize_one(self.variant, self.id);
}
Variant & variant;
@@ -3448,13 +3445,14 @@
}
constexpr explicit value_or_errc(error_type error) :
- m_error(std::forward<decltype(error)>(error))
+ m_error(std::forward<decltype(error)>(error)),
+ m_failure(true)
{
}
constexpr value_or_errc(value_or_errc && other) noexcept
{
- if (other.is_value()) {
+ if (other.success()) {
if constexpr (!std::is_void_v<Type>) {
if constexpr (!std::is_reference_v<Type>) {
::new (std::addressof(m_return_value))
@@ -3752,16 +3750,16 @@
requires(decltype(in.remaining_data()) & data) {
(context.*Function)(data);
}) {
- struct _
+ struct guard
{
decltype(in) archive;
decltype(in.remaining_data()) data;
- constexpr ~_()
+ constexpr ~guard()
{
archive.position() += data.size();
}
- } _{in, in.remaining_data()};
- return (context.*Function)(_.data);
+ } guard{in, in.remaining_data()};
+ return (context.*Function)(guard.data);
} else {
return (context.*Function)();
}
@@ -3776,16 +3774,16 @@
requires(decltype(in.remaining_data()) & data) {
Function(data);
}) {
- struct _
+ struct guard
{
decltype(in) archive;
decltype(in.remaining_data()) data;
- constexpr ~_()
+ constexpr ~guard()
{
archive.position() += data.size();
}
- } _{in, in.remaining_data()};
- return Function(_.data);
+ } guard{in, in.remaining_data()};
+ return Function(guard.data);
} else {
return Function();
}
@@ -5029,9 +5027,7 @@
serialize use();
};
- std::aligned_storage_t<sizeof(value_type),
- alignof(value_type)>
- storage;
+ alignas(value_type) std::byte storage[sizeof(value_type)];
auto object =
access::placement_new<value_type>(std::addressof(storage));
@@ -5111,9 +5107,7 @@
return errc{};
}
} else {
- std::aligned_storage_t<sizeof(value_type),
- alignof(value_type)>
- storage;
+ alignas(value_type) std::byte storage[sizeof(value_type)];
auto object = access::placement_new<value_type>(
std::addressof(storage));
@@ -5382,47 +5376,47 @@
};
auto align = [](auto v, auto a) { return (v + (a - 1)) / a * a; };
- auto h0 = big_endian{0x6a09e667u};
- auto h1 = big_endian{0xbb67ae85u};
- auto h2 = big_endian{0x3c6ef372u};
- auto h3 = big_endian{0xa54ff53au};
- auto h4 = big_endian{0x510e527fu};
- auto h5 = big_endian{0x9b05688cu};
- auto h6 = big_endian{0x1f83d9abu};
- auto h7 = big_endian{0x5be0cd19u};
-
- std::array k{big_endian{0x428a2f98u}, big_endian{0x71374491u},
- big_endian{0xb5c0fbcfu}, big_endian{0xe9b5dba5u},
- big_endian{0x3956c25bu}, big_endian{0x59f111f1u},
- big_endian{0x923f82a4u}, big_endian{0xab1c5ed5u},
- big_endian{0xd807aa98u}, big_endian{0x12835b01u},
- big_endian{0x243185beu}, big_endian{0x550c7dc3u},
- big_endian{0x72be5d74u}, big_endian{0x80deb1feu},
- big_endian{0x9bdc06a7u}, big_endian{0xc19bf174u},
- big_endian{0xe49b69c1u}, big_endian{0xefbe4786u},
- big_endian{0x0fc19dc6u}, big_endian{0x240ca1ccu},
- big_endian{0x2de92c6fu}, big_endian{0x4a7484aau},
- big_endian{0x5cb0a9dcu}, big_endian{0x76f988dau},
- big_endian{0x983e5152u}, big_endian{0xa831c66du},
- big_endian{0xb00327c8u}, big_endian{0xbf597fc7u},
- big_endian{0xc6e00bf3u}, big_endian{0xd5a79147u},
- big_endian{0x06ca6351u}, big_endian{0x14292967u},
- big_endian{0x27b70a85u}, big_endian{0x2e1b2138u},
- big_endian{0x4d2c6dfcu}, big_endian{0x53380d13u},
- big_endian{0x650a7354u}, big_endian{0x766a0abbu},
- big_endian{0x81c2c92eu}, big_endian{0x92722c85u},
- big_endian{0xa2bfe8a1u}, big_endian{0xa81a664bu},
- big_endian{0xc24b8b70u}, big_endian{0xc76c51a3u},
- big_endian{0xd192e819u}, big_endian{0xd6990624u},
- big_endian{0xf40e3585u}, big_endian{0x106aa070u},
- big_endian{0x19a4c116u}, big_endian{0x1e376c08u},
- big_endian{0x2748774cu}, big_endian{0x34b0bcb5u},
- big_endian{0x391c0cb3u}, big_endian{0x4ed8aa4au},
- big_endian{0x5b9cca4fu}, big_endian{0x682e6ff3u},
- big_endian{0x748f82eeu}, big_endian{0x78a5636fu},
- big_endian{0x84c87814u}, big_endian{0x8cc70208u},
- big_endian{0x90befffau}, big_endian{0xa4506cebu},
- big_endian{0xbef9a3f7u}, big_endian{0xc67178f2u}};
+ auto h0 = big_endian{std::uint32_t{0x6a09e667u}};
+ auto h1 = big_endian{std::uint32_t{0xbb67ae85u}};
+ auto h2 = big_endian{std::uint32_t{0x3c6ef372u}};
+ auto h3 = big_endian{std::uint32_t{0xa54ff53au}};
+ auto h4 = big_endian{std::uint32_t{0x510e527fu}};
+ auto h5 = big_endian{std::uint32_t{0x9b05688cu}};
+ auto h6 = big_endian{std::uint32_t{0x1f83d9abu}};
+ auto h7 = big_endian{std::uint32_t{0x5be0cd19u}};
+
+ std::array k{big_endian{std::uint32_t{0x428a2f98u}}, big_endian{std::uint32_t{0x71374491u}},
+ big_endian{std::uint32_t{0xb5c0fbcfu}}, big_endian{std::uint32_t{0xe9b5dba5u}},
+ big_endian{std::uint32_t{0x3956c25bu}}, big_endian{std::uint32_t{0x59f111f1u}},
+ big_endian{std::uint32_t{0x923f82a4u}}, big_endian{std::uint32_t{0xab1c5ed5u}},
+ big_endian{std::uint32_t{0xd807aa98u}}, big_endian{std::uint32_t{0x12835b01u}},
+ big_endian{std::uint32_t{0x243185beu}}, big_endian{std::uint32_t{0x550c7dc3u}},
+ big_endian{std::uint32_t{0x72be5d74u}}, big_endian{std::uint32_t{0x80deb1feu}},
+ big_endian{std::uint32_t{0x9bdc06a7u}}, big_endian{std::uint32_t{0xc19bf174u}},
+ big_endian{std::uint32_t{0xe49b69c1u}}, big_endian{std::uint32_t{0xefbe4786u}},
+ big_endian{std::uint32_t{0x0fc19dc6u}}, big_endian{std::uint32_t{0x240ca1ccu}},
+ big_endian{std::uint32_t{0x2de92c6fu}}, big_endian{std::uint32_t{0x4a7484aau}},
+ big_endian{std::uint32_t{0x5cb0a9dcu}}, big_endian{std::uint32_t{0x76f988dau}},
+ big_endian{std::uint32_t{0x983e5152u}}, big_endian{std::uint32_t{0xa831c66du}},
+ big_endian{std::uint32_t{0xb00327c8u}}, big_endian{std::uint32_t{0xbf597fc7u}},
+ big_endian{std::uint32_t{0xc6e00bf3u}}, big_endian{std::uint32_t{0xd5a79147u}},
+ big_endian{std::uint32_t{0x06ca6351u}}, big_endian{std::uint32_t{0x14292967u}},
+ big_endian{std::uint32_t{0x27b70a85u}}, big_endian{std::uint32_t{0x2e1b2138u}},
+ big_endian{std::uint32_t{0x4d2c6dfcu}}, big_endian{std::uint32_t{0x53380d13u}},
+ big_endian{std::uint32_t{0x650a7354u}}, big_endian{std::uint32_t{0x766a0abbu}},
+ big_endian{std::uint32_t{0x81c2c92eu}}, big_endian{std::uint32_t{0x92722c85u}},
+ big_endian{std::uint32_t{0xa2bfe8a1u}}, big_endian{std::uint32_t{0xa81a664bu}},
+ big_endian{std::uint32_t{0xc24b8b70u}}, big_endian{std::uint32_t{0xc76c51a3u}},
+ big_endian{std::uint32_t{0xd192e819u}}, big_endian{std::uint32_t{0xd6990624u}},
+ big_endian{std::uint32_t{0xf40e3585u}}, big_endian{std::uint32_t{0x106aa070u}},
+ big_endian{std::uint32_t{0x19a4c116u}}, big_endian{std::uint32_t{0x1e376c08u}},
+ big_endian{std::uint32_t{0x2748774cu}}, big_endian{std::uint32_t{0x34b0bcb5u}},
+ big_endian{std::uint32_t{0x391c0cb3u}}, big_endian{std::uint32_t{0x4ed8aa4au}},
+ big_endian{std::uint32_t{0x5b9cca4fu}}, big_endian{std::uint32_t{0x682e6ff3u}},
+ big_endian{std::uint32_t{0x748f82eeu}}, big_endian{std::uint32_t{0x78a5636fu}},
+ big_endian{std::uint32_t{0x84c87814u}}, big_endian{std::uint32_t{0x8cc70208u}},
+ big_endian{std::uint32_t{0x90befffau}}, big_endian{std::uint32_t{0xa4506cebu}},
+ big_endian{std::uint32_t{0xbef9a3f7u}}, big_endian{std::uint32_t{0xc67178f2u}}};
constexpr auto original_message = to_bytes<Object>();
constexpr auto chunk_size = 512 / CHAR_BIT;
+35 -14
View File
@@ -1,29 +1,35 @@
--- ceph-19.2.0/src/googletest/googletest/CMakeLists.txt.orig 2025-01-23 07:43:33.314970694 -0500
+++ ceph-19.2.0/src/googletest/googletest/CMakeLists.txt 2025-01-23 07:44:13.529250429 -0500
@@ -63,6 +63,8 @@
--- ceph-20.1.0/src/googletest/googletest/CMakeLists.txt.orig 2025-02-07 11:04:56.000000000 -0500
+++ ceph-20.1.0/src/googletest/googletest/CMakeLists.txt 2025-09-16 08:11:59.341951770 -0400
@@ -53,6 +53,8 @@
set_up_hermetic_build()
endif()
+option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)." OFF)
+
# These commands only run if this is the main project
# These commands only run if this is the main project.
if(CMAKE_PROJECT_NAME STREQUAL "gtest" OR CMAKE_PROJECT_NAME STREQUAL "googletest-distribution")
@@ -126,9 +128,9 @@
@@ -121,7 +123,7 @@
# are used for other targets, to ensure that gtest can be compiled by a user
# aggressive about warnings.
cxx_library(gtest "${cxx_strict}" src/gtest-all.cc)
-set_target_properties(gtest PROPERTIES VERSION ${GOOGLETEST_VERSION})
+#set_target_properties(gtest PROPERTIES VERSION ${GOOGLETEST_VERSION})
if(GTEST_HAS_ABSL)
target_compile_definitions(gtest PUBLIC GTEST_HAS_ABSL=1)
target_link_libraries(gtest PUBLIC
@@ -139,7 +141,7 @@
)
endif()
cxx_library(gtest_main "${cxx_strict}" src/gtest_main.cc)
-set_target_properties(gtest_main PROPERTIES VERSION ${GOOGLETEST_VERSION})
+#set_target_properties(gtest_main PROPERTIES VERSION ${GOOGLETEST_VERSION})
# If the CMake version supports it, attach header directory information
# to the targets for when we are part of a parent build (ie being pulled
# in via add_subdirectory() rather than being a standalone build).
--- ceph-19.2.0/src/googletest/googlemock/CMakeLists.txt.orig 2025-01-23 07:40:52.399853240 -0500
+++ ceph-19.2.0/src/googletest/googlemock/CMakeLists.txt 2025-01-23 07:43:18.466236642 -0500
@@ -54,6 +54,7 @@
string(REPLACE ";" "$<SEMICOLON>" dirs "${gtest_build_include_dirs}")
target_include_directories(gtest SYSTEM INTERFACE
"$<BUILD_INTERFACE:${dirs}>"
--- ceph-20.1.0/src/googletest/googlemock/CMakeLists.txt.orig 2025-02-07 11:04:56.000000000 -0500
+++ ceph-20.1.0/src/googletest/googlemock/CMakeLists.txt 2025-09-16 08:12:18.121641875 -0400
@@ -49,6 +49,7 @@
# if they are the same (the default).
add_subdirectory("${gtest_dir}" "${gmock_BINARY_DIR}/${gtest_dir}")
@@ -31,12 +37,27 @@
# These commands only run if this is the main project
if(CMAKE_PROJECT_NAME STREQUAL "gmock" OR CMAKE_PROJECT_NAME STREQUAL "googletest-distribution")
@@ -100,7 +101,7 @@
@@ -96,10 +97,10 @@
else()
cxx_library(gmock "${cxx_strict}" src/gmock-all.cc)
target_link_libraries(gmock PUBLIC gtest)
- set_target_properties(gmock PROPERTIES VERSION ${GOOGLETEST_VERSION})
+ # set_target_properties(gmock PROPERTIES VERSION ${GOOGLETEST_VERSION})
+ # set_target_properties(gmock PROPERTIES VERSION ${GOOGLETEST_VERSION})
cxx_library(gmock_main "${cxx_strict}" src/gmock_main.cc)
target_link_libraries(gmock_main PUBLIC gmock)
set_target_properties(gmock_main PROPERTIES VERSION ${GOOGLETEST_VERSION})
- set_target_properties(gmock_main PROPERTIES VERSION ${GOOGLETEST_VERSION})
+ # set_target_properties(gmock_main PROPERTIES VERSION ${GOOGLETEST_VERSION})
endif()
string(REPLACE ";" "$<SEMICOLON>" dirs "${gmock_build_include_dirs}")
--- ceph-20.1.0/src/googletest/googletest/cmake/internal_utils.cmake.orig 2025-09-22 10:12:18.037349494 -0400
+++ ceph-20.1.0/src/googletest/googletest/cmake/internal_utils.cmake 2025-09-22 10:13:13.729590249 -0400
@@ -207,7 +207,7 @@
endfunction()
function(cxx_library name cxx_flags)
- cxx_library_with_type(${name} "" "${cxx_flags}" ${ARGN})
+ cxx_library_with_type(${name} STATIC "${cxx_flags}" ${ARGN})
endfunction()
# cxx_executable_with_flags(name cxx_flags libs srcs...)
+2 -11
View File
@@ -1,5 +1,5 @@
--- ceph-19.2.0/src/test/neorados/common_tests.h.orig 2024-09-18 12:27:51.000000000 -0400
+++ ceph-19.2.0/src/test/neorados/common_tests.h 2025-01-26 19:31:32.545762972 -0500
--- ceph-20.1.0/src/test/neorados/common_tests.h.orig 2025-09-04 15:35:40.000000000 -0400
+++ ceph-20.1.0/src/test/neorados/common_tests.h 2025-09-16 08:13:03.274896767 -0400
@@ -47,6 +47,18 @@
#include "gtest/gtest.h"
@@ -19,12 +19,3 @@
/// \file test/neorados/common_tests.h
///
/// \brief Tools for testing neorados code
@@ -398,7 +410,7 @@
\
private: \
boost::asio::awaitable<void> CoTestBody() override; \
- static ::testing::TestInfo *const test_info_ GTEST_ATTRIBUTE_UNUSED_; \
+ static ::testing::TestInfo *const test_info_ __attribute__((unused)); \
}; \
\
::testing::TestInfo *const GTEST_TEST_CLASS_NAME_(test_suite_name, \
+838
View File
@@ -0,0 +1,838 @@
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal_19.h.orig 2025-07-08 07:40:29.811814549 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal_19.h 2025-07-08 07:40:29.811739290 -0400
@@ -0,0 +1,114 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "parquet/properties.h"
+#include "parquet/types.h"
+
+using parquet::ParquetCipher;
+
+namespace parquet {
+namespace encryption {
+
+constexpr int kGcmTagLength = 16;
+constexpr int kNonceLength = 12;
+
+// Module types
+constexpr int8_t kFooter = 0;
+constexpr int8_t kColumnMetaData = 1;
+constexpr int8_t kDataPage = 2;
+constexpr int8_t kDictionaryPage = 3;
+constexpr int8_t kDataPageHeader = 4;
+constexpr int8_t kDictionaryPageHeader = 5;
+constexpr int8_t kColumnIndex = 6;
+constexpr int8_t kOffsetIndex = 7;
+
+/// Performs AES encryption operations with GCM or CTR ciphers.
+class AesEncryptor {
+ public:
+ static AesEncryptor* Make(ParquetCipher::type alg_id, int key_len, bool metadata,
+ std::vector<AesEncryptor*>* all_encryptors);
+
+ ~AesEncryptor();
+
+ /// Size difference between plaintext and ciphertext, for this cipher.
+ int CiphertextSizeDelta();
+
+ /// Encrypts plaintext with the key and aad. Key length is passed only for validation.
+ /// If different from value in constructor, exception will be thrown.
+ int Encrypt(const uint8_t* plaintext, int plaintext_len, const uint8_t* key,
+ int key_len, const uint8_t* aad, int aad_len, uint8_t* ciphertext);
+
+ /// Encrypts plaintext footer, in order to compute footer signature (tag).
+ int SignedFooterEncrypt(const uint8_t* footer, int footer_len, const uint8_t* key,
+ int key_len, const uint8_t* aad, int aad_len,
+ const uint8_t* nonce, uint8_t* encrypted_footer);
+
+ void WipeOut();
+
+ private:
+ /// Can serve one key length only. Possible values: 16, 24, 32 bytes.
+ explicit AesEncryptor(ParquetCipher::type alg_id, int key_len, bool metadata);
+ // PIMPL Idiom
+ class AesEncryptorImpl;
+ std::unique_ptr<AesEncryptorImpl> impl_;
+};
+
+/// Performs AES decryption operations with GCM or CTR ciphers.
+class AesDecryptor {
+ public:
+ static AesDecryptor* Make(ParquetCipher::type alg_id, int key_len, bool metadata,
+ std::vector<AesDecryptor*>* all_decryptors);
+
+ ~AesDecryptor();
+ void WipeOut();
+
+ /// Size difference between plaintext and ciphertext, for this cipher.
+ int CiphertextSizeDelta();
+
+ /// Decrypts ciphertext with the key and aad. Key length is passed only for
+ /// validation. If different from value in constructor, exception will be thrown.
+ int Decrypt(const uint8_t* ciphertext, int ciphertext_len, const uint8_t* key,
+ int key_len, const uint8_t* aad, int aad_len, uint8_t* plaintext);
+
+ private:
+ /// Can serve one key length only. Possible values: 16, 24, 32 bytes.
+ explicit AesDecryptor(ParquetCipher::type alg_id, int key_len, bool metadata);
+ // PIMPL Idiom
+ class AesDecryptorImpl;
+ std::unique_ptr<AesDecryptorImpl> impl_;
+};
+
+std::string CreateModuleAad(const std::string& file_aad, int8_t module_type,
+ int16_t row_group_ordinal, int16_t column_ordinal,
+ int16_t page_ordinal);
+
+std::string CreateFooterAad(const std::string& aad_prefix_bytes);
+
+// Update last two bytes of page (or page header) module AAD
+void QuickUpdatePageAad(const std::string& AAD, int16_t new_page_ordinal);
+
+// Wraps OpenSSL RAND_bytes function
+void RandBytes(unsigned char* buf, int num);
+
+} // namespace encryption
+} // namespace parquet
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal_20.h.orig 2025-07-08 07:40:29.812759948 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal_20.h 2025-07-08 07:40:29.812687536 -0400
@@ -0,0 +1,141 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "arrow/util/span.h"
+#include "parquet/properties.h"
+#include "parquet/types.h"
+
+using parquet::ParquetCipher;
+
+namespace parquet::encryption {
+
+constexpr int32_t kGcmTagLength = 16;
+constexpr int32_t kNonceLength = 12;
+
+// Module types
+constexpr int8_t kFooter = 0;
+constexpr int8_t kColumnMetaData = 1;
+constexpr int8_t kDataPage = 2;
+constexpr int8_t kDictionaryPage = 3;
+constexpr int8_t kDataPageHeader = 4;
+constexpr int8_t kDictionaryPageHeader = 5;
+constexpr int8_t kColumnIndex = 6;
+constexpr int8_t kOffsetIndex = 7;
+constexpr int8_t kBloomFilterHeader = 8;
+constexpr int8_t kBloomFilterBitset = 9;
+
+/// Performs AES encryption operations with GCM or CTR ciphers.
+class PARQUET_EXPORT AesEncryptor {
+ public:
+ /// Can serve one key length only. Possible values: 16, 24, 32 bytes.
+ /// If write_length is true, prepend ciphertext length to the ciphertext
+ explicit AesEncryptor(ParquetCipher::type alg_id, int32_t key_len, bool metadata,
+ bool write_length = true);
+
+ static std::unique_ptr<AesEncryptor> Make(ParquetCipher::type alg_id, int32_t key_len,
+ bool metadata, bool write_length = true);
+
+ ~AesEncryptor();
+
+ /// The size of the ciphertext, for this cipher and the specified plaintext length.
+ [[nodiscard]] int32_t CiphertextLength(int64_t plaintext_len) const;
+
+ /// Encrypts plaintext with the key and aad. Key length is passed only for validation.
+ /// If different from value in constructor, exception will be thrown.
+ int32_t Encrypt(::arrow::util::span<const uint8_t> plaintext,
+ ::arrow::util::span<const uint8_t> key,
+ ::arrow::util::span<const uint8_t> aad,
+ ::arrow::util::span<uint8_t> ciphertext);
+
+ /// Encrypts plaintext footer, in order to compute footer signature (tag).
+ int32_t SignedFooterEncrypt(::arrow::util::span<const uint8_t> footer,
+ ::arrow::util::span<const uint8_t> key,
+ ::arrow::util::span<const uint8_t> aad,
+ ::arrow::util::span<const uint8_t> nonce,
+ ::arrow::util::span<uint8_t> encrypted_footer);
+
+ private:
+ // PIMPL Idiom
+ class AesEncryptorImpl;
+ std::unique_ptr<AesEncryptorImpl> impl_;
+};
+
+/// Performs AES decryption operations with GCM or CTR ciphers.
+class PARQUET_EXPORT AesDecryptor {
+ public:
+ /// \brief Construct an AesDecryptor
+ ///
+ /// \param alg_id the encryption algorithm to use
+ /// \param key_len key length. Possible values: 16, 24, 32 bytes.
+ /// \param metadata if true then this is a metadata decryptor
+ /// \param contains_length if true, expect ciphertext length prepended to the ciphertext
+ explicit AesDecryptor(ParquetCipher::type alg_id, int32_t key_len, bool metadata,
+ bool contains_length = true);
+
+ static std::unique_ptr<AesDecryptor> Make(ParquetCipher::type alg_id, int32_t key_len,
+ bool metadata);
+
+ ~AesDecryptor();
+
+ /// The size of the plaintext, for this cipher and the specified ciphertext length.
+ [[nodiscard]] int32_t PlaintextLength(int32_t ciphertext_len) const;
+
+ /// The size of the ciphertext, for this cipher and the specified plaintext length.
+ [[nodiscard]] int32_t CiphertextLength(int32_t plaintext_len) const;
+
+ /// Decrypts ciphertext with the key and aad. Key length is passed only for
+ /// validation. If different from value in constructor, exception will be thrown.
+ /// The caller is responsible for ensuring that the plaintext buffer is at least as
+ /// large as PlaintextLength(ciphertext_len).
+ int32_t Decrypt(::arrow::util::span<const uint8_t> ciphertext,
+ ::arrow::util::span<const uint8_t> key,
+ ::arrow::util::span<const uint8_t> aad,
+ ::arrow::util::span<uint8_t> plaintext);
+
+ private:
+ // PIMPL Idiom
+ class AesDecryptorImpl;
+ std::unique_ptr<AesDecryptorImpl> impl_;
+};
+
+std::string CreateModuleAad(const std::string& file_aad, int8_t module_type,
+ int16_t row_group_ordinal, int16_t column_ordinal,
+ int32_t page_ordinal);
+
+std::string CreateFooterAad(const std::string& aad_prefix_bytes);
+
+// Update last two bytes of page (or page header) module AAD
+void QuickUpdatePageAad(int32_t new_page_ordinal, std::string* AAD);
+
+// Wraps OpenSSL RAND_bytes function
+void RandBytes(unsigned char* buf, size_t num);
+
+// Ensure OpenSSL is initialized.
+//
+// This is only necessary in specific situations since OpenSSL otherwise
+// initializes itself automatically. For example, under Valgrind, a memory
+// leak will be reported if OpenSSL is initialized for the first time from
+// a worker thread; calling this function from the main thread prevents this.
+void EnsureBackendInitialized();
+
+} // namespace parquet::encryption
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal.h.orig 2024-10-06 07:18:41.000000000 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/encryption_internal.h 2025-07-08 07:40:29.809908891 -0400
@@ -17,98 +17,8 @@
#pragma once
-#include <memory>
-#include <string>
-#include <vector>
-
-#include "parquet/properties.h"
-#include "parquet/types.h"
-
-using parquet::ParquetCipher;
-
-namespace parquet {
-namespace encryption {
-
-constexpr int kGcmTagLength = 16;
-constexpr int kNonceLength = 12;
-
-// Module types
-constexpr int8_t kFooter = 0;
-constexpr int8_t kColumnMetaData = 1;
-constexpr int8_t kDataPage = 2;
-constexpr int8_t kDictionaryPage = 3;
-constexpr int8_t kDataPageHeader = 4;
-constexpr int8_t kDictionaryPageHeader = 5;
-constexpr int8_t kColumnIndex = 6;
-constexpr int8_t kOffsetIndex = 7;
-
-/// Performs AES encryption operations with GCM or CTR ciphers.
-class AesEncryptor {
- public:
- static AesEncryptor* Make(ParquetCipher::type alg_id, int key_len, bool metadata,
- std::vector<AesEncryptor*>* all_encryptors);
-
- ~AesEncryptor();
-
- /// Size difference between plaintext and ciphertext, for this cipher.
- int CiphertextSizeDelta();
-
- /// Encrypts plaintext with the key and aad. Key length is passed only for validation.
- /// If different from value in constructor, exception will be thrown.
- int Encrypt(const uint8_t* plaintext, int plaintext_len, const uint8_t* key,
- int key_len, const uint8_t* aad, int aad_len, uint8_t* ciphertext);
-
- /// Encrypts plaintext footer, in order to compute footer signature (tag).
- int SignedFooterEncrypt(const uint8_t* footer, int footer_len, const uint8_t* key,
- int key_len, const uint8_t* aad, int aad_len,
- const uint8_t* nonce, uint8_t* encrypted_footer);
-
- void WipeOut();
-
- private:
- /// Can serve one key length only. Possible values: 16, 24, 32 bytes.
- explicit AesEncryptor(ParquetCipher::type alg_id, int key_len, bool metadata);
- // PIMPL Idiom
- class AesEncryptorImpl;
- std::unique_ptr<AesEncryptorImpl> impl_;
-};
-
-/// Performs AES decryption operations with GCM or CTR ciphers.
-class AesDecryptor {
- public:
- static AesDecryptor* Make(ParquetCipher::type alg_id, int key_len, bool metadata,
- std::vector<AesDecryptor*>* all_decryptors);
-
- ~AesDecryptor();
- void WipeOut();
-
- /// Size difference between plaintext and ciphertext, for this cipher.
- int CiphertextSizeDelta();
-
- /// Decrypts ciphertext with the key and aad. Key length is passed only for
- /// validation. If different from value in constructor, exception will be thrown.
- int Decrypt(const uint8_t* ciphertext, int ciphertext_len, const uint8_t* key,
- int key_len, const uint8_t* aad, int aad_len, uint8_t* plaintext);
-
- private:
- /// Can serve one key length only. Possible values: 16, 24, 32 bytes.
- explicit AesDecryptor(ParquetCipher::type alg_id, int key_len, bool metadata);
- // PIMPL Idiom
- class AesDecryptorImpl;
- std::unique_ptr<AesDecryptorImpl> impl_;
-};
-
-std::string CreateModuleAad(const std::string& file_aad, int8_t module_type,
- int16_t row_group_ordinal, int16_t column_ordinal,
- int16_t page_ordinal);
-
-std::string CreateFooterAad(const std::string& aad_prefix_bytes);
-
-// Update last two bytes of page (or page header) module AAD
-void QuickUpdatePageAad(const std::string& AAD, int16_t new_page_ordinal);
-
-// Wraps OpenSSL RAND_bytes function
-void RandBytes(unsigned char* buf, int num);
-
-} // namespace encryption
-} // namespace parquet
+#if ARROW_VERSION_MAJOR < 20
+#include "encryption_internal_19.h"
+#else
+#include "encryption_internal_20.h"
+#endif
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor_19.h.orig 2025-07-08 07:40:29.814292389 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor_19.h 2025-07-08 07:40:29.813727465 -0400
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <map>
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "parquet/schema.h"
+
+namespace parquet {
+
+namespace encryption {
+class AesDecryptor;
+class AesEncryptor;
+} // namespace encryption
+
+class FileDecryptionProperties;
+
+class PARQUET_EXPORT Decryptor {
+ public:
+ Decryptor(encryption::AesDecryptor* decryptor, const std::string& key,
+ const std::string& file_aad, const std::string& aad,
+ ::arrow::MemoryPool* pool);
+
+ const std::string& file_aad() const { return file_aad_; }
+ void UpdateAad(const std::string& aad) { aad_ = aad; }
+ ::arrow::MemoryPool* pool() { return pool_; }
+
+ int CiphertextSizeDelta();
+ int Decrypt(const uint8_t* ciphertext, int ciphertext_len, uint8_t* plaintext);
+
+ private:
+ encryption::AesDecryptor* aes_decryptor_;
+ std::string key_;
+ std::string file_aad_;
+ std::string aad_;
+ ::arrow::MemoryPool* pool_;
+};
+
+class InternalFileDecryptor {
+ public:
+ explicit InternalFileDecryptor(FileDecryptionProperties* properties,
+ const std::string& file_aad,
+ ParquetCipher::type algorithm,
+ const std::string& footer_key_metadata,
+ ::arrow::MemoryPool* pool);
+
+ std::string& file_aad() { return file_aad_; }
+
+ std::string GetFooterKey();
+
+ ParquetCipher::type algorithm() { return algorithm_; }
+
+ std::string& footer_key_metadata() { return footer_key_metadata_; }
+
+ FileDecryptionProperties* properties() { return properties_; }
+
+ void WipeOutDecryptionKeys();
+
+ ::arrow::MemoryPool* pool() { return pool_; }
+
+ std::shared_ptr<Decryptor> GetFooterDecryptor();
+ std::shared_ptr<Decryptor> GetFooterDecryptorForColumnMeta(const std::string& aad = "");
+ std::shared_ptr<Decryptor> GetFooterDecryptorForColumnData(const std::string& aad = "");
+ std::shared_ptr<Decryptor> GetColumnMetaDecryptor(
+ const std::string& column_path, const std::string& column_key_metadata,
+ const std::string& aad = "");
+ std::shared_ptr<Decryptor> GetColumnDataDecryptor(
+ const std::string& column_path, const std::string& column_key_metadata,
+ const std::string& aad = "");
+
+ private:
+ FileDecryptionProperties* properties_;
+ // Concatenation of aad_prefix (if exists) and aad_file_unique
+ std::string file_aad_;
+ std::map<std::string, std::shared_ptr<Decryptor>> column_data_map_;
+ std::map<std::string, std::shared_ptr<Decryptor>> column_metadata_map_;
+
+ std::shared_ptr<Decryptor> footer_metadata_decryptor_;
+ std::shared_ptr<Decryptor> footer_data_decryptor_;
+ ParquetCipher::type algorithm_;
+ std::string footer_key_metadata_;
+ std::vector<encryption::AesDecryptor*> all_decryptors_;
+
+ /// Key must be 16, 24 or 32 bytes in length. Thus there could be up to three
+ // types of meta_decryptors and data_decryptors.
+ std::unique_ptr<encryption::AesDecryptor> meta_decryptor_[3];
+ std::unique_ptr<encryption::AesDecryptor> data_decryptor_[3];
+
+ ::arrow::MemoryPool* pool_;
+
+ std::shared_ptr<Decryptor> GetFooterDecryptor(const std::string& aad, bool metadata);
+ std::shared_ptr<Decryptor> GetColumnDecryptor(const std::string& column_path,
+ const std::string& column_key_metadata,
+ const std::string& aad,
+ bool metadata = false);
+
+ encryption::AesDecryptor* GetMetaAesDecryptor(size_t key_size);
+ encryption::AesDecryptor* GetDataAesDecryptor(size_t key_size);
+
+ int MapKeyLenToDecryptorArrayIndex(int key_len);
+};
+
+} // namespace parquet
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor_20.h.orig 2025-07-08 07:40:29.815411998 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor_20.h 2025-07-08 07:40:29.815245155 -0400
@@ -0,0 +1,148 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <memory>
+#include <mutex>
+#include <string>
+#include <vector>
+
+#include "parquet/schema.h"
+
+namespace parquet {
+
+namespace encryption {
+class AesDecryptor;
+class AesEncryptor;
+} // namespace encryption
+
+class ColumnCryptoMetaData;
+class FileDecryptionProperties;
+
+// An object handling decryption using well-known encryption parameters
+//
+// CAUTION: Decryptor objects are not thread-safe.
+class PARQUET_EXPORT Decryptor {
+ public:
+ Decryptor(std::unique_ptr<encryption::AesDecryptor> decryptor, const std::string& key,
+ const std::string& file_aad, const std::string& aad,
+ ::arrow::MemoryPool* pool);
+ ~Decryptor();
+
+ const std::string& file_aad() const { return file_aad_; }
+ void UpdateAad(const std::string& aad) { aad_ = aad; }
+ ::arrow::MemoryPool* pool() { return pool_; }
+
+ [[nodiscard]] int32_t PlaintextLength(int32_t ciphertext_len) const;
+ [[nodiscard]] int32_t CiphertextLength(int32_t plaintext_len) const;
+ int32_t Decrypt(::arrow::util::span<const uint8_t> ciphertext,
+ ::arrow::util::span<uint8_t> plaintext);
+
+ private:
+ std::unique_ptr<encryption::AesDecryptor> aes_decryptor_;
+ std::string key_;
+ std::string file_aad_;
+ std::string aad_;
+ ::arrow::MemoryPool* pool_;
+};
+
+class InternalFileDecryptor {
+ public:
+ explicit InternalFileDecryptor(std::shared_ptr<FileDecryptionProperties> properties,
+ const std::string& file_aad,
+ ParquetCipher::type algorithm,
+ const std::string& footer_key_metadata,
+ ::arrow::MemoryPool* pool);
+
+ const std::string& file_aad() const { return file_aad_; }
+
+ std::string GetFooterKey();
+
+ ParquetCipher::type algorithm() const { return algorithm_; }
+
+ const std::string& footer_key_metadata() const { return footer_key_metadata_; }
+
+ const std::shared_ptr<FileDecryptionProperties>& properties() const {
+ return properties_;
+ }
+
+ ::arrow::MemoryPool* pool() const { return pool_; }
+
+ // Get a Decryptor instance for the Parquet footer
+ std::unique_ptr<Decryptor> GetFooterDecryptor();
+
+ // Get a Decryptor instance for column chunk metadata.
+ std::unique_ptr<Decryptor> GetColumnMetaDecryptor(
+ const std::string& column_path, const std::string& column_key_metadata,
+ const std::string& aad = "") {
+ return GetColumnDecryptor(column_path, column_key_metadata, aad, /*metadata=*/true);
+ }
+
+ // Get a Decryptor instance for column chunk data.
+ std::unique_ptr<Decryptor> GetColumnDataDecryptor(
+ const std::string& column_path, const std::string& column_key_metadata,
+ const std::string& aad = "") {
+ return GetColumnDecryptor(column_path, column_key_metadata, aad, /*metadata=*/false);
+ }
+
+ // Get a Decryptor factory for column chunk metadata.
+ //
+ // This is typically useful if multi-threaded decryption is expected.
+ // This is a static function as it accepts a null `InternalFileDecryptor*`
+ // argument if the column is not encrypted.
+ static std::function<std::unique_ptr<Decryptor>()> GetColumnMetaDecryptorFactory(
+ InternalFileDecryptor*, const ColumnCryptoMetaData* crypto_metadata,
+ const std::string& aad = "");
+ // Get a Decryptor factory for column chunk data.
+ //
+ // This is typically useful if multi-threaded decryption is expected.
+ // This is a static function as it accepts a null `InternalFileDecryptor*`
+ // argument if the column is not encrypted.
+ static std::function<std::unique_ptr<Decryptor>()> GetColumnDataDecryptorFactory(
+ InternalFileDecryptor*, const ColumnCryptoMetaData* crypto_metadata,
+ const std::string& aad = "");
+
+ private:
+ std::shared_ptr<FileDecryptionProperties> properties_;
+ // Concatenation of aad_prefix (if exists) and aad_file_unique
+ std::string file_aad_;
+ ParquetCipher::type algorithm_;
+ std::string footer_key_metadata_;
+ ::arrow::MemoryPool* pool_;
+
+ // Protects footer_key_ updates
+ std::mutex mutex_;
+ std::string footer_key_;
+
+ std::string GetColumnKey(const std::string& column_path,
+ const std::string& column_key_metadata);
+
+ std::unique_ptr<Decryptor> GetFooterDecryptor(const std::string& aad, bool metadata);
+
+ std::unique_ptr<Decryptor> GetColumnDecryptor(const std::string& column_path,
+ const std::string& column_key_metadata,
+ const std::string& aad, bool metadata);
+
+ std::function<std::unique_ptr<Decryptor>()> GetColumnDecryptorFactory(
+ const ColumnCryptoMetaData* crypto_metadata, const std::string& aad, bool metadata);
+};
+
+void UpdateDecryptor(Decryptor* decryptor, int16_t row_group_ordinal,
+ int16_t column_ordinal, int8_t module_type);
+
+} // namespace parquet
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor.h.orig 2024-10-06 07:18:41.000000000 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/internal_file_decryptor.h 2025-07-08 07:40:29.813623143 -0400
@@ -17,105 +17,8 @@
#pragma once
-#include <map>
-#include <memory>
-#include <string>
-#include <vector>
-
-#include "parquet/schema.h"
-
-namespace parquet {
-
-namespace encryption {
-class AesDecryptor;
-class AesEncryptor;
-} // namespace encryption
-
-class FileDecryptionProperties;
-
-class PARQUET_EXPORT Decryptor {
- public:
- Decryptor(encryption::AesDecryptor* decryptor, const std::string& key,
- const std::string& file_aad, const std::string& aad,
- ::arrow::MemoryPool* pool);
-
- const std::string& file_aad() const { return file_aad_; }
- void UpdateAad(const std::string& aad) { aad_ = aad; }
- ::arrow::MemoryPool* pool() { return pool_; }
-
- int CiphertextSizeDelta();
- int Decrypt(const uint8_t* ciphertext, int ciphertext_len, uint8_t* plaintext);
-
- private:
- encryption::AesDecryptor* aes_decryptor_;
- std::string key_;
- std::string file_aad_;
- std::string aad_;
- ::arrow::MemoryPool* pool_;
-};
-
-class InternalFileDecryptor {
- public:
- explicit InternalFileDecryptor(FileDecryptionProperties* properties,
- const std::string& file_aad,
- ParquetCipher::type algorithm,
- const std::string& footer_key_metadata,
- ::arrow::MemoryPool* pool);
-
- std::string& file_aad() { return file_aad_; }
-
- std::string GetFooterKey();
-
- ParquetCipher::type algorithm() { return algorithm_; }
-
- std::string& footer_key_metadata() { return footer_key_metadata_; }
-
- FileDecryptionProperties* properties() { return properties_; }
-
- void WipeOutDecryptionKeys();
-
- ::arrow::MemoryPool* pool() { return pool_; }
-
- std::shared_ptr<Decryptor> GetFooterDecryptor();
- std::shared_ptr<Decryptor> GetFooterDecryptorForColumnMeta(const std::string& aad = "");
- std::shared_ptr<Decryptor> GetFooterDecryptorForColumnData(const std::string& aad = "");
- std::shared_ptr<Decryptor> GetColumnMetaDecryptor(
- const std::string& column_path, const std::string& column_key_metadata,
- const std::string& aad = "");
- std::shared_ptr<Decryptor> GetColumnDataDecryptor(
- const std::string& column_path, const std::string& column_key_metadata,
- const std::string& aad = "");
-
- private:
- FileDecryptionProperties* properties_;
- // Concatenation of aad_prefix (if exists) and aad_file_unique
- std::string file_aad_;
- std::map<std::string, std::shared_ptr<Decryptor>> column_data_map_;
- std::map<std::string, std::shared_ptr<Decryptor>> column_metadata_map_;
-
- std::shared_ptr<Decryptor> footer_metadata_decryptor_;
- std::shared_ptr<Decryptor> footer_data_decryptor_;
- ParquetCipher::type algorithm_;
- std::string footer_key_metadata_;
- std::vector<encryption::AesDecryptor*> all_decryptors_;
-
- /// Key must be 16, 24 or 32 bytes in length. Thus there could be up to three
- // types of meta_decryptors and data_decryptors.
- std::unique_ptr<encryption::AesDecryptor> meta_decryptor_[3];
- std::unique_ptr<encryption::AesDecryptor> data_decryptor_[3];
-
- ::arrow::MemoryPool* pool_;
-
- std::shared_ptr<Decryptor> GetFooterDecryptor(const std::string& aad, bool metadata);
- std::shared_ptr<Decryptor> GetColumnDecryptor(const std::string& column_path,
- const std::string& column_key_metadata,
- const std::string& aad,
- bool metadata = false);
-
- encryption::AesDecryptor* GetMetaAesDecryptor(size_t key_size);
- encryption::AesDecryptor* GetDataAesDecryptor(size_t key_size);
-
- int MapKeyLenToDecryptorArrayIndex(int key_len);
-};
-
-} // namespace parquet
+#if ARROW_VERSION_MAJOR < 20
+#include "internal_file_decryptor_19.h"
+#else
+#include "internal_file_decryptor_20.h"
+#endif
--- ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/s3select_parquet_intrf.h.orig 2024-10-06 07:18:41.000000000 -0400
+++ ceph-20.0.0-2362-ga9d20fc0/src/s3select/include/s3select_parquet_intrf.h 2025-07-08 07:40:29.816727417 -0400
@@ -1002,6 +1002,7 @@
throw ParquetException("Encrypted files cannot contain more than 32767 row groups");
}
+#if ARROW_VERSION_MAJOR < 20
// The column is encrypted
std::shared_ptr<::parquet::Decryptor> meta_decryptor;
std::shared_ptr<Decryptor> data_decryptor;
@@ -1035,6 +1036,25 @@
false,
#endif
properties_.memory_pool(), &ctx);
+#else
+ // Arrow 20+ version uses factory functions instead of shared_ptr for decryptors
+ std::function<std::unique_ptr<Decryptor>()> meta_decryptor_factory =
+ InternalFileDecryptor::GetColumnMetaDecryptorFactory(file_decryptor_.get(), crypto_metadata.get());
+ std::function<std::unique_ptr<Decryptor>()> data_decryptor_factory =
+ InternalFileDecryptor::GetColumnDataDecryptorFactory(file_decryptor_.get(), crypto_metadata.get());
+
+ const CryptoContext ctx {
+ col->has_dictionary_page(),
+ row_group_ordinal_,
+ static_cast<int16_t>(i),
+ meta_decryptor_factory,
+ data_decryptor_factory,
+ };
+
+ return PageReader::Open(stream, col->num_values(), col->compression(),
+ false,
+ properties_.memory_pool(), &ctx);
+#endif
}
private:
@@ -1071,7 +1091,9 @@
}
void Close() override {
+#if ARROW_VERSION_MAJOR < 20
if (file_decryptor_) file_decryptor_->WipeOutDecryptionKeys();
+#endif
}
std::shared_ptr<RowGroupReader> GetRowGroup(int i) override {
@@ -1249,9 +1271,17 @@
// Handle AAD prefix
EncryptionAlgorithm algo = file_crypto_metadata->encryption_algorithm();
std::string file_aad = HandleAadPrefix(file_decryption_properties, algo);
+#if ARROW_VERSION_MAJOR < 20
file_decryptor_ = std::make_shared<::parquet::InternalFileDecryptor>(
file_decryption_properties, file_aad, algo.algorithm,
file_crypto_metadata->key_metadata(), properties_.memory_pool());
+#else
+ // Arrow 20+ takes a shared_ptr to FileDecryptionProperties
+ file_decryptor_ = std::make_shared<::parquet::InternalFileDecryptor>(
+ std::shared_ptr<FileDecryptionProperties>(file_decryption_properties),
+ file_aad, algo.algorithm,
+ file_crypto_metadata->key_metadata(), properties_.memory_pool());
+#endif
int64_t metadata_offset = source_size_ - kFooterSize - footer_len + crypto_metadata_len;
uint32_t metadata_len = footer_len - crypto_metadata_len;
@@ -1282,9 +1312,18 @@
EncryptionAlgorithm algo = file_metadata_->encryption_algorithm();
// Handle AAD prefix
std::string file_aad = HandleAadPrefix(file_decryption_properties, algo);
+#if ARROW_VERSION_MAJOR < 20
file_decryptor_ = std::make_shared<::parquet::InternalFileDecryptor>(
file_decryption_properties, file_aad, algo.algorithm,
file_metadata_->footer_signing_key_metadata(), properties_.memory_pool());
+#else
+ // Arrow 20+ takes a shared_ptr to FileDecryptionProperties
+ file_decryptor_ = std::make_shared<::parquet::InternalFileDecryptor>(
+ std::shared_ptr<FileDecryptionProperties>(file_decryption_properties),
+ file_aad, algo.algorithm,
+ file_metadata_->footer_signing_key_metadata(), properties_.memory_pool());
+ // In Arrow 20+, no need to set file_decryptor in metadata
+#endif
// set the InternalFileDecryptor in the metadata as well, as it's used
// for signature verification and for ColumnChunkMetaData creation.
#if GAL_set_file_decryptor_declare_private
@@ -1,37 +0,0 @@
From 8c78a22d2cf69892570f635735d9735169b64a75 Mon Sep 17 00:00:00 2001
From: Peter Sabaini <peter.sabaini@canonical.com>
Date: Wed, 11 Sep 2024 16:56:50 +0200
Subject: [PATCH] ceph-volume: fix importlib.metadata compat
The importlib.metadata library removed older shims in releases >5.0.0
where EntryPoints objects use .select() instead of dict-like access.
Fixes: https://tracker.ceph.com/issues/68032
Signed-off-by: Peter Sabaini <peter.sabaini@canonical.com>
---
src/ceph-volume/ceph_volume/main.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/src/ceph-volume/ceph_volume/main.py b/src/ceph-volume/ceph_volume/main.py
index f8eca65ec497c..4f27f429e89e2 100644
--- a/src/ceph-volume/ceph_volume/main.py
+++ b/src/ceph-volume/ceph_volume/main.py
@@ -11,8 +11,16 @@
from importlib.metadata import entry_points
def get_entry_points(group: str): # type: ignore
- return entry_points().get(group, []) # type: ignore
+ eps = entry_points()
+ if hasattr(eps, 'select'):
+ # New importlib.metadata uses .select()
+ return eps.select(group=group)
+ else:
+ # Fallback to older EntryPoints that returns dicts
+ return eps.get(group, []) # type: ignore
+
except ImportError:
+ # Fallback to `pkg_resources` for older versions
from pkg_resources import iter_entry_points as entry_points # type: ignore
def get_entry_points(group: str): # type: ignore
@@ -0,0 +1,10 @@
--- ceph-20.0.0-2099-gc62cbb7d/src/json_spirit/json_spirit_reader_template.h.orig 2025-06-09 11:56:21.534808695 -0400
+++ ceph-20.0.0-2099-gc62cbb7d/src/json_spirit/json_spirit_reader_template.h 2025-06-09 12:01:56.358334353 -0400
@@ -136,6 +136,7 @@
{
typedef typename String_type::const_iterator Iter_type;
+ if( end - begin < 1 ) return String_type();
if( end - begin < 2 ) return String_type( begin, end );
String_type result;
@@ -1,53 +0,0 @@
From 5924df771f850c249396a37a62f97ac242bf2f96 Mon Sep 17 00:00:00 2001
From: Hector Martin <marcan@marcan.st>
Date: Wed, 20 Aug 2025 13:34:08 +0900
Subject: [PATCH 1/2] mgr/dashboard: Make saml2 robust against module load
errors
Loading saml2 can fail due to issues with the xmlsec package (possibly
subinterpreter related):
File "/usr/share/ceph/mgr/dashboard/controllers/saml2.py", line 6, in <module>
from onelogin.saml2.auth import OneLogin_Saml2_Auth
File "/lib/python3.13/site-packages/onelogin/saml2/auth.py", line 12, in <module>
import xmlsec
xmlsec.Error: (100, 'lxml & xmlsec libxml2 library version mismatch')
Instead of taking down the entire dashboard module, treat this exception
like a missing saml2 package.
Signed-off-by: Hector Martin <marcan@marcan.st>
---
src/pybind/mgr/dashboard/controllers/saml2.py | 2 +-
src/pybind/mgr/dashboard/services/sso.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/pybind/mgr/dashboard/controllers/saml2.py b/src/pybind/mgr/dashboard/controllers/saml2.py
index c11b18a27bc7..e859f558cdc4 100644
--- a/src/pybind/mgr/dashboard/controllers/saml2.py
+++ b/src/pybind/mgr/dashboard/controllers/saml2.py
@@ -8,7 +8,7 @@ try:
from onelogin.saml2.settings import OneLogin_Saml2_Settings
python_saml_imported = True
-except ImportError:
+except Exception:
python_saml_imported = False
from .. import mgr
diff --git a/src/pybind/mgr/dashboard/services/sso.py b/src/pybind/mgr/dashboard/services/sso.py
index 2290e6ea3e15..38910ca4aa34 100644
--- a/src/pybind/mgr/dashboard/services/sso.py
+++ b/src/pybind/mgr/dashboard/services/sso.py
@@ -20,7 +20,7 @@ try:
from onelogin.saml2.settings import OneLogin_Saml2_Settings as Saml2Settings
python_saml_imported = True
-except ImportError:
+except Exception:
python_saml_imported = False
--
2.50.1
+13
View File
@@ -0,0 +1,13 @@
--- ceph-20.2.0/src/CMakeLists.txt.orig 2026-01-20 14:31:56.764231793 -0500
+++ ceph-20.2.0/src/CMakeLists.txt 2026-01-20 14:35:21.258787691 -0500
@@ -1051,8 +1051,8 @@
if(WITH_RADOSGW)
if(WITH_RADOSGW_SELECT_PARQUET OR WITH_RADOSGW_ARROW_FLIGHT)
if(WITH_SYSTEM_ARROW)
- find_package(Arrow 4 REQUIRED QUIET)
- find_package(Parquet 4 REQUIRED QUIET)
+ find_package(Arrow REQUIRED QUIET)
+ find_package(Parquet REQUIRED QUIET)
else()
# find arrow's dependencies
if (WITH_SYSTEM_UTF8PROC)
+22
View File
@@ -0,0 +1,22 @@
--- ceph-20.0.0-2787-g838ba95e/src/jaegertracing/opentelemetry-cpp/api/include/opentelemetry/nostd/internal/absl/base/options.h.orig 2025-07-31 10:18:56.636528168 -0400
+++ ceph-20.0.0-2787-g838ba95e/src/jaegertracing/opentelemetry-cpp/api/include/opentelemetry/nostd/internal/absl/base/options.h 2025-07-31 10:19:12.923329646 -0400
@@ -70,7 +70,7 @@
// Include a standard library header to allow configuration based on the
// standard library in use.
#ifdef __cplusplus
-#include <ciso646>
+#include <iso646.h>
#endif
// -----------------------------------------------------------------------------
--- ceph-20.2.0/src/boost/boost/redis/adapter/detail/adapters.hpp.orig 2026-02-06 09:54:39.748078321 -0500
+++ ceph-20.2.0/src/boost/boost/redis/adapter/detail/adapters.hpp 2026-02-06 09:55:02.427664048 -0500
@@ -29,7 +29,7 @@
#include <charconv>
// See https://stackoverflow.com/a/31658120/1077832
-#include<ciso646>
+#include<iso646.h>
#ifdef _LIBCPP_VERSION
#else
#include <cstdlib>
@@ -1,44 +0,0 @@
From 76af91b516bb31ae0a79f55f315bef2c2105a06e Mon Sep 17 00:00:00 2001
From: Nizamudeen A <nia@redhat.com>
Date: Thu, 26 Jun 2025 12:55:22 +0530
Subject: [PATCH 2/2] mgr/dashboard: catch protobuf error due to mismatch in
version
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit acb0f19c4a50b2ea68f328a61a14a2da06be298b)
Conflicts:
src/pybind/mgr/dashboard/services/nvmeof_client.py
- only kept relavant portion applicable for squid
---
src/pybind/mgr/dashboard/services/nvmeof_client.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/src/pybind/mgr/dashboard/services/nvmeof_client.py b/src/pybind/mgr/dashboard/services/nvmeof_client.py
index e0ea6d1e48b3..be2b5edbaa52 100644
--- a/src/pybind/mgr/dashboard/services/nvmeof_client.py
+++ b/src/pybind/mgr/dashboard/services/nvmeof_client.py
@@ -1,3 +1,5 @@
+# pylint: disable=unexpected-keyword-arg
+
import functools
import logging
from collections.abc import Iterable
@@ -9,6 +11,14 @@ from .nvmeof_conf import NvmeofGatewaysConfig
logger = logging.getLogger("nvmeof_client")
try:
+ # if the protobuf version is newer than what we generated with
+ # proto file import will fail (because of differences between what's
+ # available in centos and ubuntu).
+ # this "hack" should be removed once we update both the
+ # distros; centos and ubuntu.
+ import os
+ os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
+
import grpc # type: ignore
import grpc._channel # type: ignore
from google.protobuf.message import Message # type: ignore
--
2.50.1
@@ -0,0 +1,10 @@
--- ceph-20.2.0/src/crimson/common/smp_helpers.h.orig 2025-12-04 10:38:34.906915674 -0500
+++ ceph-20.2.0/src/crimson/common/smp_helpers.h 2025-12-04 10:38:45.200845804 -0500
@@ -8,6 +8,7 @@
#include <optional>
#include <type_traits>
#include <vector>
+#include <coroutine>
#include <seastar/core/shared_future.hh>
#include <seastar/core/smp.hh>
+70
View File
@@ -0,0 +1,70 @@
--- ceph-20.2.0/src/common/Formatter.h.orig 2026-01-06 17:47:18.437014517 -0500
+++ ceph-20.2.0/src/common/Formatter.h 2026-01-06 17:47:53.015404074 -0500
@@ -12,6 +12,7 @@
#include <memory>
#include <vector>
#include <stdarg.h>
+#include <cstdint>
#include <sstream>
#include <map>
#include <vector>
--- ceph-20.2.0/src/mds/Locker.cc.orig 2026-01-07 10:34:12.731210930 -0500
+++ ceph-20.2.0/src/mds/Locker.cc 2026-01-07 10:34:35.627823155 -0500
@@ -4489,7 +4489,7 @@
dout(7) << "handle_client_lease client." << client << " renew on " << *dn
<< (!dn->lock.can_lease(client)?", revoking lease":"") << dendl;
if (dn->lock.can_lease(client)) {
- auto reply = make_message<MClientLease>(*m);
+ auto reply = ceph::make_message<MClientLease>(*m);
int pool = 1; // fixme.. do something smart!
reply->h.duration_ms = (int)(1000 * mdcache->client_lease_durations[pool]);
reply->h.seq = ++l->seq;
--- ceph-20.2.0/src/mds/Server.cc.orig 2026-01-07 11:27:13.532097596 -0500
+++ ceph-20.2.0/src/mds/Server.cc 2026-01-07 11:28:42.411581266 -0500
@@ -150,7 +150,7 @@
}
}
batch_reqs.clear();
- server->reply_client_request(mdr, make_message<MClientReply>(*mdr->client_request, r));
+ server->reply_client_request(mdr, ceph::make_message<MClientReply>(*mdr->client_request, r));
}
void print(std::ostream& o) const override {
o << "[batch front=" << *mdr << "]";
@@ -2142,7 +2142,7 @@
dout(20) << __func__ << ": batch head " << *mdr << dendl;
mdr->release_batch_op()->respond(r);
} else {
- reply_client_request(mdr, make_message<MClientReply>(*mdr->client_request, r));
+ reply_client_request(mdr, ceph::make_message<MClientReply>(*mdr->client_request, r));
}
} else if (mdr->internal_op > -1) {
dout(10) << __func__ << ": completing with result " << cpp_strerror(r) << " on internal " << *mdr << dendl;
@@ -2290,7 +2290,7 @@
}
- auto reply = make_message<MClientReply>(*req, 0);
+ auto reply = ceph::make_message<MClientReply>(*req, 0);
reply->set_unsafe();
// mark xlocks "done", indicating that we are exposing uncommitted changes.
@@ -2632,7 +2632,7 @@
req->get_op() != CEPH_MDS_OP_OPEN &&
req->get_op() != CEPH_MDS_OP_CREATE)) {
dout(5) << "already completed " << req->get_reqid() << dendl;
- auto reply = make_message<MClientReply>(*req, 0);
+ auto reply = ceph::make_message<MClientReply>(*req, 0);
if (created != inodeno_t()) {
bufferlist extra;
set_reply_extra_bl(req, created, extra);
--- ceph-20.2.0/src/mds/MDCache.cc.orig 2026-01-07 11:28:48.882470871 -0500
+++ ceph-20.2.0/src/mds/MDCache.cc 2026-01-07 11:29:12.405069562 -0500
@@ -10535,7 +10535,7 @@
CInode *cur = 0;
- auto reply = make_message<MDiscoverReply>(*dis);
+ auto reply = ceph::make_message<MDiscoverReply>(*dis);
snapid_t snapid = dis->get_snapid();
@@ -0,0 +1,11 @@
--- ceph-20.2.0/src/rgw/driver/dbstore/CMakeLists.txt.orig 2026-01-27 19:53:53.780108462 -0500
+++ ceph-20.2.0/src/rgw/driver/dbstore/CMakeLists.txt 2026-01-28 07:30:04.861741687 -0500
@@ -34,7 +34,7 @@
list(APPEND link_targets jaeger_base)
endif()
list(APPEND link_targets rgw_common)
-target_link_libraries(dbstore_lib PUBLIC ${link_targets})
+target_link_libraries(dbstore_lib PUBLIC ${link_targets} PRIVATE rgw_a)
set (CMAKE_LINK_LIBRARIES ${CMAKE_LINK_LIBRARIES} dbstore_lib)
@@ -0,0 +1,8 @@
--- ceph-20.2.0/src/jaegertracing/opentelemetry-cpp/CMakeLists.txt.orig 2026-02-17 08:17:06.372917033 -0500
+++ ceph-20.2.0/src/jaegertracing/opentelemetry-cpp/CMakeLists.txt 2026-02-17 08:17:44.026323618 -0500
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.1)
+cmake_minimum_required(VERSION 3.5)
# See https://cmake.org/cmake/help/v3.3/policy/CMP0057.html required by certain
# versions of gtest
+14
View File
@@ -0,0 +1,14 @@
--- ceph-20.2.0/src/rgw/rgw_lua_utils.cc.orig 2026-02-17 11:46:05.677894827 -0500
+++ ceph-20.2.0/src/rgw/rgw_lua_utils.cc 2026-02-17 11:50:06.106569226 -0500
@@ -115,7 +115,11 @@
// create new lua state together with reference to the guard
lua_State* newstate(lua_state_guard* guard) {
+#if (LUA_VERSION_NUM < 505)
lua_State* L = lua_newstate(allocator, guard);
+#else
+ lua_State* L = lua_newstate(allocator, guard, 314159);
+#endif
if (L) {
lua_atpanic(L, [](lua_State* L) -> int {
const char* msg = lua_tostring(L, -1);
@@ -1,152 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: "Max R. Carrara" <m.carrara@proxmox.com>
Date: Wed, 16 Jul 2025 13:14:39 +0200
Subject: [PATCH 58/59] pybind/mgr/restful: provide workaround for PyO3
ImportError
Move the self-signed cert generation into a separate module
inside python-common/ceph and run the module in a separate Python
process.
This provides a workaround for the ImportError thrown by PyO3 when
the `restful` module is loaded in the context of multiple Python
sub-interpreters being present. In particular, the ImportError is
thrown by the `crypto` module of the `OpenSSL` package.
Inspired by an upstream PR [0].
[0]: https://github.com/ceph/ceph/pull/62951
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
src/pybind/mgr/restful/module.py | 24 +++------
src/python-common/ceph/_crypto_wrap.py | 69 ++++++++++++++++++++++++++
2 files changed, 76 insertions(+), 17 deletions(-)
create mode 100644 src/python-common/ceph/_crypto_wrap.py
diff --git a/src/pybind/mgr/restful/module.py b/src/pybind/mgr/restful/module.py
index 0f8c78e0bd8..7f93c41f1e6 100644
--- a/src/pybind/mgr/restful/module.py
+++ b/src/pybind/mgr/restful/module.py
@@ -7,6 +7,7 @@ import json
import time
import errno
import inspect
+import subprocess
import tempfile
import threading
import traceback
@@ -19,7 +20,6 @@ from . import context
from uuid import uuid4
from pecan import jsonify, make_app
-from OpenSSL import crypto
from pecan.rest import RestController
from werkzeug.serving import make_server, make_ssl_devcert
@@ -401,24 +401,14 @@ class Module(MgrModule):
def create_self_signed_cert(self):
- # create a key pair
- pkey = crypto.PKey()
- pkey.generate_key(crypto.TYPE_RSA, 2048)
-
- # create a self-signed cert
- cert = crypto.X509()
- cert.get_subject().O = "IT"
- cert.get_subject().CN = "ceph-restful"
- cert.set_serial_number(int(uuid4()))
- cert.gmtime_adj_notBefore(0)
- cert.gmtime_adj_notAfter(10*365*24*60*60)
- cert.set_issuer(cert.get_subject())
- cert.set_pubkey(pkey)
- cert.sign(pkey, 'sha512')
+ cmd = ["python3", "-m", "ceph._crypto_wrap", "create_self_signed_cert"]
+
+ response = subprocess.run(cmd, capture_output=True, check=True)
+ response_obj = json.loads(response.stdout)
return (
- crypto.dump_certificate(crypto.FILETYPE_PEM, cert),
- crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey)
+ response_obj["cert"].encode("utf-8"),
+ response_obj["key"].encode("utf-8"),
)
diff --git a/src/python-common/ceph/_crypto_wrap.py b/src/python-common/ceph/_crypto_wrap.py
new file mode 100644
index 00000000000..16a19a5345e
--- /dev/null
+++ b/src/python-common/ceph/_crypto_wrap.py
@@ -0,0 +1,69 @@
+"""CLI wrapper for cryptographic functions of the :mod:`restful` module.
+
+To be called via :func:`subprocess.run()` as a workaround for
+:class:`ImportError`s related to PyO3's current lack of sub-interpreter
+support.
+
+Note:
+ Since this module is installed as part of the ``ceph`` package,
+ it should be called like so::
+
+ python3 -m ceph._crypto_wrap create_self_signed_cert
+"""
+
+import argparse
+import sys
+import json
+
+from argparse import Namespace
+from typing import Any
+from uuid import uuid4
+
+from OpenSSL import crypto
+
+
+def _respond(data: dict[str, Any]) -> None:
+ json.dump(data, sys.stdout)
+ sys.stdout.flush()
+
+
+def create_self_signed_cert(args: Namespace) -> None:
+ cert_key_pair = _create_self_signed_cert()
+ _respond(cert_key_pair)
+
+
+def _create_self_signed_cert() -> dict[str, str]:
+ # create a key pair
+ pubkey = crypto.PKey()
+ pubkey.generate_key(crypto.TYPE_RSA, 2048)
+
+ # create a self-signed cert
+ cert = crypto.X509()
+ cert.get_subject().O = "IT"
+ cert.get_subject().CN = "ceph-restful"
+ cert.set_serial_number(int(uuid4()))
+ cert.gmtime_adj_notBefore(0)
+ cert.gmtime_adj_notAfter(10 * 365 * 24 * 60 * 60)
+ cert.set_issuer(cert.get_subject())
+ cert.set_pubkey(pubkey)
+ cert.sign(pubkey, "sha512")
+
+ return {
+ "cert": crypto.dump_certificate(crypto.FILETYPE_PEM, cert).decode(),
+ "key": crypto.dump_privatekey(crypto.FILETYPE_PEM, pubkey).decode(),
+ }
+
+
+def main() -> None:
+ parser = argparse.ArgumentParser(prog="_crypto_wrap.py")
+ subparsers = parser.add_subparsers(required=True)
+
+ parser_cssc = subparsers.add_parser("create_self_signed_cert")
+ parser_cssc.set_defaults(func=create_self_signed_cert)
+
+ args = parser.parse_args()
+ args.func(args)
+
+
+if __name__ == "__main__":
+ main()
@@ -1,56 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: "Max R. Carrara" <m.carrara@proxmox.com>
Date: Wed, 16 Jul 2025 16:31:43 +0200
Subject: [PATCH 59/59] mgr: fix module import by making NOTIFY_TYPES in py
modules optional
If NOTIFY_TYPES isn't an attribute of the passed class, the Python
(sub-)interpreter raises an AttributeError that must be handled or cleared
explicitly via the Python C-API. Unfortunately, this isn't done here,
which means that the exception sticks around until handled.
This caused a call to PyModule::load_subclass_of() to fail and
incorrectly report the AttributeError as cause.
Checking whether the class has NOTIFY_TYPES as attribute in the first
place fixes this.
Note that there's an upstream PR [0] that wasn't backported that aimed
to fix this, but does so incorrectly, as the exception is still not
cleared there. The warnings regarding NOTIFY_TYPES missing also occurs
on Reef but doesn't cause any module imports to fail there. As the
affected Ceph code has stayed mostly the same between bookworm and
trixie releases, this suggests that some behavior between Python 3.11
and 3.13 likely changed.
Either way, avoiding the AttributeError altogether fixes this.
[0]: https://github.com/ceph/ceph/pull/57106
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
---
src/mgr/PyModule.cc | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/src/mgr/PyModule.cc b/src/mgr/PyModule.cc
index 084cf3ffc1e..e6fd269dca5 100644
--- a/src/mgr/PyModule.cc
+++ b/src/mgr/PyModule.cc
@@ -513,11 +513,13 @@ int PyModule::register_options(PyObject *cls)
int PyModule::load_notify_types()
{
- PyObject *ls = PyObject_GetAttrString(pClass, "NOTIFY_TYPES");
- if (ls == nullptr) {
- derr << "Module " << get_name() << " has missing NOTIFY_TYPES member" << dendl;
- return -EINVAL;
+ if (!PyObject_HasAttrString(pClass, "NOTIFY_TYPES")) {
+ dout(10) << "Module " << get_name() << " has no NOTIFY_TYPES member" << dendl;
+ return 0;
}
+
+ PyObject *ls = PyObject_GetAttrString(pClass, "NOTIFY_TYPES");
+
if (!PyObject_TypeCheck(ls, &PyList_Type)) {
// Relatively easy mistake for human to make, e.g. defining COMMANDS
// as a {} instead of a []
+287 -345
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1 +1 @@
SHA512 (ceph-19.2.3.tar.gz) = 278101d2df7bed5363b20c2b065d7a7b26252c8164511257e213ffaa58d509015558183de10bc9281bcbe4d9f85244bcac5bba4db9823e28df6a96d0b687d00a
SHA512 (ceph-20.2.0.tar.gz) = 5678586fe663ddc3d8ca4ded7a2b811025784abc5f493164d2f1e590608a72176a722d5984d83400c501deeb526e0a108c2e7e3d969dea7bf6ce0d0b42190ea5