ceph::errorator<> throw/catch-free, compile time-checked exceptions for seastar::future<>

1. 1
ceph::errorator<>
throw/catch-free, compile time-checked
exceptions for seastar::future<>
Radosław Zarzyński
Seastar Summit 2019
2019.11.04

2. 2
Error handling with seastar::future
● future_state_base is discriminated union between
promised value and std::exception_ptr.
● Signaling:
○ throw Exception() or
○ seastar::make_exception_future<T…>(Exception&& ex) or
○ Its second variant taking std::exception_ptr.
● Inspecting:
○ handle_exception_type<ExceptionTakingFunc>(...) or
○ handle_exception(...) and raw std::exception_ptr.

3. 3
● Throwing on current implementation imposes costs both in the terms of
○ performance (computational resources) and
○ scalability (number of threads throwing the same time).
● The scalability limitations have been only partially tackled down in GCC.
Throwing still imposes locking.
● Seastar workarounds the issue with the exception scalability hack at the price
of assuming no dlopen() after the initialization phase.
● The hack should be disabled in crimson as loading plugins is necessary to
fully mimic the interfaces of current OSD’s interfaces.
What’s wrong with throwing?

4. 4
● Zero-cost exceptions tend to be
slow when it comes to throwing...
● … but without the exception hack
they are also not scalable.
● The make_exception_future
path on GCC is throw-free after
Gleb Natapov's optimization for
std::make_exception_ptr
implementation in libstdc++.
● Programmer is not enforced to
use the optimized path – someone
still can throw.
● Is there better way than review?
Signaling
try
{
- throw __ex;
+#if __cpp_rtti && !_GLIBCXX_HAVE_CDTOR_CALLABI
+ void *__e =
__cxxabiv1::__cxa_allocate_exception(sizeof(_Ex));
+ (void)__cxxabiv1::__cxa_init_primary_exception(__e,
+ const_cast<std::type_info*>(&typeid(__ex)),
+ __exception_ptr::__dest_thunk<_Ex>);
+ new (__e) _Ex(__ex);
+ return exception_ptr(__e);
+#else
+ throw __ex;
+#endif
}
catch(...)

5. 5
● handle_exception_type()
is not suitable for a hot path
due to rethrowing
● handle_exception() doesn’t
solve the problem of
differentiating behavior basing
on exception type; it just
delegates it outside.
● Is there a throw-free approach
to match ExceptionA but not
ExceptionB?
std::tuple<T...> get() const& {
// …
if (_u.st >= state::exception_min)
std::rethrow_exception(_u.ex);
// ...
}
template <typename Func>
future<T...> handle_exception_type(Func&& func) noexcept {
// ...
return then_wrapped([func = std::forward<Func>(func)]
(auto&& fut) mutable -> future<T...>
{
try {
return make_ready_future<T...>(fut.get());
} catch(ex_type& ex) {
return futurize<func_ret>::apply(func, ex);
}
Inspecting

6. 6
● Allows for fast, throw-free type inspection of std::exception_ptr:
*ep.__cxa_exception_type() == typeid(ExceptionA);
● Compiler extension present in both GCC and Clang.
● For other compilers can be mimicked on top of try/catch.
__cxa_exception_type()
class exception_ptr {
const std::type_info* __cxa_exception_type() const;
}

7. 7
● The expression does exact matching while it’s perfectly
fine to:
fut.handle_exception_type([] (ExceptionBase&) {});
● So maybe handle_exception_exact_type()?
Drop-in replacement for handle_exception_type?

8. 8
seastar::future<ceph::bufferptr> CyanStore::get_attr(
CollectionRef ch,
const ghobject_t& oid,
std::string_view name) const;
What are the errors here?
● What will happen if object does not exist?
● How no-object is distinguished from no-key?

9. 9
seastar::future<ceph::bufferptr> CyanStore::get_attr(
// …
{
auto o = c->get_object(oid);
if (!o) {
return seastar::make_exception_future<ceph::bufferptr>(
EnoentException(fmt::format("object does not exist: {}", oid)));
}
if (auto found = o->xattr.find(name); found != o->xattr.end()) {
return seastar::make_ready_future<ceph::bufferptr>(found->second);
} else {
return seastar::make_exception_future<ceph::bufferptr>(
EnodataException(fmt::format("attr does not exist: {}/{}", oid,
name)));
}
}
What are the errors here?

10. 10
Is get_object() anyhow relevant from the perspective of error handling?
What are the errors here?

11. 11
Is get_object() anyhow relevant from the perspective of error handling?
Actually no:
What are the errors here?
Collection::ObjectRef Collection::get_object(ghobject_t oid)
{
auto o = object_hash.find(oid);
if (o == object_hash.end())
return ObjectRef();
return o->second;
}

12. 12
Summary
Not all errors happen exceptionally.
Straight exceptions are not the best tool in such
situation.

13. 13
errorator<> errorates
future<> at compile-time

14. 14
Goals
● Performance – no throw/catch neither on signaling nor
inspecting.
● Type safety – ensure proper error handling. Ignoring
errors is fine till you do it explicitly.
● Improve code readability – make errors part of signature.
● No revolution. Keep changes minimal and separated.

15. 15
● Header-only library separated from Seastar
● Java's checked exceptions for Seastar built on top of:
○ the Gleb's improvement for std::make_exception_ptr(),
○ __cxa_exception_type().
● 700 lines of hard-to-understand C++ metaprogramming
● Class template nesting derivative of seastar::future.
What errorator is?

16. 16
template <class... AllowedErrors>
struct errorator {
// …
template <class... ValuesT>
class future<ValuesT...>
: private seastar::future<ValuesT...> {
// NO ADDITIONAL DATA.
// sizeof(errorated future) == sizeof(seastar::future)
};
};
ceph::errorator<> literally

17. 17
Empty errorator aliases seastar::future.
template <>
class errorator<> {
public:
template <class... ValuesT>
using future = ::seastar::future<ValuesT...>;
};
ceph::errorator<> literally

18. 18
Composing an errorator resembles defining enum.
using get_attr_errorator = ceph::errorator<
ceph::ct_error::enoent,
ceph::ct_error::enodata>;
These errors are aliases to instances of std::error_code
wrapped with unthrowable wrapper to prevent accidental
throwing.
Usage

19. 19
CyanStore::get_attr_errorator::future<ceph::bufferptr>
CyanStore::get_attr(
/* ... */) const
{
// ...
throw ceph::ct_error::enoent; // won’t compile
throw ceph::ct_error::enoent::make(); // won’t compile
return ceph::ct_error::enoent::make();
}
Usage

20. 20
● Is not implicitly convertible into seastar::future.
● Its interface offers safe_then() and basically nothing else.
● safe_then() is like then() apart:
○ can take error handlers,
○ returns potentially differently errorated future depending on the result of error handling.
● Handling an error squeezes it from return errorator's error set but
● signaling new error anywhere inside safe_then() appends it to the set,
The errorated future

21. 21
seastar::future<PGBackend::cached_os_t > // == ceph::errorator<>::future<...>
PGBackend::_load_os(const hobject_t& oid)
return store->get_attr(coll,
ghobject_t{oid, ghobject_t::NO_GEN, shard},
OI_ATTR)
.safe_then(
[] (ceph::bufferptr&& bp) {
// optimistic path
}, ceph::errorator< ceph::ct_error::enoent,
ceph::ct_error::enodata>::all_same_way([oid, this] {
return seastar::make_ready_future<cached_os_t>(
os_cache.insert(oid,
std::make_unique<ObjectState>(object_info_t{oid}, false)));
}));
Usage

22. 22
seastar::future<PGBackend::cached_os_t > // won’t compile
PGBackend::_load_os(const hobject_t& oid)
return store->get_attr(coll,
ghobject_t{oid, ghobject_t::NO_GEN, shard},
OI_ATTR)
.safe_then(
[] (ceph::bufferptr&& bp) {
// optimistic path
}, ceph::ct_error::enoent::handle([oid, this] {
return seastar::make_ready_future <cached_os_t>(
os_cache.insert(oid,
std::make_unique<ObjectState>(object_info_t{oid}, false)));
}));
Usage – chaining

23. 23
ceph::errorator<ceph::ct_error::enodata>::future<PGBackend::cached_os_t >
PGBackend::_load_os(const hobject_t& oid)
return store->get_attr(coll,
ghobject_t{oid, ghobject_t::NO_GEN, shard},
OI_ATTR)
.safe_then(
[] (ceph::bufferptr&& bp) {
// optimistic path
}, ceph::ct_error::enoent::handle([oid, this] {
return seastar::make_ready_future <cached_os_t>(
os_cache.insert(oid,
std::make_unique<ObjectState>(object_info_t{oid}, false)));
}));
Usage – chaining

24. 24
Limitations,
problems,
plans

25. 25
__cxa_exception_data()?
● Only type can be inspected in the throw-free way.
This is fine for stateless objects.
● Currently there is no __cxa_exception_data() to cover
stateful errors.
● It might be feasible to get this feature as compiler’s
extension.

26. 26
● then_wrapped()-imposed temporary future creation,
● get() instead get_available_state() brings some
garbage.
● Converting differently errorated futures imposes move.
Performance

27. 27
● safe_then() is implemented with then_wrapped().
It provides the passed callable with a copy of the future
instead of *this.
● Getting rid of that is an opportunity for generic
optimization.
● Another possibility is to stop using then_wrapped() but
access to private members would be necessary.
More copying because of then_wrapped()

28. 28
No access to get_available_state()
● safe_then() retrieves the value with the less efficient get() which checks
and does blocking for seastar::thread. DCE doesn’t optimize this out.
// if (__builtin_expect(future.failed(), false)) {
// ea25: 48 83 bd c8 fe ff ff cmpq $0x2,-0x138(%rbp)
// ea2c: 02
// ea2d: 0f 87 f0 05 00 00 ja f023 <ceph::osd::
// ...
// /// If get() is called in a ref seastar::thread context,
// /// then it need not be available; instead, the thread will
// /// be paused until the future becomes available.
// [[gnu::always_inline]]
// std::tuple<T...> get() {
// if (!_state.available()) {
// ea3a: 0f 85 1b 05 00 00 jne ef5b <ceph::osd::
// }

29. 29
--- a/include/seastar/core/future.hh
+++ b/include/seastar/core/future.hh
@@ -729,7 +729,7 @@ class future {
promise<T...>* _promise;
future_state<T...> _state;
static constexpr bool copy_noexcept = future_state<T...>::copy_noexcept;
-private:
+protected:
future(promise<T...>* pr) noexcept : _promise(pr),
_state(std::move(pr->_local_state)) {
_promise->_future = this;
_promise->_state = &_state;
protected instead of private?

30. 30
Converting differently errorated futures requires moving.
PGBackend::stat_errorator::future<> PGBackend::stat(
const ObjectState& os,
OSDOp& osd_op)
{
// …
- return seastar::now();
+ return stat_errorator::now();
}
Extra moves

31. 31
● Errorator future is not a future from the perspective of e.g.
seastar::do_for_each().
● Rewriting future-util.hh would be painful and rudimentary
● There is already the concept of seastar::is_future<> trait but not all of
the utilities make use of it.
seastar::is_future<>
template<typename Iterator, typename AsyncAction>
GCC6_CONCEPT( requires requires (Iterator i, AsyncAction aa) {
{ futurize_apply(aa, *i) } -> future<>;
} )
inline
future<> do_for_each(Iterator begin, Iterator end, AsyncAction action) {

32. 32
● Optimizing errorator unveiled unnecessary temporary spawning on hot paths:
○ future::get_available_state(),
○ future::get(),
○ future_state::get().
● There is release candidate of patchset optimizing them out:
https://github.com/ceph/seastar/pull/9
The unnecessary temporaries

33. 33
?
Q&A
The pull request with errorator:
https://github.com/ceph/ceph/pull/30387

34. 34
● https://ceph.io/
● Twitter: @ceph
● Docs: http://docs.ceph.com/
● Mailing lists: http://lists.ceph.io/
○ ceph-announce@ceph.io → announcements
○ ceph-users@ceph.io → user discussion
○ dev@ceph.io → developer discussion
● IRC: irc.oftc.net
○ #ceph, #ceph-devel
● GitHub: https://github.com/ceph/
● YouTube ‘Ceph’ channel
FOR MORE INFORMATION