Kazu's Log

Kazuyoshi Kato is a Japanese software developer in Seattle, Washington. He is interested in programming languages, data visualization and prototyping.

Weak Linkage in Rust

Aug 13, 2018

reqwest, Rust’s HTTP crate doesn’t support HTTP/2 yet. There might be multiple missing pieces, but one of them is the lack of ALPN support. So, I’m working on that these days.

ALPN – Application-Layer Protocol Negotiation is TLS’ extention, and standardnized as RFC 7301. Basically speaking, the negotiation is happening by sending a short string over TLS’ initial packets to indicate that the protocol over TLS is HTTP/2. request is using native-tls to support TLS. This crate builds an abstraction layer over three platform-TLS crates - schannel on Windows, security-framework on macOS and openssl on Linux.

Initially I opened a pull request against native-tls to support ALPN, then Steven Fackler, the author of the crate told that;

My current thinking on how we would deal with ALPN on unsupported platforms (e.g. OSX 10.12, OpenSSL 1.0.1, Windows Vista) is that the APIs would always be present, but ALPN would just not run when it isn’t supported.

For Security.framework we’ll probably have to implement this by not actually using the security-framework but by instead using dlsym to dynamically load the symbols at runtime if present.

But there is a rabbit hole…


Rust itself supports weak linkage, however the whole linkage feature hasn’t been stabilized yet. On this GitHub issue, Alex Crichton wrote that

I personally consider this a perma-unstable issue for now.

So, it may not be stabilized soon-ish.


There are multiple dlopen wrappers in Rust’s crates. dlopen is one of them, and I fixed its minor issue. dlopen works, but it is a bit odd to see that Security.framework is loaded and dlopen opens the framework once again.


Then I realized that Rust’s libstd is having the same issue – pipe2() is available on some Unix OS-es. And the issue has been resolved by having a small macro called weak!. Interestingly this macro is copied over mio as dlsym!.


To have weak linkage, there are three options;

  1. #[linkage] — Good to have zero dependencies, but it hasn’t been stabilized yet.
  2. dlopen and other dlopen wrappers – Good to have a crate to solve the issue, but may be overkill for having a few functions
  3. weak!, dlsym! or the equivalent

I have picked #3 and now I have a new pull request against security-framework. It may be merged, soon, I hope.

Since Rust 1.28.0, unit tests can return Result. In other words, you can use ? on unit tests’ functions.

But, while working on my pull request to use that on solana, I have found 2 pitfalls.

left is 1, right is 0?

Sadly the error message regarding a failing unit test is not great.

thread '...' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`', libtest/lib.rs:326:5

There is an open pull request to address the issue. So it will be fixed at 1.30, according to #48854.

The backtrace doesn’t include the line number of ?

Thils is probably hard to address. Due to the way it uses Result, libtest, the test runnner can’t tell where the failing ? is. The pull request on solana had the discusstion about that, and eventually we settled that it would be okay. But it may be frustrating if you have multiple ?s in a long function. We shouldn’t have such a function though.

It actually reminds me Go’s errors. They are not having stracktraces by default and companies like Dropbox implement its own variants to have them. Would Result<T, E> have something similar since nice ? syntax pushes people to use more Results? I am not so sure.

Expanding Rust macros

Jul 29, 2018

I was using dlopen 0.1.2 and had the below error.

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
   --> security-framework/src/secure_transport.rs:497:10
497 | #[derive(SymBorApi)]
    |          ^^^^^^^^^

Access to extern crates through prelude? dlopen’s SymBorApi is using macros, and this error doesn’t make much sense without looking the macro’s implementation.

rustc has an option to expand macros, but the option is only available on the nightly compiler.

% rustup default nightly-x86_64-apple-darwin
info: using existing install for 'nightly-x86_64-apple-darwin'
info: default toolchain set to 'nightly-x86_64-apple-darwin'

  nightly-x86_64-apple-darwin unchanged - rustc 1.28.0-nightly (cbc4c8380 2018-06-22)

% cargo rustc --profile=check -- -Zunstable-options --pretty=expanded

Or, you can use cargo-expand. You still need the nightly compiler though.

In my case, the lack of “::” on the macro was the cause. The pull request to fix the issue has been merged. So you wouldn’t hit the issue. But if you have an error around macros, expanding macros to see the result is probably a good idea.