Connect with us

Hi, what are you looking for?

Code Life

MSYS2 Build Processes Driving You Crazy?

You’re not alone.

At work, I have the task of making builds for a particular cryptocurrency wallet for multiple platforms. Among them is Windows, and we don’t build with Visual Studio - instead, we using CMake with MSYS2 so we don’t have to rewrite the whole codebase just for Windows-specific edge cases.

There was just one problem though. When I was assigned to this, the codebase just wouldn’t pass the CMake configuring process on Windows according to the build instructions. There didn’t seem to be anything wrong at first, but later on, there were a few red herrings of an error that threw off my debugging.

Eventually, I was able to nudge it past CMake. Here’s how I did it.

Some background information about MSYS2

You don’t actually build applications inside MSYS2. You build them inside MinGW - that’s because MSYS2 is just a shell. That’s right, MSYS2 only provides you with a bash-like shell, and the rest of the GNU tools are provided by MinGW.

In particular, all the build tools are only available if you run the MinGW version of the MSYS2 shell. So every Windows project’s build instructions that don’t use Visual Studio will require you to build it under MSYS2.

MSYS2 and MinGW use the pacman package manager to install software. There are two sets of packages (well, there are actually more than that, but I’m not counting the 32-bit or Universal C Runtime or cross-platform packages). The mingw-w64-x86_64-* packages install in MinGW’s “namespace”, and the unqualified (with no crazy prefixes) packages install in MSYS2’s namespace.

Unqualified packages are visible from the MinGW program, but apparently, the mingw* packages are not visible from MSYS2, but please let me know in the comments if this is wrong, because I discovered this from trial and error.

The problem

CMake Error: File <REDACTED>/external/unbound/win32pthread-NOTFOUND does not exist.
-- Could NOT find Readline (missing: Readline_INCLUDE_DIR)
Doxygen: graphviz not found - graphs disabled

Do any of these errors look familiar to you? Well apparently, Google returns a few hits on them, but doesn’t really offer answers for any of them.

Some of these errors can be resolved by installing the relevant mingw-* packages. This is the case for Doxygen and graphviz not being found. Others, like the first two, are more subtle and cannot be resolved just by installing the package - there is a little more work you have to do in your CMake files.

Only the pthreads error was fatal, but all of them are equally annoying. Let’s start with the Readline error first.

First though, ensure that mingw-w64-x86_64-readline is already installed. The same goes for the Doxygen and graphviz packages. The MinGW packages are necessary, not the unqualified MSYS2 packages.


Your Readline package configuration probably looks something like this:

if(USE_READLINE)
find_package(Readline)
if(READLINE_FOUND AND GNU_READLINE_FOUND)
add_definitions(-DHAVE_READLINE)
include_directories(${Readline_INCLUDE_DIR})
message(STATUS "Found readline library at: ${Readline_ROOT_DIR}")
else()
message(STATUS "Could not find GNU readline library so building without readline support")
endif()
endif()

A call to find_package, followed by a check whether Readline is found (and a diagnostic printing that it will build without Readline support otherwise).

The problem here is, find_package never finds the path to Readline. This is strange, for two reasons. One, find_package has no problem finding other packages, and two, the Readline library is already installed inside MinGW. So what could be the problem?

After doing some inspection, I discovered that find_package does find the Readline library, but not the include path. Apparently, this is because it was told to look for a specific filename (readline/readline.h) that had a forward slash in it, as the path separator.

MinGW programs don’t understand UNIX paths because they are programmed to understand only Windows paths. This is to make them work under Windows.

To make things worse, MSYS only works with UNIX paths, not the Windows paths that MinGW programs use.

This causes some confusion in CMake, because the find_package function ultimately just calls the find(1) program, which is a MinGW program and only understands Windows paths.

Worse, CMake itself will only take UNIX paths for file specification no matter whether you install the MSYS or MinGW version of it. This leads to a situation where you have to mix Windows and UNIX paths in the same configuration file, which can get quite messy.

I could’ve gone that way and modified the file responsible for finding the Readline package, but since that wouldn’t be portable, I decided to implement a fallback condition instead, just for MinGW.

if(USE_READLINE)
find_package(Readline)
if(READLINE_FOUND AND GNU_READLINE_FOUND)
add_definitions(-DHAVE_READLINE)
include_directories(${Readline_INCLUDE_DIR})
message(STATUS "Found readline library at: ${Readline_ROOT_DIR}")
elseif (NOT READLINE_FOUND AND MINGW)
# Due to a problem in the package searching process for MinGW, some include paths
# are not found but library paths are.
if (BUILD_64)
set(Readline_ROOT_DIR /mingw64)
else()
set(Readline_ROOT_DIR /mingw32)
endif()
set(Readline_INCLUDE_DIR ${Readline_ROOT_DIR}/include)
message(WARNING "Readline was not found by automatic MinGW search so using default root directory: ${Readline_ROOT_DIR} and include directory ${Readline_INCLUDE_DIR}")
else()
message(STATUS "Could not find GNU readline library so building without readline support")
endif()
endif()
view raw readline-fixed.txt hosted with ❤ by GitHub

As you can see, there is a special condition in the fixed code that detects the case where find_package can’t find Readline, and if the program is being built under MinGW. In this case, I just emit a warning, hardcode the path that Readline is supposed to be, and proceed with the rest of the CMake file.

This particular case also has checks for 32- and 64-bit MinGW.


As I mentioned before, the pthreads error was the one that halted CMake execution, and not any of the other errors. In this case, it complained that it could not find the pthread library under Windows.

if (MINGW)
# There is no variable for this (probably due to the fact that the pthread
# library is implicit with a link in msys).
find_library(win32pthread
NAMES libwinpthread-1.dll)
# Proceed to copy file to build directory…
endif ()
view raw pthread-problem.txt hosted with ❤ by GitHub

This can get confusing, because Windows doesn’t use pthreads - It uses it’s own OS-specific threading library. So MinGW emulates pthreads on Windows by shipping a pthreads .dll library specifically for Windows.

This error is basically saying it can’t find that file.
First, ensure that mingw-w64-x86_64-winpthreads-git is installed - this will install the pthreads development files, and will also pull the *-libwinpthread package, which provides the /mingw64(or 32)/bin/libwinpthread-1.dll library that needs to be found.

Here, we have to use Windows paths to specify this file, because the (unshown) copy function that happens immediately after this search will not work with UNIX paths.

The result will look something like this, along with warnings and 64-bit checks:

if (MINGW)
# There is no variable for this (probably due to the fact that the pthread
# library is implicit with a link in msys).
find_library(win32pthread
NAMES libwinpthread-1.dll)
if (NOT win32pthread)
if (BUILD_64)
set(win32pthread C:\\msys64\\mingw64\\bin\\libwinpthread-1.dll)
else()
set(win32pthread C:\\msys64\\mingw32\\bin\\libwinpthread-1.dll)
endif()
message(WARNING "win32pthread library was not found using automatic MinGW search so using default location ${win32pthread}")
endif()
# Proceed to copy file to build directory…
endif ()
view raw pthread-fixed.txt hosted with ❤ by GitHub

You should now be able to build your codebase without these errors displayed, and the compiler will automatically be able to find these files itself.

Cover Image by Envato Elements

Author

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Support us

Crypto donation button by NOWPayments
All donations will be re-invested back into the website.
You can also sign up on NOWPayments to start receiving donations on your website too.

You May Also Like

Tutorials

Earlier Bitcoin Wallet used to be a bunch of private keys. In order to use new address, user had to generate new private key...

Tutorials

When you send Bitcoin from your wallet to some other address it needs some time for transaction confirmation in the blockchain.

Tutorials

To use Bitcoin, user generally needs two things: Private Key and Bitcoin Address.

Tutorials

Paper wallets if done right are one of the most secure ways of storing bitcoin especially if done on an air gapped computer.