October 8, 2015
Comparing Build Times Using Clang and GCC on Linux
When I first installed Clang and tried to use it to build C++ code with libc++, which is a different implementation of the standard C++ library, providing an alternative to the ubiquitous libstdc++, I noticed that build times where noticeably slower than those with GCC. Here is a very simple program that I tried on an Amazon Web Services virtual machine:
#include <iostream>
#include <cstdlib>
using namespace std;
void myexit()
{
cout << "Exiting...\n";
}
int main()
{
atexit( myexit );
cout << "Hey, AWS!!!\n";
return 0;
}
Building it using GCC (g++) took 0.16 seconds; Clang (clang++) with the standard libstdc++ - 0.20 seconds; clang++ with libc++ - 0.48 seconds.
Please note that at first I built LLVM, Clang, and libc++ without specifying release build, which resulted in even slower build times. Please don't fall into the same trap. Make sure you specify that you want a release build when configuring the source code, if you build from source. Clang here was installed by Yum from a repository. Build times compared similarly when using Clang built from source in Release mode. Build times for this trivial program compared about the same for the three test platforms listed below.
So, I decided to build a non-trivial software package using the various compilers with different C++ library implementations.
Build Machines Used
I used the following machines in my testing:
- An Amazon Web Services (AWS) t2.medium instance running 64-bit Amazon Linux with 4GB of RAM, Clang v. 3.6.2 built from source, and GCC v. 4.8.2 installed from the Yum repository. The libc++ library was built from the same source distribution as Clang and LLVM, downloaded from llvm.org. Clang on this machine was bootstrapped, built using Clang and linked with libc++.
- An AWS virtual machine t2.medium instance running 64-bit RedHat 7.1 with 3.6GB of RAM, Clang v. 3.4.2 installed from the Yum repository, and GCC v. 4.8.3, also from the repository. The libc++ library used was built, using this Clang 3.4.2 compiler, from version 3.7.0 source downloaded from llvm.org.
- An Azure CentOS 64-bit virtual machine with 3.5GB of RAM, Clang and libc++ v. 3.6.2 built from source and linked with libstdc++, and GCC v. 4.8.3.
As a side note, the tests were first attempted using t2.micro AWS VM instances, but it was soon discovered that build times deteriorated during the uptime of a machine, which had to do with so called CPU credits that determine the speed of an Amazon VM instance and that are too constraining for a t2.micro instance. Thus the tests had to be done on t2.medium instances, the CPU credit balance had to be watched carefully to ensure it remained above 0 during each test.
Build Times for LLVM
I decided to build LLVM from source and compare build times depending on which compiler was used, whether the source was configured using CMake or old-style configure scripts, and whether libstdc++ or libc++ library was used. We are not comparing build times of source configured using configure and that configured with CMake because different built types were performed: a Release build using CMake, and a Release + Asserts build using configure.
Build times, in minutes, on Amazon Linux are summarized in the following table:
| libstdc++ | libc++ |
| Clang/CMake | 40 | 37 |
| GCC/CMake | 39 | NA |
| Clang/configure | 45 | 42 |
| GCC/configure | 46 | NA |
Build times, in minutes, on RedHat 7.1 are summarized in the following table:
| libstdc++ | libc++ |
| Clang/CMake | 38 | 33 |
| GCC/CMake | 43 | NA |
| Clang/configure | 53 | 50 |
| GCC/configure | 50 | NA |
When configuring LLVM 3.7.0 using configure script, a minor problem was encountered. If the configure command was executed in the source directory, it failed with
configure: error: In-source builds are not allowed. Please configure from a separate build directory!
This was easy to fix, however. In this example the source was in the llvm_only_src directory. So, a build directory was created in the same directory as llvm_only_src, and the following command was issued from the build directory:
../llvm_only_src/configure
Building with Clang/CMake/libc++ on the Azure CentOS VM took 119 minutes, so the other tests with LLVM were abandoned. Instead, a few tests building HDF5 software were performed.
Building HDF5 with Clang
Unlike LLVM, which is mostly written in C++, HDF5 (stands for Hierarchical Data Format 5) is mostly C with a little bit of C++. Building it with CMake was found to be kind of tricky, so only the configure script was used. The results:
- Clang/libstdc++: 5 minutes
- Clang/libc++: 6 minutes
- GCC/libstdc++: 14 minutes
Conclusions
These results are not intended to be viewed as a solid engineering test, but they seem to suggest the following:
- Clang is faster at building C code; see the results building HDF5.
- Clang builds C++ code faster when using the libc++ implementation of the standard C++ library rather than libstdc++ commonly found on Linuxes.
- When building C++ code, Clang linking with libc++ is typically faster than GCC linking with libstdc++.
So, my initial impression that Clang linking with libc++ was too slow appears to be incorrect.