Archive for category autotools

A brilliant way to parallize your commands

The solution I’m taking about here comes from stackoverflow.com. The original question is :

I have a list/queue of 200 commands that I need to run in a shell on a Linux server. I only want to have a maximum of 10 processes running (from the queue) at once. Some processes will take a few seconds to complete, other processes will take much longer. When a process finishes I want the next command to be “popped” from the queue and executed.

Solution: write a makefile with the following content:

all: usera userb userc....
usera:
imapsync usera
userb:
imapsync userb
....

Then just run

make -j 10 -f makefile

I think the answer is really brilliant as it not only makes use of available tools that are quite mature, but also solves the problem fast with little effort. If the situation is getting more complicated, the solution can still be easily adjusted. xargs is also a good solution btw.

5 Comments

Handling unit test with C++ visibility=hidden

Recently I started working on the libbash project. I will write several articles talking about it in future. Now I’d just like to write something about the problem we encountered during development.

Our project will be a shared library. The C++ visibility support could improve the overall performance. Put simply, it hides most of the ELF symbols which would have previously (and unnecessarily) been public. Well it’s good for the library, it’s not good for unit tests(namely gtest) because they need to know the symbols.

Of course we don’t want  the unit tests to be part of our library. So we need to find some way to let the unit test know the symbols and separate them into different automake targets. Our first solution is to use hidden visibility for the library and create an internal target with default visibility for the unit test. However, that requires compiling the source code twice. Finally Petteri Räty come with a solution (He is too busy to write a post :P):

lib_LTLIBRARIES = libcppbash.la
libcppbash_la_SOURCES = blah
libcppbash_la_CXXFLAGS = $(AM_CXXFLAGS) -fvisibility=hidden -fvisibility-inlines-hidden

cppunittests_SOURCES = blah
cppunittests_LDADD = libcppbash.la $(GTEST_LIBS)
cppunittests_LDFLAGS = -static

Here’s his explanation:
libtool by default builds both a shared and a static library for our project. This is why you see it building things twice (with PIC and without). Giving -static to libtool is just telling it to use the static version. Using the static version means everything ends up in the unit test binaries.

, , ,

6 Comments