Archive for category tips

A brilliant way to parallize your commands

The solution I’m taking about here comes from The original question is :

I have a list/queue of 200 commands that I need to run in a shell on a Linux server. I only want to have a maximum of 10 processes running (from the queue) at once. Some processes will take a few seconds to complete, other processes will take much longer. When a process finishes I want the next command to be “popped” from the queue and executed.

Solution: write a makefile with the following content:

all: usera userb userc....
imapsync usera
imapsync userb

Then just run

make -j 10 -f makefile

I think the answer is really brilliant as it not only makes use of available tools that are quite mature, but also solves the problem fast with little effort. If the situation is getting more complicated, the solution can still be easily adjusted. xargs is also a good solution btw.



Use gcov and lcov to know your test coverage

As we all know, unit test has been a crucial part of software development. I feel I was walking on ice if I was coding without writing tests. But even if we have unit tests, things may still go wrong if the tests have bad coverage. Fortunately, there are many tools today to help us analyze our code and give us every detail of the quality of our unit tests. GNU gcov is such a tool for code compiled with GCC.

gcov is very easy to use. If you’re developing a C/C++ project, all you have to do is to compile your project with “-g -O0 –coverage”. “–coverage” is a synonym for-fprofile-arcs, -ftest-coverage(compiling) and-lgcov(linking). This tells the compiler to generate additional information and code needed by gcov.

OK, let’s run our tests. GCC will generate a bunch of files(profiles) that will be used later. These files know how your tests was executed and record many information such as arc transition counts and some summary information. Now we’re ready to view our test coverage now with gcov.  But gcov doesn’t give us a report that is easy to read and it’s not convenient. Here comes the tool called “LCOV”.

LCOV test coverage report for libbash

Read the rest of this entry »

, , ,


Handling unit test with C++ visibility=hidden

Recently I started working on the libbash project. I will write several articles talking about it in future. Now I’d just like to write something about the problem we encountered during development.

Our project will be a shared library. The C++ visibility support could improve the overall performance. Put simply, it hides most of the ELF symbols which would have previously (and unnecessarily) been public. Well it’s good for the library, it’s not good for unit tests(namely gtest) because they need to know the symbols.

Of course we don’t want  the unit tests to be part of our library. So we need to find some way to let the unit test know the symbols and separate them into different automake targets. Our first solution is to use hidden visibility for the library and create an internal target with default visibility for the unit test. However, that requires compiling the source code twice. Finally Petteri Räty come with a solution (He is too busy to write a post :P):

libcppbash_la_SOURCES = blah
libcppbash_la_CXXFLAGS = $(AM_CXXFLAGS) -fvisibility=hidden -fvisibility-inlines-hidden

cppunittests_SOURCES = blah
cppunittests_LDADD = $(GTEST_LIBS)
cppunittests_LDFLAGS = -static

Here’s his explanation:
libtool by default builds both a shared and a static library for our project. This is why you see it building things twice (with PIC and without). Giving -static to libtool is just telling it to use the static version. Using the static version means everything ends up in the unit test binaries.

, , ,


Using vimpager as your PAGER program

Sorry for not posting anything these days. It’s been very busy recently. And every time when I have some experience to share, the damn GFW blocks me. Anyway, I’m really glad to see that there are people waiting for my post :). So I’ll try to update my blog more frequently.

Today I’m going to talk about our PAGER program. Maybe the most commonly used PAGER program is “less” or “more”. Every time you have too much information in your terminal and you will use “command | less” to make it readable(git, man and many other commands call the PAGER program). But the above two commands do not give us a beautiful output. There’s another PAGER program called “most”, which could give you a colorful output. However, it doesn’t use vim key bindings and involves several dependencies. So I choose not to use it.

Wait a minute, think about your vim. Isn’t it amazing if we can use vim for our PAGER program? It has syntax highlighting, different color themes and the most important of all — vim features including key bindings, searching and etc.

Here is the download link for vimpager. What can it do? Let me show you my git diff and git log:

git diff with vimpagergit log with vimpager

Colorful result with vim key bindings. And it’s also very fast.

Installation is quite easy. Thanks to Daniel Gryniewicz’s comment: Add “vim-pager” use flag and emerge vim.




An easy command to monitor network connections

Lately I played mongodb a bit and I encountered this situation:

I want to figure out how many connections have been established between my client and mongodb server when I run the test code.

Though I figured out later that the log file contain such information,  I managed to use ‘watch’ to do such a job, which could be applied to other cases. Here it is:

watch -d -n 0.1 'netstat -p -an|grep mongo'

1 Comment

Valgrind – Your Memory Assistant

Valgrind is a programmer tool that allows to track memory related errors in C and C++ programs. This tool helped me greatly when I was checking out potential memory leaks.

As we all know, it’s really difficult to ensure that your program don’t have any memory leak at all. A good coding practice always helps, but usually it’s infeasible to find all bugs by our eyes. So we need a tool to automatically find these bugs. Valgrind is such a tool and is very easy to use. Before using it, you could compile your program with -g to include debugging information and -O0 to disable compiler’s optimization. Doing these will let memcheck’s error messages include exact line numbers.

Then you just use:

valgrind --leak-check=yes myprog arg1 arg2

The result will be displayed and it will tell you where the potential memory leaks are. You could fix the bugs according to it. There are more options and you could check them out on man page.

For gnome applications building with glib, things are a little different since glib has it’s own memory management library. Here are some tips I stolen from  Gnome community:

G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind --tool=memcheck --leak-check=full --leak-resolution=high --num-callers=20 --log-file=vgdump your-program

and you could also use it for libtool program. Check this out to see how to do this.