Archive for June, 2011

libbash weekly report #6

Last week I tried to do semantic predicate in our parser grammar. We hoped that it could solve most of the problems we were facing. Unfortunately, it didn’t end up well as we cannot make semantic predicate working with backtracking. This week I will migrate to ANTLR 3.4. and see if it helps solve the problem. Here are what I have done in the last week:

  • Supported braces in command arguments
  • Improved comment handling
  • Supported ANSI C Quoting
  • Supported shortcut capability for && and || in arithmetic expansion
  • Supported arithmetic expression
  • Supported break built-in
  • Improved our build system to reduce dependencies
  • Made arithmetic expansion follow POSIX
  • Improved exception hierarchy
  • Implemented shift built-in
  • Improved the ast_printer utility

A few stories are blocked due to the backtracking problem.

This week I will:

  • Upgrade to ANTLR 3.4 and see if it helps solve the backtracking problem
  • Fix errors in the CI Server
  • Break down walker grammar to reduce compile time
  • Use bash to verify test scripts
  • Support alias
  • Handle options to the local built-in
  • Upgrade to Paludis 0.64.1
  • Support thread-safety
  • Clean up warnings from doxygen

, ,

Leave a comment

libbash weekly report #5

In the last week, I focused on parser grammar improvement. So far we can generate correct metadata for 8028 ebuilds. As we have made error handling POSIX compliant, any parsing failure will cause an exception. So making the parser working properly is the first thing that should be done. To be honest, fixing bugs in the parser is not easy. The logic there is already quite complicated and backtracking make it worse. But I need to get through it. Here are what I have done in the last week:

  • Supported bash redirection for all kinds of commands
  • Supported the special parameter $-
  • Supported parsing -o and -a operators for built-in test
  • Supported brace expansion
  • Implemented eclass parse failure cache
  • Supported backslash escapes inside double quotes
  • Fixed variable indirection in arithmetic expressions
  • Supported regex match operator for keyword test
  • Tried to parse here document and improve variable expansion
  • Improved CI server configuration

This week I will:

  • Support braces in command arguments
  • Improve comment handling
  • Handle single quoted string in variable reference like $’string’
  • Support shortcut capability for && and || in arithmetic expression
  • Support arithmetic expression
  • Support break built-in
  • Support read-only built-in
  • Improve our build system to reduce dependencies
  • Make arithmetic expansion follow POSIX
  • Improve exception hierarchy
  • Implement shift built-in
  • Try boost::spirit::qi to implement a simple lexer for ANTLR

At the end of this weekly report, I’d like to mention a small tip for bash arithmetic. As you probably have known, you can’t write $(( expression )) as a bash command. But sometimes you just need to evaluate the expression and nothing else. You can certainly call the let built-in but that requires you quote your expression to avoid word splitting. Then some people invent this:

: $(( expression ))

‘:’ is a bash built-in that does nothing. So the argument gets evaluated first and then ‘:’ gets called. I really feel it unnecessary as you can always use bash arithmetic expression:

(( expression ))

This is exactly equivalent to

let "expression"

, ,

Leave a comment

libbash weekly report #4

This week I’ve made an important change to bash function implementation. So far we can generate correct metadata for 7927 ebuilds. Here’s what I have done:

  • supported not equals in arithmetic expansion
  • supported array offset expansion like ${*:1:2} and ${a[@]:1:2}
  • improved the implementation for bash functions
  • supported shopt -p
  • supported declare -p
  • supported printf
  • supported $#
  • fixed case statement with empty body
  • removed unnecessary abstractions in the implementation of arithmetic expansion
  • filed new stories from the output of instruo

In the coming week, I’ll:

  • Support bash redirections for all kinds of commands
  • support $-
  • fix problems with built-in test
  • support brace expansion
  • cache parsing failures
  • support indirect reference in arithmetic expansion
  • support escaped characters in double quoted string
  • fix bugs in arithmetic expansion
  • continue working on the story for the CI server

Till now, we have supported most of the language features that are needed for metadata generation. But there are a lot of small problems that we have to handle. The most difficult part is to improve the parser grammar. Even a small change to the grammar could break a lot of things because the grammar is already complicated. So our progress may be slowed down when I have to deal with parser grammar improvement.

Hope we can get more stuff fixed in the next couple of weeks.

, ,

Leave a comment

libbash weekly report #3

We use Scrum in our project. Petteri calls this iteration “getting hotter” because we already have a lot of features working. In this iteration we want to make them work better. Here’s what I have done in the first week (two weeks for an iteration):

  • Supported the eval built-in
  • Supported pattern matching in keyword test
  • Supported simple IO redirection
  • Supported arithmetic expression in keyword test
  • Made error handling POSIX compliant
  • Improved command list
  • Improved variable expansion and arithmetic expansion
  • Supported more Portage functions (ewarn, debug-print, etc.)
  • Cleaned the output of unit tests
  • Improved the quality of the code (explicitly disabled copying, fixed header ordering, compiled our code with -Wconversion and -Wsign-conversion, etc.)
  • Fixed mistakes in the metadata that our instruo generates
  • Examined the syntax that we can not handle in all eclasses and added corresponding stories
  • Added stories from the output of our instruo
  • Wrote a simple ebuild to install necessary dependencies (layman -a qiaomuf)
  • Created a home page based on Gentoo Infrastructure (new Gentoo project)

Before making error handling POSIX compliant, we could generate correct ebuild metadata for about 7800 ebuilds. After that we can only generate 1689 because we will stop if any error occurs.

In the next week, I’ll:

  • Make the CI server running
  • Support array index in variable expansion
  • Add more stories based on the output of instruo

I can’t tell more because I don’t know what stories I’ll add. We’ll see in the next weekly report.

, ,

Leave a comment