::::: : the wood : davidrobins.net

SCons: adding a post-processing step

Technical ·Monday November 16, 2009 @ 22:58 EST (link)

I got annoyed by this warning generated by compiling my project's Flex scanner:
flex --header-file=scanner.h -DECHO= -t scanner.l > scanner.cc
g++ -o scanner.o -c -ggdb -Wall scanner.cc
<stdout>: In function 'int yy_get_next_buffer()':
<stdout>:1340: warning: comparison between signed and unsigned integer expressions
I looked it up and it turns out that although it's not fixed in Flex itself, it is logged and fixed as Debian bug 466793 in the Debian package: the YY_INPUT macro in the generated code (scanner.cc here) should have size_t in one place where it has int. Since my system is Gentoo, not Debian, I couldn't (consistently) install the patched Flex, but I could patch the generated scanner.cc file before compilation in the SCons build file, by adding a post-processing step to the action list for .l (Flex lexer definition) files as follows:
c_file, cxx_file = SCons.Tool.createCFileBuilders(env)
cxx_file.cmdgen['.l'] += [
 r"perl -pwli -e 's/^(\t\t)int( n; \\)$/$1size_t$2/" +
 r" if /^#define YY_INPUT/ .. /^#endif/' $TARGET"]
Now the relevant build steps become:
flex --header-file=scanner.h -DECHO= -t scanner.l > scanner.cc
perl -pwli -e 's/^(\t\t)int( n; \\)$/$1size_t$2/ if /^#define YY_INPUT/ .. /^#endif/' scanner.cc
g++ -o scanner.o -c -ggdb -Wall scanner.cc
I have, of course, glossed over exactly how I knew to call SCons.Tool.createCFileBuilders, and that adding to cxx_file.cmdgen['.l'] would work and do what I wanted; as it happens, I had to browse the sources for that, although it's all in the inline documentation. On my system the SCons modules are under /usr/lib64/scons-1.2.0/SCons. It turns out that cxx_file is a SCons Builder.CompositeBuilder object, whose cmdgen field is a Builder.DictCmdGenerator whose values are SCons action lists. It might be better if it were not necessary to access the un(py)documented cmdgen field (one might assume that the add_action method would do what I wanted, but unfortunately it replaces the old action, that of running Flex in this case).

It's sufficiently elegant for a postmodern high-level language.

Books finished: Night of Power.

DVDs finished: Harry Potter and the Prisoner of Azkaban.