“We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.” - John F. Kennedy

At the office we’ve been looking at using Lua for some expression evaluation.

“Lua is a lightweight multi-paradigm programming language designed as a scripting language with extensible semantics as a primary goal. Lua is cross-platform since it is written in ANSI C and has a relatively simple C API.”

Lua has a relatively conventional syntax, small footprint, but powerful extension mechanisms which make it handy as an embedded language.

I’m not going to enumerate all the features of the language. For that you can look at other online materials, e.g. the Wikipedia entry or the first edition of the book “Programming in Lua”.

Instead as a quick taster here is the ubiquitous factorial function, written as a loop:

function factorial(n)
  local x = 1
  for i = 2,n do
    x = x * i
  end
  return x
end

“Lua” is Portuguese for moon by the way.

For a hobby project I wanted something that would demonstrate how you could embed Lua (version 5.2.3) into another program. I settled on making a simple command line filter program which will act a bit like sed, awk, or Perl (with the “-p” switch). Basically I wanted to build a program to munge standard input into standard output with a usage like:

munge "Lua expression"  output

where a Lua expression would be invoked on each input line (somehow) to produce a new output line.

I decided to write the wrapper application in C++ as that’s what we use in-house for our core components.¹ In any case, it is straightforward enough to write a C++ program that copies input to output a line at a line, so the two remaining problems to solve are:

1. how to turn the Lua expression into something that could be given the line text to munge into something else; and
2. how to invoke the Lua interpreter at all

Taking these in turn.

We are given an expression. You can’t really invoke expressions via the Lua C API. You need to have a function. So we need to turn the expression into a function. There’s some discussion of this on stackoverflow and that link mentions the ae library. However, I did my own similar thing. It is obvious that the main input to this function is a line of text - so that can be the parameter of the function. And the output from the function is obviously the munged line of text - so that can be the return value. What would be an appropriate name for the parameter? Inspired by Perl (how many people can say that?) I chose “_”. So when the user enters an expression on the command line it gets turned into:

function munge (_) return expression end

This should be ok if they have referenced “_” in the expression. In the actual program I’ve also added something to return the empty string if the expression evaluates to nil.

So now we just need to arrange to call this on each line of input, and get the result back for the output. The Lua C API is stack-based. You push functions and arguments onto the stack, invoke the function, and then pop the result off the stack. Here is a function that does that:

string lua_call_munge (const string& line)
{
  lua_getglobal(L, "munge");
  lua_pushstring(L, line.c_str());
  lua_call(L, 1, 1);
  string result = lua_tostring(L, -1);
  lua_pop(L, 1);

  return result;
}

This looks up the value of the “munge” function from a global variable and puts that on the stack, then puts the input line on the stack, then invokes the function, and finally reads the return value off the stack (popping it as well).

A question that might be arising in the reader’s mind now is how we actually get the “munge” function defined via the C API? The answer is that you call “luaL_loadstring” on the above definition - with the user’s expression substituted in. This pushes a wrapper function, which represents the code, onto the stack. You can then invoke it as above to define “munge”.

See the full code for all the details. I haven’t gone to town on the error handling, but one frill is an extra option to choose whether to assume the Lua fragment on the command line is truly an expression or actually a block of statements (with its own return). The latter is useful to do more complex processing.

Here are some examples of what you can now do (perversely I’ve used Cygwin’s GCC on Windows to build this):

Echo the input to output (you can also omit the expression completely if you want):

.\munge.exe -e "_" ...

Convert all the words to upper case (“:” is the Lua object oriented method call operator):

.\munge.exe -e "_:upper()"  ...

Append the reverse of each line to itself (“..” is the concatenation operator):

.\munge.exe -e "_ .. _:reverse()" ...

Double each number found (“gsub” calls the function you provide on each match with the match groups as arguments and replaces the match with the result):

.\munge.exe -e "_:gsub('(%d+)', function(m) return m*2 end)" ...

Sum all the numbers on each line (here I’ve used the option to provide a block of statements with their own “return”):

.\munge.exe -s "sum = 0; _:gsub('(%d+)', function(m) sum = sum + m; end); return sum" ...

At the risk of sounding like a lunatic, I am over the moon with how easy this was.

“Tcl did a lot of good in advancing the ideology of using scripting to build applications. It was the Lua of its era. But Lua is the Lua of this era.” — Jay Carlson

1. Thanks to Malcolm Parsons for reviewing the C++ code.