Lua Socket by moneycael

VIEWS: 9 PAGES: 26

More Info
									LuaSocket
behind the scenes

   Diego Nehab
Short Bio

• Graduated from PUC in CS & E, 1999;
• Worked in Tecgraf 1995-2002;
• MSc in PL with Roberto, 2001;
• 3rd year PhD candidate at Princeton;
  • Computer Graphics.
Outline of talk

• A few historical notes
• Case study: SMTP support
  • Protocol abstraction
  • Message abstraction
  • Implementation highlights
• Conclusions
Historical notes

• 1.0, 1999, 1.5k C, 200 man
• 1.1, 2000, 1.5k C, 1.3k Lua, 500 man
  • added protocol support for HTTP, SMTP, FTP
• 1.2, 2001, 2k C, 1.3k Lua, 900 man
  • buffered input and non-blocking I/O
  • UDP support
  • object oriented syntax
Historical notes

• 1.3, 2001, 2.3k C, 1.6k Lua, 1.2k man
  • streaming with callbacks
  • added select function
• 1.4, 2001-2, 2.2k C, 2.2k Lua, 1.9k man
  • LTN7
  • added URL module
  • named parameters
Current version

• 2.0, 2005, 4.6k C, 2.5k Lua, 4.7k man
  • Extensible C architecture, split in modules
  • LTN12 (sources, sinks and filters)
  • MIME support (partial but honest)      David Burgess

  • Multipart messages support
  • LTN13 (finalized exceptions)
  • Package proposal
  • Improved non-blocking code, robust to signals...
Outline of talk

• A few historical notes
• Case study: SMTP support
  • Protocol abstraction
  • Message abstraction
  • Implementation highlights
• Conclusions
SMTP (RFC2821)
[lua:roberto] telnet mail.tecgraf.puc-rio.br 25
220 tecgraf.puc-rio.br ESMTP Sendmail 8.9.3/8.9.3
helo lua
250 tecgraf.puc-rio.br Hello lua, pleased to meet you
mail from: <roberto@inf.puc-rio.br>                     from
250 <roberto@inf.puc-rio.br>... Sender ok
rcpt to: <diego@tecgraf.puc-rio.br>                     rcpt
250 <diego@tecgraf.puc-rio.br>... Recipient ok
data
354 Enter mail, end with "." on a line by itself
Subject: World domination: instructions.
                                                        body
Commence stage two.
.
250 RAA10452 Message accepted for delivery
quit
221 tecgraf.puc-rio.br closing connection
Protocol abstraction
status, error = smtp.send {
    from = "<roberto@inf.puc-rio.br>",
    rcpt = "<diego@tecgraf.puc-rio.br>",
    body = "Subject: World domination: instructions.\r\n\r\n" ..
           "Comence stage two."
}


• What if body is large?
LTN12 sources

• Use callback function that produces data;
• Returns one chunk each time called;
• Signals termination returning nil.
function ltn12.source.file(handle)
      return function()
            local chunk = handle:read(BLOCKSIZE)
            if not chunk then handle:close() end
            return chunk
      end
end
Using sources
status, message = smtp.send {
    from = "<roberto@inf.puc-rio.br>",
    rcpt = "<diego@tecgraf.puc-rio.br>",
    body = ltn12.source.file(io.open("/mail/body", "r"))
}


• What if body is complicated?
Message Format (RFC2822)
 From: Roberto Ierusalimschy <roberto@inf.puc-rio.br>
 To: Diego Nehab <diego@tecgraf.puc-rio.br>
                                                         headers
 Subject: World domination: roadmap.
 Content-Type: multipart/mixed; boundary=part


 This message contains attachments
 --part
 Content-Type: text/plain                               headers
                                                           part 1
 Please see attached roadmap.                               body
                                                           body
 --part
 Content-Type: text/html; name="roadmap.html"
 ...                                                      part 2
 --part--
Message abstraction
declaration = {
    headers = {
         subject = "World domination",
         from = "Roberto <roberto@inf.puc-rio.br>",
         to = "Diego <diego@tecgraf.puc-rio.br>"
    },
    preamble = "This message contains attachments.",
    [1] = {
         headers = { ... },
         body = "Please see attatched roadmap."
    },
    [2] = {
         headers = { ... },
         body = ltn12.source.file(io.open("/plans/roadmap.html", "r"))
    }
}
Our message API
status, message = smtp.send {
    from = "<roberto@inf.puc-rio.br>",
    rcpt = "<diego@tecgraf.puc-rio.br>",
    body = smtp.message(declaration)
}


• Transform declaration into an LTN12 source;
• Pass source as body to sending function.
How hard is it?

•   Message structure is recursive;
•   Need to return chunks but mantain context;
•   Nightmare to write in C!;
•   Use coroutines;
•   Write function recursively, naturally;
•   Call yield with each chunk;
•   Next call resumes wherever we left.
Zoom in on attachments
[2] = {
    headers = {
          ["content-type"] = 'text/html; name="roadmap.html"',
          ["content-disposition"] = 'attachment; filename ="roadmap.html"'
    },
    body = ltn12.source.file(io.open("/plans/roadmap.html", "r"))
}




• Would like to send PDF;
• Binary data has to be encoded (Base64);
• Want to encode on-the-fly.
LTN12 filters and chains

• Filters process data one chunk at a time;
• MIME module provides common filters:
  • base64, quoted-printable, stuffing, line-wrap...
• Can chain two filters together: factory
  • Produce a filter with the composite effect
• Can chain a filter with a source: factory
  • Produce a source that returns filtered data.
Zoom in on attachments
[2] = {
    headers = {
          ["content-type"] = 'application/pdf; name="roadmap.pdf"',
          ["content-disposition"] = 'attachment; filename ="roadmap.pdf"',
          ["content-description"] = 'Detailed world domination plan',
          ["content-transfer-encoding"] = 'BASE64'
    },
    body = ltn12.source.chain(
          ltn12.source.file(io.open("/plans/roadmap.pdf", "r")),
          ltn12.filter.chain(
              mime.encode("base64"),
              mime.wrap("base64")
          )
    )
}
Creating filters: high-level

• Chunks can be broken arbitrarily;
• Filters have to keep context between calls;
function ltn12.filter.cycle(low, ctx, extra)
      return function(chunk)
            local ret
            ret, ctx = low(ctx, chunk, extra)
            return ret
      end
end
function mime.normalize(marker)
      return ltn12.filter.cycle(mime.eol, 0, marker)
end
Creating filters: low-level
int eol(lua_State *L) {
    int ctx = luaL_checkint(L, 1);
    size_t isize = 0;
    const char *input = luaL_optlstring(L, 2, NULL, &isize);
    const char *last = input + isize;
    const char *marker = luaL_optstring(L, 3, CRLF);
    luaL_Buffer buffer;
    luaL_buffinit(L, &buffer);
    while (input < last)
       ctx = translate(*input++, ctx, marker, &buffer);
    luaL_pushresult(&buffer);
    lua_pushnumber(L, ctx);
    return 2;
}
Creating filters: low-level
#define candidate(c) (c == CR || c == LF)
int translate(int c, int last, const char *mark, luaL_Buffer *buffer) {
    if (candidate(c)) {
        if (candidate(last)) {
            if (c == last) luaL_addstring(buffer, mark);
            return 0;
        } else {
            luaL_addstring(buffer, mark);
            return c;
        }
    } else {
        luaL_putchar(buffer, c);
        return 0;
    }
}
SMTP dependencies


   socket      tp




              smtp   ltn12




              mime
Error checking

• Function return convention
      • Return nil, followed by message on error;
function metat.__index:greet(domain)
      local r, e = self.tp:check("2..")
      if not r then return nil, e end
      r, e = self.tp:command("HELO", domain)
      if not r then return nil, e end
      return self.tp:check("2..")
end


• Tedious, error prone, virotic, not finalized.
LTN13 exceptions

• try = newtry(finalizer): factory;
   • On success, try returns all arguments;
  • On failure, throws the second argument;
  • Calls finalizer before raising the exception.
• foo = protect(bar): factory;
   • foo executes bar in a protected environment;
   • Returns nil followed by any thrown error.
No 'if' statements
function metat.__index:greet(domain)
      self.try(self.tp:check("2.."))
      self.try(self.tp:command("HELO", domain))
      return self.try(self.tp:check("2.."))
end


• Internal functions throw exceptions;
• try calls tp.close() on error;
• External functions can be protected.
Conclusions

• Hope you like our API, we do;
• It is easy to implement;
   • Function factories + closures, coroutines
• It is fast;
   • Time critical in C, management in Lua;
• Questions?

								
To top