node-pgwest

Document Sample
node-pgwest Powered By Docstoc
					A COMMAND PROMPT TALK
Node.js and postgres
         An Introductory Glance
          With Live Code, Later

 2010 AURYNN SHAW, COMMAND PROMPT INC.
Hi!
                    What is Node.js



Fast

Pure event-driven

server-side

javascript programming environment
                    Ew, javascript



Built on Google‟s V8

Runs on modern unix-likes (Linux, OSX, Freebsd, Cygwin)

Speaks network natively, and easily (Hello world is in http!)

Ends up Similar to Python, Perl, Ruby, ETc.
               Did I mention fast?




Single thread can do many thousands of connections

standard library doesn‟t block
               Did I mention fast?



Single thread can do many thousands of connections

standard library doesn‟t block

Keeps the slow tasks from blocking

Keeps the quick tasks quick
                     blocking?

# Standard synchronous processing
fh = open(“somefile”,”r”)
contents = fh.read()
# what if somefile is huge, 1GB+ ?
print contents
Every step waits
How is node different?
               Did I mention fast?



Single thread can do many thousands of connections

standard library doesn‟t block

Keeps the slow tasks from blocking

Keeps the quick tasks quick
Really good for anything I/O
While slow tasks run, node
        carries on
                      Non-blocking!

fs.open(“somefile”, function (fd) {
  // When the file descriptor gets returned.
  var buffer = null;
   fs.read(fd, buffer, length, position, function (e, bytes) {
       // bytes read are here.
       // buffer will be not-null.
   });
});
As always, there‟s caveats to
             this
                    Some caveats




node is entirely single-threaded

Event-driven programming is *really* different
                A maze of callbacks

fs.open(“/tmp/hello”, function (fd) {
  var buffer = null;
   fs.read(fd, buffer, length, position, function (e, bytes) {
       // bytes read are here.
       // buffer will be not-null.
   });
});
                    Some caveats



node is entirely single-threaded

Event-driven programming is *really* different

Javascript‟s prototype model is also really different

The syntax has warts
          Build a bridge out of her?

/*
This allows you to kind of inherit properly.
*/
var o = function () {};
var p = function () {
  /* Because „this‟ changes */
   var self = this;
   o.call(this);
};
p.prototype = o;
p.prototype.constructor = p;
So why is this important?
             Why is it important?




Javascript owns the frontend
             Why is it important?




Javascript owns the frontend

More jquery and javascript programmers every day
               Why is it important?




Javascript owns the frontend

More jquery and javascript programmers every day

Javascript is fast
Faster every day
And now, we can use
     postgres!
               When I got here...




postgres.js worked.
               When I got here...




postgres.js worked.

Used the simple query protocol
               When I got here...




postgres.js worked.

Used the simple query protocol

Used SQL escape to prevent injection attacks.
               whyyyy




SOURCE: http://www.flickr.com/photos/striatic/2192192956/
But it did work!
And so I wanted to make it
         BETTER
So, I had to ask,
How do I not allow this?
Well...
                to prepare a query




postgres.js Didn‟t use libpq.
SOURCE: http://www.flickr.com/photos/tanaka/2319325408/
                to prepare a query




postgres.js Didn‟t use libpq.

Speaks pure protocol
                to prepare a query




postgres.js Didn‟t use libpq.

Speaks pure protocol

...So I had to learn the protocol.
Wax on, wax off
    MONTAGE!
Sadly, reality was a lot less
          exciting
             Complexity Abounds




Simple protocol sends one message to pg

Easy to queue, internally.
REALLY SIMPLE MESSAGE QUEUE




    QUERY

            QUERY

            QUERY

            QUERY

            QUERY

            QUERY

                    RESULTS
                    Really, though



Simple protocol sends one message to PG

Easy to queue, internally.

Prepared statements require at least 5

Parse, describe, execute, bind, flush
THIS TYPE OF QUEUE DOESN‟T WORK




      SYNC

             EXECUTE

               BIND

             DESCRIBE

              PARSE

              QUERY

                        QUERY
   And while all this is happening




You cannot block the main thread

And postgres expects a synchronous request/response

But it does give responses...
   And while all this is happening




You cannot block the main thread
   And while all this is happening




You cannot block the main thread

And postgres expects a synchronous request/response
   And while all this is happening




You cannot block the main thread

And postgres expects a synchronous request/response

But it does give responses...
So, we buffer message sets
   KIND OF LIKE THIS




SIMPLE
         PREPARED
          GROUP
          SIMPLE

          SIMPLE
         PREPARED
          GROUP
          SIMPLE

                    SIMPLE
So we never block waiting for
         the DB
the application keeps
     responding
But that‟s all a bit on the
  technobabbly side


           SO
Let‟s look codE!
                 Using postgres.js

/* Node setup */
var pg = require(“lib/postgres-pure.js”),
   sys = require(“sys”);

var db = new
pg.connect("pgsql://test:12345@localhost:5432/template1");

/* We now have a working DB handle.
   Or, we have an error. Either way.
*/
                  Using postgres.js

/* We now have a working DB handle. */
/* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {
   // Wait, a function?
   console.log(sys.inspect(rs));
});
Why a function?
              What‟s going on here?

/* We now have a working DB handle. */
/* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {
// Wait, a function?
console.log(sys.inspect(rs));
});
                          What you get

1
3
// A little while later
2
  It‟s happening asynchronously!



The function argument is what‟s going to happen when the DB comes
back

db.query returns immediately

and your app keeps on going!
this isn‟t the cool bit
                  Prepared queries

/* Prepared queries work the same way */

db.prepare(“select ?::int as foobar;”, function (sth) {
   sth.execute(1, function (err, rs) {
       console.log(sys.inspect(rs));
   });
   sth.execute(2, function (err, rs) {
       console.log(sys.inspect(rs));
   });
});
                   Prepared queries

/* which will return */
[ { foobar: 1 } ]
[ { foobar: 2 } ]
                       Or this!

/*
This becomes a prepared statement behind the scenes
*/
db.query(“select ?::int as foobar;”,
   1, // A single argument
   function (err, rs) {
      console.log(sys.inspect(rs));
});
                    It would return

[ { foobar: 1 } ]
Sub-ideal Aspects
                      Bad, bad, bad.




postgres.js is still very alpha.

Node API is in flux

Learning Node takes a while
Cool bits!
         Things that are pleasing




clean, perl DBI-based API.

Official support from Command prompt

Performance!
                        Vroom

[ananke:~/src/postgres-js] aurynn> time node benchmark.js
40000
[ { count: 40001 } ]
node benchmark.js 40000 18.85s user 2.75s system 87% cpu
24.602 total

Just over 2122 queries/second.
Some Stuff we can‟t wait to
           add
                 Shiny things!




Notify support
                  PG notifications

/* Event-driven frameworks are ideal for this, and
   the notification setup can be handled behind the
   scenes.
*/

db.on(“<NOTIFYNAME>”, function (payload) {
   // Do something useful as a result of
   // the notification from PG
});
Payload is also arbitrary data
                 Shiny things!




Notify support

Stability
                        Shiny things!




Notify support

Stability

Better error handling
                        Shiny things!



Notify support

Stability

Better error handling

More test cases
                        Shiny things!


Notify support

Stability

Better error handling

More test cases

Better type coercion
But the most important thing
         we‟d like?
Users.
You guys. People who care.
So that‟s postgres.js.
Any questions?
Okay. LIVE CODE!
 Shout out ideas!
     FORK IT, AND HELP!
               UNSTABLE

 https://github.com/aurynn/postgres-js/



         NOT QUITE AS UNSTABLE
https://github.com/commandprompt/postg
                  res-js/

				
DOCUMENT INFO
Categories:
Tags:
Stats:
views:7
posted:9/13/2011
language:English
pages:85