Web Applications A KISS Introduction
Web Applications with Python
Fetching, parsing, text processing Database client – mySQL, etc., for building dynamic information on-the-fly Python CGI web pages – or even web servers!
Use urllib or urllib2 library
Python is used frequently to parse data pages – regular expressions are very important Nothing new here
>>> from urllib2 import * >>> file = urlopen("http://www.mizzao.tk/~mao") >>> file.read() '<html>\n<head>\n<title>A Simple Page</title>\n</head>\n<body>\nNothing to see...\n</body>\n</html>‘ >>> file.read.split(‘\n’) (all the lines at once) >>> file.readline() (one line at a time)
Say there is a webpage you want to pull certain information from every day…use Python! You already know how to parse webpages though – and that isn’t really a web application Lots of complex parsers that are beyond the scope of this demo On to the interesting stuff!
A database is a program that manages a set of data (gee, who would have guessed) Most databases are relational SQL databases SQL – Structured Query Language, a standardized syntax for telling a database what to do – although Microsoft’s SQL is different from everyone else’s
mySQL is the most widely used opensource database – Python supports it! Uses library MySQLdb – a mySQL API for Python Must install this yourself – on Linux with a package manager or by tarball, on Windows with an archive
How SQL Databases Work
Databases take some sort of connection as communication – usually through TCP/IP Most databases are set up as a group of schemas, each containing a set of tables Each table contains a set of columns, each with a different data type Rows are the entries in a table
The library is called _mysql, and it implements the standard Python Database API Specification Almost all database modules for python implement the API, which defines the set of functions connect() takes parameters to establish a connection, and select_db() chooses the schema to work with
>>> from _mysql import * >>> db = connect(host="localhost",user="mao",passwd="") >>> db.select_db("test")
Some Python Database Commands
Command _mysql.connect(params) conn.select_db(db) conn.query(string) conn.store_result() What it does Returns a new object that represents the database Selects the schema (database) to use after connecting Queries the database with a SQL syntactic string Returns all the results of the last query at once
conn.fetch_row(maxrows) conn.info() conn.close()
Returns an iterator over the results of the last query
Returns up to maxrows of the query Returns some information about the last query Closes the database connection
Some Simple Database Examples
>>> db.query("CREATE TABLE students (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, lname TEXT, fname TEXT)") >>> db.query("INSERT INTO students (lname, fname) VALUES ('Mao', 'Andrew')") >>> db.query("INSERT INTO students (lname, fname) VALUES ('Walker', 'James')") >>> db.query("INSERT INTO students (lname, fname) VALUES ('Cohen', 'Ian')") >>> db.query("SELECT * FROM students") >>> people = db.store_result() >>> people.fetch_row() (('1', 'Mao', 'Andrew'),) >>> people.fetch_row() (('2', 'Walker', 'James'),) >>> people.fetch_row() (('3', 'Cohen', 'Ian'),) >>> people.fetch_row() () >>> db.close()
What Do We Do Now?
So we have access to a database and we can get data from it – but how do we get that across the web? Answer – CGI! CGI stands for Common Gateway Interface – Allows any script (perl, bash, python) to output to HTTP. It’s how you submit your CSE 121 homework
Using Python through CGI
First, you must set up the web server to use handle Python scripts with CGI With Apache, this is easy – 2 lines in httpd.conf
AddHandler cgi-script .cgi .py … <Directory /> Options ExecCGI </Directory>
A Web Python Hello World!
When we run a Python program as a CGI script, the output goes out directly and the file must be executable (+x). The first line we have to print are the HTTP headers – this tells the browser to read it as either text, or render it as HTML. Any line after that comprises the actual HTML or text itself
# Required header that tells the browser how to render the text. print "Content-Type: text/plain\n\n"
# Print a simple message to the display window. print "Hello, World!\n"
Lo and Behold…
Notice that the first line went through as HTTP headers.
Tying It All Together…
We want to be able to receive some input to our python program, and then output something based on that input. The python cgi library can find all the HTTP POST and GET variables. We can then use these to query the database, etc.
A Final Simple Program
#!/usr/bin/python import cgi,_mysql,cgitb; cgitb.enable() search = cgi.FieldStorage() # returns a dictionary print "Content-Type: text/plain\n\n" if not (search.has_key("fname") and search.has_key("lname")): print "Need a first and last name!" exit fname,lname = search["fname"].value,search["lname"].value conn = _mysql.connect(host="localhost",user="mao") conn.select_db("test") conn.query("SELECT * FROM students WHERE lname = '"+lname+"' AND fname = '"+fname+"'") people = conn.store_result() row = people.fetch_row() if row == (): print fname,lname,"is not a student.\n" else: print fname,lname,"is a student.\n" conn.close()
cgitb is a library that dumps errors to the output – you don’t have to go searching for it in your log file. cgi.FieldStorage() is a pseudo-dictionary containing all the POST and GET variables and their values.
HTML Form Interface
<html> <head> <title>Student Search</title> </head> <body> <form action = "cgi-bin/search.py" method = 'GET'> First Name:<input type = 'text' name = 'fname'> <br> Last Name:<input type = 'text' name = 'lname'> <br> <input type = 'submit' value = 'Find this Person'> </body> </html