| Author: | Dave Kuhlman |
|---|---|
| Address: | dkuhlman@rexx.com http://www.rexx.com/~dkuhlman |
| Revision: | 1.1b |
| Date: | Mar. 13, 2008 |
| Copyright: | Copyright (c) 2007 Dave Kuhlman. All Rights Reserved. This software is subject to the provisions of the MIT License http://www.opensource.org/licenses/mit-license.php. |
Abstract
This document provides an outline of an introductory course on programming in Python.
Contents
Introductions
Practical matters
Starting the Python interactive interpreter. Also, IPython and Idle.
Running scripts
Editors -- Choose an editor which you can configure so that it indents with 4 spaces, not tab characters. For a list of editors for Python, see: http://wiki.python.org/moin/PythonEditors. A few possible editors:
Interactive interpreters:
IDEs -- Also see http://en.wikipedia.org/wiki/List_of_integrated_development_environments_for_Python:
Where else to get help:
Python standard documentation -- http://www.python.org/doc/.
You will also find links to tutorials there.
FAQs -- http://www.python.org/doc/faq/.
Special interest groups (SIGs) -- http://www.python.org/sigs/
Other python related mailing lists and lists for specific applications (for example, Zope, Twisted, etc). Try: http://dir.gmane.org/search.php?match=python.
http://sourceforge.net -- Lots of projects. Search for "python".
USENET -- comp.lang.python. Can also be accessed through Gmane: http://dir.gmane.org/gmane.comp.python.general.
The Python tutor email list -- http://mail.python.org/mailman/listinfo/tutor
Local documentation:
On MS Windows, the Python documentation is installed with the standard installation.
Install the standard Python documentation on your machine from http://www.python.org/doc/.
pydoc. Example, on the command line, type: pydoc re.
Import a module, then view its .__doc__ attribute.
At the interactive prompt, use help(obj). You might need to import it first. Example:
>>> import urllib >>> help(urllib)
In IPython, the question mark operator gives help. Example:
In [13]: open?
Type: builtin_function_or_method
Base Class: <type 'builtin_function_or_method'>
String Form: <built-in function open>
Namespace: Python builtin
Docstring:
open(name[, mode[, buffering]]) -> file object
Open a file using the file() type, returns a file object.
Constructor Docstring:
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
Callable: Yes
Call def: Calling definition not available.Call docstring:
x.__call__(...) <==> x(...)
A general description of Python:
Python represents block structure and nested block structure with indentation, not with begin and end brackets.
Benefits of the use of indentation to indicate structure:
Editor considerations -- The standard is 4 spaces (no tabs) for each indentation level. You will need a text editor that helps you respect that.
Doc strings are like comments, but they are carried with executing code. Doc strings can be viewed with several tools, e.g. help(), obj.__doc__, and, in IPython, ?.
We can use triple-quoting to create doc strings that span multiple lines.
There are also tools that extract and format doc strings, for example:
See: http://docs.python.org/ref/operators.html. Python defines the following operators:
+ - * ** / // % << >> & | ^ ~ < > <= >= == != <>
The comparison operators <> and != are alternate spellings of the same operator. != is the preferred spelling; <> is obsolescent.
Logical operators:
and or is not in
There are also (1) the dot operator, (2) the subscript operator [], and the function/method call operator ().
For information on the precedences of operators, see Summary of operators -- http://docs.python.org/ref/summary.html, which is reproduced below.
The following table summarizes the operator precedences in Python, from lowest precedence (least binding) to highest precedence (most binding). Operators in the same box have the same precedence. Unless the syntax is explicitly given, operators are binary. Operators in the same box group left to right (except for comparisons, including tests, which all have the same precedence and chain from left to right -- see section 5.9 -- and exponentiation, which groups from right to left):
Operator Description
======================== ==================
lambda Lambda expression
or Boolean OR
and Boolean AND
not x Boolean NOT
in, not in Membership tests
is, is not Identity tests
<, <=, >, >=, <>, !=, == Comparisons
| Bitwise OR
^ Bitwise XOR
& Bitwise AND
<<, >> Shifts
+, - Addition and subtraction
*, /, % Multiplication, division, remainder
+x, -x Positive, negative
~x Bitwise not
** Exponentiation
x.attribute Attribute reference
x[index] Subscription
x[index:index] Slicing
f(arguments...) Function call
(expressions...) Binding or tuple display
[expressions...] List display
{key:datum...} Dictionary display
`expressions...` String conversion
Note that most operators result in calls to methods with special names, for example __add__, __sub__, __mul__, etc. See Special method names http://docs.python.org/ref/specialnames.html
Later, we will see how these operators can be emulated in classes that you define yourself, through the use of these special names.
For more on lexical matters and Python styles, see:
Understanding the Python execution model -- How Python evaluates and executes your code.
Evaluating expressions.
Creating names/variables -- Binding -- The following all create names (variables) and bind values (objects) to them: (1) assignment, (2) function definition, (3) class definition, (4) function and method call, (5) importing a module, ...
First class objects -- Almost all objects in Python are first class. Definition: An object is first class if: (1) we can put it in a structured object; (2) we can pass it to a function; (3) we can return it from a function.
References -- Objects (or references to them) can be shared. What does this mean?
For information on built-in data types, see section Built-in Types -- http://docs.python.org/lib/types.html in the Python standard documentation.
The numeric types are:
See 2.3.4 Numeric Types -- int, float, long, complex -- http://docs.python.org/lib/typesnumeric.html.
Python does mixed arithmetic.
Integer division truncates.
Scientific and heavily numeric programming -- High level Python and Jython are not very efficient for numerical programming. But, there are libraries that help:
List -- A list is a dynamic array/sequence. A list is mutable.
List constructors: [], list().
Tuples -- A tuple is a sequence. A tuple is immutable.
Tuple constructors: (), but really a comma and tuple().
Tuples are like lists, but are not mutable.
Python lists are (1) heterogeneous (2) indexable, and (3) dynamic (i.e. we can add to a list and make it longer).
Notes on sequence constructors:
The length of a tuple or list (or other container): len(mylist).
Subscription:
Operations on tuples -- No operations that change the tuple, since tuples are immutable. We can do iteration and subscription. We can do "contains" (the in operator) and get the length (the len() operator). We can use certain boolean operators.
Operations on lists -- Operations similar to tuples plus:
List operators -- +, *, etc.
For more operations and operators on sequences, see: http://docs.python.org/lib/typesseq.html.
Exercises:
Create an empty list. Append 4 strings to the list. Then pop one item off the end of the list. Solution:
In [25]: a = []
In [26]: a.append('aaa')
In [27]: a.append('bbb')
In [28]: a.append('ccc')
In [29]: a.append('ddd')
In [30]: print a
['aaa', 'bbb', 'ccc', 'ddd']
In [31]: a.pop()
Out[31]: 'ddd'
Use the for statement to print the items in the list. Solution:
In [32]: for item in a: ....: print item ....: aaa bbb ccc
Use the string join operation to concatenate the items in the list. Solution:
In [33]: '||'.join(a) Out[33]: 'aaa||bbb||ccc'
Use lists containing three (3) elements to create and show a tree:
In [37]: b = ['bb', None, None] In [38]: c = ['cc', None, None] In [39]: root = ['aa', b, c] In [40]: In [40]: In [40]: def show_tree(t): ....: if not t: ....: return ....: print t[0] ....: show_tree(t[1]) ....: show_tree(t[2]) ....: ....: In [41]: show_tree(root) aa bb cc
Note that we will learn a better way to represent tree structures when we cover implementing classes in Python.
Strings are sequences. They are immutable. They are indexable.
For operations on strings, see http://docs.python.org/lib/string-methods.html or use:
>>> help(str)
Or:
>>> dir("abc")
String operations (methods).
String operators, e.g. +, <, <=, ==, etc..
Constructors/literals:
Escape characters in strings -- \t, \n, \\, etc.
String formatting -- See: 2.3.6.2 String Formatting Operations -- http://docs.python.org/lib/typesseq-strings.html. Examples:
In [18]: name = 'dave' In [19]: size = 25 In [20]: factor = 3.45 In [21]: print 'Name: %s Size: %d Factor: %3.4f' % (name, size, factor, ) Name: dave Size: 25 Factor: 3.4500 In [25]: print 'Name: %s Size: %d Factor: %08.4f' % (name, size, factor, ) Name: dave Size: 25 Factor: 003.4500
If the right-hand argument to the formatting operator is a dictionary, then you can (actually, must) use the names of keys in the dictionary in your format strings. Examples:
In [115]: values = {'vegetable': 'chard', 'fruit': 'nectarine'}
In [116]: 'I love %(vegetable)s and I love %(fruit)s.' % values
Out[116]: 'I love chard and I love nectarine.'
Also consider using the right justify and left justify operations. Examples: mystring.rjust(20), mystring.ljust(20, ':').
Exercises:
Use a literal to create a string containing (1) a single quote, (2) a double quote, (3) both a single and double quote. Solutions:
"Some 'quoted' text." 'Some "quoted" text.' 'Some "quoted" \'extra\' text.'
Write a string literal that spans multiple lines. Solution:
"""This string spans several lines because it is a little long. """
Use the string join operation to create a string that contains a colon as a separator. Solution:
>>> content = []
>>> content.append('finch')
>>> content.append('sparrow')
>>> content.append('thrush')
>>> content.append('jay')
>>> contentstr = ':'.join(content)
>>> print contentstr
finch:sparrow:thrush:jay
Use string formatting to produce a string containing your last and first names, separated by a comma. Solution:
>>> first = 'Dave' >>> last = 'Kuhlman' >>> full = '%s, %s' % (last, first, ) >>> print full Kuhlman, Dave
Incrementally building up large strings from lots of small strings -- Since strings in Python are immutable, appending to a string requires a re-allocation. So, it is faster to append to a list, then use join. Example:
In [25]: strlist = []
In [26]: strlist.append('Line #1')
In [27]: strlist.append('Line #2')
In [28]: strlist.append('Line #3')
In [29]: str = '\n'.join(strlist)
In [30]: print str
Line #1
Line #2
Line #3
A dictionary is a collection, whose values are accessible by key. It is a collection of name-value pairs.
The order of elements in a dictionary is undefined. But, we can iterate over (1) the keys, (2) the values, and (3) the items (key-value pairs) in a dictionary. We can set the value of a key and we can get the value associated with a key.
Keys must be immutable objects: ints, strings, tuples, ...
Literals for constructing dictionaries:
d1 = {}
d2 = {key1: value1, key2: value2, }
Constructor for dictionaries -- dict() is a factory function that constructs dictionaries. Some examples, which were taken from 2.1 Built-in Functions -- http://docs.python.org/lib/built-in-funcs.html.
dict({'one': 2, 'two': 3})
dict({'one': 2, 'two': 3}.items())
dict({'one': 2, 'two': 3}.iteritems())
dict(zip(('one', 'two'), (2, 3)))
dict([['two', 3], ['one', 2]])
dict(one=2, two=3)
dict([(['one', 'two'][i-2], i) for i in (2, 3)])
For operations on dictionaries, see http://docs.python.org/lib/typesmapping.html or use:
>>> help({})
Or:
>>> dir({})
Indexing -- Access or add items to a dictionary with the indexing operator [ ]. Example:
In [102]: dict1 = {}
In [103]: dict1['name'] = 'dave'
In [104]: dict1['category'] = 38
In [105]: dict1
Out[105]: {'category': 38, 'name': 'dave'}
Test for the existence of a key in a dictionary with the in operator. Example:
In [106]: 'name' in dict1 Out[106]: True
Some of the operations produce the keys, the values, and the items (pairs) in a dictionary. Examples:
In [43]: d = {'aa': 111, 'bb': 222}
In [44]: d.keys()
Out[44]: ['aa', 'bb']
In [45]: d.values()
Out[45]: [111, 222]
In [46]: d.items()
Out[46]: [('aa', 111), ('bb', 222)]
For large dictionaries, use methods iterkeys(), itervalues(), and iteritems(). Example:
In [47]:
In [47]: d = {'aa': 111, 'bb': 222}
In [48]: for key in d.iterkeys():
....: print key
....:
....:
aa
bb
To test for the existence of a key in a dictionary, use in. Example:
>>> d = {'tomato': 101, 'cucumber': 102}
>>> k = 'tomato'
>>> k in d
True
You can often avoid the need for a test by using method get. Example:
>>> d = {'tomato': 101, 'cucumber': 102}
>>> d.get('tomato', -1)
101
>>> d.get('chard', -1)
-1
>>> if d.get('eggplant') is None:
... print 'missing'
...
missing
Exercises:
Write a literal that defines a dictionary using both string literals and variables containing strings. Solution:
>>> first = 'Dave'
>>> last = 'Kuhlman'
>>> name_dict = {first: last, 'Elvis': 'Presley'}
>>> print name_dict
{'Dave': 'Kuhlman', 'Elvis': 'Presley'}
Write statements that iterate over (1) the keys, (2) the values, and (3) the items in a dictionary. (Note: Requires introduction of the for statement.) Solutions:
>>> d = {'aa': 111, 'bb': 222, 'cc': 333}
>>> for key in d.keys():
... print key
...
aa
cc
bb
>>> for value in d.values():
... print value
...
111
333
222
>>> for item in d.items():
... print item
...
('aa', 111)
('cc', 333)
('bb', 222)
>>> for key, value in d.items():
... print key, '::', value
...
aa :: 111
cc :: 333
bb :: 222
Additional notes on dictionaries:
You can use iterkeys(), itervalues(), iteritems()`` to obtain iterators over keys, values, and items.
A dictionary itself is iterable: it iterates over its keys. So, the following two lines are equivalent:
for k in myDict: print k for k in myDict.iterkeys(): print k
The in operator tests for a key in a dictionary. Example:
In [52]: mydict = {'peach': 'sweet', 'lemon': 'tangy'}
In [53]: key = 'peach'
In [54]: if key in mydict:
....: print mydict[key]
....:
sweet
Open a file with the open factory method. Example:
In [28]: f = open('mylog.txt', 'w')
In [29]: f.write('message #1\n')
In [30]: f.write('message #2\n')
In [31]: f.write('message #3\n')
In [32]: f.close()
In [33]: f = file('mylog.txt', 'r')
In [34]: for line in f:
....: print line,
....:
message #1
message #2
message #3
In [35]: f.close()
Notes:
Use the (built-in) open(path, mode) function to open a file and create a file object. You could also use file(), but open() is recommended.
A file object supports the iterator protocol and, therefore, can be used in a for statement. It iterates over the lines in the file. This is most likely only useful for text files.
open is a factory method that creates file objects. Use it to open files for reading, writing, and appending. Examples:
infile = open('myfile.txt', 'r') # open for reading
outfile = open('myfile.txt', 'w') # open for (over-) writing
log = open('myfile.txt', 'a') # open for appending to existing content
When you have finished with a file, close it. Examples:
infile.close() outfile.close()
file is the file type and can be used as a constructor to create file objects. But, open is preferred.
Lines read from a text file have a newline. Strip it off with something like: line.rstrip('\n').
For binary files you should add the binary mode, for example: rb, wb. For more about modes, see the description of the open() function at Built-in Functions -- http://docs.python.org/lib/built-in-funcs.html.
Learn more about file objects and the methods they provide at: 2.3.9 File Objects -- http://docs.python.org/lib/bltin-file-objects.html.
You can also append to an existing file. Example:
In [39]: f = open('mylog.txt', 'a')
In [40]: f.write('message #4\n')
In [41]: f.close()
In [42]: f = file('mylog.txt', 'r')
In [43]: for line in f:
....: print line,
....:
message #1
message #2
message #3
message #4
In [44]: f.close()
Exercises:
Read all of the lines of a file into a list. Print the 3rd and 5th lines in the file/list. Solution:
In [55]: f = open('tmp1.txt', 'r')
In [56]: lines = f.readlines()
In [57]: f.close()
In [58]: lines
Out[58]: ['the\n', 'big\n', 'brown\n', 'dog\n', 'had\n', 'long\n', 'hair\n']
In [59]: print lines[2]
brown
In [61]: print lines[4]
had
More notes:
Other built-in data types are described in section Built-in Types -- http://docs.python.org/lib/types.html in the Python standard documentation.
The unique value None is used to indicate "no value", "nothing", "non-existence", etc. There is only one None value.
Use is to test for None. Example:
>>> flag = None >>> >>> if flag is None: ... print 'clear' ... clear >>> if flag is not None: ... print 'hello' ... >>>
True and False are the boolean values.
The following values also count as false: 0, None, the empty string, an empty list, an empty dictionary, any empty container, etc.
Jython has True and False, but their values are int 1 and 0.
A set is an unordered collection of immutable objects. A set does not contain duplicates.
Sets support several set operations, for example: union, intersection, difference, ...
A frozenset is like a set, except that a frozenset is immutable. Therefore, a frozenset is hash-able and can be used as a key in a dictionary.
Create a set with the set constructor. Examples:
>>> a = set()
>>> a
set([])
>>> a.add('aa')
>>> a.add('bb')
>>> a
set(['aa', 'bb'])
>>> b = set([11, 22])
>>> b
set([11, 22])
For more information on sets, see: Set Types -- set, frozenset -- http://docs.python.org/lib/types-set.html
Jython 2.2.1 does not have sets. Consider using java.util.HashSet, java.util.LinkedHashSet, or java.util.TreeSet, each of which implement the java.util.Set interface. This Jython example uses class HashSet:
>>> # Jython, not Python
>>>
>>> from java.util import HashSet
>>> a = HashSet()
>>> a.add('aa')
1
>>> a.add('bb')
1
>>> a
[aa, bb]
Structured code -- Python programs are made up of expressions, statements, functions, classes, modules, and packages.
Python objects are first-class objects.
Expressions are evaluated.
Statements and executed.
Functions
Object-oriented programming in Python. Modeling "real world" objects. (1) Encapsulation; (2) data hiding; (3) inheritance. Polymorphism.
Classes -- (1) encapsulation; (2) data hiding; (3) inheritance.
An overview of the structure of a typical class: (1) methods; (2) the constructor; (3) class (static) variables; (4) super/sub-classes.
Form -- target = expression.
Possible targets:
Identifier
Tuple or list -- Can be nested. Left and right sides must have equivalent structure. Example:
>>> x, y, z = 11, 22, 33 >>> [x, y, z] = 111, 222, 333 >>> a, (b, c) = 11, (22, 33) >>> a, B = 11, (22, 33)
This feature can be used to simulate an enum:
In [22]: LITTLE, MEDIUM, LARGE = range(1, 4) In [23]: LITTLE Out[23]: 1 In [24]: MEDIUM Out[24]: 2
Subscription of a sequence, dictionary, etc.
A slice of a sequence -- Note that the sequence must be mutable.
Attribute reference -- Example:
>>> class MyClass: ... pass ... >>> anObj = MyClass() >>> anObj.desc = 'pretty' >>> print anObj.desc pretty
There is also augmented assignment. Examples:
>>> index = 0 >>> index += 1 >>> index += 5 >>> index += f(x) >>> index -= 1 >>> index *= 3
Things to note:
Assignment to a name creates a new variable (if it does not exist in the namespace) and a binding. Specifically, it binds a value to the new name. Calling a function also does this to the (formal) parameters.
In Python, a language with dynamic typing, the data type is associated with the value, not the variable, as in statically typed languages.
Assignment can also cause sharing of an object. Example:
obj1 = A() obj2 = obj1
Check to determine that the same object is shared with id(obj) or the is operator.
You can also do multiple assignment in a single statement. Example:
a = b = 3
You can interchange (swap) the value of two variables using assignment and packing/unpacking:
>>> a = 111 >>> b = 222 >>> a, b = b, a >>> a 222 >>> b 111
Make module available.
What import does:
Where import looks for modules:
sys.path shows where it looks.
Packages need a file named __init__.py.
Extensions -- To determine what extensions import looks for, do:
>>> import imp
>>> imp.get_suffixes()
[('.so', 'rb', 3), ('module.so', 'rb', 3), ('.py', 'U', 1), ('.pyc', 'rb', 2)]
Forms of the import statement:
More notes on the import statement:
The import statement and packages -- A file named __init__.py is required in a package. This file is evaluated the first time either the package is imported or a file in the package is imported. Question: What is made available when you do import aPackage? Answer: All variables (names) that are global inside the __init__.py module in that package. But, see notes on the use of __all__: The import statement -- http://docs.python.org/ref/import.html
The use of if __name__ == "__main__": -- Makes a module both import-able and executable.
Using dots in the import statement -- From the Python language reference manual:
"Hierarchical module names:when the module names contains one or more dots, the module search path is carried out differently. The sequence of identifiers up to the last dot is used to find a package; the final identifier is then searched inside the package. A package is generally a subdirectory of a directory on sys.path that has a file __init__.py."
See: The import statement -- http://docs.python.org/ref/import.html
Exercises:
Import a module from the standard library, for example re.
Import an element from a module from the standard library, for example import compile from the re module.
Create a simple Python package with a single module in it. Solution:
Create a directory named simplepackage in the current directory.
Create an (empty) __init__.py in the new directory.
Create an simple.py in the new directory.
Add a simple function name test1 in simple.py.
Import using any of the following:
>>> import simplepackage.simple >>> from simplepackage import simple >>> from simplepackage.simple import test1 >>> from simplepackage.simple import test1 as mytest
Arguments to print:
String formatting -- Arguments are a tuple. Reference: 2.3.6.2 String Formatting Operations -- http://docs.python.org/lib/typesseq-strings.html.
Can also use sys.stdout. Note that a carriage return is not automatically added. Example:
>>> import sys
>>> sys.stdout.write('hello\n')
Controlling the destination and format of print -- Replace sys.stdout with an instance of any class that implements the method write taking one parameter. Example:
import sys
class Writer:
def __init__(self, file_name):
self.out_file = file(file_name, 'a')
def write(self, msg):
self.out_file.write('[[%s]]' % msg)
def close(self):
self.out_file.close()
def test():
writer = Writer('outputfile.txt')
save_stdout = sys.stdout
sys.stdout = writer
print 'hello'
print 'goodbye'
writer.close()
# Show the output.
tmp_file = file('outputfile.txt')
sys.stdout = save_stdout
content = tmp_file.read()
tmp_file.close()
print content
test()
There is an alternative form of the print statement that takes a file-like object, in particular an object that has a write method. For example:
In [1]: outfile = open('tmp.log', 'w')
In [2]: print >> outfile, 'Message #1'
In [3]: print >> outfile, 'Message #2'
In [4]: print >> outfile, 'Message #3'
In [5]: outfile.close()
In [6]:
In [6]: infile = open('tmp.log', 'r')
In [7]: for line in infile:
...: print 'Line:', line.rstrip('\n')
...:
Line: Message #1
Line: Message #2
Line: Message #3
In [8]: infile.close()
Conditions -- Expressions -- Anything that returns a value. Compare with eval() and exec.
Truth values:
Operators:
and and or
not
is and is not -- The identical object. Cf. a is b and id(a) == id(b). Useful to test for None, for example:
if x is None:
...
if x is not None:
...
in and not in -- Can be used to test for existence of a key in a dictionary. Example:
>>> d = {'aa': 111, 'bb': 222}
>>> 'aa' in d
True
>>> 'aa' not in d
False
>>> 'xx' in d
False
Comparison operators, for example ==, !=, <, <=, ...
An if expression. Example:
>>> a = 'aa' >>> b = 'bb' >>> x = 'yes' if a == b else 'no' >>> x 'no'
Exercises:
Iterate over a sequence or an "iterable" object.
Form -- for x in y:.
Iterator -- Some notes on what it means to be iterable:
Some ways to produce iterators (see http://docs.python.org/lib/built-in-funcs.html):
iter()
enumerate()
some_dict.iterkeys(), some_dict.itervalues(), some_dict.iteritems().
Use a sequence in an iterator context, for example in a for statement. Lists, tuples, dictionaries, and strings can be used in an iterator context to produce an iterator.
Generator expressions -- Latest Python only. Syntactically like list comprehensions (surrounded by parens instead of square brackets), but use lazy evaluation.
A class that implements the iterator protocol -- Example:
class A:
def __init__(self):
self.data = [11,22,33]
self.idx = 0
def __iter__(self):
return self
def next(self):
if self.idx < len(self.data):
x = self.data[self.idx]
self.idx +=1
return x
else:
raise StopIteration
def test():
a = A()
for x in a:
print x
test()
The for statement can also have an optional else: clause. The else: clause is executed if the for statement completes normally, that is if a break statement is not executed.
Helpful functions with for:
enumerate(iterable) -- Returns an iterable that produces pairs (tuples) containing count and value. Example:
for count, value in enumerate([11,22,33]):
print count, value
range([start,] stop[, step]) and xrange([start,] stop[, step]).
List comprehensions revisited -- Since list comprehensions create lists, they are useful in for statements, although you should consider using a generator expression instead. Two forms:
Exercises:
Write a list comprehension that returns all the keys in a dictionary whose associated values are greater than zero.
Write a list comprehension that produces even integers from 0 to 10. Use a for statement to iterate over those values. Solution:
for x in [y for y in range(10) if y % 2 == 0]:
print 'x: %s' % x
But, note that in the previous exercise, a generator expression would be better. A generator expression is like a list comprehension, except that, instead of creating the entire list, it produces a generator that can be used to produce all the elements.
The break and continue statements are often useful in a for statement.
The for statement can also have an optional else: clause. The else: clause is executed if the for statement completes normally, that is if a break statement is not executed.
Form:
while condition:
block
Exercises:
Write a while statement that prints integers from zero to 5. Solution:
count = 0
while count < 5:
count += 1
print count
The break and continue statements are often useful in a while statement.
The while statement can also have an optional else: clause. The else: clause is executed if the while statement completes normally, that is if a break statement is not executed.
The break statement exits from a loop.
The continue statement causes execution to immediately continue at the start of the loop.
Can be used in for and while.
Exercises:
Using break, write a while statement that prints integers from zero to 5. Solution:
count = 0
while True:
count += 1
if count > 5:
break
print count
Using continue, write a while statement that processes only even integers from 0 to 10. Note: % is the modulo operator. Solution:
count = 0
while count < 10:
count += 1
if count % 2 == 0:
continue
print count
Exceptions are a systematic and consistent way of processing errors and "unusual" events in Python.
Caught and un-caught exceptions -- Uncaught exceptions terminate a program.
The try: statement catches an exception.
Almost all errors in Python are exceptions.
Evaluation (execution model) of the try statement -- When an exception occurs in the try block, even if inside a nested function call, execution of the try block ends and the except clauses are searched for a matching exception.
Tracebacks -- Also see the traceback module: http://docs.python.org/lib/module-traceback.html
Exceptions are classes.
Exception classes -- Sub-classing, args.
An exception class in an except: clause catches instances of that exception class and all sub-classes, but not super-classes.
Built-in exception classes -- See:
User defined exception classes -- Sub-classes of Exception.
Example:
try:
raise RuntimeError('this silly error')
except RuntimeError, e:
print "[[[%s]]]" % e
Reference: http://docs.python.org/lib/module-exceptions.html
You can also get the arguments passed to the constructor of an exception object. In the above example, these would be:
e.args
Why would you define your own exception class? One answer: You want a user of your code to catch your exception and no others.
Catching an exception by exception class catches exceptions of that class and all its subclasses. So:
except SomeExceptionClass, e:
matches and catches an exception if SomeExceptionClass is the exception class or a base class (superclass) of the exception class. So:
class MyE(ValueError):
pass
try:
raise MyE
except ValueError:
print 'caught exception'
will print "caught exception", because ValueError is a base class of MyE.
Exercises:
Write a very simple, empty exception sub-class. Solution:
class MyE(Exception):
pass
Write a try:except: statement that raises your exception and also catches it. Solution:
try:
raise MyE('hello there dave')
except MyE, e:
print e
Throw or raise an exception.
Forms:
The raise statement takes:
See http://docs.python.org/ref/raise.html.
For a list of built-in exceptions, see http://docs.python.org/lib/module-exceptions.html.
The following example defines an exception sub-class and throws an instance of that sub-class. It also shows how to pass and catch multiple arguments to the exception:
class NotsobadError(Exception):
pass
def test(x):
try:
if x == 0:
raise NotsobadError('a moderately bad error', 'not too bad')
except NotsobadError, e:
print 'Error args: %s' % (e.args, )
test(0)
The following example does a small amount of processing of the arguments:
class NotsobadError(Exception):
"""An exception class.
"""
def get_args(self):
return '::::'.join(self.args)
def test(x):
try:
if x == 0:
raise NotsobadError('a moderately bad error', 'not too bad')
except NotsobadError, e:
print 'Error args: {{{%s}}}' % (e.get_args(), )
test(0)
The del statement can be used to:
If name is listed in a global statement, then del removes name from the global namespace.
Names can be a (nested) list. Examples:
>>> del a >>> del a, b, c
We can also delete items of a list or dictionary (and perhaps from other objects that we can subscript). Examples:
In [9]:d = {'aa': 111, 'bb': 222, 'cc': 333}
In [10]:print d
{'aa': 111, 'cc': 333, 'bb': 222}
In [11]:del d['bb']
In [12]:print d
{'aa': 111, 'cc': 333}
In [13]:
In [13]:a = [111, 222, 333, 444]
In [14]:print a
[111, 222, 333, 444]
In [15]:del a[1]
In [16]:print a
[111, 333, 444]
And, we can delete an attribute from an instance. Example:
In [17]:class A: ....: pass ....: In [18]:a = A() In [19]:a.x = 123 In [20]:dir(a) Out[20]:['__doc__', '__module__', 'x'] In [21]:print a.x 123 In [22]:del a.x In [23]:dir(a) Out[23]:['__doc__', '__module__'] In [24]:print a.x ---------------------------------------------- exceptions.AttributeError Traceback (most recent call last) /home/dkuhlman/a1/Python/Test/<console> AttributeError: A instance has no attribute 'x'
The def statement is used to define functions and methods.
The def statement is evaluated. It produces a function/method (object) and binds it to a variable in the current name-space.
The return statement is used to return values from a function.
The return statement takes zero or more values, separated by commas. Using commas actually returns a single tuple.
The default value is None.
To return multiple values, use an expression list. Don't forget that (assignment) unpacking can be used to capture multiple values. Returning multiple items separated by commas is equivalent to returning a tuple. Example:
In [8]: def test(x, y): ...: return x * 3, y * 4 ...: In [9]: a, b = test(3, 4) In [10]: print a 9 In [11]: print b 16
Default values -- Example:
In [53]: def t(max=5): ....: for val in range(max): ....: print val ....: ....: In [54]: t(3) 0 1 2 In [55]: t() 0 1 2 3 4
Note: If a function has an argument with a default value, then all "normal" arguments must proceed the arguments with default values. More completely, arguments must be given from left to right in the following order:
List arguments -- *args. It's a tuple.
Keyword arguments -- **kwargs. It's a dictionary.
Passing lists to a function as multiple arguments -- some_func(*aList). Effectively, this syntax causes Python to unroll the arguments.
Return values:
Local variables:
Things to know about functions:
Functions are first-class -- You can store them in a structure, pass them to a function, and return them from a function.
Function calls can take keyword arguments. Example:
>>> test(size=25)
Formal arguments to a function can have default values. Example:
>>> def test(size=0):
...
Do not use mutable objects as default values.
You can "capture" remaining arguments with *args, and **kwargs. (Spelling is not significant.) Example:
In [13]: def test(size, *args, **kwargs):
....: print size
....: print args
....: print kwargs
....:
....:
In [14]: test(32, 'aa', 'bb', otherparam='xyz')
32
('aa', 'bb')
{'otherparam': 'xyz'}
Normal arguments must come before default arguments which must come before keyword arguments.
A function that does not explicitly return a value, returns None.
In order to set the value of a global variable, declare the variable with global.
Exercises:
Write a function that takes a single argument, prints the value of the argument, and returns the argument as a string. Solution:
>>> def t(x): ... print 'x: %s' % x ... return '[[%s]]' % x ... >>> t(3) x: 3 '[[3]]'
Write a function that takes a variable number of arguments and prints them all. Solution:
>>> def t(*args):
... for arg in args:
... print 'arg: %s' % arg
...
>>> t('aa', 'bb', 'cc')
arg: aa
arg: bb
arg: cc
Write a function that prints the names and values of keyword arguments passed to it. Solution:
>>> def t(**kwargs): ... for key in kwargs.keys(): ... print 'key: %s value: %s' % (key, kwargs[key], ) ... >>> t(arg1=11, arg2=22) key: arg1 value: 11 key: arg2 value: 22
By default, assignment in a function or method creates local variables.
Reference (not assignment) to a variable, accesses a local variable if it has already been created, else accesses a global variable.
In order to assign a value to a global variable, declare the variable as global at the beginning of the function or method.
If in a function or method, you both reference and assign to a variable, then you must either:
The global statement declares one or more variables, separated by commas, to be global.
Some examples:
In [1]: In [1]: X = 3 In [2]: def t(): ...: print X ...: In [3]: In [3]: t() 3 In [4]: def s(): ...: X = 4 ...: In [5]: In [5]: In [5]: s() In [6]: t() 3 In [7]: X = -1 In [8]: def u(): ...: global X ...: X = 5 ...: In [9]: In [9]: u() In [10]: t() 5 In [16]: def v(): ....: x = X ....: X = 6 ....: return x ....: In [17]: In [17]: v() ------------------------------------------------------------ Traceback (most recent call last): File "<ipython console>", line 1, in <module> File "<ipython console>", line 2, in v UnboundLocalError: local variable 'X' referenced before assignment In [18]: def w(): ....: global X ....: x = X ....: X = 7 ....: return x ....: In [19]: In [19]: w() Out[19]: 5 In [20]: X Out[20]: 7
Add docstrings as a triple-quoted string beginning with the first line of a function or method. See epydoc for a suggested format.
Use a lambda, as a convenience, when you need a function that both:
A lambda can take multiple arguments and can return (like a function) multiple values. Example:
In [79]: a = lambda x, y: (x * 3, y * 4, (x, y)) In [80]: In [81]: a(3, 4) Out[81]: (9, 16, (3, 4))
Suggestion: In some cases, a lambda may be useful as an event handler.
Example:
class Test:
def __init__(self, first='', last=''):
self.first = first
self.last = last
def test(self, formatter):
"""
Test for lambdas.
formatter is a function taking 2 arguments, first and last
names. It should return the formatted name.
"""
msg = 'My name is %s' % (formatter(self.first, self.last),)
print msg
def test():
t = Test('Dave', 'Kuhlman')
t.test(lambda first, last: '%s %s' % (first, last, ))
t.test(lambda first, last: '%s, %s' % (last, first, ))
test()
Reference: http://docs.python.org/ref/lambdas.html
Concepts:
An object satisfies the iterator protocol if it does the following:
For more information on iterators, see the section on iterator types in the Python Library Reference -- http://docs.python.org/lib/typeiter.html.
For more on yield expressions and on the next() and send() generator methods, as well as others, see Yield expression -- http://docs.python.org/ref/yieldexpr.html in the Python reference manual.
A function or method containing a yield statement implements a generator. Adding the yield statement to a function or method turns that function or method into one which, when called, returns a generator, i.e. an object that implements the iterator protocol.
A generator (a function containing yield) provides a convenient way to implement a filter. But, also consider:
Here are a few examples:
# For Jython
from __future__ import generators # Note 1
def simplegenerator():
yield 'aaa' # Note 2
yield 'bbb'
yield 'ccc'
def list_tripler(somelist):
for item in somelist:
item *= 3
yield item
def limit_iterator(somelist, max):
for item in somelist:
if item > max:
return # Note 3
yield item
def test():
print '1.', '-' * 30
it = simplegenerator()
for item in it:
print item
print '2.', '-' * 30
alist = range(5)
it = list_tripler(alist)
for item in it:
print item
print '3.', '-' * 30
alist = range(8)
it = limit_iterator(alist, 4)
for item in it:
print item
print '4.', '-' * 30
it = simplegenerator()
try:
print it.next() # Note 4
print it.next()
print it.next()
print it.next()
except StopIteration, exp: # Note 5
print 'reached end of sequence'
if __name__ == '__main__':
test()
Notes:
In order to use generators and the yield statement in Jython 2.2, we need to add the following import statement:
from __future__ import generators
The yield statement returns a value. When the next item is requested and the iterator is "resumed", execution continues immediately after the yield statement.
We can terminate the sequence generated by an iterator by using a return statement with no value.
To resume a generator, use the generator's next() or send() methods. send() is like next(), but provides a value to the yield expression.
We can alternatively obtain the items in a sequence by calling the iterator's next() method. Since an iterator is a first-class object, we can save it in a data structure and can pass it around for use at different locations and times in our program.
When an iterator is exhausted or empty, it throws the StopIteration exception, which we can catch.
And here is the output from running the above example:
$ python test_iterator.py 1. ------------------------------ aaa bbb ccc 2. ------------------------------ 0 3 6 9 12 3. ------------------------------ 0 1 2 3 4 4. ------------------------------ aaa bbb ccc reached end of sequence
An instance of a class which implements the __iter__ method, returning an iterator, is iterable. For example, it can be used in a for statement or in a list comprehension, or in a generator expression, or as an argument to the iter() built-in method. But, notice that the class most likely implements a generator method which can be called directly.
Examples -- The following code implements an iterator that produces all the objects in a tree of objects:
class Node:
def __init__(self, data, children=None):
self.initlevel = 0
self.data = data
if children is None:
self.children = []
else:
self.children = children
def set_initlevel(self, initlevel): self.initlevel = initlevel
def get_initlevel(self): return self.initlevel
def addchild(self, child):
self.children.append(child)
def get_data(self):
return self.data
def get_children(self):
return self.children
def show_tree(self, level):
self.show_level(level)
print 'data: %s' % (self.data, )
for child in self.children:
child.show_tree(level + 1)
def show_level(self, level):
print ' ' * level,
#
# Generator method #1
# This generator turns instances of this class into iterable objects.
#
def walk_tree(self, level):
yield (level, self, )
for child in self.get_children():
for level1, tree1 in child.walk_tree(level+1):
yield level1, tree1
def __iter__(self):
return self.walk_tree(self.initlevel)
#
# Generator method #2
# This generator uses a support function (walk_list) which calls
# this function to recursively walk the tree.
# If effect, this iterates over the support function, which
# iterates over this function.
#
def walk_tree(tree, level):
yield (level, tree)
for child in walk_list(tree.get_children(), level+1):
yield child
def walk_list(trees, level):
for tree in trees:
for tree in walk_tree(tree, level):
yield tree
#
# Generator method #3
# This generator is like method #2, but calls itself (as an iterator),
# rather than calling a support function.
#
def walk_tree_recur(tree, level):
yield (level, tree,)
for child in tree.get_children():
for level1, tree1 in walk_tree_recur(child, level+1):
yield (level1, tree1, )
def show_level(level):
print ' ' * level,
def test():
a7 = Node('777')
a6 = Node('666')
a5 = Node('555')
a4 = Node('444')
a3 = Node('333', [a4, a5])
a2 = Node('222', [a6, a7])
a1 = Node('111', [a2, a3])
initLevel = 2
a1.show_tree(initLevel)
print '=' * 40
for level, item in walk_tree(a1, initLevel):
show_level(level)
print 'item:', item.get_data()
print '=' * 40
for level, item in walk_tree_recur(a1, initLevel):
show_level(level)
print 'item:', item.get_data()
print '=' * 40
a1.set_initlevel(initLevel)
for level, item in a1:
show_level(level)
print 'item:', item.get_data()
iter1 = iter(a1)
print iter1
print iter1.next()
print iter1.next()
print iter1.next()
print iter1.next()
print iter1.next()
print iter1.next()
print iter1.next()
## print iter1.next()
return a1
if __name__ == '__main__':
test()
Notes:
A module is a Python source code file.
A module can be imported. When imported, the module is evaluated, and a module object is created. The module object has attributes. The following attributes are of special interest:
A module can be run.
To make a module both import-able and run-able, use the following idiom (at the end of the module):
def main():
o
o
o
if __name__ == '__main__':
main()
Where Python looks for modules:
Notes about modules and objects:
Add docstrings as a triple-quoted string at or near the top of the file. See epydoc for a suggested format.
A package is a directory on the file system which contains a file named __init__.py.
The __init__.py file:
Why is it there? -- It makes modules in the directory "import-able".
Can __init__.py be empty? -- Yes. Or, just include a comment.
When is it evaluated? -- It is evaluated the first time that an application imports anything from that directory/package.
What can you do with it? -- Any code that should be executed exactly once and during import. For example:
Define a variable named __all__ to specify the list of names that will be imported by from my_package import *. For example, if the following is present in my_package/__init__.py:
__all__ = ['func1', 'func2',]
Then, from my_package import * will import func1 and func2, but not other names defined in my_package.
Note that __all__ can be used at the module level, as well as at the package level.
pdb -- The Python debugger:
Start the debugger by running an expression:
pdb.run('expression')
Example:
if __name__ == '__main__':
import pdb
pdb.run('main()')
Start up the debugger at a specific location with the following:
import pdb; pdb.set_trace()
Example:
if __name__ == '__main__':
import pdb
pdb.set_trace()
main()
Get help from within the debugger. For example:
(Pdb) help (Pdb) help next
Can also embed IPython into your code. See http://ipython.scipy.org/doc/manual/manual.html.
Inspecting:
Miscellaneous tools:
Classes model the behavior of objects in the "real" world. Methods implement the behaviors of these types of objects. Member variables hold (current) state.
In [104]: class A: .....: pass .....: In [105]: a = A()
To define a new style class (recommended), inherit from object or from another class that does. Example:
In [21]: class A(object): ....: pass ....: In [22]: In [22]: a = A() In [23]: a Out[23]: <__main__.A object at 0x82fbfcc>
A method is a function defined in class scope and with first parameter self:
In [106]: class B: .....: def show(self): .....: print 'hello from B' .....: In [107]: b = B() In [108]: b.show() hello from B
The constructor is a method named __init__.
Exercise: Define a class with a member variable name and a show method. Use print to show the name. Solution:
In [109]: class A:
.....: def __init__(self, name):
.....: self.name = name
.....: def show(self):
.....: print 'name: "%s"' % self.name
.....:
In [111]: a = A('dave')
In [112]: a.show()
name: "dave"
Notes:
Defining member variables -- Member variables are created with assignment. Example:
class A:
def __init__(self, name):
self.name = name
A small gotcha -- Do this:
In [28]: class A: ....: def __init__(self, items=None): ....: if items is None: ....: self.items = [] ....: else: ....: self.items = items
Do not do this:
In [29]: class B: ....: def __init__(self, items=[]): # wrong. list ctor evaluated only once. ....: self.items = items
In the second example, the def statement and the list constructor are evaluated only once. Therefore, the item member variable of all instances of class B, will share the same value, which is most likely not what you want.
Defining methods
Calling methods:
Referencing super-classes -- Use the name of the super-class, for example:
In [39]: class B(A): ....: def __init__(self, name, size): ....: A.__init__(self, name) ....: self.self = size
Calling the constructor of the super-class.
You can also use multiple inheritance. Example:
class C(A, B):
...
Python searches super-classes in left-to-right depth-first order.
For more information on inheritance, see the tutorial in the standard Python documentation set: 9.5 Inheritance and 9.5.1 Multiple Inheritance.
Watch out for problems with inheriting from classes that have a common base class.
Also called static data.
Define at class level with assignment. Example:
class A:
size = 5
def get_size(self):
return A.size
Reference with classname.variable.
Caution: self.variable = x creates a new member variable.
Instance (plain) methods:
Class methods:
Static methods:
Notes on decorators:
A decorator of the for @afunc is the same as m = afunc(m). So, this:
@afunc def m(self): pass
is the same as:
def m(self): pass m = afunc(m)
You can use @classmethod and @staticmethod (instead of the classmethod and staticmethod built-in functions) to declare class methods and static methods.
Decorators are not yet available in Jython.
Example:
class B:
def dup_string(x):
s1 = '%s%s' % (x, x,)
return s1
dup_string = staticmethod(dup_string)
B.dup_string('abcd')
'abcdabcd'
An alternative way to implement static methods -- Use a "plain", module-level function. For example:
In [1]: def inc_count(): ...: A.count += 1 ...: In [2]: In [2]: def dec_count(): ...: A.count -= 1 ...: In [3]: In [3]: class A: ...: count = 0 ...: def get_count(self): ...: return A.count ...: In [4]: In [4]: a = A() In [5]: a.get_count() Out[5]: 0 In [6]: In [6]: In [6]: inc_count() In [7]: inc_count() In [8]: a.get_count() Out[8]: 2 In [9]: In [9]: b = A() In [10]: b.get_count() Out[10]: 2
The property built-in function enables us to write classes in a way that does not require a user of the class to use getters and setters. Example:
class TestProperty(object):
def __init__(self, description):
self._description = description
def set_description(self, description):
print 'setting description'
self._description = description
def get_description(self):
print 'getting description'
return self._description
description = property(get_description, set_description)
For more information on properties, see Built-in Functions -- http://docs.python.org/lib/built-in-funcs.html#l2h-57
In Python, to implement an interface is to implement a method with a specific name and a specific arguments.
"Duck typing" -- If it walks like a duck and quacks like a duck ...
One way to define an "interface" is to define a class containing methods that have a header and a doc string but no implementation.
Additional notes on interfaces:
A new-style class is one that sub-classes object or a class that sub-classes object or another new-style class.
You can sub-class Python's built-in datatypes.
A simple example -- the following class extends the list datatype:
class C(list):
def get_len(self):
return len(self)
c = C((11,22,33))
c.get_len()
c = C((11,22,33,44,55,66,77,88))
p