Style

Introduction

• Good programming style is hard to teach because the rules tend to be either banal or overly restrictive
  • “Make methods short, but not too short” isn't useful
  • Neither is, “No method shall be longer than sixty lines.”
  • Doesn't help that the strength of programmers' opinions is inversely proportional to the amount of data they have
• But readable code is less likely to contain errors, and much easier to maintain
• This lecture presents and motivates style guidelines
  • For a comprehensive guide to structuring large Unix applications, see [Spinellis 2003]
• Send comments

You Can Skip This Lecture If...

• You know what 7±2 has to do with readability
• You know what a docstring is, and what it should contain
• You know what traceability is
• You know how to automatically check for style violations
• Send comments

Reading is Learning

• People doing creative work in almost every field routinely inspect, dissect, and critique what's come before
  • But most student programmers only ever read short fragments in textbooks
  • Like reading sonnets, then trying to write a novel
• Knowing how to read code is as useful as knowing how to read a proof
  • Have to do it in order to figure out how to make specific changes to specific programs
  • A good way to learn new things
• Send comments

Seven Plus or Minus

• The average person's short term memory can hold 7±2 items [Hock 2004]
  • Seven random digits (as in phone numbers)
  • Seven tasks that still have to be done
• If we try to remember more than that, we:
  • Make mistakes, or
  • Create chunks so that we can remember things at a higher level
    • Common chord progressions in music
    • “Castled kingside” instead of the positions of five separate pieces
  • 

    Figure 10.1: Chunking in Short-Term Memory

• Send comments

The Mind's Eye

• [Chase & Simon 1973] studied what happened when novice and master chess players were shown actual and random positions
  • Masters remember actual positions better
  • But they were worse at remembering random ones
  • Their minds “see” patterns that aren't there


Figure 10.2: Actual Chess Position



Figure 10.3: Retention of Actual Chess Position



Figure 10.4: Random Chess Position



Figure 10.5: Retention of Random Chess Position

• Send comments

What Does This Have to Do With Programming?

• When reading and writing code, you have to keep a bunch of facts straight for a short period of time
  • What do this function's parameters mean?
  • What does this loop's index refer to?
• The more odds and ends readers have to keep track of, the more errors they will make
  • Goal of style rules is therefore to reduce the number of things the reader has to juggle mentally
• The greater a difference is, the more likely we are to notice it
  • So every semantic difference ought to be visually distinct…
  • …and every difference in naming or layout ought to mean something
• Most important thing is to be consistent
  • Anything consistent is readable after a while
  • Just watch kids learning to read French, Punjabi, and Korean
• Send comments

Python Style Guide

• Taken from PEP-008: Python Style Guide
  • Stick to this unless you have hard data that proves something else is better
• Basic layout
  • Indent blocks using four spaces
  • Keep lines less than 80 characters long
  • Separate functions with two blank lines
  • Separate logical chunks of long functions with a single blank line
  • Put comments on lines of their own, rather than to the right of code

Rule	Good	Bad
No whitespace immediately inside parentheses	max(candidates[sublist])	max( candidates[ sublist ] )
…or before the parenthesis starting indexing or slicing		max (candidates [sublist] )
No whitespace immediately before comma or colon	if limit > 0: print minimum, limit	if limit > 0 : print minimum , limit
Use space around arithmetic and in-place operators	x += 3 * 5	x+=3*5
No spaces when specifying default parameter values	def integrate(func, start=0.0, interval=1.0)	def integrate(func, start = 0.0, interval = 1.0)
Never use names that are distinguished only by "l", "1", "0", or "O"	tempo_long and tempo_init	tempo_l and tempo_1
Short lower-case names for modules (i.e., files)	geology	Geology or geology_package
Upper case with underscores for constants	TOLERANCE or MAX_AREA	Tolerance or MaxArea
Camel case for class names	SingleVariableIntegrator	single_variable_integrator
Lowercase with underscores for function and method names	divide_region	divRegion
…and member variables	max_so_far	maxSoFar
Use is and is not when comparing to special values	if current is not None:	if current != None:
Use isinstance when checking types	if isinstance(current, Rock):	if type(current) == Rock:
Table 10.1: Basic Python Style Rules

• Rules relating to classes will become clear soon
• Send comments

Naming

• Names of files, classes, methods, variables, and other things are the most visible clue to purpose
  • A variable called temperature shouldn't be used to store the number of pottery shards found at a dig site
• Choose names that are both meaningful and readable
  • current_surface_temperature_of_probe is meaningful, but not readable
  • cstp is easier to read, but hard to understand…
  • …and easy to confuse with ctsp
• If you must abbreviate, be consistent
  • curr_ave_temp instead of current_average_temperature is OK…
  • …but only if no one else is using curnt_av_tmp
• Send comments

Scope and Size

• The smaller the scope of a name, the more compact it can be
  • It's OK to use i and j for indices in tightly-nested for loops
  • But not OK if the loop bodies are several pages long
    • Of course, they shouldn't be anyway…
• The wider the scope of a name, the more descriptive it has to be
  • Call a class ExperimentalRecord, rather than ER or ExpRec
• Send comments

The Difference It Makes

• Before:

  import sys, os
  import reader, splitter, transpose
  a=[]
  b=[]
  c=[]
  d=sys.argv[1]
  a=reader.rdlines(d)
  b=splitter.splitsec(a)
  c=d.split('.')
  for i in range(len(b)):
      if os.path.isfile('%s.%d.dat'%(c[0],i+1)):
          print '%s.%d.dat already exists!'%(c[0],i+1)
          break
      else:
          output=file('%s.%d.dat'%(c[0],i+1),'w')
          print>>output,transpose.txpose(b[i])
          output.close()
  

• After:

  import sys, os
  import reader, splitter, transpose
  
  input_file_name = sys.argv[1]
  lines = reader.read_lines_from_file(input_file_name)
  sections = splitter.split_into_sections(lines)
  file_name_stem = input_file_name.split('.')[0]
  for i in range(len(sections)):
      output_file_name = '%s.%d.dat' % (file_name_stem, i+1)
      if os.path.isfile(output_file_name):
          print '%s already exists!' % output_file_name
          break
      else:
          output = file(output_file_name, 'w')
          print >> output, transpose.transpose(sections[i])
          output.close()
  

• Send comments

Function Length

• Every function should do exactly one job
  • Should be able to describe that job in a single memorable sentence
  • If that sentence is five phrases joined with “and”, the function should be split up
• A good way to judge size and scope is the notion of a program slice
  • The subset of names in scope at a particular statement that are needed to understand what that statement does
  • If the slice is much smaller than the method itself, the method is probably bloated
• A thousand one-line methods are not an improvement over one thousand-line method
• Send comments

What Does This Function Do?

• Key questions:
  • What are the function's arguments for?
  • What does it return?
  • What side effects does it have?

# What's missing, and what's extra?
def diff_filelist(dir_path, manifest,
                  ignore=[os.curdir, os.pardir, '.svn']):

    def show_diff(title, diff):
        if diff:
            print title
            for d in diff:
                print '\t' + d

    expected = Set()
    inf = open(manifest, 'r')
    for line in inf:
        expected.add(line.strip())
    inf.close()

    actual = Set()
    contents = os.listdir(dir_path)
    for c in contents:
        if c not in ignore:
            actual.add(c)

    show_diff('missing:', expected - actual)
    show_diff('surplus:', actual - expected)

• Send comments

Ways to Answer the Question

• Read comments
  • Not particularly helpful in this case
• Guess based on names
  • Find the difference between two file lists?
  • But dir_path suggests “directory path”
• Read the function body
  • Define a helper function that displays diff if it isn't empty
  • Make a set holding the lines in the manifest file
  • Make a set holding the contents of the directory specified by dir_path
    • Ignoring certain things
  • Use the helper function to show the differences between these two sets
• Notice how “explaining” is the same as “creating chunks”
• Send comments

Other Sources of Information

• External documentation
  • There is none
• Calls to the function
  • One in the bottom of the same file

  if __name__ == '__main__':
      if len(sys.argv) != 3:
          print >> sys.stderr, "usage: diff_filelist directory_path manifest_file"
          sys.exit(1)
      diff_filelist(sys.argv[1], sys.argv[2])

  • Not much new information there
• Use grep or “Find in Files” to search for others
  • A few examples of how something is used is often sufficient
  • Except when it's misleading
• Send comments

Idioms

• Every language (human or computer) has idioms
  • E.g., “face the music” does not mean “look at the orchestra”
• Older versions of Python couldn't iterate directly over a file using for line in input:
  • Solution: an “infinite” loop with a break to handle end-of-input

  count = 0
  while 1:
      line = infile.readline()
      if not line:
          break
      count += 1
  

  • Uses 1 instead of True because older Pythons didn't define True
• You can figure out what the code is doing even if you don't know the idiom…
  • …but knowing the idiom makes it a lot simpler for your brain to chunk what it's reading
• Learn idioms by reading:
  • Books (e.g., the “Effective” series from Addison-Wesley)
  • Other people's code
• Send comments

Style Tools

• Many tools exist to check and enforce style guidelines
• Make these tools part of your project's build
  • Don't check anything into version control unless it passes a style check
  • Take their warnings very seriously when debugging
    • If someone is sloppy enough to make style mistakes, they're probably making serious mistakes too
• Send comments

Python Style Tools

• PyLint parses programs to create an abstract syntax tree
  • 

    Figure 10.6: Abstract Syntax Tree

  • Then searches the tree for matches to problem patterns
• PyChecker imports the module (or modules)
  • If the code doesn't run, PyChecker can't analyze it
• Both tools allow users to:
  • Customize built-in checks (e.g., specify maximum length of a function)
  • Define entirely new checks
• Send comments

Documentation

• Requirements
  • What needs the software is supposed to meet
  • “If more than 100 events arrive in a second, the seismograph interface must store them in a queue“ tells you to look for a seismograph interface, and a queue that feeds it data
• User guide
  • Actually just another way to specify requirements
• Architectural descriptions
  • Architecture is what you draw on the whiteboard when explaining the program to other people
  • Shows relationships between major modules, data flow, etc.
  • Gives other programmers a mental map of how everything fits together
• Send comments

More On Documentation

• At start of functions (or classes, or methods) to explain what they're for
  • “This function returns the first pair of non-overlapping subsequences that match the input pattern, or null otherwise”
  • The programmer's guide to the software
• Embedded in code to explain tricky bits
  • “This is a while loop, instead of a for, because it may delete items from the list as it goes”
  • In general, if you need to explain the code, you ought to simplify it instead
• Send comments

Traceability

• Traceability is the key to reproducibility
  • …which is the key to debuggability
  • Where did your data come from?
  • What did you do to it?
  • Using which versions of which programs?
• Always include version control information in source files
  • Most version control systems will update text like $Revision: 1.1 $ when you submit changes
  • The file carries its identity with it when it's printed, archived, or emailed
• Usually put the keyword inside a string, so that version information is available inside the program
  • By convention, Python calls this string __version__

  __version__ = "$Revision: 1.1 $"
  
  if __name__ == '__main__':
      print __version__
  

• Send comments

Tracing Data

• Every original data file should go under version control
  • Record author and date of last change as well as revision number
• Carry this information through when processing files
  • E.g., input file biomes.dat has a header $Revision: 1.1 $
  • Is processed by version 37 of ecoanalyzer.py
  • So include the following comment header in the output file

  # $Revision: 1.1 $
  # From: biomes.dat version 421
  # By: ecoanalyzer.py version 37
  # Parameters: sliding_average 20 trim False
  # On: 2006-02-22 12:14:07 EST
  

  • The $Revision: 1.1 $ header will be expanded when the file is first checked in
• Send comments

Embedding Documentation

• Embedded documentation is more likely to be up to date than external documentation
  • A simple form of literate programming
  • It's right in front of programmers as they make changes
• Javadoc translates specially-formatted comments into HTML
  • Java

    /**
     * Returns the least common ancestor of two species based on DNA
     * comparison, with certainty no less than the specified threshold.
     * Note that getConcestor(X, X, t) returns X for any threshold.
     *
     * @param left        one of the base species for the search
     * @param right       the other base species for the search
     * @param threshold   the degree of certainty required
     * @return            the common ancestor, or null if none is found
     * @see               Species
     */
    public Species getConcestor(Species left, Species right, float threshold) {
        ...implementation...
    }
    

  • Documentation web page

    getConcestor

    public Species getConcestor(Species left, Species right, float threshold)

    Returns the least common ancestor of two species based on DNA comparison, with certainty no less than the specified threshold. Note that getConcestor(X, X, t) returns X for any threshold.

    Parameters:

    left - one of the base species for the search

    right - the other base species for the search

    threshold - the degree of certainty required

    Parameters:

    the common ancestor, or null if none is found

    See Also:

    Image

• Send comments

Docstrings

• Python uses documentation strings (or docstrings) instead of comments
  • A string at the start of a module or function that isn't assigned to anything becomes the object's __doc__ attribute
  • Unlike a comment, it's there at runtime

'''This module provides functions that search and compare genomes.
All functions assume that their input arguments are in valid CCSN-2
format; unless specifically noted, they do not modify their arguments,
print, or have other side effects.
'''

__version__ = '$Revision: 1.1 $'

def get_concestor(left, right, threshold):
    '''Find the least common ancestor of two species.

    This function searches for a least common ancestor based on DNA
    comparison with certainty no less than the specified threshold.
    If one can be found, it is returned; otherwise, the function
    returns None.  get_concestor(X, X, t) returns X for any threshold.

    left      : one of the base species for the search
    right     : the other base species for the search
    threshold : the degree of certainty required
    '''

    pass # implementation would go here

$ python
>>> import genome
>>> print genome.__doc__
This module provides functions that search and compare genomes.
All functions assume that their input arguments are in valid CCSN-2
format; unless specifically noted, they do not modify their arguments,
print, or have other side effects.

>>> print genome.get_concestor.__doc__
Find the least common ancestor of two species.

    This function searches for a least common ancestor based on DNA
    comparison with certainty no less than the specified threshold.
    If one can be found, it is returned; otherwise, the function
    returns None.  get_concestor(X, X, t) returns X for any threshold.

    left      : one of the base species for the search
    right     : the other base species for the search
    threshold : the degree of certainty required

• Python's Docutils will extract, format, and cross-reference docstrings
• Send comments

Summary

• Code and documentation decay over time [Eick et al 2001]
• To prevent this, must:
  • Make good style a habit
  • Back it up with automated checks
• Remember, we don't actually write programs for the benefit of computers
  • It takes a lot of very sophisticated software to translate our programs into a form computers understand
• We write programs for other people
  • Our colleagues
  • Our future selves
• Send comments