Scipy includes a handy lookup table class, but it’s sort of hidden. Here is an example of how to create a lookup table using interp1d:

from scipy.interpolate import interp1
from numpy import *

ddeltah = L/1000.
h_sampled = arange(0., L+deltah/10, deltah)
q_sampled = q(h_sampled,L,D,1e-4)
h_interp = interp1d(q_sampled, h_sampled, kind='linear')
print h_interp(0.514234)

First, create a Numpy array h_sampled to store the h-values for the lookup table. The function q(h,L,D,dt) is not shown, but it takes an array of h values and returns a floating-point array of q values in the range from 0 to 1. These two arrays are then used to construct the lookup table. The table uses linear interpolation to compute values between the known points. Higher-order interpolations can be used, but I don’t need them in this case. Higher-order interpolations can introduce distortions for certain types of data, so they are best avoided unless you really understand the data being fitted. Below is a plot illustrating the results of a look-up table:

The solid line is the function h(q), the black dots are the values stored in the lookup table, and the red triangles are values interpolated from the lookup table. The function h(q) was approximated with a lookup table that was created by computing q(h) with an evenly spaced array of h values. Notice how the points stored in the table are spaced more closely together for smaller values of q. This is a consequence of using a linear “h” spacing with a nonlinear function. You could work around this by using a non-uniform array of “h” values to create the lookup table. This is another reason to understand the function you are approximating, instead of blindly placing values into the table.

If you want to understand more about how lookup tables work, or don’t want to install SciPy, check out this interpolated lookup table class. I wouldn’t use it for serious math, since it doesn’t use Numpy and is probably not very fast.

This time, I made use of a site.cfg file. Copy the file site.cfg.example to site.cfg and edit. At the end of the file, uncomment the [mkl] section and set the path to your library. Mine looks like:

[mkl]
library_dirs = /opt/intel/mkl/10.0.1.014/lib/em64t/
lapack_libs = mkl_lapack
mkl_libs = mkl, guide

Configure the build with the following command. I’m not 100% sure that you need to specify the compilers at this point, but it never hurts to be consistent:

python setup.py config --compiler=intel --fcompiler=intel

After configuration, build the library:

python setup.py build --compiler=intel --fcompiler=intel --verbose > install_output.txt

Finally, install in my home directory since I don’t have admin privileges on this cluster:

python setup.py install --home=~

This seems like a better method than the one I used before. It doesn’t require typing such a long command line. It helps that the site.cfg.example file from numpy-1.2.1 is a little easier to follow.

<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian">
<Collection>
<DataSet timestep="131" group="" part="0" file="particles_step131.vtu"/>
<DataSet timestep="417" group="" part="0" file="particles_step417.vtu"/>
<DataSet timestep="923" group="" part="0" file="particles_step923.vtu"/>
<DataSet timestep="1744" group="" part="0" file="particles_step1744.vtu"/>
<DataSet timestep="5113" group="" part="0" file="particles_step5113.vtu"/>
<DataSet timestep="17613" group="" part="0" file="particles_step17613.vtu"/>
<DataSet timestep="29999" group="" part="0" file="particles_step29999.vtu"/>
</Collection>
</VTKFile>

You can read in a single .pvd file from Paraview, and all the timestep files will automatically be read in.

In the near future, I plan to publish a Python class that handles the details of generating valid XML, which you can use as an example or as a foundation to create more advanced VTK output.

Python handles integer math differently than floating point math.  If you type a number without a decimal point, Python treats it as an integer.  All math performed only with integers results in integers. For example, 1/2 evaluates to 0 while 1./2. evaluates to 0.5.  If you mix integers and floats, Python will  produce a floating point result (1/2.=0.5), but you must be very careful.  For example, you might expect the expression 4/3*3.14159 to yield a floating point result.  It does yield a floating point number, but not the one you were expecting!  4/3*3.14159 yields 3.14159.  What happened?  Python works from left to right.  4/3 evaluates to the integer “1″.  1*3.14159 evaluates to  3.14159.  For comparison, 4./3.*3.14159 evaluates to 4.1887866.  Here’s the problem with this particular aspect of Python: according to the rules of math, 4/3*3.14159 is exactly the same expression as 4*3.14159/3, but in Python they yield different results if you forget the decimal points!  4*3.14159 evaluates to a floating point, so (4*3.14159)/3 yields the “correct” floating point value.

Lesson Learned: be explicit about specifying all floats if you are doing floating-point math!  Sometimes I get lazy and leave a trailing decimal point off of a number when doing a floating point calculation, knowing that the results are “upcast” into floats. Not any more!

Note: this unexpected behavior goes away in Python 3.0

Like all things Python, profiling is easier than you think.  Python 2.4 has the profile module, and Python 2.5 has both profile and cProfile.  cProfile is written in C for lower overhead, and it’s the recommended version.  I am stuck with profile, because the cluster that I am working with still uses Python 2.4.  You can read the docs for more details, but I will quickly outline what I find to be the most helpful usage.  For example, say I want to profile the file run_sim.py.  I use the following command line:

python -m profile -o profile_file run_sim.py

The argument -m tells Python to run the library module profile as a script.  The argument -o is an option that tells profile to write the profile data to the specified file name.  The last argument, of course, is the name of the script to analyze.  Once the run is finished,  you can analyze the results.  I prefer to do this interactively, and I highly recommend the iPython shell.  The following is an example of an interactive session in iPython.

In [1]: import pstats
In [2]: p = pstats.Stats('profile_file')
In [3]: p.sort_stats('cumulative').print_stats(10)
Fri Oct 24 17:18:37 2008    profile_file

         278914972 function calls (277313577 primitive calls) in 4618.950 CPU seconds

   Ordered by: cumulative time
   List reduced from 550 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000 4618.950 4618.950 profile:0(execfile('run_sim.py'))
        1   12.740   12.740 4618.950 4618.950 run_sim.py:3(?)
        1    0.000    0.000 4618.950 4618.950 :0(execfile)
    129/1    0.000    0.000 4618.950 4618.950 <string>:1(?)
   100000 1416.240    0.014 4513.100    0.045 ~/BrownianDynamics.py:212(timeStep)
 12130610 1144.580    0.000 1620.700    0.000 ~/Brownian_Dynamics/trunk/BrownianDynamics.py:105(hasXYZCollision)
 36484167  497.980    0.000  779.550    0.000 /usr/lib64/python2.4/random.py:506(gauss)
   200000   22.760    0.000  673.340    0.003 ~/lib64/python/scipy/integrate/quadrature.py:182(simps)
   300000  545.730    0.002  637.640    0.002 ~/lib64/python/scipy/integrate/quadrature.py:152(_basic_simps)

This is the profile of a Brownian Dynamics simulation.  I have sorted the results by cumulative time, to see what parts of the code take longest to run.  The first five “calls” are not that useful–they show the profiler calling the “master” script, which in turn calls a “slave” script that actually does the work.  The fifth call is where things get interesting–it shows the runtime of the script that actually does the computations.  The sixth call is to the method “hasXYZCollision” within that script.  Clearly, this is the most time-consuming part of the code, which is why I have spend the past few posts optimizing it!  The next call is to a random number generator–not much I can do about that right now.  The final two calls demonstrate that you have to read the profile carefully.  The function “simps” calls the function “_basic_simps”, so the total time spend doing numerical integration is actually 673 seconds, not 673+637 seconds.

I have found the Python profiler to be extremely useful.  Since “premature optimization is the root of all evil,” the profiler is an necessary tool to find and improve those bits of code that are truly inefficient.

• Delete the numpy-1.0.4/build directory after every build attempt.  Doing “python setup.py clean” is not effective.  I kept getting errors about undefined symbols when I tried to “import numpy” on the Python command line.  It was looking for symbols in the old locations, even though I had just rebuilt the code using the new library locations.  It turned out that I needed to delete the build directory in order to force a complete bottom-up rebuild.
• The use of “setup.py” from distutils is not well documented online.  The best thing to do is run “python setup.py –help-commands” to get a list of available commands.  Then run “python setup.py <cmd> –help” to get help for that specific command.  You can string commands together on the command line, as I will show in the example below.
• When you test the new numpy, make sure you are not in the numpy-1.0.4 directory!  If you are in the numpy source directory, when you import numpy, you will get the message “Running from numpy source directory.” and you will not be able to load any symbols from numpy.
• On 64-bit architectures, you need to compile position-independent library code.  For some reason, distutils does not do this automatically, and the compilation will fail with an error similar to the following:

  relocation R_X86_64_PC32 against `_various_library_symbols'
  can not be used when making a shared object; recompile with -fPIC

  Edit the file numpy-1.0.4/numpy/distutils/intelccompiler.py.  Change the line cc_exe=”icc” to cc_exe=”icc -fPIC” (line 11 in my version of numpy).

• Make sure your LD_LIBRARY_PATH is pointing at the right location, or the compiled code won’t be able to find shared libraries.

Here is the command line I used to build numpy:

python setup.py config –library-dirs=/apps/intel/mkl/10.0.1.014/lib/em64t/ –library-dirs=/apps/intel/cce/10.1.008/lib/ –compiler=intel –fcompiler=intel build –verbose install –home=~ > install_output.txt

Explanation: “config” is a command that accepts arguments –library-dirs to set the path where icc searches for shared libraries, –compiler to set the C compiler, and –fcompiler to set the Fortran compiler.  Technically, I think you can omit the “build” command.  The “install” command installs numpy, and the –home=~ installs it in the user directory (I don’t have admin privileges on this machine).  Finally, > install_output.txt redirects the lengthy output to a text file for debugging purposes.

I hope this saves somebody some time!

    d2 = (x-self.x[0:i])*(x-self.x[0:i]) + (y-self.y[0:i])*(y-self.y[0:i]) + (z-self.z[0:i])*(z-self.z[0:i])

For some reason, I used the code (x-self.x)*(x-self.x) instead of (x-self.x)**2.  Upon further reflection, I realized that (x-self.x)*(x-self.x) computes the difference between array elements twice, and then multiplies the results.  Using a “power function” should enable the interpreter to compute the difference only once, and then multiply each element times itself.  Here is the updated code, using Python’s power operator:

d2 = (x-self.x[0:i])**2 + (y-self.y[0:i])**2 + (z-self.z[0:i])**2

The previous version of the code ran in 46.3 seconds on a representative problem–the updated version ran in only 35.8 seconds on the same problem.  That’s a pretty good improvement.  Out of curiosity, I decided to try Numpy’s power() function in place of Python’s power operator:

two = i
d2 = power(x-self.x[0:i],two) + power(y-self.y[0:i],two) + power(z-self.z[0:i],two)nt_(2)

This version took 55.5 seconds to execute.  Using a Numpy integer scalar instead of a Python integer made no difference.  Numpy’s power() function is quite sophisticated–it can take two array arguments and compute element-by-element powers.  I suspect that this flexibility results in some overhead that slows it down compared to Python’s built-in power operator.

On a side note: you should be aware that haze can also damage moving light fixtures that use a fan to cool the power supply or motors.  The fan sucks in haze and coats the internal components with a film of oil.  Eventually, something overheats and the fixture can actually catch fire.  Fortunately, we have older High End Studiospots and Trackspots that apparently don’t use forced-air cooling, so apparently the haze doesn’t really get inside.  Be warned!

# Oldest, slowest method
for j in range(0,i):
    if ((x-self.x[j])*(x-self.x[j]) + (y-self.y[j])*(y-self.y[j]) + (z-self.z[j])*(z-self.z[j])) < R_squared:
        return True
for j in range(i+1,len(self.x)):
    if ((x-self.x[j])*(x-self.x[j]) + (y-self.y[j])*(y-self.y[j]) + (z-self.z[j])*(z-self.z[j])) < R_squared:
        return True
return False

This code starts with lists of the coordinates (self.x, self.y, self.z) of identical spheres in three-dimensional space.  A sphere “i” is moved to a new location (x,y,z), and we need to know if it collides with any of the other spheres.  This algorithm uses the Pythagorean Theorem to compute the center-to-center distance between spheres.  If this distance is less than twice the sphere radius, then the new sphere intersects with one of the old ones, so we return “True,” otherwise return “False.”  We loop through every sphere in the list (except for the current one) and test for a collision.  Now, we will perform the same operation using Numpy:

if i>0:
    d2 = (x-self.x[0:i])*(x-self.x[0:i]) + (y-self.y[0:i])*(y-self.y[0:i]) + (z-self.z[0:i])*(z-self.z[0:i])
    if min(d2) < R_squared:
        return True
if i+1 < len(self.x):
    d2 = (x-self.x[i+1:len(self.x)])*(x-self.x[i+1:len(self.x)]) + (y-self.y[i+1:len(self.x)])*(y-self.y[i+1:len(self.x)]) + (z-self.z[i+1:len(self.x)])*(z-self.z[i+1:len(self.x)])
    if min(d2) < R_squared:
        return True

The basic collision detection test remains the same, but now the math is implemented using Numpy operaters on Numpy arrays.  Notice that the loops are gone–Numpy operators can operate element-by-element on an array so that an entire “for” loop can be written in one line.  Not only is the syntax cleaner, but it in this case, it sped up the code by almost a factor of ten!  Numpy is written in C and carefully optimized to perform fast operations on arrays.  When you need to operate on a whole array, the operations take place at the speed of compiled C.  Note that I had to add some “if” statements to avoid creating zero-element arrays for the cases i=0 and i=len(array)-1.

I’m going to speculate on another reason why Numpy might be so much faster: the compiled Numpy code can take advantage of parallel execution in the CPU.  Modern CPUs have some level of instruction-level parallel execution, even within a single core.  A good compiler will find sections of code that don’t have to run sequentially, and compile them in a way that takes advantage of this.  The collision detection is great example of code that can be executed in parallel–whether or not sphere “i” collides with sphere “i-1″ has nothing to do with whether or not it collides with sphere “i+1.”

Here is an extensive example of how to optimize numerical operations in Python.