Storing large Numpy arrays on disk: Python Pickle vs. HDF5
Sunday, January 10th, 2010 Posted in Python, Scientific computing, Software development | No Comments »In a previous post, I described how Python's Pickle module is fast and convenient for storing all sorts of data on disk. More recently, I showed how to profile the memory usage of Python code. In recent weeks, I've uncovered ...
f2py: binding Fortran and Python
Wednesday, September 23rd, 2009 Posted in Fortran, Python, Software development | No Comments »I have recently started using f2py to call Fortran from Python. I have found this useful for two reasons: speeding up Python scripts by calling compiled Fortran code, and using Python as a unit testing framework for Fortran modules. Unfortunately, ...
Profiling memory usage of Python code
Friday, April 17th, 2009 Posted in Linux, Python, Scientific computing, Software development | 2 Comments »In a previous post, I explained how to use the Python profiler. The profile is great for finding out which parts of the code run the slowest, or are called most often. However, the profiler doesn't give any information about ...
Lookup tables and spline fitting in Python
Wednesday, April 8th, 2009 Posted in Python, Scientific computing, Software development | No Comments »Lookup tables and spline fitting are widely used by scientific programmers. A particular function may not have an analytic solution--in other words, it can't be expressed as an equation of elementary functions. This might happen if the function were empirically ...
3D Plotting Software for Python::Part 1::PyX
Friday, March 20th, 2009 Posted in Python, Scientific computing, Software development | 3 Comments »There are lots of good open-source tools that you can use to make high-resolution, publication-quality 2D plots. Personally, I like to use Python, numpy, and matplotlib. Unfortunately, it is much harder to find a good tools to make 3D plots. ...
The Python configparser: a way to read simple data files
Tuesday, February 3rd, 2009 Posted in Linux, Software development | 1 Comment »My simulation library, which is written mostly in Python, needs a lot of data and parameters in order to run. In some cases, I just hard-code the values in the script that calls the library, and in other ...
How to put formatted, highlighted code in a Wordpress post
Saturday, January 17th, 2009 Posted in Software development, Web design | No Comments »I found two complementary plugins that enable me to put highlighted formatted code in a Wordpress page or post. Here's an example of what they do: [sourcecode language="python"] # Plot flux at continuum boundary pylab.figure() pylab.hold(True) pylab.plot(nd_times, Jl_BD, 'b-', label="Flux from BD simulation") pylab.plot(nd_times, Jl_BD+Jl_BD_std, 'b.', ...
Deploying Python applications on Windows
Thursday, January 15th, 2009 Posted in Linux, Python, Software development | No Comments »Writing applications in Python on a Linux system is almost too easy. Deploying Python apps on other Linux systems is not hard, because most Linux systems already have Python, with its core libraries and tools, installed. Most Linux systems also ...
Optimizing Python code for fast math
Friday, January 9th, 2009 Posted in Python, Scientific computing, Software development | 1 Comment »I spent some time today profiling a Brownian dynamics simulation written in Python to see how I could make it faster before starting some long runs on a Linux cluster. In the sections below, I have attempted to quantify ...
Redirecting standard output from Python: another example
Wednesday, January 7th, 2009 Posted in Python, Software development | No Comments »I wrote a previous post about how to redirect standard output from a Python script to a GUI window. In this post, I will give an even simpler example of to redirect standard output to a log file. During the ...
Tools for Python software development
Tuesday, January 6th, 2009 Posted in Linux, Python, Software development | No Comments »I have found a few tools over the years that I find extremely useful for developing software. Python is my language of choice at the moment, but I'm sure these tools will be handy for any language. Subversion is an ...
Using Python to generate XML files for visualization in Paraview
Thursday, November 13th, 2008 Posted in Linux, Python, Software development | No Comments »VTK is an open-source software system for "3D computer graphics, image processing, and visualization" developed by by Kitware. VTK is the foundation of Paraview, an industrial-strength CFD visualization tool that I have found to be very useful. I ...
Unexpected integer/float math behavior in Python
Thursday, November 6th, 2008 Posted in Python, Software development | No Comments »I wasted some time today tracking down a bug in one of my programs. It turned out to be "unexpected behavior" rather than a bug. I was aware of this aspect of the language, but I made an assumption and ...
Even faster collision detection in Python using Numpy
Thursday, October 16th, 2008 Posted in Python, Scientific computing, Software development | No Comments »Last night, in the shower, I realized that my collision detection routine could be even faster. Here is a representative snippet of code from my previous post: d2 = (x-self.x[0:i])*(x-self.x[0:i]) + (y-self.y[0:i])*(y-self.y[0:i]) + (z-self.z[0:i])*(z-self.z[0:i]) For some reason, I ...
Speeding up Python math with Numpy: collision detection example
Sunday, October 12th, 2008 Posted in Python, Software development | 1 Comment »Python is a very-high-level language. That makes it easy to write code quickly, but the program may not be as fast as a program compiled from a lower-level language. For this reason, many scientific programs are written in Fortran or ...
Python Pickle: Painless binary storage for Python objects
Monday, September 15th, 2008 Posted in Python, Software development | 2 Comments »The pickle module provided with Python is so useful that I'm surprised I haven't used it before. Pickle allows you to save an entire data structure (such as an object) to disk as a binary file in a effortless ...
Python threads are easy (with example)
Tuesday, August 19th, 2008 Posted in Software development | No Comments »It's remarkably easy to spawn a Python thread. However, before doing so, I caution you that a Python thread is not the same thing as an OS thread. Python threads run within the Python interpreter, but the Python interpreter always ...
Fun with threads in Python and wxPython
Monday, August 11th, 2008 Posted in Software development | No Comments »I have finally gotten back to programming in the last couple of days. Our project has finally started to generate a lot of data, so I've been refactoring and improving my code that reads data stored in LabView binaries. Today ...
Reading Labview binary files with Python
Wednesday, June 25th, 2008 Posted in Scientific computing, Software development | 4 Comments »My research group uses Labview 7.1 to write custom data acquisition (DAQ) software. I code everything else in Python, so I need to get data from Labview into Python for processing. Our DAQ program produces Labview binary files, so ...
Server move completed and general update
Sunday, June 22nd, 2008 Posted in Linux, Software development, Stage Lighting | No Comments »The site is back online after a server move. Actually I can't blame the server move for the downtime, because I went out of town and then had a lot of catching up to do, and as a result I ...
