<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	>

<channel>
	<title>shocksolution.com: scientific computing, modeling, and simulation</title>
	<atom:link href="http://www.shocksolution.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.shocksolution.com</link>
	<description>Modeling, simulation, and engineering</description>
	<pubDate>Wed, 18 Aug 2010 23:34:01 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.7.1</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Using Microsoft Word for Technical Documents</title>
		<link>http://www.shocksolution.com/2010/08/18/using-microsoft-word-for-technical-documents/</link>
		<comments>http://www.shocksolution.com/2010/08/18/using-microsoft-word-for-technical-documents/#comments</comments>
		<pubDate>Wed, 18 Aug 2010 23:30:34 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Scientific computing]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=438</guid>
		<description><![CDATA[Microsoft Word is not the best tool for doing technical writing.  However, sometimes we are required to use Word because we need to collaborate with others who want to use Word.  Right now, I am using Word 2007  on Windows XP to write several mathematical papers.  In general, it is a big improvement from previous [...]]]></description>
			<content:encoded><![CDATA[<p>Microsoft Word is not the best tool for doing technical writing.  However, sometimes we are required to use Word because we need to collaborate with others who want to use Word.  Right now, I am using Word 2007  on Windows XP to write several mathematical papers.  In general, it is a big improvement from previous versions.  The new integrated equation editor is outstanding&#8211;except for the major bug I&#8217;ll discuss below.  Here is a brief &#8220;FAQ&#8221; you will want to read if you are using Word for technical writing.</p>
<p><strong>Q:</strong> How do I enter multi-line equations in Word 2007?</p>
<p><strong>A: </strong>Press Shift-Enter where you want a line break to appear</p>
<p><strong>Q:</strong> How do I prevent a page break from splitting a table into two parts?</p>
<p><strong>A:</strong> Select the table.  On the Home tab, click the little box in the lower-right corner of the Paragraph box.  On the paragraph dialog, choose the &#8220;Lines and Page Breaks&#8221; tab.  Check the box for &#8220;Keep Lines Together.&#8221;</p>
<p><strong>Q:</strong> How do I automatically number headings in Word 2007?  For example: Section 1, Subsection 1.1, 1.2, 1.3, Section 2, etc.</p>
<p><strong>A:</strong> It&#8217;s not obvious.  <a title="Numbering headings in Word 2007" href="http://www.dummies.com/how-to/content/numbering-headings-in-word-2007-multilevel-lists0.html">See this page from dummies.com about number headings</a>.</p>
<p><strong>Q:</strong> In Word 2007, why do equations sometimes appear as blank spaces or question marks when I print or save my document as a PDF file?</p>
<p><strong>A.</strong> This occurs when Word is installed on Windows XP Pro.  See the following Microsoft tech support item to find out how to install missing scripts:</p>
<p><a title="Microsoft Support Item 960985" href="http://support.microsoft.com/kb/960985" target="_blank">The characters in an equation are not printed</a>&#8230;</p>
<p>You may also have an outdated printer driver:</p>
<p><a title="Microsoft Support Item 920228" href="http://support.microsoft.com/kb/920228">Microsoft Support Item 920228</a></p>
<p><strong>Q:</strong> Why do equations created in Word 2007 disappear when I open the document in Word 2008 for Mac?</p>
<p><strong>A.</strong> <a title="Word for Mac does not support Word 2007 equations" href="http://tinyurl.com/2knb6l">Word for Mac does not support equations written in the new Word 2007 equation editor</a>. Unfortunately, neither does PowerPoint 2007 on the PC.  You can work around this by inserting equations into Word the old-fashioned way: go to the Insert tab, click on Object (found in the &#8220;Text&#8221; box towards the right side of the tab), and choose &#8220;Microsoft Equation 3.0&#8243; from the list in the dialog box.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/08/18/using-microsoft-word-for-technical-documents/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Reading an array from a text file with Fortran 90/95</title>
		<link>http://www.shocksolution.com/2010/05/21/reading-an-array-from-a-text-file-with-fortran-9095/</link>
		<comments>http://www.shocksolution.com/2010/05/21/reading-an-array-from-a-text-file-with-fortran-9095/#comments</comments>
		<pubDate>Fri, 21 May 2010 20:59:26 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=428</guid>
		<description><![CDATA[If you&#8217;re used to coding in more modern languages, Fortran I/O can seem a little bizarre.  Strings in Fortran are much more difficult to work with, since they are fixed-length rather than null-terminated.  The following example illustrates a simple way to read an array of numbers from a text file when the array length [...]]]></description>
			<content:encoded><![CDATA[<p>If you&#8217;re used to coding in more modern languages, Fortran I/O can seem a little bizarre.  Strings in Fortran are much more difficult to work with, since they are fixed-length rather than null-terminated.  The following example illustrates a simple way to read an array of numbers from a text file when the array length is unknown at compile time.</p>
<pre class="brush: cpp">
program io_test
      real, dimension(:), allocatable :: x
      integer :: n

      open (unit=99, file=&#039;array.txt&#039;, status=&#039;old&#039;, action=&#039;read&#039;)
      read(99, *), n
      allocate(x(n))
      read(99,*) x

      write(*,*) x
end
</pre>
<p>Here is the text file that the array is read from.  The integer on the first line is the number of elements to read from the next line.</p>
<pre class="brush: text">
10
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
</pre>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/05/21/reading-an-array-from-a-text-file-with-fortran-9095/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Managing a pool of MPI processes with Python and Pypar</title>
		<link>http://www.shocksolution.com/2010/04/17/managing-a-pool-of-mpi-processes-with-python-and-pypar/</link>
		<comments>http://www.shocksolution.com/2010/04/17/managing-a-pool-of-mpi-processes-with-python-and-pypar/#comments</comments>
		<pubDate>Sat, 17 Apr 2010 18:52:13 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=403</guid>
		<description><![CDATA[MPI is a standard for communication between multiple processes in parallel computing.  These processes can be running on different cores, CPUs, or entirely different computers in a grid.  MPI is a standard, and there are many implementations available (many are open source).  The features of MPI can be accessed from Python with [...]]]></description>
			<content:encoded><![CDATA[<p>MPI is a standard for communication between multiple processes in parallel computing.  These processes can be running on different cores, CPUs, or entirely different computers in a grid.  MPI is a standard, and there are many implementations available (many are open source).  The features of MPI can be accessed from Python with the packages <a title="pypar" href="http://code.google.com/p/pypar/" target="_blank">pypar</a> and <a title="mpi4py" href="http://mpi4py.scipy.org/" target="_blank">mpi4py</a>.  Here I present a Python script that implements a &#8220;process pool&#8221; <a title="Design patterns on Wikipedia" href="http://en.wikipedia.org/wiki/Design_pattern_(computer_science)">design pattern</a> using pypar.  I have observed a pattern often enough in my own work that I wrote this framework to avoid reinventing the wheel every time I come across it.</p>
<p>This pattern is useful for any <a title="Embarassingly Parallel" href="http://en.wikipedia.org/wiki/Embarrassingly_parallel">embarrassingly parallel</a> problem.  This describes a computing task that can be easily accelerated by running multiple parallel processes that do not not need to interact with one another.  For example, I have large scientific data sets from several runs of an experiment that need to be analyzed.  Since the data from each run can be analyzed independently from the other runs, I can analyze all the data sets at once on a parallel machine.   The code below implements a &#8220;master-worker&#8221; paradigm that requires at least three processes to accelerate the calculation.  The first process be comes the master, which does no calculation but hands out tasks to the workers.  The rest of the processes are workers, which receive a chunk of work, finish it, return the result to the master process, and then wait for more work.</p>
<pre>
<pre class="brush: python">#!/usr/bin/env python
from numpy import *
import pypar
import time

# Constants
MASTER_PROCESS = 0
WORK_TAG = 1
DIE_TAG = 2

MPI_myID = pypar.rank()

### Master Process ###
if MPI_myID == MASTER_PROCESS:
    num_processors = pypar.size()
    print &quot;Master process found &quot; + str(num_processors) + &quot; worker processors.&quot;

    # Create a list of dummy arrays to pass to the worker processes
    work_size = 10
    work_array = range(0,work_size)
    for i in range(len(work_array)):
        work_array[i] = arange(0.0, 10.0)

    # Dispatch jobs to worker processes
    work_index = 0
    num_completed = 0

    # Start all worker processes
    for i in range(1, min(num_processors, work_size)):
        pypar.send(work_index, i, tag=WORK_TAG)
        pypar.send(work_array[work_index], i)
        print &quot;Sent work index &quot; + str(work_index) + &quot; to processor &quot; + str(i)
        work_index += 1

    # Receive results from each worker, and send it new data
    for i in range(num_processors, work_size):
        results, status = pypar.receive(source=pypar.any_source, tag=pypar.any_tag, return_status=True)
        index = status.tag
        proc = status.source
        num_completed += 1
        work_index += 1
        pypar.send(work_index, proc, tag=WORK_TAG)
        pypar.send(work_array[work_index], proc)
        print &quot;Sent work index &quot; + str(work_index) + &quot; to processor &quot; + str(proc)

    # Get results from remaining worker processes
    while num_completed &lt; work_size-1:
        results, status = pypar.receive(source=pypar.any_source, tag=pypar.any_tag, return_status=True)
        num_completed += 1

    # Shut down worker processes
    for proc in range(1, num_processors):
        print &quot;Stopping worker process &quot; + str(proc)
        pypar.send(-1, proc, tag=DIE_TAG)

else:
    ### Worker Processes ###
    continue_working = True
    while continue_working:

        work_index, status =  pypar.receive(source=MASTER_PROCESS, tag=pypar.any_tag, \
                return_status=True)

        if status.tag == DIE_TAG:
            continue_working = False
        else:
            work_array, status = pypar.receive(source=MASTER_PROCESS, tag=pypar.any_tag, \
                return_status=True)
            work_index = status.tag

            # Code below simulates a task running
            time.sleep(random.random_integers(low=0, high=5))
            result_array = work_array.copy()

            pypar.send(result_array, destination=MASTER_PROCESS, tag=work_index)
    #### while
#### if worker

pypar.finalize()</pre>
</pre>
<div><span style="font-family: 'Courier New', monospace; font-size: small;"><span><br />
</span></span></div>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/04/17/managing-a-pool-of-mpi-processes-with-python-and-pypar/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Constrained least-squares fitting with Python</title>
		<link>http://www.shocksolution.com/2010/03/05/constrained-least-squares-fitting-with-python/</link>
		<comments>http://www.shocksolution.com/2010/03/05/constrained-least-squares-fitting-with-python/#comments</comments>
		<pubDate>Fri, 05 Mar 2010 19:27:25 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Python]]></category>

		<category><![CDATA[Scientific computing]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=399</guid>
		<description><![CDATA[Scipy contains a good least-squares fitting routine, leastsq(), which implements a modified Levenberg-Marquardt algorithm.  I just learned that it also has a constrained least-squared routine called fmin_slsqp().   I am using simple upper and lower bound constraints, but it&#8217;s also possible to specify more complex functional constraints.
What I did not realize, at first, is that fmin_slsqp [...]]]></description>
			<content:encoded><![CDATA[<p><a title="SciPy home" href="http://scipy.org">Scipy</a> contains a good least-squares fitting routine, <a title="scipy.optimize.leastsq" href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html#scipy-optimize-leastsq"><strong>leastsq()</strong></a>, which implements a modified Levenberg-Marquardt algorithm.  I just learned that it also has a constrained least-squared routine called <a title="scipy.optimize.fmin_slsqp" href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html#scipy.optimize.fmin_slsqp" target="_blank"><strong>fmin_slsqp()</strong></a>.   I am using simple upper and lower bound constraints, but it&#8217;s also possible to specify more complex functional constraints.</p>
<p>What I did not realize, at first, is that<strong> fmin_slsqp</strong> requires a different type of objective function than <strong>leastsq</strong>.   <strong>leastsq</strong> requires you to write a function that returns a vector of residuals, and <strong>leastsq</strong> automatically squares and sums the residuals.  <strong>fmin_slsqp</strong> is actually more flexible, in that it can use any objective function that returns a single scalar value.  To implement least-squares curve fitting, your objective function will need to find the residual at each data point, square the values, and sum them up.  Hopefully this tip will save you some time.</p>
<p>Check out the <a title="scipy optimization tutorial" href="http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html">scipy optimization tutorial</a> for more examples.  Here is the original <a title="SLSQP reference" href="http://abs-5.me.washington.edu/dynOpt/p262-kraft.pdf">paper by Dieter Kraft</a> which introduces the algorithm used by fmin_slsqp.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/03/05/constrained-least-squares-fitting-with-python/feed/</wfw:commentRss>
		</item>
		<item>
		<title>A preliminary review of the Lehigh Rendition console</title>
		<link>http://www.shocksolution.com/2010/01/17/lehigh-rendition-console-review/</link>
		<comments>http://www.shocksolution.com/2010/01/17/lehigh-rendition-console-review/#comments</comments>
		<pubDate>Sun, 17 Jan 2010 17:35:09 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Stage Lighting]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=387</guid>
		<description><![CDATA[In my last post, I explained why I don&#8217;t think the ETC Element is a good replacement for the ETC Express.  When I was at LDI 2009, I ran across the Rendition series of consoles from Lehigh Lighting.   Based on the literature I picked up, and the extensive demonstration that I received from the [...]]]></description>
			<content:encoded><![CDATA[<p>In my last post, I explained why I don&#8217;t think the ETC Element is a good replacement for the ETC Express.  When I was at LDI 2009, I ran across the <a title="Lehigh Rendition" href="http://www.lehighdim.com/rendition.htm">Rendition</a> series of consoles from <a href="http://www.lehighdim.com/" target="_blank">Lehigh Lighting</a>.   Based on the literature I picked up, and the extensive demonstration that I received from the Lehigh representative, it looks like the Rendition might be the true heir to the Express.  It is targeted at a similar market: small theaters, churches, and schools.  The physical layout will be familiar to anyone with theater console experience: 24 or 48 submasters on the left,  dual cue playback controls in the center, and hardkeys on the right, with channel faders (48 or 96) along the upper right portion of the desk.  The submasters are traditional theater-style subs.  Each sub records a fixed look rather than an independent cuelist.  Shows can be saved to a flash drive via a USB port.<br />
<a title="Lehigh Rendition 24/48 by Oanjao, on Flickr" href="http://www.flickr.com/photos/a_mirror_dimly/4281567195/"><img src="http://farm3.static.flickr.com/2694/4281567195_952c07ffc0.jpg" alt="Lehigh Rendition 24/48" width="500" height="183" /></a><br />
While the console hardware is finalized, several additional features will be added in an upcoming version of the operation system.  The console currently supports one external monitor, but dual monitors will soon be supported.  An Ethernet port is already included on the console.  In a future software release, it will be possible to add wings with additional submasters or channel faders.  The advantage of Ethernet over USB is that a wing can be placed hundreds of feet from the console, and connected using existing Ethernet lines.  This feature would allow a wing to to be used as remote focus unit.  Using a wireless Ethernet bridge opens up even more possibilities.</p>
<p>The main difference between the two models, the 24/48 and 48/96, is the number of hardware faders.  The software capabilities are identical.   The price of the console depends on how many channels of conventional dimming are &#8220;unlocked.&#8221;  You can choose from 125, 250, or 512 conventional channels.  All models support 1024 channels of moving-light control.  The two hardware DMX ports each support 512 channels, but two additional universes can be accessed via Ethernet (I&#8217;m not sure of the details on how this works).  The console seems to support all the basic and advanced cue and effect functions that you&#8217;d expect: multiple cue lists, flexible cue timing options, macros, subroutines, and effects.  I can&#8217;t really comment further about that, since I would need to spend a lot of time with the console to accurately gauge its reliability and ease of use.  Moving lights are also supported, with fixture profiles for easy patching and a trackball for focusing.  Once again, it looks promising, but the only way to really evaluate these features is to try to use them and see how intuitive the process is.  <a title="Lehigh Rendition offline editor" href="http://www.lehighdim.com/offline_editors.htm">Download the offline editing software and try it for yourself</a>.</p>
<p>A more sophisticated console called the Rendition Pro is expected to be available in the second quarter of 2010.  The Pro is aimed at more experienced users, and appears to compete with the ETC Eos and Ion.  It will run the same software as the Rendition, so many of the features that are expected to be added to the Rendition are actually being developed for the Pro.  The layout is similar to the Rendition, with playback masters on the left side, traditional cue playback controls in the lower center, and keys to the lower right. Two LCD displays with encoder wheels and softkeys occupy the space above the keypad, with no hard channel faders.  Further, the submasters  have LCD displays and additional buttons&#8211;probably &#8220;Go&#8221; and &#8220;Pause&#8221;  buttons to support a cue stack on each sub.</p>
<p>From my limited experience, both of these consoles series are worthy of further investigation.  They seem to be solidly built and well-engineered.  The Lehigh personnel that I spoke to were helpful and friendly, and I appreciate their assistance in answering my questions.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/01/17/lehigh-rendition-console-review/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Storing large Numpy arrays on disk: Python Pickle vs. HDF5</title>
		<link>http://www.shocksolution.com/2010/01/10/storing-large-numpy-arrays-on-disk-python-pickle-vs-hdf5adsf/</link>
		<comments>http://www.shocksolution.com/2010/01/10/storing-large-numpy-arrays-on-disk-python-pickle-vs-hdf5adsf/#comments</comments>
		<pubDate>Sun, 10 Jan 2010 23:00:20 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Python]]></category>

		<category><![CDATA[Scientific computing]]></category>

		<category><![CDATA[Software development]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=381</guid>
		<description><![CDATA[In a previous post, I described how Python&#8217;s Pickle module is fast and convenient for storing all sorts of data on disk. More recently, I showed how to profile the memory usage of Python code.  In recent weeks, I&#8217;ve uncovered a serious limitation in the Pickle module when storing large amounts of data: Pickle requires [...]]]></description>
			<content:encoded><![CDATA[<p><a title="Python data storage with Pickle" href="http://www.shocksolution.com/2008/09/15/python-pickle-painless-binary-storage-for-python-objects/" target="_blank">In a previous post, I described how Python&#8217;s Pickle module is fast and convenient</a> for storing all sorts of data on disk. More recently, I showed <a title="Profiling Python code" href="http://www.shocksolution.com/2009/04/17/profiling-memory-usage-of-python-code/" target="_blank">how to profile the memory usage of Python code</a>.  In recent weeks, I&#8217;ve uncovered a serious limitation in the Pickle module when storing large amounts of data: Pickle requires a large amount of memory to save a data structure to disk. Fortunately, there is an open standard called HDF, which defines a binary file format that is designed to efficiently store large scientific data sets. I will demonstrate both approaches, and profile them to see how much memory is required. I am writing the HDF file using the PyTables interface. Here&#8217;s the little test program I&#8217;ve been using:</p>
<pre><code>#!/usr/bin/env python

from numpy import *

array_len = 10000000

a = zeros(array_len, dtype=float)

#import pickle
#f = open('test.pickle', 'wb')
#pickle.dump(a, f, pickle.HIGHEST_PROTOCOL)
#f.close()

import tables
h5file = tables.openFile('test.h5', mode='w', title="Test Array")
root = h5file.root
h5file.createArray(root, "test", a)
h5file.close()</code></pre>
<p>I first ran the program with both the pickle and the HDF code commented out, and profiled RAM usage with Valgrind and Massif (see my post about profiling memory usage of Python code). Massif shows that the program uses about 80MB of RAM. I then uncommented the Pickle code, and profiled the program again. Look at how the memory usage almost triples!</p>
<pre> MB
232.2^                                                                  #
  |                                                                  #
  |                                                                  #
  |                                                                  #
  |                                                                  #
  |                                                                  #
  |                                                                  #
  |                                                         @        #
  |                                                         @        #
  |                                                         @        #
  |                                                         @        #
  |                                                         @        #
  |                                                         @        #
  |                                                ,        @        #  .
  |                                                @        @        #  :
  |                                                @        @        #  :
  |                                                @        @        #  :
  |                                                @        @        #  :
  |                                                @        @        #  :
  |                                                @        @        #  :
0 +-----------------------------------------------------------------------&gt;Mi
  0                                                                   161.9</pre>
<p>I then commented out the Pickle code and uncommented the HDF code, and ran the profile again. Notice how efficient the HDF library is:</p>
<pre>    MB
81.62^                              ,.., .....,.. .,...,... .,......,..,:  .
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |                              @::@ :::::@:: :@:::@::: :@::::::@::#:  :
   |              .,..,. .....    @::@ :::::@:: :@:::@::: :@::::::@::#:  :::
 0 +-----------------------------------------------------------------------&gt;Mi
   0                                                                   246.6</pre>
<p>I didn&#8217;t benchmark the speed because, for my application, it doesn&#8217;t really matter, because the data gets dumped to disk relatively infrequently. It&#8217;s interesting to note that the data files on disk are almost identical in size: 76.29MB.</p>
<p>Why does Pickle consume so much more memory? The reason is that HDF is a binary data pipe, while Pickle is an object serialization protocol. Pickle actually consists of a simple virtual machine (VM) that translates an object into a series of opcodes and writes them to disk. To unpickle something, the VM reads and interprets the opcodes and reconstructs an object. The downside of this approach is that the VM has to construct a complete copy of the object in memory before it writes it to disk. I&#8217;m not sure why the pickle operation in my example code seems to require three times as much memory as the array itself. 99% of the time, memory usage isn&#8217;t a problem, but it becomes a real issue when you have hundreds of megabytes of data in a single array. Fortunately, HDF exists to efficiently handle large binary data sets, and the Pytables package makes it easy to access in a very Pythonic way.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2010/01/10/storing-large-numpy-arrays-on-disk-python-pickle-vs-hdf5adsf/feed/</wfw:commentRss>
		</item>
		<item>
		<title>My article published in Technologies for Worship Magazine</title>
		<link>http://www.shocksolution.com/2009/12/14/my-article-published-in-technologies-for-worship-magazine/</link>
		<comments>http://www.shocksolution.com/2009/12/14/my-article-published-in-technologies-for-worship-magazine/#comments</comments>
		<pubDate>Tue, 15 Dec 2009 04:49:09 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Stage Lighting]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=375</guid>
		<description><![CDATA[Head over to Technologies for Worship Magazine and check out my recently-published article about backlighting.  It&#8217;s my first publication in print (outside of technical journals).  I&#8217;m looking forward to writing more articles for a general audience!
]]></description>
			<content:encoded><![CDATA[<p>Head over to <a title="Technologies for Worship Magazine" href="https://www.tfwm.com/1109backlighting" target="_blank">Technologies for Worship Magazine</a> and check out my recently-published article about backlighting.  It&#8217;s my first publication in print (outside of technical journals).  I&#8217;m looking forward to writing more articles for a general audience!</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2009/12/14/my-article-published-in-technologies-for-worship-magazine/feed/</wfw:commentRss>
		</item>
		<item>
		<title>First impression: ETC Element Console Review</title>
		<link>http://www.shocksolution.com/2009/12/08/first-impression-etc-element-console-review/</link>
		<comments>http://www.shocksolution.com/2009/12/08/first-impression-etc-element-console-review/#comments</comments>
		<pubDate>Wed, 09 Dec 2009 04:12:04 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Stage Lighting]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=373</guid>
		<description><![CDATA[We recently got an ETC Element to replace an ETC Express 72/144 that had become unreliable due to a hardware problem.  I&#8217;d like to point out that ETC equipment is generally very durable, but this particular console only lasted about 8 years due to abuse by the high-school students who use the console during the [...]]]></description>
			<content:encoded><![CDATA[<p>We recently got an <a title="ETC Element" href="http://www.etcconnect.com/product.overview.aspx?ID=22010" target="_blank">ETC Element</a> to replace an <a title="ETC Express" href="http://www.etcconnect.com/product.overview.aspx?ID=20012&amp;lang=us&amp;region=1" target="_blank">ETC Express</a> 72/144 that had become unreliable due to a hardware problem.  I&#8217;d like to point out that ETC equipment is generally very durable, but this particular console only lasted about 8 years due to abuse by the high-school students who use the console during the week.  The Express was a great console.  It had a hardware fader for each of the channels that we commonly used, and a bank of 20 submasters.  Recording a submaster was as simple as creating a look onstage with the channel faders, then pressing &#8220;Record&#8221; and hitting the submaster bump button.  It had one external LCD screen for output, which had a very simple character-based interface.  Just about anyone could learn to use the Express.</p>
<p>The Element is, in many ways, a much more sophisticated console.  While the Express didn&#8217;t have any real capability to run moving lights, the Element has many features to facilitate the use of moving lights and programmable LED fixtures.  It supports two external LCD screens with a full graphical interface, a keyboard, and a mouse. Unfortunately, the Element has lost all of the simplicity that made the Express so successful in its target market.  Let&#8217;s start with the physical design.</p>
<p>The greatest disadvantage of the Element, especially for untrained users, is that all of the forty faders (60 in some models) are multi-function.  With the bank switch in position 1, the faders represent channels 1-40.  Position 2 changes the fader function to channels 41-80, and Position 3 changes the faders to channels 81-120.  Position 4 changes the fader function to submasters.  While it&#8217;s nice to have 40 subs, it&#8217;s also extremely confusing to figure out which channel fader or submaster is currently controlling the output.  Here&#8217;s an example.</p>
<p>Let&#8217; s say that you use faders 2, 17, and 31 to set a look on Bank 1.  If you switch to Bank 2, faders 2, 17, and 31 are still up, but they are not controlling anything.  Bring those faders down, and continue setting up the look on channels 41-80 using Bank 2.  Now switch back to Bank 1 to tweak the look on channels 1-40…but the faders are no longer up!  The LEDs on the bump buttons will flash to show which channel levels no longer match the fader positions.  To take control of these channels, bring the fader up until it matches the current channel level.  Then you can control the channel again.  Are you confused yet?  Did I mention that the faders have no numbers?  I find this slows me down when creating a look, and makes it harder to edit a look &#8220;on the fly.&#8221;  To top it off, the bank selector is a little plastic twist switch, and it&#8217;s hard to see which bank it&#8217;s pointing at.</p>
<p>With many consoles, you can work around the limited number of faders by adding an expansion wing with additional controls.  Unfortunately, the Element does not support expansions&#8211;you have to upgrade to the Ion to get that feature!  I&#8217;m not thrilled with the layout of the keys on the Element keyboard, either.  It&#8217;s difficult to guess where a key is going to be&#8211;they don&#8217;t seem to be grouped in any particular way.  Compare to the Hog 1000, in which the keys are grouped as &#8220;verbs&#8221; and &#8220;nouns&#8221; for command-line programming.  One final gripe about the hardware: it&#8217;s difficult to run my finger down the row of bump buttons to  find a particular light&#8211;this was easy to do on the old Express.</p>
<p>So, after all my complaining, I want to leave you with a great feature.  I love the &#8220;Exclusive&#8221; submaster setting that prevents the output from a sub from being recorded in subsequent cues and submasters.  I use this all the time to keep some lights up onstage or in the house without having them recorded into cues.  I&#8217;m sure there are more little gems built into the Express.</p>
<p>Here is my conclusion about the Element: it is NOT a replacement for the Express.  Do not expect this console to be a &#8220;drop-in&#8221; replacement for any traditional theater console.  The learning curve will be very steep for brand-new users, and slightly less steep for Express users.  In the end, the Element will be far more capable than the Express&#8211;but that&#8217;s not the target market for the Express.  If you are buying a console for a school, small church, or community theater where the users are generally inexperienced, there are probably better choices than the Element.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2009/12/08/first-impression-etc-element-console-review/feed/</wfw:commentRss>
		</item>
		<item>
		<title>LDI 2009 Report</title>
		<link>http://www.shocksolution.com/2009/11/22/ldi-2009-report/</link>
		<comments>http://www.shocksolution.com/2009/11/22/ldi-2009-report/#comments</comments>
		<pubDate>Mon, 23 Nov 2009 03:15:54 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Stage Lighting]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=362</guid>
		<description><![CDATA[I attended LDI (Live Design International) last Friday in Orlando, FL.  This was my second time at the show (read my report from 2007).  Overall, I&#8217;d say this year&#8217;s show was scaled back from 2007, which is not surprising considering how the economic recession has affected the entertainment industry.  There were fewer booths this year, [...]]]></description>
			<content:encoded><![CDATA[<p>I attended <a title="LDI" href="http://www.ldishow.com/LDI09/public/enter.aspx" target="_blank">LDI</a> (Live Design International) last Friday in Orlando, FL.  This was my second time at the show (<a title="LDI 2007 Report" href="http://www.shocksolution.com/2007/11/18/ldi-reportpart-1eye-candy/" target="_self">read my report from 2007</a>).  Overall, I&#8217;d say this year&#8217;s show was scaled back from 2007, which is not surprising considering how the economic recession has affected the entertainment industry.  There were fewer booths this year, and they weren&#8217;t as fancy.  A few manufacturers, such as Chauvet and Coemar, brought pretty spectacular booths.  Other companies brought large booths (such as ETC and Martin), but they were not as flashy.  I&#8217;m not sure if Friday is normally a slow day for this conference, but there wasn&#8217;t much of a crowd.  I didn&#8217;t have to wait in line to see anything.</p>
<p>In 2007, I went to LDI to try out control consoles to replace our Hog 1000.  That plan fell through when the economy tanked, so this year I was just looking around.  Apparently a lot of people were, because the sales people were rather aggressive about jumping onto any apparent interest in their product.  One girl tried to scan my badge as I walked by&#8211;I don&#8217;t even know what she was selling!  Actually, that&#8217;s a common occurrence at LDI&#8211;it&#8217;s hard to tell what the actual product is in the booth.  All the larger booths seemed to consist of truss, truss warmers, banners, drapes, moving lights, and video.  I often had to stop and look for a minute to figure out which one was actually their product!</p>
<p>I will post some photos when I get a chance.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2009/11/22/ldi-2009-report/feed/</wfw:commentRss>
		</item>
		<item>
		<title>Building and linking to a shared Fortran library</title>
		<link>http://www.shocksolution.com/2009/10/26/building-and-linking-to-a-shared-fortran-library/</link>
		<comments>http://www.shocksolution.com/2009/10/26/building-and-linking-to-a-shared-fortran-library/#comments</comments>
		<pubDate>Mon, 26 Oct 2009 22:34:02 +0000</pubDate>
		<dc:creator>craig</dc:creator>
		
		<category><![CDATA[Fortran]]></category>

		<category><![CDATA[Linux]]></category>

		<category><![CDATA[Scientific computing]]></category>

		<guid isPermaLink="false">http://www.shocksolution.com/?p=360</guid>
		<description><![CDATA[I&#8217;m using GNU Fortran (gfortran) to build several shared libraries, and then dynamically linking to them from a Fortran program.  The process is a little different than what I&#8217;m used to for C libraries, so I thought I&#8217;d explain it.  Unlike C, there is no need to #include header files when compiling code that relies [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;m using GNU Fortran (<a title="gFortran" href="http://gcc.gnu.org/onlinedocs/gfortran/index.html" target="_blank">gfortran</a>) to build several shared libraries, and then dynamically linking to them from a Fortran program.  The process is a little different than what I&#8217;m used to for C libraries, so I thought I&#8217;d explain it.  Unlike C, there is no need to #include header files when compiling code that relies on functions defined in an external library.  Likewise, there is no need to use -l or -L linking flags to tell the linker about s hared libraries (at least when they&#8217;re in the same directory).  In fact, the whole process requires a lot less command-line options than I had expected.</p>
<h2>Building the Fortran shared libraries</h2>
<p>I want to test a library called <strong>my_library</strong>.  However, this library relies upon another library, which is called <strong>cfdrc_user_access</strong>.  I want to compile a unit-test program that calls my library and makes sure it is working correctly.</p>
<pre>gfortran -shared -fPIC -o cfdrc_user_access.so cfdrc_user_access.f90
gfortran -shared -fPIC -o cfdrc_user.so my_library.f90</pre>
<h2>Linking the shared libraries with a Fortran program</h2>
<p>First, compile the program to an object file (.o), using the -c flag as shown on the first line.  Then use gfortran to link the object file with the shared libraries created in the previous step:</p>
<pre>gfortran -c blocking.f90
gfortran -o blocking blocking.o cfdrc_user.so cfdrc_user_access.so</pre>
<p>Remember, unless the shared libraries are in a directory that&#8217;s already on your  dynamic linker path, you  will probably need to <a title="about shared libraries" href="http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html" target="_blank">modify your LD_LIBRARY_PATH</a> in order to run the program.</p>
<pre>export LD_LIBRARY_PATH=/home/yourname/current_working_directory/
./blocking</pre>
<p>That worked for me&#8211;hopefully it helps you.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.shocksolution.com/2009/10/26/building-and-linking-to-a-shared-fortran-library/feed/</wfw:commentRss>
		</item>
	</channel>
</rss>
