1. If you just upgraded to Lion and haven’t done this yet, install Xcode from the Mac App Store.
2. Install gfortran prebuilt for Lion: http://r.research.att.com/gfortran-4.2.3.dmg
3. If you haven’t done this already, install and set up MacPorts.
4. Install OpenMPI via MacPorts with the command sudo port install openmpi. This can take a long time (hours), since it depends on also doing a separate MacPorts install of GCC 4.4. Starting this install is a good way to end the day.
5. Download SUNDIALS 2.4.0 (the whole package, sundials-2.4.0.tar.gz) from the SUNDIALS download page. Move it to wherever you like and extract it (double-clicking works fine).
6. Open a terminal window, cd into the SUNDIALS directory, and run the following (I set my CC to gcc for the sake of safety, even though Lion defaults to LLVM):

./configure CC=gcc F77=gfortran \
--prefix=/usr/local/sundials \
--enable-shared \
--with-mpicc=/opt/local/bin/openmpicc \
--with-mpif77=/opt/local/bin/openmpif77
make
sudo make install

That’s it; SUNDIALS should now be installed in /usr/local/sundials. You can use the prefix /usr/local if you like, but this way it’s conveniently kept separated from everything else.

To use it, it’s a matter of making sure that /usr/local/sundials/include is in your header search path, /usr/local/sundials/lib is in your library search path, and you give the linker -l[library name] (that’s a “hyphen lowercase ell”) for any libraries you need. For example, for the SUNDIALS example file cvsFoodWeb_ASAi_kry.c, I needed -lsundials_nvecserial -lsundials_cvodes

In Xcode, you can do this via the following steps:

• Make a new Application/Command-Line Tool project.
• Add your source code, e.g., cvsFoodWeb_ASAi_kry.c
• In the project settings, add /usr/local/sundials/include to Header Search Paths. Oddly, this causes Header Search Paths to remain empty, but a User Header Search Paths field appears with your desired path.
• Check Always Search User Paths
• Add your library flags, e.g., -lsundials_nvecserial -lsundials_cvodes, to Other Linker Flags.

If everything has gone according to plan, the code should now build and run successfully.

So far, these steps only work for serial code: when I get an MPI example compiling and running, I’ll post those steps as well.

Turns out this is really easy to do. In fact, they even do it on this tutorial page for JavaScript developers. But since Googling “pause processing.js” came up short and the relevant functions didn’t strike me as relevant in the reference docs, here’s what I came up with before I found the documentation:

<p>
  <button id="toggleBtn" onClick="toggle();">Pause</button>
</p>

<script type="text/javascript">
// If you attached weird "Processing language" code
// to a canvas with id "sketch":
var processing = Processing.getInstanceById("sketch");

// If you're doing everything in raw JavaScript like me,
// you probably did something like this:
var processing = new Processing( ... );
// ...

var looping = true;
function toggle()
{
  if(looping)
  {
    processing.noLoop();
    document.getElementById("toggleBtn").textContent = "Play";
  }
  else
  {
    processing.loop();
    document.getElementById("toggleBtn").textContent = "Pause";
  }
  looping = !looping;
}
</script>

This post is about my long-overdue first steps with Python, which, like Spanish, I have been on the verge of starting to learn for about six years.

First impression: my god, I should have done this six years ago. At last I have a go-to scripting language. Today I am (by and large) saying goodbye to bash, R, Matlab, and, when performance doesn’t much, Java/C++/Objective-C.

More details: the “multi-paradigm” buzz is legit. It’s a nice dynamic (Really Dynamic) object-oriented language, but functional constructs—which I use sparingly, not being a Functional Programming Genius—seem well done. And you never need to use objects at all: procedural style, just with functions, is also fine. It’s very expressive—lots of sophistication with few lines of code—but the code still comes out quite readable (depending, of course, on the author). In general, it seems to have lots of features without suffering from syntax excess. And there are lots of well-supported libraries, because apparently lots of people feel the way I’m starting to.

The useful stuff: although Python 3 itself seems ready to go, the package support isn’t quite there yet, so I’m going to stick with Python 2.7.1 for the time being.

Beyond Python, I think I’ll need four basic packages to start with: numpy, scipy, matplotlib, and rpy2.

To do numerical work—the kind of things many people use Matlab for—you’ll want numpy and scipy, which seem very mature and use fast C libraries like LAPACK under the hood. For graphics, many people seem to use matplotlib, which emulates Matlab’s plotting abstractions. Finally, because (despite being an atrocious language) R has lots of useful packages for statistical work, rpy2 lets you use those packages—including graphics—from within the cozy environs of Python.

Now, some actual instructions for installation and getting started with the language…

EASIEST WAY TO INSTALL EVERYTHING

After going through all the headaches documented below, I discovered that a company called Enthought maintains an up-to-date Python distribution for scientific computing that includes everything I need (except rpy2, which seemed to install easily enough atop it). They charge money for commercial/individual use, but provide a free download for academic use: follow the instructions at the distribution website to get a full 64-bit Python distribution with NumPy, SciPy, matplotlib, etc.

Only one bit of manual configuration was needed: the automated shell script didn’t correctly add their python to my path, so I had to add this to my shell profile (for most Mac OS X users, this is ~/.bash_profile):

export PATH="/Library/Frameworks/EPD64.framework/Versions/7.0/bin:${PATH}"

Then, either open a new Terminal or type source ~/.bash_profile, and you can install, e.g., rpy2 with

easy_install rpy2

I don’t recommend following the rest of these instructions—skip to the end. But if you like doing things the hard way, you can…

Installing Python 2.7.1

On Mac OS X 10.6, this just means downloading the fat 64-bit/32-bit installer10.6 installer from the Python website and running the installer. I did a custom install, checking the box about modifying my profile, which seemed to do the correct thing to my PATH in my .zprofile so that 2.7.1 gets run and not the system-included 2.6.1.

NOTE: I did this all with 64-bit first, which worked fine until I tried to use matplotlib. Unless you want to build matplotlib yourself from source, use the 32-bit version. The matplotlib binaries don’t work on 64-bit.

BUT WAIT: it turns out even the “32-bit installer” is actually 64-bit, and you need to configure it to run in 64-bit mode anyway. (See below.)

Fixing a Bug

There’s a bug in Python 2.7 on Mac OS X that prevents tabs from properly inserting in interactive mode; I found the solution on stackoverflow, which amounts to putting this in, e.g., ~/.pystartup:

import readline
if 'libedit' in readline.__doc__:
   readline.parse_and_bind("bind ^I ed-insert")

and then setting the PYTHONSTARTUP environment variable (in your ~/.bash_profile, ~/.zprofile, etc.):

export PYTHONSTARTUP=$HOME/.pystartup

Installing pip to make everything else easier

Somehow the existence of a nice Python package repository escaped my notice for the first 24 hours: it’s called PyPI, and everything I was looking for is there. To install things easily from PyPI, you can use a tool called pip.

Before installing pip, you need setuptools, which can be installed via the instructions on PyPI: first download the file, then run

cd ~/Downloads
sh ./setuptools-0.6c11-py2.7.egg

Then you can install pip. I first downloaded the .tar.gz from the bottom of the pip homepage, and then ran:

cd ~/Downloads
tar xzf pip-0.8.2.tar.gz
cd pip-0.8.2
python setup.py build
python setup.py install

Now, installing packages won’t be annoying.

Installing a FORTRAN compiler

Why would you want to do that? Because NumPy and SciPy require them.

Run this installer from AT&T Research.

Installing packages

Refreshingly, all the packages I wanted to install Just Worked. Everything except matplotlib can be successfully installed via pip.

I did the following at my shell prompt:

pip install numpy
pip install scipy
pip install rpy2

The last one depended on the fact that I had R installed in the standard location that you get with the OS X package installer for R. Nice to see that everything seems to be getting built fat (32-bit and 64-bit).

Installing matplotlib

The matplotlib package, it turns out, is a pain in the ass to build from scratch, and doesn’t work quite right from a pip install. Following the instructions at the matplotlib website led me to the following sequence of steps:

(1) Download matplotlib-1.0.1-python.org-32bit-py2.7-macosx10.3.dmg from the matplotlib Sourceforge site

(2) Run the installer.

(3) Add this line to your shell profile, since this matplotlib doesn’t work right in 64-bit mode, and I didn’t have the patience to figure out how to build a 64-bit one from scratch:

alias python="arch -i386 python"

(4) Change to a native Mac OS X plotting backend, since the default one is broken in this configuration, by putting the following into a file located at ~/.matplotlib/matplotlibrc:

backend: macosx

By the way, this is why I hate open-source software. (I love it for many other reasons.)

Actually learning Python

I found the tutorial at the Python website to be quite readable and useful. I’ve also have glanced at pieces of Dive Into Python.

Of course, getting comfortable with a language means actually Doing Something, all the while looking up syntax and reference documentation every half-line of code until you remember things. With Python, it has taken me about a day to feel confident that I can basically do what I want—contrast this with R, the inner workings of which will always be a mystery to me, particularly now that I’m leaving it behind.

Here’s a little example of how easy it is to process CSV files efficiently, with a nice dictionary mapping from column names to values:

import csv
csvfile = open("/path/to/file", mode='rU')
reader = csv.DictReader(csvfile) # csv.reader to return a list
for row in reader:
    for k, v in row.items():
        # Do something with keys and values
    # Or do something with the whole row
csvfile.close()

Bizarrely, indentation is syntax: Python knows your for loop is over because you write a line at a lower indentation level. But this has a wonderful effect: fewer lines of code, meaning you can see more of your logic in less space, and you can deploy blank lines exclusively for “writing in paragraphs” and not for silly things like “end” or “}”.

The Python equivalent of the missing semicolon seems to be the missing colon—these appear when introducing an if, for, class, etc., etc.; at this point I’m still forgetting them about 30% of the time.

The official Python people seem to encourage you to use spaces for indentation, not tabs, but the important thing is to be consistent so you don’t confuse the interpreter. I, for one, will use tabs. Commence rotten tomato throwing by people with stars upon thars.

git revert

to roll back changes to the last committed version of something—that’s how you’d do it in Subversion—but this isn’t right. This works instead:

git reset --hard HEAD

Classical Revolution Ann Arbor
Tuesday, May 5
8 PM
Kerrytown Concert House, 415 N Fourth Ave

XgridLite.prefPane.zip

Post your success/lack thereof in the comments!

XgridLite.prefPane.zip

Then decompress and drag the prefPane into your /Library/PreferencePanes or ~/Library/PreferencePanes

Let me know how it goes!

FYI, the weirdness I ran into goes something like this (as described in a message I just sent to the xgrid-users mailing list):

Hi everyone (particularly Apple folks),

After being out of the loop for a while, I finally got around to getting XgridLite working on Leopard. (Test version about to be posted at code.edbaskerville.com, official release probably by Monday.)

In doing so, I ran into one weirdness that I thought someone might be able to explain. XgridLite simply calls xgridctl to get and set controller state, which seems simple enough. In Tiger, it was sufficient to do this as root using a setuid tool.

In Leopard, however, I found that status calls always returned that the controller was off:

daemon              state               pid
======              =====               ===
xgridcontrollerd    stopped

This was happening inside the setuid tool even when querying it via sudo as a regular user worked fine:

Agnos:~ ebaskerv$ sudo xgridctl c status
daemon              state               pid
======              =====               ===
xgridcontrollerd    running             12656

To get to the bottom of this, I activated the root user on my system and tried it from a root shell, and, sure enough, I’m being told the controller is stopped:

sh-3.2# xgridctl c status
daemon              state               pid
======              =====               ===
xgridcontrollerd    stopped

Miraculously, though, if you run xgridctl via sudo as root, all is fine:

sh-3.2# sudo xgridctl c status
daemon              state               pid
======              =====               ===
xgridcontrollerd    running             12656

This gave me the workaround I needed: run xgridctl via sudo as root inside XgridLite. But I still have no idea *why* this is happening, so I’m writing purely out of curiosity. Why is it that running xgridctl status via sudo gives the correct status, while running as root directly does not? One more wrinkle: running xgridctl c start/stop seem to work fine inside my setuid tool (WITHOUT running via sudo), but do NOT work via a root shell!

Any wisdom would be appreciated.

Thanks,
Ed