We recently got an 8 core Mac Pro, and I wanted to see if I could take more advantage of all those cores when resizing images.

One of the things that confused me when I first opened activity monitor, is that it had 16 processors listed. A little digging turns up that this relates to Intel’s Hyperthreading technology, where they present the OS twice the cores and hand some extra concurrency on the chip. There is some debate out there as to whether or not it makes a difference.

It’s frustrating to have all that power and watch a CPU utilization tool look like this:

But that is what you get when a tool is not written to be parallel across processing units. Much is made over Python’s limit to threading and the GIL, however it seems that multiple threads never take as much advantage of multicore horsepower as do multiple processes.

Thanks to Python’s multiprocessing library it was very easy to create a worker pool to handle the resizing. The results are impressive. The test task was to resize a folder of 350 jpeg files by 50% and save them to a folder.

This shows you the performance gain by doing the image resizing in parallel. It goes from a 6 minute task to a 30 second task.

What is interesting is that even though there are only 8 true cores, there is a 40% increase in speed using all 16 virtual cores, but almost no advantage in going any higher than that. I’d say hyperthreading makes a difference in this case.

Now this is more like it:

A couple assorted notes:

With 8 workers, there is a roughly 10% increase in performance seen when turning off hyperthreading (via processor prefpane from dev-tools). It may be that the overhead of managing threads that aren’t doing much for you detracts from the overall performance?? Not sure.

Photoshop batch automate on the same task takes about 6 minutes, which seems to refute Adobe’s implication that muliprocessing doesn’t often gain you much.

There is one gotcha however, and that is that there does seem to be some sort of memory leak with the multiprocessing module. With just one worker in the pool, you can see a steadily increasing memory use that isn’t present in the same PIL code that isn’t run through the multiprocessing module. This is probably a manifestation of this bug

Finally, the script that I used to do the tests is available here.

So here is a quick script that I keep on my path for viewing hg diff, formated with pygments – I’m sure it would be very easy to adopt this to git.

here is the gist

Dreamhost provides free hosting to nonprofits, and for now they seem to have thrown in a virtual private server. This is hard not to take advantage of, and I’m hoping that the VPS is better performing than the shared hosting – but my concern is that they all share the same mysql servers. So here is how I got pinax up and running in its comfy and modern Python web stack.

At first I tried to go the route of custom install of python2.6 – but got bogged down trying to get either passengerwsgi or modwsgi to work properly – they both seemed to have fatal dead ends in the end.

The first thing you need to do is associate a domain with your PS and set it up as a directory of a user.

so that you have a user and directory like /home/django_user/example.com/

In the Dreamhost panel private server section “Web Server Configuration” uncheck “DreamHost Managed:”. This will prevent any accidental changes you make in the panel from undoing the customizations you need to make to the apache config.

Next you need to create an admin user in the Dreamhost control panel for your PS, lets say djangoadmin, this probably should be a different user than is set up for the domain. This user will just be configuring system level software, namely set the python version to use, install the mysql bindings, PIL, and modwsgi packages.

login as the admin user and then:

mkdir src

# This is to set python 2.5 as the default - if you don't care you can skip this step - but I'm assuming this is done
cd /usr/bin/
sudo rm python
sudo ln -s python2.5 python


cd ~/src
# note that this is specific for 2.5 - use the right egg for your python version
wget http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c11-py2.5.egg#md5=64c94f3bf7a72a13ec83e0b24f2749b2
sudo sh setuptools-0.6c11-py2.5.egg

You next need to download the MySQL-python tarball from http://sourceforge.net/projects/mysql-python/ as there is no easy way to fetch it from the command line. FTP it to your src/ directory of the admin user then:

tar xvzf MySQL-python-1.2.3c1.tar.gz
cd MySQL-python-1.2.3c1
sudo python setup.py install

cd ..
wget http://effbot.org/media/downloads/Imaging-1.1.6.tar.gz
tar xvzf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
python setup.py build --force
sudo python setup.py install
python selftest.py

You should see: 57 tests passed. Next:

sudo easy_install pip
sudo pip install virtualenv
# virtualenvwrapper is primarily useful if you are going to do 
# dev work or host many projects - I don't assume you fetch this
sudo pip install virtualenvwrapper

cd ~/src
wget http://modwsgi.googlecode.com/files/mod_wsgi-3.0c5.tar.gz
tar xvzf mod_wsgi-3.0c5.tar.gz 

./configure --with-apxs=/usr/local/dh/apache2/template/sbin/apxs --with-python=/usr/bin/python
make
sudo make install

# note that your private server ID replaces the 99999, 
# sub your favorite editor of choice (vi,emacs etc)

sudo pico /usr/local/dh/apache2/apache2-ps99999/etc/httpd.conf

find the section on modules and add:

LoadModule wsgi_module /dh/apache2/template/lib/modules/mod_wsgi.so

Now your base python environment is configured, don’t log out of this user – as we will need to make more changes to the httpd.conf file later. Instead open up another terminal session and login as your django_user

The first thing we need to do is create some folders we will use later

mkdir src envs projects wsgi-scripts

Then we will grab pinax, install it, and activate the virtualenv

cd ~/src
wget http://downloads.pinaxproject.com/Pinax-0.7.1-bundle.tar.gz
tar xvzf Pinax-0.7.1-bundle.tar.gz
python Pinax-0.7.1-bundle/scripts/pinax-boot.py ~/envs/pinax_test/
cd ~
source envs/pinax_test/bin/activate

#optional - update django to current release version
pip install -U django

Now we will create a basic pinax project

pinax-admin clone_project basic_project projects/basictest
cd projects/basictest/

Before you continue, go back to the Dreamhost panel and create a mysql database for your project – note the name and password, and the host name.

Edit the settings.py in your basictest project to use these database settings

Now lets see if we can get pinax up and running:

chmod +x manage.py 
./manage.py syncdb
./manage.py runserver 0.0.0.0:8000

You should now be able to go to http://example.com:8000 and see the pinax welcome screen, ctrl-C to stop the dev server.

To get pinax working with mod_wsgi you need two parts, a wsgi script, and changes to the apache configuration

create a file called pinax_basictest.wsgi in your wsgi-scripts folder. Its contents should look like (cobbled from wsgi docs and pinax deployment sample wsgi file):

import os
import sys
import site
sys.stdout = sys.stderr

# not needed in all environments - but is in others
os.environ['PYTHON_EGG_CACHE'] = '/tmp'

# this forces the virtualenv site-packages to be higher priority than system site-packages
site.addsitedir('/home/django_user/envs/pinax_test/lib/python2.5/site-packages')

sys.path.insert(0,'/home/django_user/projects/')

from django.conf import settings
os.environ['DJANGO_SETTINGS_MODULE'] = 'basictest.settings'

sys.path.insert(0, os.path.join(settings.PINAX_ROOT, "apps"))
sys.path.insert(0, os.path.join(settings.PROJECT_ROOT, "apps"))


import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()


# uncomment below to do a sanity check on the wsgi setup

# def test_wsgi(environ, start_response):
#     status = '200 OK'
#     output = 'Hello World! wsgi py \n' + sys.version + '\n' + '\n'.join(sys.path)
# 
#     response_headers = [('Content-type', 'text/plain'),
#                         ('Content-Length', str(len(output)))]
#     start_response(status, response_headers)
# 
#     return [output]
# 
# application = test_wsgi

Next you need to switch back to your admin user terminal (you didn’t close it right?) and edit your httpd.conf file again

sudo pico /usr/local/dh/apache2/apache2-ps99999/etc/httpd.conf

go to the very end of the file, and before the last closing /virtualhost tag (so inside the virtualhost) add the following

# wsgi
<Directory /home/django_user/wsgi-scripts>
Order allow,deny
Allow from all
</Directory>


# absolute path
WSGIScriptAlias / /home/django_user/wsgi-scripts/pinax_basictest.wsgi 

<Directory "/home/django_user/projects/basictest/media">
    Allow from all
    Order allow,deny
</Directory>


<Directory "/home/django_user/envs/pinax_test/lib/python2.5/site-packages/django/contrib/admin/media">
    Allow from all
    Order allow,deny
</Directory>

Alias "/media/" "/home/django_user/projects/basictest/media/"

# note some use hyphen - others _
Alias "/admin_media/" "/home/django_user/envs/pinax_test/lib/python2.5/site-packages/django/contrib/admin/media/"

Finally, still while in the admin user terminal – restart apache (remember to change for your private server id):

sudo /etc/init.d/httpd2 restart apache2-ps999999

Now you should be able to go to http://example.com and see the pinax welcome screen

If you get a 500 error:

• look in your apache error log which will be at /home/django_user/logs/example.com/http/error.log
• uncomment the sanity check in the wsgi script to see if you can get basic hello world working, along with some info about what python environment is being used by wsgi

First I check for the user and create it if it doesn’t exist

#!bash
user_exists=`dscl . -read /Users/remote GeneratedUID | grep -c GeneratedUID`
if [ $user_exists -ne 1 ]; then
    echo "creating remote user"
    sudo dscl . -create /Users/remote
    dscl . -create /Users/remote UserShell /bin/bash
    sudo dscl . -create /Users/remote RealName "remote"
    dscl . -create /Users/remote UniqueID 509
    dscl . -create /Users/remote PrimaryGroupID 1000
    dscl . -create /Users/remote NFSHomeDirectory /Local/Users/remote
    dscl . -passwd /Users/remote remote
fi

because these are fully managed machines, I know what UIDs are available. (For a method that checks for available UID I’ve posted a script from Andrew Mortensen below)

The next part of the script will check if the user is logged in as “remote” and on campus using a regular expression (our two subnets are 10.5.5.X and 10.6.6.X). You could also check a router, DHCP server, or internal DNS as other approaches. If they are on campus I use a display utility called BigHonkingText to throw up a message and then kill the loginwindow.

#!bash
user="$1"

if [ "$user" == "remote" ]; then
    IP=`ifconfig | grep "inet " | grep -v 127.0.0.1 | awk 'NR>1{exit};{ print $2 }'`
    echo $IP
    if [[ "$IP" =~ 10.[5,6].[5,6].[0-9]* ]]; then
        echo "on campus"
    /usr/local/bin/BigHonkingText "account not allowed on campus"
    killall loginwindow
        # reboot
    fi
fi

Here is the script from Andrew Mortensen:

#!bash
#! /bin/sh

# create a template user

export PATH=/bin:/usr/bin:/sbin:/usr/sbin

# arbitrary uid
N_UID=501

# arbitrary gid
N_GID=501

# user name
N_USERNAME="$1"

# home
N_HOME="/var/${N_USERNAME}"

# system default user home template
SYSHOMETEMPLATE="/System/Library/User Template/English.lproj"

# make sure the uid and gid are available
while [ true ]; do
    user="`dscl . -search /users UniqueID ${N_UID} 2>/dev/null`"

    if [ -z "${user}" ]; then
    break
    fi

    N_UID=$((${N_UID} + 1));
done

while [ true ]; do
    group=`dscl . -search /groups PrimaryGroupID ${N_GID} 2>/dev/null`

    if [ -z "${group}" ]; then
    break
    fi

    N_GID=$((${N_GID} + 1));
done

# create user
dscl . <<EOF
create "/users/${N_USERNAME}"
create "/users/${N_USERNAME}" AppleMetaNodeLocation /Local/Default
create "/users/${N_USERNAME}" GeneratedUID `uuidgen`
create "/users/${N_USERNAME}" UniqueID ${N_UID}
create "/users/${N_USERNAME}" PrimaryGroupID ${N_GID}
create "/users/${N_USERNAME}" Password "*"
create "/users/${N_USERNAME}" RecordName ${N_USERNAME}
create "/users/${N_USERNAME}" RecordType dsRecTypeNative:users
create "/users/${N_USERNAME}" NFSHomeDirectory ${N_HOME}
create "/users/${N_USERNAME}" RealName "Template User"
create "/users/${N_USERNAME}" UserShell /bin/bash
EOF
if [ $? -ne 0 ]; then
    logger -is Creation of ${N_USERNAME} failed.

    # destroy account
    dscl . -delete "/users/${N_USERNAME}" 2>/dev/null
    exit 2
fi

# create group
dscl . <<EOF
create "/groups/${N_USERNAME}"
create "/groups/${N_USERNAME}" AppleMetaNodeLocation /Local/Default
create "/groups/${N_USERNAME}" GeneratedUID `uuidgen`
create "/groups/${N_USERNAME}" PrimaryGroupID ${N_GID}
create "/groups/${N_USERNAME}" RecordName ${N_USERNAME}
create "/groups/${N_USERNAME}" RecordType dsRecTypeNative:groups
create "/groups/${N_USERNAME}" Password "*"
create "/groups/${N_USERNAME}" GroupMembership ${N_USERNAME}
EOF
if [ $? -ne 0 ]; then
    logger -is Creation of ${N_USERNAME} failed.

    # destroy account
    dscl . -delete "/users/${N_USERNAME}" 2>/dev/null
    dscl . -delete "/groups/${N_USERNAME}" 2>/dev/null
    exit 2
fi

# make home directory
mkdir -m 0700 -p ${N_HOME}
ditto --rsrc "${SYSHOMETEMPLATE}" "${N_HOME}"

if [ $? -ne 0 ]; then
    logger -is Creation of ${N_USERNAME} failed.

    # destroy account
    dscl . -delete "/users/${N_USERNAME}" 2>/dev/null
    dscl . -delete "/groups/${N_USERNAME}" 2>/dev/null
    exit 2
fi

chown -R ${N_USERNAME}:${N_USERNAME} ${N_HOME}

logger -i Creation of template user ${N_USERNAME} succeeded.

exit 0

The crux of the issue is that the version of MySQL that comes with Leopard Server does not include the header and library files needed for linking external libraries with. This applies to 10.5, but a similar process should work with 10.6.

If you attempt to build the MySQLdb python module you’ll get errors like:

building '_mysql' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c _mysql.c -o build/temp.macosx-10.5-i386-2.5/_mysql.o -fno-omit-frame-pointer -pipe -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT
_mysql.c:36:23: error: my_config.h: No such file or directory
_mysql.c:36:23: error: my_config.h: No such file or directory
_mysql.c:38:19: error: mysql.h: No such file or directory
_mysql.c:38:19: error: mysql.h: No such file or directory

Apple has a http://support.apple.com/kb/TA25017 on the issue – and the solution is to download these separately. You can also just install a different version of mysql – but for a variety of reasons, I wanted to continue to run the installed version.

So here are the commands that will build you a working mysql python binding:

(assumes you have dev tools installed)

grab the binding library source:

http://sourceforge.net/projects/mysql-python/files/

mv path/to/MySQL-python-1.2.3c1.tar.gz /tmp
cd /tmp
tar xvzf MySQL-python-1.2.3c1.tar.gz
curl -O http://www.opensource.apple.com/other/MySQL-43.binaries.tar.gz
tar xzvf MySQL-43.binaries.tar.gz
tar xvzf MySQL-43.binaries/MySQL-43.root.tar.gz
cd usr
sudo ditto usr/include/ /usr/include/
sudo ditto usr/lib/ /usr/lib/
cd /tmp/MySQL-python-1.2.3c1
ARCHFLAGS='-arch i386 -arch x86_64'
export ARCHFLAGS
# edit site.cfg to point to point to /usr/bin/mysql_config
python setup.py build
sudo python setup.py install

To verify this worked – fire up a python console and ‘import MySQLdb’

I’ve been mostly using transmit/SFTP to do editing of remote files with my local editor of choice (TextMate). Before I switched to TextMate I was using TextWrangler. The problem with using Transmit (or any FTP client) for editing is that if you are already where you need to be in a SSH session – you have to navigate to that same location in the FTP client before opening the file for local editing.

One feature TextWrangler has that TextMate does not is the ability to open files from a SFTP URL and save them back remotely when you do CMD-S.

So putting the following in the .bash_profile on the server will let you execute “redit somefile” which will cause a reverse ssh command to be sent you telling TextWrangler to open the file.

Some prereqs:

• You have to have key based SSH auth working in BOTH directions
• You have to have the TextWrangler CLI tool (edit) installed
• Since there is a good chance that the local user you are logged in as is different than the one you are shelling into as, you have to hard code that – so unless your name is preston – you will have to change that.

`

SSH_CLIENT_ADDRESS=`echo $SSH_CONNECTION | cut -f1 -d" "`
export SSH_CLIENT_ADDRESS

SSH_HOST_ADDRESS=`echo $SSH_CONNECTION | cut -f3 -d" "`
export SSH_HOST_ADDRESS

ME=`whoami`

function redit() {
    ssh preston@$SSH_CLIENT_ADDRESS edit \"sftp://$ME@$SSH_HOST_ADDRESS/$PWD/"$@"\"
}

`

I use Butler as my quicklaunch keyboard macro tool – but you could use your favorite one to run the following script while reading mail.

#!applescript
tell application "Mail"
    activate
    get message viewer 1's selected mailboxes
    repeat with thisBox in result
        set read status of (messages of thisBox whose read status is false) to true
    end repeat

end tell

I find this almost critical for getting through mailing list traffic.

So OE-Cake now has a beta for windows – but not OS X.

We still have the binary from their old version that expired some time ago – we were launching it with ARD with the below trick, but now I’ve moved it into a read-only applescript application that the students can launch themselves:

try
    do shell script "sudo date 1103100008" user name "admin" password "*****" with administrator privileges
end try

do shell script "open /Applications/OE-CAKE\\!.app"
delay 10
do shell script "launchctl stop org.ntp.ntpd" user name "admin" password "*****" with administrator privileges
do shell script "launchctl start org.ntp.ntpd" user name "admin" password "*****" with administrator privileges

I have a mac mini set up as a HTPC. The music end of this was a bit neglected as it was originally set up to use a custom onscreen menu control to switch between several different iTunes libraries and then have basic control within them.

Now that we have iPhones in the house – I consolidated the decade + worth of iTunes libraries into one.

I wanted a way to control the speakers based on iTunes state.

So this little python script runs every 10 seconds, here is a quick outline of what it does.

If iTunes is playing, but the speakers are off – then adjust some volumes on the speakers, turn them on, then rewind the track.

If iTunes is paused for more than 2 minutes, and the speakers are on – and they were turned on by this autodetect system, then turn the speakers off.

The turning on and off is done by my original HTPC applescript which controls an irTrans USB IR transceiver through the iRed software.

Now we just fire up the remote app on the phone, and within a few seconds the speakers are on and the music is playing.

#!/usr/bin/env python
# encoding: utf-8
"""

Created by Preston Holmes on 2009-09-05.
preston@ptone.com
Copyright (c) 2009

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"""

import sys
import os
from appscript import *
from subprocess import Popen, call, STDOUT, PIPE
import plistlib
import time

prefs = '/Library/Preferences/com.ptone.tuneswatcher.plist'
vol_target = '/Users/Preston/Documents/HTPC/targetvolume'

def sh(cmd):
    return Popen(cmd,shell=True,stdout=PIPE,stderr=PIPE).communicate()[0]

def itunes_running_():
    for p in psi.process.ProcessTable().items():
        try:
            if "Applications/iTunes" in p.command:
                return True
        except:
            pass
    return False

def init_prefs ():
    d = {'state':'unknown','last_state_change':time.time(),'auto_on':False}
    plistlib.writePlist (d,prefs)

def itunes_running():
    pcount = sh('ps -xa | grep -c iTunes').strip()
    return int(pcount) > 3

def getSettings():
    settings_file = '/Users/preston/Documents/HTPC/settings'
    f = open(settings_file,'r')
    flines = f.readlines()
    f.close()
    settingsDict = {}
    for aLine in flines:
        aLine = aLine.rstrip('\n')
        #print aLine
        if aLine != '':
            apair = aLine.split(' ')
            settingsDict[apair[0]] = apair[1]
    return settingsDict

def read_prefs ():
    if not os.path.exists (prefs):
        init_prefs()
    d = plistlib.readPlist (prefs)
    htpc_settings = getSettings()
    d.update(htpc_settings)
    return d

def save_prefs(d):
    plistlib.writePlist (d,prefs)

def main():
    d = read_prefs()
    # print d
    is_running = itunes_running()
    if is_running:
        tunes = app("iTunes")
        # print "tunes.running"
        playing = (tunes.player_state() == k.playing)
        # print tunes.player_state() 
        # print playing
        if playing:  
            # print "tunes.playing"
            if d['speakers'] == 'off':
                # turn on speakers
                tunes.pause()
                sh ("""osascript -e 'tell application "HTPC" to power_speakers()'""")
                if int(d['volume']) < 40:
                    open(vol_target,'w').write("50")
                    sh ("""osascript -e 'tell application "HTPC" to setVolume()'""")
                tunes.sound_volume.set(60)
                tunes.player_position.set(0)
                tunes.play()
                d['auto_on'] = True
                save_prefs(d)
                # start streamer here?
            if d['state'] != 'playing':
                d['state'] = 'playing'
                d['last_state_change'] = time.time()
                save_prefs(d)


        else: # iTunes not playing
            if d['state'] == 'playing':
                d['state'] = 'stopped'
                d['last_state_change'] = time.time()
                save_prefs(d)
            # check if last change was 5 minutes ago and turn off speakers
            if d['auto_on'] and ((time.time() - d['last_state_change']) > 10):
                if d['speakers'] == 'on':
                    # turn speakers off only if turned on automatically
                    sh ("""osascript -e 'tell application "HTPC" to power_speakers()'""")
                    sh ("""osascript -e 'tell application "HTPC" to calibrateVolume()'""")
                d['auto_on'] = False
                save_prefs(d)
        # print (tunes.player_state() == k.stopped)


if __name__ == '__main__':
    main()

First off we have a student web proxy that works based on a whitelist concept. So we have to allow any web traffic explicitly. With the help of little snitch and one incomplete help page from google, here is what I’ve added to our global whitelist and seems to be allowing all the features:

• kh.google.com
• www.keyhole.com
• mw2.google.com
• earth.google.com
• auth.keyhole.com
• maps.google.com
• khmdb.google.com

It also want access to www.google.com – but we don’t allow students to full google access – but Google Earth still seems to run fine.

The next annoyance was Google’s softwareupdate which wants to run for all users and update Google Earth on launch.

Adding the following to our loginhook fixed that:

mkdir -P $nethomedir/Library/Google/
touch $nethomedir/Library/Google/GoogleSoftwareUpdate
chown root $nethomedir/Library/Google/GoogleSoftwareUpdate
chmod 644 $nethomedir/Library/Google/GoogleSoftwareUpdate

the $nethomedir var is fetched from dscl earlier in the script

Students need to have stored the proxy password in their keychain – but most have already done that when visiting a web page earlier. They then just need to allow Google Earth to access their keychain.

Note that these are not strong passwords as they are the definition of “dictionary” passwords…

import sys
import os
import random

dfile = '/usr/share/dict/web2'

def main():
    for x in range(130):
        winner = False
        MAX = 234936
        suffix = random.randint (10,100)
        while not winner:
            choice = random.randint(1,MAX)
            f = open(dfile)
            for i,l in enumerate(f):
                l = l.strip()
                length = len(l)
                if i < choice:
                    continue
                if length > 3 and length < 7:
                    winner = '%s%d' % (l,suffix)
                    break
        f.close()
        print winner.lower()


if __name__ == '__main__':
    main()

These are documented here:

http://www.list.org/site.html

They are located in /usr/share/mailman/bin/

so you may want to add that to your path in .bash_profile

most have decent help summary with –help arg at command line

while the remove_members command accepts addresses on the command line itself:

remove_members [options] [listname] [addr1 ...]

remove_members staff frank@foo.com

The add_members command specifically wants file input. Sometimes I just want to add 2-3 so I use this trick:

add_members -r - staff <<-EOF
>sue@foo.com
>bob@bar.com
> <cntl-d>

First for me their was the discovery of cntl-a which jumps one back to the beginning of a line, but just as often I wanted to delete a long path as an opt to a long command.

The first thing is to set your default Editor in your environment variables. I use TextMate – but you could use textwrangler, VI, Emacs.

add a line like this to your ~/.bash_profile

export EDITOR=”mate -w”

then close your terminal session or “source ~/.bash_profile”

Now when you are in the middle of typing a long command, or after hitting the up arrow, press cntl-x and hold it, then hit “e”

boom – your current command opens up in your editor, you can use all the features of that editor, and when you save and close that file – the command will be executed back in your shell.

Since I’ve integrated this tip into my workflow – I find I use it all the time.

Only downside is GUI editors won’t work for SSH since you are in the remote hosts env – there is probably a tricky way to reverse-ssh the editor command back to you, but I haven’t explored that.

There are a number of defaults that can be set to make these not appear. The preferred way is to manage these with MCX, but for now I’ve deployed these as a login script (its lazy-easy because its all in just one text file)

This will set defaults so that iWork and iLife don’t bug the user. There are also defaults to turn off auto update check in perian, disable the welcome movie for Safari 4 and to set the standard file format for iWork 09 to use the iWork 08 package style file format which is more efficient when saving over the network. Finally there is a default to tell iTunes not to look for shared music.

Most of these should be pretty straight forward to read – if you have others, please post them in the comments.

TODO: make these into MCX manifests?

# !/bin/bash
echo Setting iPrefs...

sudo -u $1 defaults write org.perian.Perian NextRunDate -date '4000-12-31 16:00:00 -0800'

# iWork

sudo -u $1 defaults -currentHost write com.apple.iWork SFLDefaultsAutoUpdateCheck -bool FALSE
defaults write /Library/Preferences/com.apple.iWork09 DisplayedRegistrationWindowCount -int 3
defaults write /Library/Preferences/com.apple.iWork09 ShouldNotSendRegistration -bool TRUE
chown admin:admin /Library/Preferences/com.apple.iWork09.plist
chmod 644 /Library/Preferences/com.apple.iWork09.plist

sudo -u $1 defaults write com.apple.iWork.Keynote DisplayedRegistrationWindowCount -int 2
sudo -u $1 defaults write com.apple.iWork.Keynote RegistrationHasBeenSent -bool TRUE
sudo -u $1 defaults -currentHost write com.apple.iWork.Keynote FirstRunFlag -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Keynote dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Keynote DisableSingleFileFormat -bool TRUE

sudo -u $1 defaults write com.apple.iWork.Pages DisplayedRegistrationWindowCount -int 2
sudo -u $1 defaults write com.apple.iWork.Pages RegistrationHasBeenSent -bool TRUE
sudo -u $1 defaults -currentHost write com.apple.iWork.Pages FirstRunFlag -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Pages dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Pages DisableSingleFileFormat -bool TRUE

sudo -u $1 defaults write com.apple.iWork.Numbers DisplayedRegistrationWindowCount -int 2
sudo -u $1 defaults write com.apple.iWork.Numbers RegistrationHasBeenSent -bool TRUE
sudo -u $1 defaults -currentHost write com.apple.iWork.Numbers FirstRunFlag -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Numbers dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iWork.Numbers DisableSingleFileFormat -bool TRUE

# Garageband
sudo -u $1 defaults write com.apple.garageband dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.garageband DfPrefs_NoDocumentPanel -bool TRUE
sudo -u $1 defaults write com.apple.garageband DfPrefs_NextUpgradeCheck -date 'Aug 10, 2012 12:00:00 PM'
sudo -u $1 defaults write com.apple.garageband DfPrefs_AskAboutUpgrades -dict '5.1' 'NO'

# iMovie
sudo -u $1 defaults write com.apple.iMovie8 dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iMovie8 AutoUpgradeCheck -bool FALSE
sudo -u $1 defaults write com.apple.iMovie8 NextUpgradeCheck -date 'Aug 10, 2012 12:00:00 PM'
# no proof that this one is recognized - but it is in the others
sudo -u $1 defaults write com.apple.iMovie8 SFLDefaultsAutoUpdateCheck -bool FALSE

# iPhoto
sudo -u $1 defaults write com.apple.iPhoto CheckForUpdates -bool FALSE
sudo -u $1 defaults write com.apple.iPhoto dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iPhoto GeocodeLookupPreference -int 0
sudo -u $1 defaults write com.apple.iPhoto NextUpgradeCheck -date 'Aug 10, 2012 12:00:00 PM'
# not so sure about these:
sudo -u $1 defaults write com.apple.iPhoto SFLDefaultsAutoUpdateCheck -bool FALSE
sudo -u $1 defaults write com.apple.iPhoto UpdateSplash -int 8

# iTunes
sudo -u $1 defaults write com.apple.iTunes lookForSharedMusic -bool FALSE

# iWeb
sudo -u $1 defaults write com.apple.iWeb dontShowWhatsNew -bool TRUE
sudo -u $1 defaults write com.apple.iWeb dismissDotMacUpsellWindow -bool TRUE
sudo -u $1 defaults write com.apple.iWeb SFLDefaultsAutoUpdateCheck -bool FALSE
sudo -u $1 defaults -currentHost write com.apple.iWeb FirstRunFlag -bool TRUE 

# Safari
sudo -u $1 defaults write com.apple.Safari LastDisplayedWelcomePageVersionString '4.0'

You can get lost in the ins and outs of git vs hg, choosing a database engine (SQL vs Key-Value etc), javascript frameworks, PIP vs Buildout and integrating with virtualenv.

They all have their merit, especially to people who sit in this world all day long, but none of them directly produce work product for you. They are tools – and its best to learn them bit by bit, not all at once, or you will be overwhelmed.