Enter automation

I set about making a quick script to automate the process of setting up a new website template based on the boilerplate (which I'll refer to as h5bp from here on).  The steps it goes through are as follows (Bash-specific but could be retro-fitted to Windows without too much trouble).

• Checks to see if git is installed.  This is required as the latest h5bp version is available from github.
• If git isn't installed, the script will fail gracefully and provide instructions on how to install git.
• Checks for a single parameter.  If found, the script will use this as the name of the directory to create the project in, relative to "."
• If no parameter is specified, the destination folder is setup to a folder named after the current date and time.
• Checks to see if the destination folder exists (almost impossible if using the date and time method).  If it's not found, it is created.
• Initialises a new git local repository in the new project directory.
• Clones the latest h5bp version into the project directory.
• Clones the latest h5bp ant build script into the project directory,
• Cleans up the directory structure a bit, e.g. renames the ant build script from "ant-build-script" to "build"  I've found this to be a required step before the build script will run properly.
• Converts the h5bp markdown documentation to HTML, if pandoc is found on the local system.
• Does a bit more cleanup and moves some non-critical files into a directory called "_exclude"
• Configures some global git options, i.e. the user name and user email address.
• Adds all the project files to the new git repository.
• Does a full commit of the new files.
• Finishes up with a small message showing some steps that may be helpful to new users.

Care to share?

Sure.  If you're interested in the script, here it is in its entirety.  On my system I have it installed as an alias by putting into ~/.bash_profile inside a function called h5bp.  If you want to use the script in a stand-alone way, that works fine, too.  For the purposes of this article, I'll show the code in its function-based form.

The script

# function to create a new website based on the HTML5 BoilerPlate (http://h5bp.com)
function h5bp {
	if [[ -z `which git` ]]; then
		echo "   The git binaries were not found on this system and are required before this script will run properly."
		echo "   If you are sure git is installed, please investigate why the 'git' command wasn't found, then try again."
		echo "   To do this, you can try running \"echo \`which git\`\" (without the double quotes but keep the single ones!)."
		echo "   If you don't have git installed, please download and install it by following the instructions located at http://git-scm.com/download."
	else
		echo "git binary found at `which git`."
		echo "Creating new website."
		if [ -z $1 ]; then
			newFolderName=`date +%Y-%m-%d_%H-%M-%S`
			echo "No folder name specified, using current date and time: $newFolderName."
		else
			newFolderName=$1
			echo "Using '$newFolderName' as destination folder."
		fi
	        if [ ! -d $newFolderName ]; then
				echo "Destination folder not found, creating it now."
				mkdir $newFolderName
				cd $newFolderName
	        else
				cd $newFolderName
	        fi
	        echo "Initialising new git repository in `pwd`."
	        git init --quiet
	        echo "Getting latest h5bp build from github."
	        git clone https://github.com/h5bp/html5-boilerplate.git
	        echo "Getting latest h5bp build script."
	        git clone https://github.com/h5bp/ant-build-script.git
	    echo "Moving files into place."
		mv html5-boilerplate/* ./
		rm -Rf html5-boilerplate/
		mv ant-build-script/ build/
		mkdir _exclude/
		if [[ -z `which pandoc` ]]; then
			echo "Unable to convert the readme.md file to HTML for viewing.  You can fix this by installing 'pandoc' from http://johnmacfarlane.net/pandoc/."
		else
			echo "Converting markdown documentation to HTML."
			pandoc readme.md -o readme.html
			mv readme.html _exclude/
		fi
		mv readme.md _exclude/
		echo "Setting git global options."
		git config --global user.name "Put your name here" --quiet
		git config --global user.email you@yourdomain.com --quiet
		echo "Adding new files to new git repository."
		git add *
		echo "Running initial project commit."
		git commit -m "Initial h5bp project creation" --quiet
		echo ""
		echo "You should edit humans.txt in the '$newFolderName' directory before going any further."
		echo "Also, if you're not familiar with how to use the h5bp, you should go through the readme file in the _exclude/ directory (readme.md, or readme.html if you have pandoc installed)."
		echo ""
		echo "Done!"
	fi
}

Can we see it in action?

Of course you can.  I've recorded the script in action and put it up on YouTube - see below.  To view the video in HD so that your viewing experience doesn't suffer from small-video-itis, I'd highly recommend watching it on the YouTube website (opens in a new window).

While setting up our partner technology centre recently, I found myself switching back and forth between networks so often that I was constantly having to change my laptop’s IP address. For reasons that are outside the scope of this article I’m unable to use the option for an alternate network configuration.

Powershell to the rescue

The solution? Two small Powershell scripts - one to setup my network connection for our corporate LAN, the other to setup my network connection for the PTC. The scripts are shown below - feel free to use them in any way you like.

PTC configuration script

$index = (gwmi Win32_NetworkAdapter | where {$_.netconnectionid -eq “Local Area Connection”}).InterfaceIndex
$NetInterface = Get-WmiObject Win32_NetworkAdapterConfiguration | where {$_.InterfaceIndex -eq $index}
$NetInterface.EnableStatic(“10.0.0.10”, “255.255.255.0”)
$NetInterface.SetDynamicDNSRegistration(“FALSE”)

Corporate LAN configuration script

This script just resets the network adapter back to DHCP

$index = (gwmi Win32_NetworkAdapter | where {$_.netconnectionid -eq “Local Area Connection”}).InterfaceIndex
$NetInterface = Get-WmiObject Win32_NetworkAdapterConfiguration | where {$_.InterfaceIndex -eq $index}
$NetInterface.EnableDHCP()
$NetInterface.SetDynamicDNSRegistration(“TRUE”)

Extra stuff

Although I don’t need them in my PTC environment (it’s internal only, with no internet access), you can also use the snippets below to add some extra functionality.

$NetInterface.SetGateways($gateway)

$NetInterface.SetDNSServerSearchOrder($dns)
# (e.g. “10.10.1.1” for single server or “10.10.1.1,10.10.1.2” for multiple servers)

$NetInterface.SetDynamicDNSRegistration($registerDns)

This is a quick update to that post that shows how to setup Hazel to also process subfolders as, up until now, the Hazel configuration I had only processed a single level of folders.

The Hazel Rule

The configuration is pretty easy. Simply add a new rule at the top of your Hazel rules, making 100% sure it is above the rule that runs to automatically convert the video files.

The rule order looks like this:

The rule itself looks like this:

Once that rule is in place and the order is correct, the following rule will run once a new subfolder is found in the appropriate location.

This article is going to be a long one so sit back, relax (if you can) and get ready to do some EE hacking … ok, not hacking, creation. :)

Assumptions

I’m going to make a few assumptions for this article, as follows.

1. You know your way around the ExpressionEngine control panel and are familiar with terms like template, etc.
2. You know what jQuery is have an idea of what it is used for.

The Situation

Recently I’ve been doing some work on the redesign of the website for Erin King Photographer. Part of the redesign is the requirement for Erin to do 99% of the site management herself, without the need to refer back to me.

I realised that I’d need absolute control over the layout, styling & site architecture for this to work - that meant Wordpress wasn’t an option for this version of Erin’s site. Now, I’m not saying Wordpress couldn’t do the job (we still run Erin’s blog part of the site on Wordpress. In my opinion, though, Wordpress forces a shift in focus from design to content management - good for some but not what I wanted.

The Desired Layout

It looks simple (and is, when rendered), but this is the layout we’re trying to get to. Don’t forget this must be built & rendered dynamically, not hard-coded in the page’s markup.

The Decision

Some time ago I decided the new site should run on ExpressionEngine. The ability to control absolutely everything and yet still have the power of a full CMS made the decision pretty easy (plus, Digital Formula runs on ExpressionEngine).

The Problem

The first issue I ran into (which I knew was coming) was that ExpressionEngine’s native file module, while powerful in its simplicity, does lack a couple of what you’d expect to be standard features. For example, it’s difficult to control the order that files are displayed, unless they all have a different date and time in the database. It’s also impossible to manually set the file order in the control panel and have that order obeyed during page rendering.

The Solution

So, the point of this article is to show how to create a completely dynamic grid-based product page using the following things.

1. ExpressionEngine. This is the basis for the site itself.
2. The ExpressionEngine File Module.
3. The brilliant Matrix plugin by Pixel and Tonic.
4. Zack Grossbart’s awesome Dynamic Grid Layout jQuery Plugin.
5. The Image Sizer plugin for ExpressionEngine.

Note: Because EE isn’t free (some consider it very expensive), I’m going to make the assumption that paying USD $55 for the Matrix plugin isn’t a major problem, either.

Ok, so how do we proceed?

There are a few steps to cover off first - please make sure you’ve done these before getting stuck in.

1. Setup ExpressionEngine as per your requirements. Because these settings are site-specific, I’m not going to cover EE configuration here.
2. Enable the EE File module.
3. Setup an upload location for the images in the product section (remember, this article is for a products page). Don’t forget to set the appropriate permissions on the upload location …
4. Install and register the Matrix fieldtype.
5. Install and enable the Image Sizer plugin.
6. Upload some files to the products upload location. You don’t need to add captions to the files as we’ll be using Matrix for that.
7. Upload and reference the Dynamic Grid Layout JavaScript in the relevant pages on your site.

Matrix Configuration

Now that all the preparatory stuff is done, we can configure Matrix. The channel that has the products info must have a custom field of type ‘Matrix’. The example I’m working with is called Product Images and has three field options within the field:

1. Product Image (cell type = File), called product_image - make sure the Allowed Directory is set to the name of your upload location if you want to guarantee the files end up in the right place.
2. Long caption (cell type = Text), called caption.
3. Short caption (cell type = Text), called short_caption.

Channel Entry

Create a channel entry in the relevant channel and, in the “Product Images” field, create a number of Matrix rows that reference files you uploaded earlier. Make sure the Matrix rows have info in both the Caption and Short Caption columns. If you want to make absolutely sure these fields are filled in, enforce it by setting those columns to required in the Matrix configuration.

Once the entry is live, take note of the entry ID - we’ll be using it in our template.

Template Code

Up till this point, you’ve done the following.

1. Upload the product images.
2. Configured Matrix.
3. Created a channel entry and assigned some of the product images to the Matrix field.

Now you’re ready to create a template to show these files referenced by the entry you created a minute ago. The template code is as follows.

{exp:channel:entries channel="products" entry_id="25" dynamic="no"}
    <h2>{title}</h2><br />
    <div id="basicgrid">  
        {product_images}
            <div class="cell">
                <div class="product">
                    <a href="{product_image}" class="fancybox" title="{caption}">
                        {exp:imgsizer:size src="{product_image}" width="200"}
                            <img src="{sized}" alt="{short_caption}" width="{width}" height="{height}" />
                        {/exp:imgsizer:size}
                    </a>
                    <br />
                    <span class="product-caption">{short_caption}</span>
                </div>
            </div>
        {/product_images}
    </div>
{/exp:channel:entries}

Those of you with keen eyes may have noticed the reference to “FancyBox”. This isn’t a requirement for the layout but will give your users the ability to click the images and be shown a nice popup containing the full-size image. If you don’t want the images to be clickable, remove the anchor tag. Likewise, if you don’t want to use FancyBox, remove the class=”fancybox” attribute on the anchor tag.

Line-by-line Explanations

{exp:channel:entries channel="products" entry_id="1" dynamic="no"}

This line is where we tell the EE template which channel and entry to pull from the database. In this example, we’re using entry number 1 from the “products” channel.

<div id="basicgrid">

This is a critical line for the grid formatting that we’ll be setting up shortly. The div id doesn’t have to be “basicgrid” but you do need to remember what you put here.

{product_images}
            <div class="cell">

These are the opening tags for the Matrix field (remember we called it product_images earlier) and opens a div that will be used by the grid script later. Unless you edit the multigrid.js file, this div have a class of “cell”.

<a href="{product_image}" class="fancybox" title="{caption}">
    {exp:imgsizer:size src="{product_image}" width="200"}
        <img src="{sized}" alt="{short_caption}" width="{width}" height="{height}" />
    {/exp:imgsizer:size}
</a>

These lines create an anchor that can be clicked to show the full-sized image in a nice FancyBox popup. Within the anchor we’re taking the product_image field and applying the Image Sizer plugin to it to create a thumbnail 200px wide.

{/product_images}
    </div>
{/exp:channel:entries}

Finally, don’t forget to close the Matrix product_images and channel entries tags …

Will it work, yet?

Yes … sort-of. At this point the template will pull entry number 1 from the product channel and display the Matrix images associated with it on the page. However, the images will be displayed top-down without any grid formatting. The next step is to apply the grid formatting JavaScript so that the grid layout works properly.

Reformatting the product images

If you don’t have one already, your site’s pages will need to run a JavaScript file that can call jQuery functions. I’m happy to help you if you don’t know what this means (Contact Me) but, for those that do, the jQuery code is as follows.

$(document).ready(function() {
    /* jQuery grid */
    /* parameter order: number of columns, column width, column height, cell spacing */
    /* all sizes are measured in em */
    alignGrid("basicgrid", 4, 18, 18, 0.25);
});

The example above uses numbers I pulled out of thin air; you’ll need to modify them to fit your site’s design.

NOW will it work?

Yes. If all plugins are installed & configured properly and you have the correct content in your channel entry, your product images will be taken from the database, displayed on the page then formatted into a nice grid.

Now, I’m aware that this could be done manually by using an HTML table but remember that the point of this is for Erin to be able to upload & remove product images, re-order them, edit captions and descriptions etc, all without asking for any help from the site’s designer (me).

Problem solved. :)

The full set is available on the Digital Formula Flickr Pro site, along with information on how high/print quality images can be ordered.  Enjoy.

Why a plugin?

I'm assuming that someone else out there will find this useful.  I'm sure plugins like this exist already but, if they do, I can't find them.

For an example of what the plugin looks like when it finds an outdated entry, please see this article: Move ExpressionEngine to a different server (and yes, the content there is actually outdated).

Can I get it?

Of course you can.  If you want to download Entry Age and try it out, you can do by clicking the following link: download Entry Age 1.0 now.

Hope it helps someone.

Unfortunately, the PuTTY keys don't work for me as I'm using OS X ... insert appropriate Mac-hating statement here, if you must.  ;-)  Needless to say, I had to find a way of using those keys so I could access the servers without any problems.

Fortunately for me, I've encountered this situation before.  While testing some connectivity issues recently I kept getting denied access.  The problem is that the keys need to be converted to OpenSSH format before they'll work with command-line ssh in OS X.

To fix the issue, you'll need PuTTYGen from the PuTTY website (don't worry, it's free).  If you don't have it already, grab it, and follow the simple steps below (I'll include the step to download, just in case you missed it).

Note: You'll need access to a Windows machine for the below steps to work.

1. Download PuTTYGen from the PuTTY website.
2. Run PuTTYGen.exe, and open the .ppk file using the open option on the File menu.
3. From the Conversions menu, export the key in OpenSSH format - it's what you need for the key to work with OS X.

Once you've done all that, you should be able to run ssh from the OS X command line and connect without any problems.

Note: There are a ton of people who have experienced this issue and nearly every other website will tell you that you need to set the permissions on the .ssh file to 700.  In most cases 600 is enough.  If you don't do this, you can receive an error like this one:

That's all there is to it - easy.

To make the change easy, I'll explain the main things that have changed between 2008 and 2008 R2 from an unattended install perspective.

Assumptions

• You've already downloaded SQL Server 2008 R2 Express. This article will assume you're installing the 32-bit (x86) version although I see no reason why the steps below won't work with the 64-bit (x64) version.
• Your system meets the requirements for installing SQL Server 2008 R2 Express.  Note that this includes installing the .NET Framework 3.5 SP1, Windows Installer 4.5 and Windows Powershell.
• You are installing from C:\Install\SQL2008 (you can change this as necessary but please make sure you update the paths in the examples below).
• You'll save your configuration file as C:\Install\sql-2008-express.ini

Extracting installation files

To get the extracted installation files, run the following command and follow the wizard - easy.  Note that I've extracted mine to C:\Install\SQL2008\Extracted.

C:\Install\SQL2008\SQLEXPRWT_x86_ENU.exe /x

Getting the default configuration file

This is the key difference between the previous version (2008) and this one (2008 R2).  Run the following command either from the command prompt or from the Start » Run dialog and then follow the steps below carefully - you'll see that we don't actually want to complete the install right now.

C:\Install\SQL2008\SQLEXPRWT_x86_ENU.exe /UIMODE=Normal /ACTION=INSTALL

This will start the installation with all parts of the installation visible.  If you just ran the executable, you wouldn't see the screen that shows where the default configuration file is.

From here, follow the wizard, do a new install and configure the options as per your requirements, making sure that the steps on the left include the item "Ready to Install".  Note that this will only present if you ran the command above to set the UIMODE and ACTION parameters.

If your system is prepared properly, you'll see all items in the prerequisite checks pass successfully.

Important!

When you reach the step called "Ready to Install" - STOP.  Near the bottom of the installation window is a box labelled "Configuration file path:".  Take a copy of this file - you'll need it later.

Cancel the install

Click the "Cancel" button now - we're not actually going to complete the install at this point.  You could, of course, but the whole point of this article is to install SQL Server 2008 Express R2 without any user input.

Preparing the configuration file

Make the following changes to the ConfigurationFile.ini that you took a copy of.

1. INDICATEPROGRESS="True" - by default this is set to "False"
2. QUIETSIMPLE="True" - by default this is set to "False"
3. Remove the line that says UIMODE="Normal".  If this line isn't removed, the installation will fail.

For your reference, I've made the default and modified files available for download.

The installation

Now that your system is ready, the installation files and configuration files are ready and the changes made, you're ready to run the installation.  From above, remember that I'm assuming you're installing from C:\Install\SQL2008 and that your configuration file is C:\Install\sql-2008-express.ini - now you just need to run the command below.

C:\Install\SQL2008\Extracted\setup.exe /CONFIGURATIONFILE=C:\Install\sql-2008-express.ini /IAcceptSQLServerLicenseTerms

Take note of the /IAcceptSQLServerLicenseTerms switch - it automatically accepts the EULA.  Without this switch on the command line, the install will fail.

That's it!

If you've done everything correctly, SQL Server 2008 Express R2 will install without any user interaction.  Please note that my requirements aren't for a completely silent install so this isn't a 100% silent install.  You can make it that way by setting the QUIET parameter to "True", though.

If you have any questions about this, please let me know in the comments.  Thanks.

What is the chosen alternative?

I've been a long-time user of Evernote and its ability to sync between Mac (all my hardware), Windows (at work) and iPhone (in my pocket) makes it a sensible choice for the transition.  However, I needed to find a way to transfer all 2628 of my bookmarks to Evernote.

First attempt

My first go was to use the web service from Dr. Palaniraja but that didn't do some of the things I needed.  For example:

• It didn't keep my tags, despite the website saying it would (this effectively made it useless for me)
• It required a lot of manual input.  I don't mind this so much but less technical users probably would.

I was looking for an automated way to transfer everything.  My chosen method is an AppleScript that I found online at Veritrope, which I then heavily modified to do more of the things I wanted.

My changes are as follows:

• The original script verified every URL before adding it to Evernote.  Conceptually this is fine but I wanted to offer the option of not doing that.
• The original script used the bookmark's URL as the note title.  To me this is a little odd so I modified it to use the bookmark's actual title as the note's title, too.
• The original script required some manual editing to set your location i.e. Japan, U.S. or outside Japan/U.S. I'm in Australia which meant manually changing the location.  This is easy but I wanted to prompt the user for this information.
• The original script saved the bookmark's entire page content as the content of the new note.  This is a great idea but I wanted the option of not doing this, if necessary.  The main reason for this is so that Evernote's basic account holders don't blow the cap on their monthly usage.
• The original script, if verifying URLs, saved all the bad URLs to a single note.  I modified mine to save them as individual notes in a special, local-only, notebook called "Bad Bookmark List".

The script

Here's the entire script.  All you need to do is:

• Export your bookmarks from Delicious using their export/download page.
• Copy/paste the script below into AppleScript Editor (Mac only!) and run it.

The only manual change necessary, if required, is to change the name of the destination notebooks for the good and bad bookmarks.  These two settings are evNotebook and badNotebook respectively.

Warning: Be aware that if you run this script over a big bookmark list and select to verify URLs, it may take a long time to run!

-- v1.30
-- http://veritrope.com
-- Delicious Bookmark Importer for Evernote
-- February 18, 2011
-- Status, Latest Updates, and Comments Collected at:
-- http://veritrope.com/code/delicious-bookmark-importer-for-evernote/
-- 
-- v2.0
-- June 14th, 2011
-- Chris Rasmussen, digitalformula.net
-- Changes:
-- Added prompt for verifying URL
-- Added prompt for saving notes with actual title as title or URL as title
-- Added prompt for location i.e. U.S. or Japan or everywhere else
-- Added prompt for saving the bookmark or full page as the note's content (recommended for Evernote premium users, only)
-- 		
-- Note : The above prompts would usually be too many but since this script is probably intended to be run once only,
-- more options are better than none
-- 
-- v2.1
-- June 14th, 2011
-- Chris Rasmussen, digitalformula.net
-- Changes from 2.0:
-- Bad bookmarks are now added to a notebook called 'Bad Bookmark list' (createNote method changed to allow that)

-- *** Make changes in this section, only ***

-- title for dialog prompts
property scriptTitle : "Import Delicious bookmarks into Evernote"
-- set the variable below to the name of the Evernote folder to use for imported bookmarks
property evNotebook : "Delicious Import"

-- note for bad bookmarks - local only (won't be synchronised!)
property badNotebook : "Bad Bookmark List"

-- *** Stop editing here, please :) ***

-- firstly, create the 'bad' notebook
-- we do this now because otherwise AppleScript will, by default, create a synchronised notebook (bad idea if heaps of your bookmarks are invalid)
-- also, an error will be thrown if the notebook already exists - to prevent that, we do a quick check to see if a matching notebook can be found
tell application "Evernote"
	if (not (notebook named badNotebook exists)) then
		create notebook badNotebook with type local only
	end if
end tell

-- prompt the user for their location
global formatDates
property locationOptions : {"U.S.", "Japan", "Outside Japan/U.S."}
set location to (choose from list locationOptions with prompt "Please choose your location:" with title scriptTitle) as string

if location = "U.S." then
	set formatDates to "YES"
else if (location = "Japan") then
	set formatDates to "JP"
else
	set formatDates to "NO"
end if

-- verify URLs?
-- if the user responds in the affirmative, all URLs will be verified as working before they're added as notes
-- if the users responds in the negative, URLs are not verified and will be added even if they don't work
global verify
property verifyOptions : {"Yes - Verify all URLs", "No - Just import everything"}
set verifyResult to (choose from list verifyOptions with prompt "Verify URLs before import?" with title scriptTitle) as string

if verifyResult = "Yes - Verify all URLs" then
	set verify to true
else
	set verify to false
end if

-- use bookmark title or URL as new note's title?
-- the original script added notes with the bookmark's URL as the note title
-- this will allow the user to keep this option, if they like, or have the notes' title as the bookmark title (better, in my opinion)
global titleType
property titleOptions : {"Use bookmark's URL", "Use bookmark's title"}
set titleResult to (choose from list titleOptions with prompt "Use bookmark URL or title as note title?" with title scriptTitle) as string

if titleResult = "Use bookmark's URL" then
	set titleType to "url"
else
	set titleType to "title"
end if

-- save the page's full content as the note's content?
-- if the user responds in the affirmative, the page's entire content will be scraped and saved as the note's body
-- this is recommended for Evernote premium users, only
global contentType
property pageOptions : {"Yes - Save full page", "No - Save summary only"}
set pageResult to (choose from list pageOptions with prompt "Save full page as note's content?  This is recommended for Evernote premium users or small bookmark lists, only" with title scriptTitle) as string

if pageResult = "Yes - Save full page" then
	set contentType to "full"
else
	set contentType to "summary"
end if

property linkList : {}
property theTags : {}
property linkExists : ""
property badList : {}

-- Main program begins

-- reset items
set AppleScript's text item delimiters to ""
set linkList to {}
set badList to {}

-- select the Delicious export file
set deliciousFile to (choose file with prompt "Choose Delicious Bookmark Export File")

-- open the Delicious export file
set deliciousData to read deliciousFile
set deliciousBody to my extractBetween(deliciousData, "
", "
")

-- get a list of all the links
set delDelimiter to "
"
set deliciousRAW to read deliciousFile using delimiter delDelimiter
my processLinks(deliciousRAW, linkList)

-- process each link into a seperate webclipped note in Evernote, keeping tags and dates intact
my evernoteImport(linkList, theTags)

-- add bad links to a note containing all the bad URLs
if badList is not {} then
	
	set old_delim to AppleScript's text item delimiters
	set AppleScript's text item delimiters to return
	set badList to badList as text
	set AppleScript's text item delimiters to old_delim
	set badLinks to "List of URLs Returning Errors: " & return & (current date) & return & return & badList as Unicode text
	
	tell application "Evernote"
		create note with text badLinks title "List of URLs Returning Errors" notebook evNotebook
	end tell
end if

-- child methods

-- get the anchor href info
on processLinks(deliciousRAW, linkList)
	repeat with i from 1 to (length of deliciousRAW)
		set theItem to item i of deliciousRAW
		if theItem contains "A HREF" then
			set end of linkList to theItem
		end if
	end repeat
end processLinks

-- process each link
on evernoteImport(linkList, theTags)
	repeat with i from 1 to (length of linkList)
		set theItem to item i of linkList
		set theURL to my extractBetween(theItem, "A HREF=\"", "\"")
		set linkExists to "true" as boolean
		set epochseconds to my extractBetween(theItem, "ADD_DATE=\"", "\"")
		set deliciousTags to my extractBetween(theItem, "TAGS=\"", "\"")
		set itemTitle to my extractBetween(theItem, "\">", "</A>")
		-- change the delimiters to parse tags
		set oldDelims to AppleScript's text item delimiters
		set AppleScript's text item delimiters to ","
		set theTags to text items of deliciousTags as list
		-- change the delimiters back to what they were before
		set AppleScript's text item delimiters to oldDelims
		
		-- if the user selected to verify all URLs, the URL will be verified here
		if verify then
			my verify_Link(theURL)
		else
			set linkExists to true
		end if
		
		if linkExists is not false then
			-- add the bookmark to the good list
			if titleType = "title" then
				my createNote(itemTitle, theURL, theTags, epochseconds, evNotebook)
			else
				my createNote(theURL, theURL, theTags, epochseconds, evNotebook)
			end if
			set theTags to {}
		else
			-- add the bookmark to the bad list
			if titleType = "title" then
				my createNote(itemTitle, theURL, theTags, epochseconds, badNotebook)
			else
				my createNote(theURL, theURL, theTags, epochseconds, badNotebook)
			end if
			set theTags to {}
		end if
		
		if verify then
			-- delay is only required if we're verifying all links
			delay 3
		end if
		
	end repeat
end evernoteImport

-- verify working URL, if required
on verify_Link(theURL)
	try
		set the_page to do shell script "curl -siL \"" & theURL & "\""
		set the_code to items 10 thru 12 of the_page as string
		if the_code is greater than 400 then
			set linkExists to "false" as boolean
			copy theURL to the end of badList
		end if
	end try
end verify_Link

-- create the note
on createNote(itemTitle, theURL, theTags, epochseconds, notebookName)
	tell application "Evernote"
		try
			set createDate to my epoch2datetime(epochseconds)
			-- create the note based on user's choice of summary or entire page content
			if contentType = "full" then
				-- save entire page (this is the original script)
				set n to create note from url theURL created createDate notebook notebookName tags theTags
				set source URL of n to theURL
			else
				-- save summary only (new option, good for Evernote basic account holders)
				set n to create note title itemTitle with text theURL created createDate tags theTags notebook notebookName
				set source URL of n to theURL
			end if
		end try
	end tell
end createNote

-- convert UNIX epoch time
on epoch2datetime(epochseconds)
	-- adapted from script found at Erik's Lab (http://erikslab.com/2006/09/05/how-to-convert-an-epoch-time-to-a-meaningful-date-and-time/)
	set myshell1 to "date -r "
	if formatDates is "YES" then
		-- formatted for U.S.
		set myshell2 to " \"+%m/%d/%Y %l:%M %p\"" -- Formatted for U.S.
	else if formatDates is "JP" then
		-- formatted for Japan
		set myshell2 to " \"+%Y/%m/%d %l:%M %p\""
	else
		-- formatted for everyone outside Japan/U.S.
		set myshell2 to " \"+%d/%m/%Y %l:%M %p\""
	end if
	set theDatetime to do shell script (myshell1 & epochseconds & myshell2)
	return date theDatetime
end epoch2datetime

-- extract text between specified delimiters
on extractBetween(SearchText, startText, endText)
	set tid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to startText
	set endItems to text of text item -1 of SearchText
	set AppleScript's text item delimiters to endText
	set beginningToEnd to text of text item 1 of endItems
	set AppleScript's text item delimiters to tid
	return beginningToEnd
end extractBetween

Thanks to Justin from Veritrope for the original idea.

The idea of GeekTool is to use the often under-utilised desktop area of OS X to display useful info.  Some people use it to display images but most use it to run Bash shell commands to interrogate and display various system-level statistics.  For example, my desktop looks like this (click for larger version):

Before you ask, the desktop background is an InterfaceLift image called The Solutionist.  :)

Now, I’m going to assume that you’ve already downloaded and installed GeekTool 3 from the GeekTool homepage.  If you haven’t done that already, do it now, please.

Once you’ve got it installed, open System Preferences and click the GeekTool preference pane that should now be available.  For this article, please drag a "Shell" Geeklet onto your desktop.  Configure it any way you’d like in terms of position - for the sake of comparison, the screenshot below shows the settings for my date and weather Geeklet.

Because there are so many sites out there that try and teach Bash scripting and how to do all the stuff inside it, I’m only going to continue from here by showing what each of my scripts looks like.  The processes, network traffic and uptime scripts were borrowed in their original form from the Geeklets list on Mac OS X tips.  Credit for those must go to the original authors.  Here are the scripts:

date ‘+%H:%M’

top -l 1 | awk '/CPU usage/ {print "u:", $3, "s:", $5, "i:", $7}’

# day
_day=`date +%a | tr '[:lower:]' '[:upper:]'`

# degrees symbol: ˚

# get the temperature and humidity and write them to a temporary file
~/_Applications/lynx/lynx -dump http://www.bom.gov.au/vic/observations/melbourne.shtml | awk '/\[36\]Melbourne/ { print $3 }; /\[36\]Melbourne/ { print $6, "%"; };' > /tmp/weather-temp

_temp=`sed -n 1p /tmp/weather-temp`
_humidity=`sed -n 2p /tmp/weather-temp`

# month
_month=`date +%b`
_date=`date +%d`

# do some checks to make sure data returned for the checks that require an internet connection
if [[ -z $_temp ]]; then
	_temp="-"
fi
if [[ -z $_humidity ]]; then
	_humidity="-"
fi

echo "$_day\t\t\t`echo $_temp | tr -d " "` C"
echo "$_month $_date\t`echo $_humidity | tr -d " "` HUMIDITY" | tr '[:lower:]' ‘[:upper:]’

df -hl | grep 'disk0s2' | awk '{print "t:", $2, "f:", $4, "p:", $5; }'

top -l 1 | awk '/PhysMem/ {print "u:", $8, "f:", $10}'

The "Start Teh Music ..." widget you see next to the dock is created by Bowtie, definitely the best iTunes notification/info widget app out there.  It will change to show the title and artist of the track that’s currently playing, once one starts.  I’ve got the Aussie V8 Supercars on the T.V. right now ... no music needed.  :)

Here’s the same desktop again, this time labelled so that you know where each script is (click for larger version).

Granted, my choice of system info, fonts, colours etc is fairly boring and minimal compared to some but they give me what I need without destroying what I reckon is one of the best desktop background images I’ve ever found.

There are a ton of ways to accomplish the scheduling a script to run.  If you're hardcore you can use Cron (I won't cover that here), you can purchase some scheduling software for OS X or you can do what I do - use iCal.

But iCal is a calendar ... isn't it?

Yes, it is.  However, because it's a calendar, you can add events and set reminders for them.  See where I’m going with this?  Correct - you can run an AppleScript as part of an iCal event’s alarm.  If you read the article about my AppleScript backups, you’ll remember that I said that the idea was to do things with built-in apps, if possible.  Every OS X installation includes iCal by default so it just makes sense to use it here.

Won’t my calendar get messy?

iCal allows you to setup separate calendars for different types of events so no, it won’t get messy if you don’t want it to.  For example, I have a calendar called ‘Backups’ and it holds ... wait for it ... backups.  :-)  My backup script runs each day at 8:00 p.m. without me having to do anything manually.

Here’s an example of what one of my backup events looks like.  Note that the event is set to happen 15 minutes after the actual backup starts, with the reminder event triggering 15 minutes before that, i.e. at the actual time the backup is to start.

It really is that simple.  You don’t need anything other than what comes with your OS X installation to do any of this either - that makes it completely free.

If you end up using any of this, I’d love to hear about it.  Thanks for reading!

Because I’m in I.T. I’m quite paranoid about my data and its safety.  This is partly due to being responsible for the data for companies that would seriously lose hundreds of millions in capital without that data and partly due to losing all my own data many years ago.  At the time I was working on a website editor that, after some semi-public testing, I had been informed was better than the best commercial website editor at the time.  I lost the code, all the files that related to the project and all my backups ... my backup software had a bug that killed my ability to restore from tape.  Sigh.

So anyway, I’ve used many methods of backing up my data but I’m overly paranoid about backup software not doing what I want when I *really* want it to and I therefore do my own backups, ‘on the side’, so to speak.

How?

I’m a big fan of doing things as simply as possible and in this case the simplest solution is to take the data I want to backup, archive it in a widely-used format (ZIP) and put it somewhere I can easily get it back.  Naturally this means offsite, outside my house, and in today’s world that means in ‘the cloud’.  Dropbox, in this case.

Now, if you’ve used Dropbox and have given it a bit of a thrashing, you’ll know that as soon as you throw a LOT of files at it, the indexing process really starts to take over, chewing CPU and disk I/O cycles.  For that reason, my solution involving archiving the data into a ZIP file before giving it to Dropbox works nicely.

No, really, how?

My current hardware platform is Mac.  For that reason I figured the best way to back things up without installing anything extra was to write an AppleScript to handle things.  You might argue that I should’ve written it using standard Unix .sh or Bash script so that I could use it on Linux but that’s not a requirement.  Plus, although I wrote this primarily for my own use, I had always planned on making this script publicly available and AppleScript is very easy to understand, even if you’re not a tech-head, like me.  Don’t expect a production-quality script as I’m not trying to sell it so please be aware that there are no warranties provided with anything you download from here.

Is it configurable?

A lot of the settings are, yes.  However, this is a backup script and I therefore chose to keep a number of things ‘set in concrete’, if that makes any sense.  However, being an editable script means that you can just change stuff, if you want.  Here’s what you can change.

• Prompt on overwrite, if a destination archive already exists
• The ‘base’ path of your backup destination, by default Dropbox
• The list of paths that need backing up

In my opinion, you don’t really need to change much more than that.  :)

Can we see the script, now?

Well, OK.  Since we’re up to that part, I want to point out a few things.

• If Dropbox isn’t running, a warning will appear.  This is simply because I truly believe you should be backing up to an offsite location.
• Old backups are never.  Did you read that?  Make sure you do as the potential for filling up your free 2GB Dropbox is quite real if you choose to backup a LOT of data each time.
• Destination folders are created, if they don’t exist.
• Growl notifications are shown for everything important.

The Script

One last thing - you can download the script, if you want to read it offline.

-- Backup.scpt
--
-- April 2011, Chris Rasmussen
-- http://digitalformula.net | http://twitter.com/digitalformula
--
-- Used to archive specified files to relevant backup folders to a specified folder, preferably Dropbox or some other cloud-based sync service
--
-- The backup is done by archiving source folders so that indexing doesn't take too long (a known issue with Dropbox)

-- set some variables for machine-specific information
-- get the username of the current logged in user
global computerName
set computerName to (computer name of (system info))
global userHome
set userHome to (home directory of (system info))

-- ***
-- *** Makes changes in this section only
-- ***

-- options
global PromptForOverwrite
-- set this to true if you want to prompted before overwriting any existing archived backups in the destination folder
-- since we're using destination folder names based on date *and* time, existing backups shouldn't ever really exist
set PromptForOverwrite to false

-- set this to the parent path of where backups will go
-- unless you change this below, the default is your Dropbox
-- IMPORTANT : if you set this to anything but 'userHome' you MUST make sure a colon (:) is added to the end of the path
global BasePath
set BasePath to userHome & "Dropbox" & ":"

-- create the list of items to backup
-- for readability the list definition has been split onto multiple lines using the 'continuation character' (Option + lower-case L)
-- the first parameter is the name of the application, the second parameter is the full Apple path its files are in
-- please don't put spaces in the application name for now ... sorry
-- this is the same as:
-- set TheBackupList to {{"SnippetsApp", "Snippets"}, {"HazelApp", "Hazel"}}
-- for the downloadable version, the data to be backed up is specified one item per line
--
-- to add a new item to the list, copy the last item onto a new line - make sure the curly braces, commas and carriage returns match!

set TheBackupList to {¬
	{"SnippetsApp", " HD:Users:chris:Library:Application Support:Snippets"}, ¬
	{"HazelApp", " HD:Users:chris:Library:Application Support:Hazel"}, ¬
	{"KeePass", " HD:Users:chris:Data:KeePass"}, ¬
	{"AppleScripts", " HD:Users:chris:Library:Scripts"}, ¬
	{"PrefsFolder", " HD:Users:chris:Library:Preferences"} ¬
		}

-- ***
-- *** That's it - stop editing!
-- ***

-- see if Growl is available on this system
-- this variable is global so that the newBackup class has access to it
global GrowlAvailable
tell application "System Events" to set GrowlAvailable to exists application process "GrowlHelperApp"

if GrowlAvailable then
	-- enable the script to send Growl notifications
	tell application "GrowlHelperApp"
		set the allNotificationsList to {"removed", "success", "aborted", "no_existing", "created"}
		set the enabledNotificationsList to {"removed", "success", "aborted", "no_existing", "created"}
		register as application "Backup Database" all notifications allNotificationsList default notifications enabledNotificationsList icon of application "Script Editor"
	end tell
end if

-- see if Dropbox is available
global DropboxAvailable
tell application "System Events" to set DropboxAvailable to exists application process "Dropbox"

-- see if Dropbox is available
if not DropboxAvailable then
	display dialog "Oops!  Dropbox is either not installed or not running on this computer.  Dropbox is highly recommended as the backup destination so that backup archives are created 'offsite'.\n\nPlease download and install Dropbox from http://www.dropbox.com or, if you've already got it, look into why Dropbox can't be accessed ...k?  :)" buttons {"Ok"} with icon stop with title "Oh no!"
end if

-- class to store settings relating to the objects to be backed up
-- by using a class we can do multiple backups by simple accessing the class and 'running' it
on BackupObject(appName, folderName)
	
	script backupObjectDetails
		
		-- get the current date and format it nicely
		property CurrentDate : (year of (current date)) & "-" & (month of (current date) as integer) & "-" & (day of (current date)) as string
		property CurrentTime : (hours of (current date)) & (minutes of (current date)) & (seconds of (current date)) as string
		
		-- although it's not an explicitly specified configuration option, you can uncomment the following line if you don't want to use destination folders that are named based on date & time
		-- property ApplicationName : appName
		-- if you uncommented the line above, make sure you comment the one below or your changes won't take effect
		property ApplicationName : CurrentDate & "-" & CurrentTime & "-" & appName
		property ApplicationNameShort : appName
		
		property SourceParent : userHome & "Library:Application Support:" as string
		property DestParent : BasePath & "Backups:" & computerName & ":" as string
		
		property SourcePath : folderName
		property DestPath : DestParent & ApplicationName
		
		property AppFolder : folderName
		
		property DestDB : DestPath & ":" & ApplicationName & ".zip"
		property FileCount : 0
		
		-- flag variable to tell the script if it should continue once all conditions are evaluated
		property GoodToGo : false
		
		-- method to copy the master database to the backup folder
		to copyDatabase to destinationFolder
			
			-- try and create the master Backups directory
			try
				tell application "Finder" to set destResults to make new folder at (BasePath as string) with properties {name:"Backups"}
			end try
			
			-- try and create the machine-specific backup directory
			try
				tell application "Finder" to set destResults to make new folder at (BasePath & "Backups" as string) with properties {name:computerName}
			end try
			
			-- first check if the folder containing the source files exists
			-- if it doesn't, we've got nowhere to copy files from
			tell application "Finder" to if exists SourcePath then
				
				-- then check if the destination folder exists
				-- if it doesn't, the script will create it
				tell application "Finder" to if exists DestPath then
					
					-- check if a backup copy of the database file already exists
					-- if it doesn't, the database will be copied immediately
					tell application "Finder" to if exists DestDB then
						
						if PromptForOverwrite then
							
							-- an existing backup does exist
							-- ask the user if they want to remove it first
							set continueResult to display dialog "Archived backup of " & ApplicationName & " database already exists in " & DestPath & ".  Delete existing backup and continue?" buttons {"Yes", "No"} with icon caution with title "Backup already exists"
							if button returned of continueResult is "No" then
								if GrowlAvailable then tell application "GrowlHelperApp" to notify with name "aborted" title "Backup " & ApplicationName & " Database" description ApplicationName & " database backup aborted." application name "Backup Database"
								
								-- don't continue
								set GoodToGo to false
								
							else
								
								-- user specified to remove the existing backup first
								delete file DestDB
								if GrowlAvailable then tell application "GrowlHelperApp" to notify with name "removed" title "Backup " & ApplicationName & " Database" description "Old " & ApplicationName & " database backups removed." application name "Backup Database"
								
								-- ok to continue
								set GoodToGo to true
								
							end if
						else
							-- ok to continue
							set GoodToGo to true
							
						end if
					else
						-- an existing backup doesn't exist so just do the backup now
						if GrowlAvailable then tell application "GrowlHelperApp" to notify with name "no_existing" title "Backup " & ApplicationName & " Database" description "No existing " & ApplicationName & " database backups found." application name "Backup Database"
						
						-- ok to continue
						set GoodToGo to true
						
					end if
				else
					-- destination directory doesn't exist
					-- try and create it then do the copy
					try
						tell application "Finder" to set destResults to make new folder at (BasePath & "Backups:" & computerName as string) with properties {name:ApplicationName}
					on error
						display dialog "Oops!  The destination folder, " & DestPath & ", could not be created.  Please try again later." buttons {"Ok"} with icon stop with title "Oh no!"
						return
					end try
					
					-- ok to continue
					set GoodToGo to true
					
				end if
				
			else
				display dialog "Oops!  The source folder you selected, " & SourcePath & ", ain't there!  Please check it and try again." buttons {"Ok"} with icon stop with title "Oh no!"
				
			end if
			
			if GoodToGo then
				
				-- see if there is more than 1 file in the selected folder
				-- we need to do this so the notifications show the correct file counts when archiving is finished
				tell application "Finder"
					-- create the ZIP file
					do shell script "zip -r " & POSIX path of DestDB & " -9 \"" & POSIX path of SourcePath & "\""
					try
						set SourceData to every item in entire contents of folder SourcePath
						if class of SourceData is list then
							-- the resulting object will be a list if there is more than 1 file
							set FileCount to (count of items in SourceData)
						else
							-- only 1 file exists
							set FileCount to 1
						end if
					on error
						display dialog "No files found in selected folder!"
					end try
				end tell
				
				-- backup successful - time to display some stats
				if GrowlAvailable then tell application "GrowlHelperApp" to notify with name "success" title ApplicationNameShort & " backup successful" description (FileCount as string) & " " & ApplicationNameShort & " files archived successfully" application name "Backup Database"
				
			else
				-- GoodToGo is false
				display dialog "Hmmm you either aborted the backup or something broke while the backup was running.  If you didn't abort, please make sure file permissions are correct and that there aren't any locked files in the source or destination folders then try running the script again ... k?  :)" buttons {"Ok"} with icon stop with title "Oh no!"
				
			end if
			
		end copyDatabase
		
	end script
	
end BackupObject

-- go through the list and backup everything up
-- excessive comments for readability

-- get the number of objects we're backing up
set NumberOfObjects to count of TheBackupList

-- set the list pointer to the first object MINUS 1
-- AppleScripts are 1-based, not 0-based!
set CurrentObject to 0
-- go through the list and do the actual backup part
repeat until CurrentObject is equal to (NumberOfObjects)
	
	-- move the list pointer to the first object
	-- this is done here because of AppleScript's 1-based lists
	set CurrentObject to CurrentObject + 1
	
	-- get the current object containing the name of the app and the path to be backed up
	set TempObject to item CurrentObject of TheBackupList
	
	-- create the backup object that contains all information about what to backup ...
	set TheBackupObject to BackupObject(item 1 of TempObject, item 2 of TempObject)
	-- ... then back it up!
	tell TheBackupObject to copyDatabase to (DestPath of TheBackupObject)
	
	-- rinse and repeat until there's nothing else to wash ... err ... backup
	
end repeat

It’s a decent-sized script, huh?  There are a boat load of comments so in reality I’d say the script itself isn’t too bad.  It really does work, though, I promise.  I use it every day to backup the data that’s in this particular script already i.e. my Snippets database, my Hazel configuration, KeePass database, all my AppleScripts and my account preferences.

If you do download the script or use it yourself, I’d like to hear about it.  Even if you grab it, tear it to shreds, change everything except the idea etc, still let me know.  Thanks!