My Cognitive Imperialism

I've gone all professional and stuff

2011-07-13T17:45:00.000-07:00

Howdy, I'm no longer posting to this blog. You can read more of my amazing adventures in code at:
http://www.reignite.com.au/blog/
Where I blog under my professional name: Surrey.
Click around the site and you'll see I work at Reignite. My blog entries there will be more formal and less chatty. Here, I can be irreverent and silly, there I have my business suit on so present my more reasonable and sensible cases.

I may even reveal the amazing truth that is AMF using GWT. That's a javascript AMF client I wrote / adapted to make use of the existing services I've already written.

Magic Words, Magical Writers

2011-02-09T18:50:00.000-08:00

Applauds Wildly, and Contest with Today's Post

Not only am I a Flex / Java nerd, but I'm a fantasy writing nerd too! I wrote a novel, then I read magicalwords.net and rewrote my novel because of all the useful info and tips I got. Hooray. Now I'm linking back to it so others can find it and so I can go in the draw to win a cool prize.

Pushing Live Streams From Flash to iPhone

2010-10-10T23:14:00.000-07:00

iPhone == anathema
So I've had to do some work with the iPhone regarding apps and that means looking at objective-C. First let me suck yuck. I started playing with OC code the other night when 1995 called and asked if it could please have its programming language back. Anyway, as a Java programmer and a Flex programmer I have to say OC is nasty for gui dev and for basic app dev work. Maybe it is really good for larger applications? I don't know, but for little mobile apps it is cumbersome, ugly and overly fiddly. That aside this post is actually about integrating an iPhone app into an existing Flex multi-media app, so I'll leave the OC coding to Mac Fanboys (as I'm a Java fanboy) and move on.

Flash Video Encoding
The existing app I have is a social networking site that allows you to stream live audio and video to an audience. No, this is not a porn web cam, it is for musicians to broadcast their jam sessions or min-gigs to their fans.
So the performer logs on, clicks "yes allow flash to use my camera" and they start streaming to the server. A fan logs on and clicks on the "watch my fave act" and they get the performer's stream playing for their pleasure. The iPhone extension allows a person to watch the same performance on their iPhone rather than on their Flash enabled device (as in everything else in the entire universe, including TVs and refrigerators).
But, there is a problem: iPhone only reads H264 encoded video with aac / mp3 audio. Flash will also read that, but at present only creates Sorenson Spark / Speex (in my case) encoded video / audio. That means the performer is producing a stream that is unintelligible to the iPhone. There is, to my knowledge, no way to load additional codecs onto the iPhone and I don't think Apple will let you write an app that does. So how the heck do you let an iPhone participate in these new fangled interwebs that Steve Jobs has heard so much about (and yet doesn't like)? Oh and yes I am being contentious on purpose because I'm like that.

Transcoding
The magic word we are looking for is transcoder. We take the flash video in one end and spit out h264. This is easier said than done. There is only one transcoder in the whole wide world, I know you don't believe me but it's true, and it is called FFMpeg. Just try to find another application / library able to convert videos via a command line or with an API, go on I dare you.
Somewhere in my back end I need to use FFMpeg to create the iPhone stream otherwise I can just kiss my rich-media app goodbye.

Xuggler
Hooray for the Java open source community! There is a lovely Java JNI wrapper for FFMpeg called Xuggler. It even has some guts in it that let you read a stream from an rtmp source and publish back to an rtmp location. As it happens my back end uses Wowza media server, also Java, so it is fairly easy to call Xuggler from a Wowza application. What isn't so easy is finding any halfway decent documentation for Wowza. Also, Wowza isn't open source so it is a bit of a mystery as to what happens inside it. I think there might be elves involved. Anyway here is the basic glue I used (which I stole from a forum post on the Wowza forums, so thank whoever that was):

public class TheApp extends ModuleBase {
   ... stuff ...

   public void publish(IClient client, RequestFunction function, AMFDataList params){
   IMediaStream stream = getStream(client, function);
   stream.addClientListener(new StreamWatcher());
   ModuleCore.publish(client, function, params);
   }
   ... other stuff ...
}

Ok so the above code adds a listener to the stream so that when the stream is actually published I get notified and a method in the StreamWatcher class is called (that's my class BTW)

public class StreamWatcher implements IMediaStreamActionNotify2 {

   public void onPublish(IMediaStream stream, String streamName, boolean isRecord, boolean isAppend) {
   H264Transcoder tc = new H264Transcoder(stream.getName());
   tc.start();
   }
}

Above you can see that when the stream is publish I create a new transcoder (which is a thread) and start it.

public class H264Transcoder extends Thread {

   public void run() {
try {
Thread.sleep(5000L);
RTMPPublishH264();
} catch (ParseException e) {
System.out.println(e);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

   public void testRTMPPublishH264() throws ParseException {
   Converter converter = new Converter();
   Options options = converter.defineOptions();
   String args = ""
   /* ffmpeg -i rtmp://www.site.com.au:1935/ppv/streamName
* -re
* -acodec libfaac
* -ar 22050
* -vcodec libx264
* -fpre default.ffpreset
* -f flv rtmp://www.site.com.au:1935/live2/streamName-iphone
   */
   CommandLine cmdLine = converter.parseOptions(options, args);
   converter.run(cmdLine);
   }
}

Wow, so the above isn't complete, but I've included the really important bits. The reason it isn't complete is because I haven't perfected it yet. Maybe I never will. Oh, who am I kidding, of course I will, but not just now.

Important Points
The important points for this:
Xuggler can't connect to specific application instances, only to the _definst_ instance. That means the rtmp url above has an application called "ppv" in which a stream called "streamName" is published. Normally I'd have a named app instance that I'd publish into so I could track shared objects and the like, but that didn't work with Xuggler. Also the output stream I've called streamName-iphone, is in a different application, again the _definst_ instance. The reason for that is my ppv application has a bunch of rich user interaction stuff like chat and it gets info from the server about the performer and the people involved need to have valid accounts and so on. I've only included the actual ffmpeg command line code that I have personally used and it works. The preset I used is the default one that comes with Xuggler for h264 so I'll probably have to tweak that to get the best performance.

I may add an update post once I get all the kinks worked out. For example I'd love to package this up as a simple jar + config arrangement, but I do have to work for a living so I may not get the time.

A Date with Flex

2010-08-26T19:25:00.000-07:00

Date Objects
So in computing we always need to deal with dates. All my tables for example have created date and modified date columns so each record is timestamped. Then there are dates for reports, dates for events and schedules and calendar dates for people to deal with. But why is it there is always so much trouble dealing with dates?
A Date is Unique
At its heart the Date object is an unique instant in time. It is universal and absolute.
What does that mean?
It means that, say in Java when I say Date myDate = new Date(); I am creating an object that stores the moment of its own creation. The date doesn't matter what timezone I'm in, it doesn't matter what language I speak or anything like that. Deep inside the Date object, and this holds true regardless of programming language or database, there is a long integer that is the milliseconds since an epoch. The usual epoch is 1/1/1970 which is often called Unix time, but it could also be UTC which was in 1972. It isn't all that important when the epoch was. The important point is there is no timezone offset. Timezone and locale are human decorators added to a Date to make it relative to the human's location. Regardless of where the computer is or the person programming it the number of milliseconds since epoch is the same. If it is 10am for me in Australia and I call my friend in Greenwich (who won't be pleased) they would tell me it was 2am. It is actually the same Date, the same time, the same instant in existence and it is that which the Date object captures. We then apply formatting to that Date to produce something relative to where we are.
So What?
This is important because it means that if I need to create a date for my friend in Greenwich, for example I need to set an alarm for them to wake up at 8am things get twisted. In Australia, if I just create a Date and set the hour to 8am and save it the alarm will go off at midnight! So I need to see a Greenwich clock and select 8am on that. In Java I can do something like this:

Calendar c = GregorianCalendar.getInstance(zone);

Where zone is Greenwich timezone. Then I can set the hour on the calendar to 8am and get the Date from the calendar and bingo it's done.
Flex
Flex has a Date object but no Calendar. So when I create a Date object in flex it automagically get the client computer's timezone/locale settings. Inside the Date it is still milliseconds since epoch and you can get that by using the various UTC related methods to get the universal time. The date picker and date formatter all by default work of locale dates / times. That means you need to write your own code to display dates in other timezones. In the above example where I need to set an alarm for my friend in Greenwich I would need to make sure that I was viewing the Date in the correct timezone.
How?
There are a number of ways you can do it but I think it boils down to two ways which are both correct depending on your exact situation.

Enter and send the date / time as a string. It is pretty easy for me to simply type 8am and send that as a string to be stored and seen by my friend. And if all I wanted to do was tell him when I'd set his alarm that would be fine. However what if the server and/or client needed to operate on that time? For example I actually needed to tell the computer what time to play the alarm and also how long the snooze lasts and so on? I'd need to start parsing the string and I'd have to tell the computer what timezone the string was in. If I was setting lots of alarms for people all over the world things would get very messy very quickly. So...
On my client I have a timezone picker that selects what timezone to display dates to me in. When I want to set the 8am alarm for my friend in Greenwich I tell my client I want to choose dates in Greenwich time, pick the date and send it. Behind the scenes I will have selected the date that locally would look something like 4pm to me, but would equate to 8am to my friend. Either way it is the same number of milliseconds since epoch.

OMG WTF?
Yeah it can get confusing. Because all the date stuff in Flex hides the timezone you need to do some actual work if you want to operate in timezones other than what is set on your computer. To do the modification might look like this:

var selectedDate : Date = new Date(2010,7,27,8); // 8am 27th Aug 2010 (month is 0 based index)

But that date is in my timezone which is no good. But what if I do this:

selectedDate.minutes -= selectedDate.timezoneOffset;

selectedDate.minutes += myFriendsTimezoneOffset;

In the first line I subtract my timezone offset. The timezone offset is the number of minutes difference between my timezone and UTC (or +0 hours). So if I subtract the timezone offset to the selected date's minutes I will get a Date that is the time I selected (8am) as it would be in UTC, that is 8am UTC which, since I'm in Australia at +8hrs means selectedDate would be 4pm my time.
In the second line I add my friend's timezone offset. This will make the Date 8am in his timezone (which just happens to be +0 since he is at Greenwich). The important thing to note is that the selectedDate object now has a UTC time (milliseconds since epoch) that is equivalent to 8am Greenwich and 4pm Australia time. It can be confusing but stick with it.
Over The Wire
When you serialize the date and send it to the server you are actually sending a UTC time. There is no timezone sent with the Date. If you remember a Date is an instance in time that is unique and absolute, a timezone / locale is something we decorate that Date with so it is relative to us humans. Computers need a single representation of Date so they can compare them and sort them and perform calculations on them. If you have chosen a Date and it is important that other people know what timezone you were in when you chose that Date you need to communicate this separately.
I hope this has helped someone.

Flex File Upload to End All Uploads

2010-06-09T19:49:00.000-07:00

Back to the Future
I previously said I had implemented an FTP based solution for file upload. Well I did and it works pretty well but it's a bit fiddly and tricky to get working so I kept looking for solutions. I found one. Hooray! Flex 4 and FileReference in Flash 10 with Blaze provide the solution.
Big Irritation
A big irritation of mine with http uploads using multi part form stuff is that it is an all or nothing proposal. Once you fire up the connection and start uploading you have to wait for the whole thing to go up. For small files this isn't such a problem but if you upload a file that takes a minute or two you often don't know if it is working (from a client perspective) until you decide that 3 hours is too long for a file to take. There are heaps of issues like socket disconnected by peer, socket read timeout and so forth. To get things working you need to check special upload timeouts and browser timeouts and virus checkers / indexers on the server and other mysterious Mac OS X related muffery. To use file upload over http in a true enterprise application you need some guarantees that the upload will happen and that data isn't lost. You need a reliable system so you can tell your paying customers that their data will get to the server and if it doesn't you know what happened and you can fix it and keep going. Anything less is amateur hour mickey mouse stuff.
Flash 10 FileReference
Flash player 10 allows you to call fileReference.load() to load a selected file. This puts all the file's bytes into the data property. You can then send that as a byte[] through AMF via Blaze to the server where you can save the file. The problem with this simplistic approach is you don't get any progress events and if it is a big file you still end up with all the problems of http upload. The solution is really simple; break the byte array into manageable chunks and send them up one at a time waiting for a response between each one. This way you get a progress event and the system doesn't just freeze up until it's finished. If you are clever you can also enable pause / resume.
I'm Clever
Yeah well so my mother says anyway. But I wrote a nice little flex library and java library which go together to provide file upload services. The flex part takes a byteArray and file name and queries the java service to see what progress the file is up to. The java service replies with a progress event which either says 0 bytes uploaded, some bytes uploaded or complete. You can use that to initialise your progress bar in flex. Then the flex library moves to the correct position in the byteArray and sends a chunk up to java which is written to file and a response sent with progress info. This continues until the file is complete at which point the flex library dispatches a complete event and you can do whatever you like with the file. The java service has some methods that let you get an input stream to the uploaded file and also delete the file once your done with it. If the user cancels midway through the upload (either on purpose or due to error) they can re-upload the file and it will resume from where you left off. Errors are also communicated back to the flex client so you can auto retry or do whatever. Hopefully I'll attach this stuff to my blog. If you like it or find bugs (you probably will) then let me know.
LINKS:
applicationContext-upload.xml
FlexFileUpload.swc
reignite-upload.jar
fileupload.xml
HOW?
add the jar file to your lib folder
include the swc in your flex project
fileupload.xml goes in your web root or docroot whatever you call it
applicationContext-upload.xml is a spring context file so put it where you load them from
If you don't use spring you should, but if you don't, you can just create a facade that adapts the upload service and creates an instance and whatever. Hey you're at least as clever as me so either use spring or take 5 minutes to figure out a way around it (I did for another project that used ejb)
in your remoting-config.xml

<destination id="fileUpload">
<properties>
<factory>spring</factory>
<source>uploadService</source>
</properties>
</destination>
You'll see I'm using the spring factory, if you don't have it do a google search for spring factory for blaze there's also an ejb factory if you are that way inclined.
The fileupload.xml has 2 things in it. The first is the name of the destination you just configured in the remoting-config.xml and the other is the endpoint context path for your channel of choice.
Now to use it in your code:
  1. Create an instance of the au.com.reignite.upload.FileUploadManager and remember to declare it in a place that will have scope for the duration of your upload. Same as FileReference below. You need to pass in some start up arguments when creating FileUploadManager: url to fileupload.xml, securityToken which you wont use so hand in null and repositoryId which is a unique identifier for this user's uploads. I use user id for this so the same user can resume their upload. If you don't use a different id for each user then you'll get stuffed up files if more than one user uploads the same file at the same time.
  2. create a FileReference and get the user to select a file. Once selected call load(). Once complete you can do the rest.
  3. Once the file reference has done its load and you have the data do this:

uploadManager.addEventListener(FileUploadErrorEvent.UPLOAD_ERROR, handleUploadError);
uploadManager.addEventListener(ProgressEvent.PROGRESS, fileUploadProgressHandler);
uploadManager.addEventListener(FileUploadCompleteEvent.COMPLETE, completeUpload);
These are the events that get thrown so you had better handle them. Pretty self explanatory really.
  4. call uploadManager.upload(fileRef.data, fileRef.name); which starts it all off. You'll get at least one progress event and a maximum of one complete event.
IF a user cancels or there is an error and you want to start over make sure you call uploadManager.cancel() otherwise you'll break it. So that means if they close the popup your doing the upload in also make sure you call cancel otherwise the upload continues.
Maybe if I get bored I'll post an example app.

I'm back with egg on my face and all

2010-06-01T22:37:00.000-07:00

So I figured I was posting off into oblivion happily talking to myself but it turns out some people read my blog and commented and I didn't think to check back. Oops.
Anyway the good news is I have bonus solutions to tricky problems.
Clustered Blaze Messaging III
I previously talked about a couple of different ways to get around the problem of having Blaze behind a hardware firewall that did round robin load balancing.

Just a recap: the client subscription lives with the servlet context on the machine they first sent their subscription to. This means when their next poll request hits the load balancer they tend to get fired off to the other server and have no subscription. I tried to use JGroups and Tomcat session replication but both methods suck a fair bit and kept falling over.

The answer to my woes has been database guaranteed subscription and message queuing.
In Brief
Client A sends subscribe request to server A. Server A creates an entry in the database (replicated using mysql replication) for that subscription. Client A then polls and hits server B which looks in the database for a subscription and finds it. Server B then checks the database message queue for any messages client A has not received and sends them or waits the prescribed time for a message to enter the queue.
So you see the messages get stored in a database table and the subscription keeps track of the last message id received. Join the dots and you've got a messaging system that relies on MySQL replication (or whatever DB you are using) which is very reliable. I say reliable because in the nearly 2 years this particular MySQL replication set up has been operating it only got out of sync once and that was due to an OS failure (at Christmas while I was on leave of course so I was called up and spent a lovely holiday reinitialising databases instead of drinking beer and stuffing my gob).
Now I will try to add the code:
In messaging-config.xml:

<?xml version="1.0" encoding="UTF-8"?>
<service id="message-service" class="flex.messaging.services.MessageService">
<adapters>
<adapter-definition id="actionscript" class="au.com.reignite.amf.DatabaseMessagingAdapter"
default="true" />
</adapters>
<default-channels>
<channel ref="my-polling-amf" />
</default-channels>
<destination id="projectLockService"/>
</service>
Everything is pretty normal except the DatabaseMessagingAdapter which is the thing that does the database magic.
There's a stack of stuff in the DatabaseMessagingAdapter that is a bit specific to my particular architecture so I'll just show the important bits:
DatabaseMessagingAdapter

public class DatabaseMessagingAdapter extends MessagingAdapter
Obviously I extend the MessagingAdapter.

@Override
public Object invoke(Message message) {

ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out;
try {
out = new ObjectOutputStream(bos);
out.writeObject(message);
getMessageOrchestrator().storeMessage(bos.toByteArray(),
message.getBody().toString());
} catch (IOException e) {
LogWriter.error(getClass(), "Failed to serialize message: " + e, e);
}

return null;
}
Here I override the invoke() method because it is what Blaze calls when it wants to store the message in its internal store. Instead you can see I serialize the message to a byte array and use my "MessageOrchestroator" to store it. This is just my class that saves the bytes to the database in the following table:

DROP TABLE IF EXISTS `truenorth`.`blaze_message`;
CREATE TABLE `truenorth`.`blaze_message` (
  `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `MESSAGE_BYTES` blob NOT NULL,
  `MESSAGE_BODY` text,
  `CREATED_DATE` datetime NOT NULL,
  `MODIFIED_DATE` datetime NOT NULL,
  PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=54 DEFAULT CHARSET=latin1;
My messages are in XML so I store the message bytes and the message body as a string so I can inspect them with my greedy eyes.
Back to DatabaseMessagingAdapter:

@Override
public boolean handlesSubscriptions() {
return true;
}
Of course it handles subscriptions too! so I return true (the default is false).

@Override
public Object manage(CommandMessage commandMessage) {

if (commandMessage.getOperation() == CommandMessage.SUBSCRIBE_OPERATION) {

MessageDestination destination = (MessageDestination) getDestination();

SubscriptionManager subscriptionManager = destination
.getSubscriptionManager();
String selectorExpr = (String) commandMessage
.getHeader(CommandMessage.SELECTOR_HEADER);
subscriptionManager.removeSubscriber(commandMessage.getClientId(),
selectorExpr, (String) commandMessage
.getHeader(AsyncMessage.SUBTOPIC_HEADER_NAME),
(String) commandMessage.getHeader(Message.ENDPOINT_HEADER));
getMessageOrchestrator().storeSubscription(
FlexContext.getFlexClient().getId(),
commandMessage.getClientId().toString(), selectorExpr);
} else if (commandMessage.getOperation() == CommandMessage.UNSUBSCRIBE_OPERATION) {
if (FlexContext.getFlexClient() != null) {

getMessageOrchestrator().deleteSubscription(

FlexContext.getFlexClient().getId());
}
}
return null;
}
This is the bit that manages subscriptions, as darkly foreshadowed by me above. If it is a subscribe operation I clear out Blazes storage of the subscription otherwise it annoys the heck out of me by denying people who shouldn't be denied. By I grab my trusty MessageOrchestrator and store my own subscription in a table like:

DROP TABLE IF EXISTS `truenorth`.`subscription`;
CREATE TABLE `truenorth`.`subscription` (
  `ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `CLIENT_ID` varchar(100) NOT NULL,
  `MESSAGE_CLIENT_ID` varchar(100) NOT NULL,
  `SELECTOR` varchar(255) DEFAULT NULL,
  `EXPIRY_DATE` datetime NOT NULL,
  `LAST_MESSAGE_ID` int(10) unsigned NOT NULL,
  `CREATED_DATE` datetime NOT NULL,
  `MODIFIED_DATE` datetime NOT NULL,
  PRIMARY KEY (`ID`),
  UNIQUE KEY `Index_CLIENT_ID_UNIQUE` (`CLIENT_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The important bit to note is I store the client_id which is the id of the instance of flash running on the client machine and the message_client_id which is the instance of the consumer object running in the flash client on the client machine. You can also see I keep the last message id sent and some datetime things.
Ok so that's how I store subscriptions and messages. The next piece of the puzzle is how I handle poll requests.
Poll requests
In my services-config.xml I have a special endpoint for polling: DatabaseConfigurableAMFEndpoint
public class DatabaseConfigurableAMFEndpoint extends ConfigurableAMFEndpoint {
I extend my own ConfigurableAMFEndpoint so I can use my super special filter chain to do my filthy security processing (see previous blogs)

@Override
protected FlushResult handleFlexClientPoll(FlexClient flexClient,
CommandMessage pollCommand) {

I'll skip including all the code because it's long and boring so I'll summarise.
I load my subscription, which is the data stored in the above table not the Blaze subscription based on the flex client id (that's as opposed to the message client id). If I don't get a record I throw a subscription error. Assuming they have a subscription I start a "do" loop where I load up the messages stored in the above table that have an id greater than the subscription's last received message. Each of those messages that match the subscription selector (I use the JMSSelector object that comes with Blaze just like Blaze does) I rehydrate and add to my results to send. If there are any results I break the loop and immediately send them. If there were no messages I wait a little bit (5 seconds) and try again. I keep doing that until the configured poll timeout (50 seconds) at which point I release the thread and let the client poll again. I then update their subscription with the last message they received and increment their expiry date.

To finish it all off I have a scheduled task (I use quartz with spring) that every few minutes culls any expired subscriptions and deletes messages that all subscriptions have received.

Victory

This method of messaging has been working now for about 10 months without failure. The concerns I have about it that it is fairly heavy on hitting the DB so I made sure the DB is on the same machine as the app server and the two machines in the cluster are directly connected via Gigabit Ethernet on the same switch (actually 2 switches cross linked for redundancy). I've also got a fair query cache set up so I'm not always pulling the same messages out of the DB and I'm only really catering to a max of about 10 concurrent users. Having said all that I'd imagine I could have a bog load more concurrent users and still have no trouble. A big benefit is the messages are held by the DB not in memory by the tomcat instance which means tomcat can fall over and people still get their messages.

Next Time

I've also now come up with what I consider the ultimate file upload solution even more betterer than ftp and I've set my blog to email me when someone comments so I can actually respond.

Get around file upload in Flex and Java

2009-10-26T19:17:00.000-07:00

Http File Upload

My goodness me but I've had trouble with file upload for web sites. All sorts of issues ranging from timeouts to size restrictions to firewalls and now most recently Mac OS X 10.5 server just likes to keep a hold of uploaded file streams before releasing them to Tomcat.

The Scenario

Mac OS X 10.5 server on a mac mini with apache 2 and tomcat. Apache is has my web app as a virtual host which lives in Tomcat (6.18 or something). I've got a flex 3.4 client trying to do an http upload to my app. Seems to work up to a point then my Java app spits the dummy saying stream connection timeout! Bah! I traced it out and found that when my servlet should be reading the file input stream the file is not in my temporary directory yet. Mac OS X has subverted the file for its own nefarious purposes and left me hanging. I tried to stop indexing, stop finder, stop any sort of virus scanning and so on but no good. OS X keeps holding onto the file. Seems to only happen on multi-page pdf and word documents. Jpgs and pngs are passed straight through.

The Fix

So I've had enough with http file upload and its various vagaries and config problems and OSes sticking their noses in where they aren't wanted. I decided to create a custom FTP solution. Everyone loves FTP! It was actually designed to upload files. But how do I do it?

Server Side

Hooray for Apache FTP Server. It's an open source Java ftp server based on the mina project. So I grabbed it source code and all and began butchering. Firstly it uses Spring for configuration and my app uses Spring for configuration so that is nice. Secondly it allows you to implement your own user manager and your own list of commands. So I created a user manager that would authenticate with my app's authentication system and added a custom command that would save uploaded files into my own file system and data base (all files are actually referenced in the app by DB entry and the DB entry points to the file system). That made for a neat little wrap.

UH OH Flex Sucks Big Ones

Flex is a fantastic bit of work from Adobe; an object oriented (mostly) language and framework for developing rich internet applications. Yum. If it's an app your building (not someones home page) then flex kicks seven kinds of poo out of html. But still, in some ways it sucks pretty hard. The hardest suck is its socket API. You can create a socket and connect to a server on any old port then read and write bytes as you'd expect. However the Flash player hides all the good stuff from you. So you can say socket.write(mybytes) and then socket.flush() but do you think the Flash player will actually send the bytes over the wire when you flush? WRONG, FAIL. The bytes go when Flash says they can go and you wont know when that is. More importantly in the case of my FTP solution you can merrily send bytes onto the socket and flush them and then close the socket only to find that the ftp server closes the socket before Flash has finished flushing! That means you get truncated files and no progress bar on the client side.

Hooray For Open Source

"No worries" says I. I've got my FTP server code and Eclipse so I can do anything. I in fact alter the server code to send back status events with the number of bytes actually read. Flex then listens to these on the control socket and updates a progress bar. Once Flex knows that all the bytes have actually been written and received it closes the socket and logs off the FTP session. The really good thing about this is that I've been able to configure the FTP server so that the only commands available are those required to authenticate and my custom upload command. So no mister hacker you can't just start mucking about with my ftp server and create accounts etc... In fact all you can do is upload files that conform to my app's specification and they aren't executed files.

That's It

So that is the solution to my http upload woes. If you or a person you know suffers from file upload issues I can post some code and such. But since no-one follows my blog anyway I can't be bothered unless requested.

New Improved Industrial Strength Whoop Tooshy

2009-08-11T00:31:00.000-07:00

As advertised in my earlier blog that noone read I found my Blaze cluster solution didn't work.

The Reason:
So creating a second subscription on the second server in the cluster stopped the "no active subscription" error which is nice, but it also tended to mean you didn't get your messages when polling against that server.
Clearly I needed a solution, one that actually works (rather than the half arsed hack I did before).

The Solution:
I still used the configurableAMFEndpoint and the ClusterMessagingFilterThingy but I changed some code in the filter. Now When a subscribe command comes in I store in the Tomcat session the cluster address of the server receiving the subscribe command (I still use the flexClientId + endpoint as the key). Then I extended the AMFEndPoint and overrode the handleFlexClientPollCommand() method to check the Tomcat session to see if the current server was the subscription server or some other server. If it was the subscription server I just continue on as normal. Otherwise I use an extended MessageService to grab the cluster manager and send the poll command to the subscription server. The how of how I did that is a kind of magic. I had to extend the JGroupsCluster object and some other JGroups thingy and I had to modify the Blaze JGroupsCluster to allow my extended object access to some of its guts. The real trick lies in once your subscription server has the poll request you need to launch a new thread to do the poll otherwise the JGroups cluster is blocked by the poll wait time (in my case 50 seconds because Mac Safari has a 60 second timeout on open but unanswered requests).

All The Code:
All the code can be added, but I'd need someone to actually ask for it because there is a fair bit of it and I'm essentially pretty lazy. No. You only think you understand how lazy I am, in reality I'm even more lazy than what you think. Kind of like estimating time to perform any given task in a project; you think you've added enough padding but no matter how much you add you always need to double it.

CIAO.

Hardware Load Balanced BlazeDS Cluster on Tomcat

2009-07-19T06:29:00.000-07:00

Blaze DS:
Blaze is an open source remoting and messaging system from Adobe. I use the Java version because that's the crazy devil-may-care fool I am. Blaze is really cool (assuming first you find web technologies cool) and has made my life considerably easier. In the past I used web services (Axis anyone?) but when developing rich Internet applications I found web services to be too cumbersome in terms of transferring stacks of complex data as objects. Blaze also does "push" which means I can have multi-user real time apps where the users can interact with shared data. Of course Blaze, being from Adobe, is used in particular to communicate between Flex and my services. Flex is fantastic too! It does of course have draw backs, but in all pretty good.

Clustering:
There comes a time in every young server's life when thoughts begin to turn toward other servers and they get together to share certain aspects of their life. I had to take two Mac Minis with OS X Server, each with Tomcat, MySQL, of course Blaze and I had to turn them into a mighty cluster. The Blaze manual says that HttpFlexSession does not cluster so you need to make sure that once a client reaches a server it should "stick" to that server so it has access to the appropriate HttpFlexSession. That would be wonderful except this is the real world where clusters sit behind a hardware load balancer and we need to not only balance load but also provide failover. This is tricky because it means that a client doesn't know which machine they will hit and they also don't have direct access to the server IP addresses so no you can't use the built in Flex failover.

JGroups:
BlazeDS uses JGroups to ensure that messages generated or sent to one server reaches all the others in the cluster. It is really easy to set up and the Blaze manual show you how... Except it is wrong. oops. Caused me trouble, but I got over it. The problem is in the jgroups-tcp.xml explanation. The num_initial_members="2" is explained as having a value equal to the number of members in the element. An important point to creating a JGroups cluster that can survive restarts and such is setting up the ports well.
<TCP ... start_port="7800" end_port="7804" ...>

Not that I've created a range of ports for the TCP element. Because when old Bessy fires up port 7800 may be in use! It may be in use by JGroups because it hasn't been released yet or because some other mysterious creature wants it. Anyway it is more rugged to give a port range. Then in the TCPPING element
<TCPPING ... initial_hosts="10.1.1.1[7800]" port_range="5" num_initial_members="2" />

So it will look for host 10.1.1.1 at port 7800 but if that fails it will check ports 7801,7802,7803 and 7804. This makes it all a little more able to deal with the ups and downs of modern server life.

Tomcat Clusters:
My first hurdle was to ensure that the Tomcat sessions were being replicated across servers. The HttpFlexSession is stored in the Tomcat session so by replicating the Tomcat session I get the HttpFlexSession! Hooray! Of course that didn't really work in my case. So anyway to set up a Tomcat cluster so it replicates sessions is really easy (unless you are reading this out of frustration hoping to find a secret answer).
In your web.xml but just below your tags. Then in the server.xml in the Tomcat conf directly uncomment the tag. I also put the sendChannelOptions="4" attribute in because I wanted to make sure that the session replication occurred before the request returned (4 = synchronous while the default is 8 which is async). That's all there is to it. Though if you are dong things a bit trickier you better check the Tomcat doco.

The Cluster Problem:
I use Blaze messaging to push updates to the users of my app. That means the client creates a Consumer and subscribes to a channel / destination at an endpoint. The endpoint is the load balancer IP address and the load balancer decides which server the Consumer subscribes to. But then when the Consumer polls (I use "long polling") it hits the load balancer again and might end up polling ther server it is not subscribed to and you get an error back and you probably start crying (that's what I did). This is simply because the subscription is held in the SubscriptionManager on the Destination which is created in the MessageBrokerServlet and so is specific to the server and not held in the HttpFlexSession let alone the Tomcat session. Ack! To make things a little interesting the subscription is created by the FlexClient which is held in the HttpFlexSession but because the FlexClient and MessagingClient are not added using the setAttribute() method of the Tomcat session the Tomcat server replication doesn't copy it across.

Solution:
Warning, the following doesn't really work :(
The following doesn't work as advertised I'm sorry (not that anyone reads this blog anyway). The configurable endpoint is good and I still use the cluster filter bizzo but the subscribing on another server thing fails somewhat. It works a bit, but also doesn't a bit. A shame really. Anyway check out my other post on how to make is good and worky.
Here is what I did: I rolled my sleeves up spat on my hands and brewed a strong cup of harden up. Also I hacked up Blaze and made it excellent.
1. Create ConfigurableAMFEndpoint.java

package flex.messaging.endpoints;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

import flex.messaging.config.ConfigMap;
import flex.messaging.endpoints.amf.AMFFilter;
import flex.messaging.endpoints.amf.EndPointAwareFilter;

/**
* @author surrey
*
*/
public class ConfigurableAMFEndpoint extends AMFEndpoint {

private List filters = new ArrayList();

protected AMFFilter createFilterChain() {
AMFFilter firstFilter = super.createFilterChain();
AMFFilter lastFilter = getLastFilter(firstFilter);
for (AMFFilter filter : filters) {
// add this endpoint on those filters that take an endpoint.
if (filter instanceof EndPointAwareFilter) {
((EndPointAwareFilter) filter).setEndpoint(this);
}
lastFilter.setNext(filter);
}
return firstFilter;
}

private AMFFilter getLastFilter(AMFFilter filterChain) {
AMFFilter last = filterChain;
while (last.getNext() != null) {
last = last.getNext();
}
return last;
}

public void initialize(String id, ConfigMap properties) {
super.initialize(id, properties);
ConfigMap filterMap = properties.getPropertyAsMap("filter-chain", null);
if (filterMap != null && !filterMap.isEmpty()) {
List filterNames = filterMap.getPropertyAsList("filter-class",
null);
if (filterNames != null) {
for (Iterator iter = filterNames.iterator(); iter.hasNext();) {
String className = (String) iter.next();
try {
filters.add((AMFFilter) Class.forName(className)
.newInstance());
} catch (InstantiationException e) {
log.error("Could not instantiate filter: " + className,
e);
} catch (IllegalAccessException e) {
log.error("No public constructor for filter: "
+ className, e);
} catch (ClassNotFoundException e) {
log.error("Could not find filter class: " + className,
e);
}
}
}
}
}

}

This endpoint extends the AMFEndpoint by allowing me to add extra filters to the Blaze filter chain.
2. I modified MessageBrokerFilter to have:
if (next != null) {
next.invoke(context);
}

// Service the message.
outMessage = endpoint.serviceMessage(inMessage);
So that the chain continues on if I've added any.
3. I created an EndPointAwareFilter which I could extend by my own filters so that they automagically get their EndPoint added to them. I've created a couple other filters too such as a security filter which allows me to use custom security with AMF headers. That's another story though because I had to hack up some Flex code and add more functionality to Blaze.

package flex.messaging.endpoints.amf;

import flex.messaging.endpoints.AMFEndpoint;

/**
* An abstract AMFFilter that other filters should extend to gain access to
* their endpoint. This is only populated when using a ConfigurableAMFEndpoint.
*
* @author Surrey
*
*/
public abstract class EndPointAwareFilter extends AMFFilter {

private AMFEndpoint endpoint;

public AMFEndpoint getEndpoint() {
return endpoint;
}

public void setEndpoint(AMFEndpoint endpoint) {
this.endpoint = endpoint;
}
}

4.. I created ClusterMessagingFilter.java

package au.com.truenorth.amf.filter;

import java.io.IOException;
import java.lang.reflect.Array;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import javax.servlet.http.HttpSession;

import flex.messaging.Destination;
import flex.messaging.FlexContext;
import flex.messaging.MessageDestination;
import flex.messaging.endpoints.amf.EndPointAwareFilter;
import flex.messaging.io.amf.ActionContext;
import flex.messaging.io.amf.MessageBody;
import flex.messaging.messages.AsyncMessage;
import flex.messaging.messages.CommandMessage;
import flex.messaging.services.Service;

public class ClusterMessagingFilter extends EndPointAwareFilter {

public ClusterMessagingFilter() {
super();
}

@SuppressWarnings("unchecked")
@Override
public void invoke(ActionContext context) throws IOException {

Object data = getData(context);

if (data instanceof CommandMessage) {
CommandMessage command = (CommandMessage) data;
int operation = command.getOperation();
Object clientId = FlexContext.getFlexClient().getId();
String endpointId = getEndpoint().getId();

if (operation == CommandMessage.POLL_OPERATION) {
// add a subscription if appropriate
HttpSession sess = FlexContext.getHttpRequest().getSession();
Object att = sess
.getAttribute(clientId.toString() + endpointId);
if (att != null && att instanceof List) {
List> destList = (List>) att;
for (Map subscribeMap : destList) {
MessageDestination msgDest = getMessageDestination(subscribeMap);
Object cId = subscribeMap.get("clientId");
if (msgDest != null
&& msgDest.getSubscriptionManager()
.getSubscriber(cId) == null) {
Object selectorExpr = subscribeMap
.get("selectorExpr");
String selectorString = getStringOrNull(selectorExpr);
Object subtopic = subscribeMap
.get("subtopicString");
String subtopicString = getStringOrNull(subtopic);

msgDest.getSubscriptionManager().addSubscriber(cId,
selectorString, subtopicString,
subscribeMap.get("endpointId").toString());
}
}
sess.removeAttribute(clientId.toString() + endpointId);
}
} else if (operation == CommandMessage.SUBSCRIBE_OPERATION) {
// get httpSession and stick a flag in with everything needed to
// make a subscription
Object att = FlexContext.getHttpRequest().getSession()
.getAttribute(clientId.toString() + endpointId);
List> destList = (List>) att;
if (destList == null) {
destList = new ArrayList>();
}

Map subscribeMap = createSubscribeMap(command);
destList.add(subscribeMap);
FlexContext.getHttpRequest().getSession().setAttribute(
clientId.toString() + endpointId, destList);
}
}
if (next != null) {
next.invoke(context);
}
}

private Map createSubscribeMap(CommandMessage command) {
Map subscribeMap = new HashMap();

subscribeMap.put("clientId", command.getClientId());
subscribeMap.put("endpointId", getEndpoint().getId());
subscribeMap.put("subtopicString", command
.getHeader(AsyncMessage.SUBTOPIC_HEADER_NAME));
subscribeMap.put("selectorExpr", command
.getHeader(CommandMessage.SELECTOR_HEADER));
subscribeMap.put("destination", command.getDestination());
return subscribeMap;
}

private String getStringOrNull(Object selectorExpr) {
String selectorString = null;
if (selectorExpr != null) {
selectorString = selectorExpr.toString();
}
return selectorString;
}

private MessageDestination getMessageDestination(
Map subscribeMap) {
String destId = subscribeMap.get("destination").toString();
MessageDestination msgDest = null;
for (Iterator iter = getEndpoint().getMessageBroker().getServices()
.keySet().iterator(); iter.hasNext();) {
Service service = (Service) getEndpoint().getMessageBroker()
.getServices().get((String) iter.next());
Destination dest = service.getDestination(destId);
if (dest != null && dest instanceof MessageDestination) {
msgDest = (MessageDestination) dest;
break;
}
}
return msgDest;
}

@SuppressWarnings("unchecked")
protected Object getData(ActionContext context) {
// get the in message to inspect the headers
MessageBody request = context.getRequestMessageBody();

Object data = request.getData();
if (data instanceof List) {
data = ((List) data).get(0);
} else if (data.getClass().isArray()) {
data = Array.get(data, 0);
}
return data;
}

}

This filter is a little messier than I like and the honest to goodness production version is a bit nicer but you get the picture. Essentially when a client subscribes to a server that server adds all the required info to subscribe into the Tomcat session so that when a poll hits another server I can get that info out and create a subscription on that server and this all happens before any servicing of the poll occurs so I don't get any dumb "you are not subscribed" errors.

The most amazing thing is this works! I was actually able to sit there with my colleague logged in to the system with both of us using it while I turned off first one server, brought it back up then turned off the other server. The whole time the system just kept polling and as users we didn't even know the server had died and come back up. Brilliant.

Oh the services-config.xml

<clusters><br />        <cluster id="default-cluster" properties="jgroups-tcp.xml" default="false" balancing="false"><br />    </clusters><br /><!-- the above is added so I can use clusters.  Note that I set default="false" because I'm a control freak -->
<channel-definition id="my-polling-amf" class="mx.messaging.channels.AMFChannel">
         <endpoint url="@AMF_POLLING_END_POINT_BASE@" class="flex.messaging.endpoints.ConfigurableAMFEndpoint">
         <properties>
             <filter-chain>
                 <filter-class>au.com.truenorth.amf.filter.ClusterMessagingFilter</filter-class>
             </filter-chain>
             <polling-enabled>true</polling-enabled>
             <wait-interval-millis>50000</wait-interval-millis>
             <polling-interval-seconds>0</polling-interval-seconds>
             <max-waiting-poll-requests>50</max-waiting-poll-requests>
             <serialization>
                 <type-marshaller>au.com.truenorth.service.bean.NumberHandlingTypeMarshaller</type-marshaller>
             </serialization>
         </properties>
     </endpoint>
<!-- This is my channel definition where I set up the long polling and configure the ClusterMessagingFilter -->
<!-- You'll also note, if you are clever, the NumberHandlingTypeMarshaller which allows me to send Null numbers to Flex.  I also modified Blaze to turn NaN numbers into null Numbers (like Integer) coming into Java so my Hibernate wouldn't go spaz when id fields were NaN when they should be null. --></channel-definition>

Well this has been a long one but I hope at least one person finds it helpful.

FOP Positive Descender Underline Bug

2009-01-18T18:38:00.000-08:00

Well here is my cowardly effort at giving back to the open source community. But first! What is the open source community?

Nerds:
It's all about nerds. I'm one and so are you if you are reading this after doing a google search for a solution to the positive descender underline bug in Apache FOP. But if not I'll give a quick run down. In the world of computer programming there is a movement which holds dear the concept that ideas and information are free. Free as in unshackled, free as in free to be exchanged. Sometimes you have to pay money for the info and ideas, but if / when you do you should have access to the whole idea. The same way when you buy a car you are allowed to open the bonnet and tinker with the guts. So programmers often allow others to see their work and we are all enriched by it. For a more detailed discussion: open source [wikipedia]

The Bug:
The bug I have solved manifests itself in a bit of software called Apache FOP. This software allows me to take some stuff drawn on the screen in Adobe Flash and turn it into pdf documents or png images. The problem happens when you try to underline text in an embedded font which has a positive descender. A positive descender means that part of the text of the font hangs below the baseline. For example in Lucida Sans the p, y, j and g all have little squiggly bits hanging below the base of the other characters. In this instance FOP fails to draw an underline because it throws an error that says it can't draw a border with a negative height.

Suspect Math:
The cause is a bit of suspect math in the underline

public abstract class AbstractPathOrientedRenderer extends PrintRenderer {
...
protected void renderTextDecoration(FontMetrics fm, int fontsize, InlineArea inline,
int baseline, int startx) {
boolean hasTextDeco = inline.hasUnderline()
|| inline.hasOverline()
|| inline.hasLineThrough();
if (hasTextDeco) {
endTextObject();
float descender = fm.getDescender(fontsize) / 1000f;
float capHeight = fm.getCapHeight(fontsize) / 1000f;
float halfLineWidth = descender / -8f / 2f;
float endx = (startx + inline.getIPD()) / 1000f;
if (inline.hasUnderline()) {
Color ct = (Color) inline.getTrait(Trait.UNDERLINE_COLOR);
float y = baseline - descender / 2f;
drawBorderLine(startx / 1000f, (y - halfLineWidth) / 1000f,
endx, (y + halfLineWidth) / 1000f,
true, true, Constants.EN_SOLID, ct);
}...

So you can see the bolded parts involve some calculation of a y position based on the descender value of the font. If the descender value is negative it all works nicely, if it is positive we end of with an inverted set of values in the call to drawBorderLine() method.

My Solution:
Here you see what I changed:

protected void renderTextDecoration(FontMetrics fm, int fontsize, InlineArea inline,
int baseline, int startx) {
boolean hasTextDeco = inline.hasUnderline()
|| inline.hasOverline()
|| inline.hasLineThrough();
if (hasTextDeco) {
endTextObject();
float descender = Math.abs(fm.getDescender(fontsize) / 1000f);
float capHeight = fm.getCapHeight(fontsize) / 1000f;
float halfLineWidth = descender / 16f;
float endx = (startx + inline.getIPD()) / 1000f;
if (inline.hasUnderline()) {
Color ct = (Color) inline.getTrait(Trait.UNDERLINE_COLOR);
float y = baseline + descender / 2f;
drawBorderLine(startx / 1000f, (y - halfLineWidth) / 1000f,
endx, (y + halfLineWidth) / 1000f,
true, true, Constants.EN_SOLID, ct);
}...

Now the bolded bits firstly show I get the absolute value of the descender, I divide it by 16 (rather than by 8 then by 2) and finally I add half the descender value to the baseline to get the y position.
The short of it is that the code now works for both negative and positive value descenders. The world rejoice!

Did I submit this back to the opensource project? No, because I've already hacked up my copy of FOP beyond recognition to reduce the on disk output size of pdfs using large svg images (by reducing the number of decimal points of precision from 8 to 3) and using a file backed caching system to step around the memory issues associated with printing said large pdfs. Feel free to ask me for more details about those fixes, if I don't get around to writing about them here.

Woodwork, My Bedside Table

2009-01-14T21:16:00.000-08:00

I love making stuff. By trade I make software but I have trouble explaining to non-software types what it is that I have done. It is difficult to show my wife the amazing indexed search system I implemented or explain how I overcame the font underline issues in Apache FOP when using a font with a positive decender value. But wait! Woodwork on the other hand produces a tangible item that can be touched and felt, smelt and tasted... though you probably shouldn't.

Project 1: My Computer Desk
I used to live in Melbourne but since I failed to grow gills and/or a protective layer of fur and blubber (working on the blubber) I moved to somewhere sunny and warm. In the move I threw out my old student desk because I was struck by a sever case of good taste. I was left without a desk, but gained a large room I could work in. The short of it is that I designed and built a computer desk with three draws on one side and a shelf on the other for my computer. It turned out ok though it is clear there is a good reason we pay cabinet makers to build such things. If you cared to read my previous post you would understand that I spent a lot of time, little money and no experience in creating my desk. I really needed to spend more time I think.

Project 2: My Bedside Table
In this new house I am in there is a "granny flat" where my parents stayed while they visited. In it we put a bed but alas, no table beside it! So I am making a bedside table. I've got a couple more tools and a computer desks worth of experience.

The design:

A marvel of engineering and sensible elegance no? The whole thing is to be made of pine then stained and sealed. I've already made a start.
What I've done:

Here you can see one completed shelf and the four frame pieces for the second shelf. I'm using lap joins to cross hatch the frame because I think I'm tricky. To cut out the waste of the lap joins I initially used a hand tenon saw to cut each edge then make three cuts in the waste. I then used a chisel to clear the waste. This is what my big book of wood working tells me to do. What the big book doesn't say is that if you are an inexperienced tit with about seven thumbs, most of them on your left hand, you have little chance of making rebates that will fit together. I had to learn that the hard way.

Difficulties:
None of my joins worked. Each plank had what looked like a suitable rebate but when put together they didn't sit flat and were really hard to squeeze in to place. Not to fear I'm a software engineer! When experience lets me down (I have so little) I look for someone else's technology to save me. I essentially wanted to download the Apache Joint Maker 4 Pine but that doesn't exist. Instead I used my Christmas gift vouchers to buy a router. I set the depth and ran the router over each of my chiseled rebates and it sheared off the wood I'd missed and now all the joins meet perfectly.

The actual shelf bit is made of three planks joined side by side with glue (I just need it strong enough to hold while I attach them) and joined to two of the cross pieces by screws hidden in pocket holes.

Next:
The next step is to make the legs and cut the housing holes for the shleves then a lick of stain and estapol and she'll be right.

Another project?
After I've done this I think I might make a little TV stand for my spare TV. Really just another excuse to use power tools and learn a bit more. I've told people I'm going to retire early (about 40) and they say "But what will you do? Wont you get bored?" (they actually misspell won't when speaking... somehow) As if the only thing I do is work! Pah. I intend to make stuff out of wood and build a car (from the ground up, not just do up an existing one) and race a car and probably crash a car and injure myself with power tools. What more could a man want?

I'll update my blog with my progress. Also I might write a bit about how it is that I intend to retire so young.

Three Exchangable Comodities

2009-01-07T22:20:00.000-08:00

In my world view, soon surely to be yours too, there are generally speaking three exchangable comodities that any human may possess. These can be used in some combination to achieve nearly anything you might choose to do.

Time
Money
Experience / Knowledge

These must be considered in the context of a specific task. For that task then a person has a pool of each resource and a lack in one can be compensated for by an abundance in another.

Time:
Given enough time a person can do anything. The sad thing being that many tasks require so much time that no one person can supply it. For example, say I want to build a house: With enough time I could build the mud bricks I'd need, then lay them (having designed the house) then make tiles / shingles and so on and so forth. This is of course how they did it many years ago, possibly as early as 1982, though I'm not sure humans had discovered shelter yet...

Money:
Ah, money. Many people really don't understand what money is, but this three comodity model is reasonably good at demonstrating it. Money is an abstract representation of applicable resource. It is a way of trading your experience and time for someone else's experience and time. In the above example it would take me many long hours (lifetimes?) to build a house. However I am quite skilled at developing software solutions to business problems. I get paid to use my skill over time so I can use that money to exchange for someone else's skill and time to build my house. Lovely. That really is what money is about. How else can I trade my knowledge with a builder? What use has a brick layer for a multi-user distributed database application? Well arguments could be made, but it general they would prefer $$$, though beer is a close second.

Experience / Knowledge:
The final piece of the puzzle. If you have plenty of experience doing something, generally speaking the less time you need to do it. If you can do it you don't need to pay someone else to do it. To do a basic service of my car takes me a good 3-4 hours. That's changing oil, filters, plugs and checking brakes etc... A good mechanic can probably do it in 1 hour (maybe less? heck I'm not even experienced enough to know.)

So time, money and experience are exchangable. Or at least that is how I look at most problems I run into. I use this theory to help evaluate various decisions I have to make.

Service my car:

Time: 3-4 hours
Money: $100
Experience: not much.

If I want to spend less money I'd need either enough time to develope and build my own oil filter or lubricant or I'd need enough experience and knowledge to find good cheaper substitutes.
To reduce time I could pay a mechanic or just know how to do everything without need to constantly look up my books.
To reduce my experience required I could pay a mechanic or spend longer learning everything.

I guess you get the picture.

Cognitive Imperial, I

2009-01-07T18:26:00.000-08:00

Cognitive imperialism is a fancy-pants way of describing the act of convincing others to a particular way of thinking; that is building an empire of thought.

Yes, I did major in Arts at university and so undertook philosophy, psychology, literature and other perverse studies. Fear not however! I quickly saw the error of my ways and changed to a double major of computing and marketing in the school of business (I actually wanted a job when I finished).

So this blog is my cognitive imperialism, my attempt to express my world view in a way that conquors the thoughts of others and sways them to hold my beliefs. Failing that I'll just discuss my thoughts on cars, motor-sport, computing, home improvement and possibly current events relating to free thought. So in other words much like every other blog. I will attempt to provide some level of contraversial or maybe thought provoking content, or in the case of my computing discussions, useful solutions to ugly problems.

So with no more time wasted on introductions I will present my first mind expanding thought:
Farts, along with sudden groin hits, are surely the only universal comedy. I recon that if aliens were to land and we wanted to give them a good giggle all we need to do is wait for a dramatic pause in the greetings and let loose with a good, loud and squeeky fart. What a way to break the galactic ice?

That aside I'll be back to discuss my current wood-working project: a bedside table! Hooray for excitement.