Monday, October 26, 2009

Get around file upload in Flex and Java

Http File Upload
My goodness me but I've had trouble with file upload for web sites. All sorts of issues ranging from timeouts to size restrictions to firewalls and now most recently Mac OS X 10.5 server just likes to keep a hold of uploaded file streams before releasing them to Tomcat.

The Scenario
Mac OS X 10.5 server on a mac mini with apache 2 and tomcat. Apache is has my web app as a virtual host which lives in Tomcat (6.18 or something). I've got a flex 3.4 client trying to do an http upload to my app. Seems to work up to a point then my Java app spits the dummy saying stream connection timeout! Bah! I traced it out and found that when my servlet should be reading the file input stream the file is not in my temporary directory yet. Mac OS X has subverted the file for its own nefarious purposes and left me hanging. I tried to stop indexing, stop finder, stop any sort of virus scanning and so on but no good. OS X keeps holding onto the file. Seems to only happen on multi-page pdf and word documents. Jpgs and pngs are passed straight through.

The Fix
So I've had enough with http file upload and its various vagaries and config problems and OSes sticking their noses in where they aren't wanted. I decided to create a custom FTP solution. Everyone loves FTP! It was actually designed to upload files. But how do I do it?

Server Side
Hooray for Apache FTP Server. It's an open source Java ftp server based on the mina project. So I grabbed it source code and all and began butchering. Firstly it uses Spring for configuration and my app uses Spring for configuration so that is nice. Secondly it allows you to implement your own user manager and your own list of commands. So I created a user manager that would authenticate with my app's authentication system and added a custom command that would save uploaded files into my own file system and data base (all files are actually referenced in the app by DB entry and the DB entry points to the file system). That made for a neat little wrap.

UH OH Flex Sucks Big Ones
Flex is a fantastic bit of work from Adobe; an object oriented (mostly) language and framework for developing rich internet applications. Yum. If it's an app your building (not someones home page) then flex kicks seven kinds of poo out of html. But still, in some ways it sucks pretty hard. The hardest suck is its socket API. You can create a socket and connect to a server on any old port then read and write bytes as you'd expect. However the Flash player hides all the good stuff from you. So you can say socket.write(mybytes) and then socket.flush() but do you think the Flash player will actually send the bytes over the wire when you flush? WRONG, FAIL. The bytes go when Flash says they can go and you wont know when that is. More importantly in the case of my FTP solution you can merrily send bytes onto the socket and flush them and then close the socket only to find that the ftp server closes the socket before Flash has finished flushing! That means you get truncated files and no progress bar on the client side.

Hooray For Open Source
"No worries" says I. I've got my FTP server code and Eclipse so I can do anything. I in fact alter the server code to send back status events with the number of bytes actually read. Flex then listens to these on the control socket and updates a progress bar. Once Flex knows that all the bytes have actually been written and received it closes the socket and logs off the FTP session. The really good thing about this is that I've been able to configure the FTP server so that the only commands available are those required to authenticate and my custom upload command. So no mister hacker you can't just start mucking about with my ftp server and create accounts etc... In fact all you can do is upload files that conform to my app's specification and they aren't executed files.

That's It
So that is the solution to my http upload woes. If you or a person you know suffers from file upload issues I can post some code and such. But since no-one follows my blog anyway I can't be bothered unless requested.