iPhone == anathema
So I've had to do some work with the iPhone regarding apps and that means looking at objective-C. First let me suck yuck. I started playing with OC code the other night when 1995 called and asked if it could please have its programming language back. Anyway, as a Java programmer and a Flex programmer I have to say OC is nasty for gui dev and for basic app dev work. Maybe it is really good for larger applications? I don't know, but for little mobile apps it is cumbersome, ugly and overly fiddly. That aside this post is actually about integrating an iPhone app into an existing Flex multi-media app, so I'll leave the OC coding to Mac Fanboys (as I'm a Java fanboy) and move on.
Flash Video Encoding
The existing app I have is a social networking site that allows you to stream live audio and video to an audience. No, this is not a porn web cam, it is for musicians to broadcast their jam sessions or min-gigs to their fans.
So the performer logs on, clicks "yes allow flash to use my camera" and they start streaming to the server. A fan logs on and clicks on the "watch my fave act" and they get the performer's stream playing for their pleasure. The iPhone extension allows a person to watch the same performance on their iPhone rather than on their Flash enabled device (as in everything else in the entire universe, including TVs and refrigerators).
But, there is a problem: iPhone only reads H264 encoded video with aac / mp3 audio. Flash will also read that, but at present only creates Sorenson Spark / Speex (in my case) encoded video / audio. That means the performer is producing a stream that is unintelligible to the iPhone. There is, to my knowledge, no way to load additional codecs onto the iPhone and I don't think Apple will let you write an app that does. So how the heck do you let an iPhone participate in these new fangled interwebs that Steve Jobs has heard so much about (and yet doesn't like)? Oh and yes I am being contentious on purpose because I'm like that.
Transcoding
The magic word we are looking for is transcoder. We take the flash video in one end and spit out h264. This is easier said than done. There is only one transcoder in the whole wide world, I know you don't believe me but it's true, and it is called FFMpeg. Just try to find another application / library able to convert videos via a command line or with an API, go on I dare you.
Somewhere in my back end I need to use FFMpeg to create the iPhone stream otherwise I can just kiss my rich-media app goodbye.
Xuggler
Hooray for the Java open source community! There is a lovely Java JNI wrapper for FFMpeg called Xuggler. It even has some guts in it that let you read a stream from an rtmp source and publish back to an rtmp location. As it happens my back end uses Wowza media server, also Java, so it is fairly easy to call Xuggler from a Wowza application. What isn't so easy is finding any halfway decent documentation for Wowza. Also, Wowza isn't open source so it is a bit of a mystery as to what happens inside it. I think there might be elves involved. Anyway here is the basic glue I used (which I stole from a forum post on the Wowza forums, so thank whoever that was):
public class TheApp extends ModuleBase {
... stuff ...
public void publish(IClient client, RequestFunction function, AMFDataList params){
IMediaStream stream = getStream(client, function);
stream.addClientListener(new StreamWatcher());
ModuleCore.publish(client, function, params);
}
... other stuff ...
}
Ok so the above code adds a listener to the stream so that when the stream is actually published I get notified and a method in the StreamWatcher class is called (that's my class BTW)
public class StreamWatcher implements IMediaStreamActionNotify2 {
public void onPublish(IMediaStream stream, String streamName, boolean isRecord, boolean isAppend) {
H264Transcoder tc = new H264Transcoder(stream.getName());
tc.start();
}
}
Above you can see that when the stream is publish I create a new transcoder (which is a thread) and start it.
public class H264Transcoder extends Thread {
public void run() {
try {
Thread.sleep(5000L);
RTMPPublishH264();
} catch (ParseException e) {
System.out.println(e);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public void testRTMPPublishH264() throws ParseException {
Converter converter = new Converter();
Options options = converter.defineOptions();
String args = ""
/* ffmpeg -i rtmp://www.site.com.au:1935/ppv/streamName
* -re
* -acodec libfaac
* -ar 22050
* -vcodec libx264
* -fpre default.ffpreset
* -f flv rtmp://www.site.com.au:1935/live2/streamName-iphone
*/
CommandLine cmdLine = converter.parseOptions(options, args);
converter.run(cmdLine);
}
}
Wow, so the above isn't complete, but I've included the really important bits. The reason it isn't complete is because I haven't perfected it yet. Maybe I never will. Oh, who am I kidding, of course I will, but not just now.
Important Points
The important points for this:
Xuggler can't connect to specific application instances, only to the _definst_ instance. That means the rtmp url above has an application called "ppv" in which a stream called "streamName" is published. Normally I'd have a named app instance that I'd publish into so I could track shared objects and the like, but that didn't work with Xuggler. Also the output stream I've called streamName-iphone, is in a different application, again the _definst_ instance. The reason for that is my ppv application has a bunch of rich user interaction stuff like chat and it gets info from the server about the performer and the people involved need to have valid accounts and so on. I've only included the actual ffmpeg command line code that I have personally used and it works. The preset I used is the default one that comes with Xuggler for h264 so I'll probably have to tweak that to get the best performance.
I may add an update post once I get all the kinks worked out. For example I'd love to package this up as a simple jar + config arrangement, but I do have to work for a living so I may not get the time.
