Speeding up Perl Functions in XS

For the longest time I was scared to mess around with XS, but I started messing around with XS in my free time the past couple of days. I've been using Term::VT102 as my sandbox. I am using this for two reasons: (a) I use it pretty heavily in some of my software and would enjoy some speedups, and (b) the code looks like it can be ported to XS without much resistance once I have the tuits.

One of my current goals is to make an entire port of Term::VT102 in XS (Term::VT102::XS). It is progressing very slowly. However, in an act of impatience, I took a current copy of Term::VT102 and converted one of its functions to XS to see how much faster it was. I replaced

	sub attr_unpack {
	shift if ref($_[0]); # called in object context, ditch the object
	my $data = shift;
	my ($num, $fg, $bg, $bo, $fa, $st, $ul, $bl, $rv);
	$num = unpack ('S', $data);
	($fg, $bg, $bo, $fa, $st, $ul, $bl, $rv) = (
	$num & 7,
	($num >> 4) & 7,
	($num >> 8) & 1,
	($num >> 9) & 1,
	($num >> 10) & 1,
	($num >> 11) & 1,
	($num >> 12) & 1,
	($num >> 13) & 1
	);
	return ($fg, $bg, $bo, $fa, $st, $ul, $bl, $rv);
	}

view raw gistfile1.pl hosted with ❤ by GitHub

...with...

	SV*
	attr_unpack(self, sv_buf)
	SV *self
	SV *sv_buf
	PPCODE:
	char *buf = SvPV_nolen(sv_buf);
	EXTEND(SP, 8);
	mPUSHs( newSViv( buf[0] & 7 ) );
	mPUSHs( newSViv( (buf[0] >> 4) & 7) );
	mPUSHs( newSViv( buf[1] & 1) );
	mPUSHs( newSViv( (buf[1] >> 1) & 1) );
	mPUSHs( newSViv( (buf[1] >> 2) & 1) );
	mPUSHs( newSViv( (buf[1] >> 3) & 1) );
	mPUSHs( newSViv( (buf[1] >> 4) & 1) );
	mPUSHs( newSViv( (buf[1] >> 5) & 1) );

view raw gistfile2.c hosted with ❤ by GitHub

and ran a test script through Devel::NYTProf.

The pure-perl version ran in this much time (9.44s is the time in line):

The XS version ran in this much time:

A speedup that reduced the amount of time in that sub to nearly one ninth the time! Believe it or not, this function was one of the more expensive calls in my project code. I know I cheated by requiring $self for the XS function (rendering it completely incompatible but I don't know how to overload it quite yet), but I hope that doesn't make too much of a difference, speed-wise. If it speeds up a single function to 12% of the time (inclusive), imagine how much faster it will be with the entire thing ported to XS.

Termcast on the Web (screencast follow-up)

Here is a follow-up screencast demonstrating the tools I discussed in the previous post. Contact me directly if you want to try it out or see it demonstrated live!



Termcast on the Web from Jason May on Vimeo.

Termcast on the Web

The History of Termcast
In the NetHack community, people like to broadcast their games over telnet so others can watch, learn, and/or be impressed. Some wanted to broadcast more than just NetHack games.

Thus, termcast was born.

Anything one could do on a shell, one could do on termcast.  This allowed for activities like showing off tricks in a favorite editor, the thought process of the broadcaster when writing in a programming language, knowledge of various shell commands, CompSci homework, installation processes, the list goes on.

Branching from Telnet

One day while goofing off on CPAN, I discovered HTML::FromANSI, which does just what it sounds like: turn ANSI into HTML. This immediately inspired me to write a proof-of-concept of a web-based termcast. I have never dealt with VT emulators before, so I had no idea if this would even work. So I hacked on something that had a single listening socket, to which I used doy++'s App::Termcast to connect. It served a single page that used a javascript setInterval() timer to update every 0.2s or so. And when I ran it, it worked! It rendered very slowly and killed my CPU, but who cares: I could see the terminal display update live as I did things on the connected terminal!  Though, it seemed pretty daunting to make this in any way efficient, so I gave up on continuing any work on it.

Until now.

Termcast on the web with true streaming
I am now working on App::Termcast::Server and its web component, App::Termcast::Server::Web. It uses clkao++'s Web::Hippie, which utilizes a nice selection of modern streaming tools for the web to process communication from server to client, such as WebSockets, MXHR and (long) polling. It will transparently choose one depending on what the connecting browser supports. For instance, at the time of this writing, Chrome and Safari supports WebSockets, but Firefox doesn't, so the javascript would choose MXHR polling to do the heavy lifting in Firefox.

I am still trying different things out with the application. My first implementation involved a table with 80 <td>s inside 24 <tr>s, with javascript updating individual rows. The process of changing the foreground color and value of the cell (not to mention up to 1920 at the same time, it was a huge bottleneck, so I decided to learn HTML5 and use that if the browser supported it. Now it uses <canvas> and updates cells with the whole different logic from the table implementation, and is noticeably faster to render on-the-fly as well.

I think I am quickly catching up to the functionality of the original termcast application. Though, in my opinion, one of my killer features is the ability to resize your terminal and see the terminal on the web application change sizes as well, so you don't have to assume 80x24 or anything.

I am happy with the progress I am making with the web component of App::Termcast::Server, but I hope others will start writing other components for it. I started on a telnet component, but it doesn't really work. Also, I wrote a small IRC bot (currently somewhat brittle) that announces termcast connections, which is pretty neat.

As always, I am happy to get feedback, and feel free to contribute in any way you can.

Patches welcome!