Apache FLV streaming done right!

After 2 days of researching, I've just found how to do FLV streaming the right way!
Let's wrap up current (I.2009) situation:

Why?

• FLV is still very common container format for videos. h264 is more efficient but is supported by Flash since version 9.115
  
• FLV streaming means you can seek to any position during video, and browser (flash player) will buffer only from this position to the end.
• Streaming allows people to skip boring parts or see video ending without loading the whole file (saves bandwidth).
  
• Lighttpd has FLV streaming built in.
• Apache doesn't.
• There is number of PHP and Perl scripts that allow to implement streaming. These work by loading the given flv file, and pushing it's content from given position to the end.
  
• It's bloated! PHP uses memory, Apache PHP module uses memory, every single streaming connection is a running php script (try empty php file with just <?=memory_get_usage(true)?> )

Behind the scenes

1. Flash player requests the file, starts loading i.e.: http://host/video.flv
2. When user clicks any position on the "scrubber" (seek bar) player stops buffering and starts buffering from url: http://host/video.flv?start=1503320
3. The parameter is byte count from where buffering should start.
4. Server fetches the file, truncates it and rewrites FLV header.
   

Solution: flv streaming in apache:

1. Download apache mod_flv module by Paul Querna or my tuned version.
2. Install apxs tool (also avaliable in RPMs for most systems as httpd-devel)
3. Compile and install your module with the following command:
   
   apxs -c -i ./mod_flvx.c
   
   
4. Add the following 2 lines to your httpd.conf or create a dedicated /etc/httpd/conf.d/mod_flvx.conf (path depends on your distribution):
   
   LoadModule flvx_module modules/mod_flvx.so
   AddHandler flv-stream .flv
   
   
5. Restart Apache (i.e. with service httpd restart)

Tuning:

1. I've modified the module to fix content-length and add last-modified headers. I've also added a fix when offset is set to wrong position (i.e. past video length).
2. You can download the tuned module here.

Things to remember:

• To use flv streaming on the web, you need to use a pseudo-streaming compliant Flash player. Flowplayer does the job quite well.
• Streaming requires that your FLV has embedded keyframe markers (meta-data). You can inject any FLV with this data using flvtool2:
  
  flvtool2 -U video.flv
  
  
• Generated streams will work briliantly in Flash player, but you won't be able to play them back in standalone player (VLC, WMP etc.). So don't be suprised when you "wget" a video with ?start=1000 and the file doesn't play. That is because they don't have starting meta-data and players will have hard time deciding which codec to use.

Have fuN!

Troubleshooting mailman - receiving posts, but not sending messages

After some struggle, I've just solved quite strange problem with mailman. I will share my new knowledge with you.

So the symptoms are:

1. You or other people are sending emails to the group
2. They get through your SMTP server (Exim here, with a line in log similar to:
   

   => list <list@domain.com> R=mailman_router T=mailman_transport

   
   
3. It goes directly to mailman script via pipe - you can verify that (i.e. exim -d -v list@domain.com)
4. Mailman doesn't send a bit, and nothing appears in it's logs from the moment it receives new message. It doesn't start sending it out, and nothing new is written in smtp.log, post.log, nor vette.log.
   

We have the following environment:

1. Fedora 7 system
2. Exim 4.62 as the mail exchanger (installed from RPM) - probably doesn't matter at all
   
3. Mailman 2.1.9 (from RPM)
4. On Exim, we use mailman_transport, working on pipe driver, executing mailman command /var/mailman/mail/mailman
   
5. Local dns server, everything above on this very machine

Resolution:
It turned out to be a problem with localhost dns name resolving. When I tried to use ping localhost it started to ping wrong address. The problem was in /etc/hosts - you have to make sure, that localhost is pointed to 127.0.0.1. It might be a problem in Mailman itself, because no problem was logged. Still - it is quite important to set up proper localhost (loopback) binding.
After fixing hosts, I could ping myself with ping localhost, and mailman magically started to work.


Other steps worth trying when mailman is dead:

• Check if your exim routing is set properly: exim -bt list@domain.com - this should result in printing something with "mailman" in it, for example:
  

  list@domain.com
  router = mailman_router, transport = mailman_transport
  

• use the delivery debug in exim: exim -d -v list@domain.com This will tell you if the message actually arrives at mailman router and then mailman transport. After running the command, it will stop near Recipients: - there you have to press CTRL+D.
  You should look for something similar to:
  

  mailman_transport transport entered
  direct command:
  argv[0] = /var/mailman/mail/mailman
  argv[1] = ${if def:local_part_suffix {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} {post}}
  argv[2] = $local_part
  direct command after expansion:
  argv[0] = /var/mailman/mail/mailman
  argv[1] = post
  argv[2] = list
  
  ....
  
  mailman_transport transport yielded 0
  

  This means the command executed successfully, and mailman actually gets the message.
  

Useful pages:

• Mailman troubleshooting FAQ
• Mailman with multiple domains problem
• Great Exim cheat sheet (useful commands)
• Debugging exim delivery
• Another small Exim cheat sheet
• HOWTO: Using Exim and Mailman together

If you still have problems, write me and I'll gladly help.

Best RSS tools for a professionals

Here is my list of essential RSS tools that will simplify your life. They are supplementary, every one has it's purpose in this world of information :)

1. Firefox RSS Ticker
• small and sexy
  
• non-distracting
  
• does exactly what you expect it to do
  
Headlines scrolling like stock quotes - it has been invented ages ago, and works nicely. Turn on randomization, and there you go - news coming straight to you. I believe it is the essence of RSS. Information is coming to you, not the other way round and you can be sure, it's as relevant as possible.






2. Google Reader
   So all around these desktop, windowed small RSS readers, here comes web-based RSS reader. It's a little by the wind, but amazingly efficient! It's google... and they did it again. It's advantages are:
   
   • It's on the web, always there when you need it
   • It tracks what you have read, no matter if you're in your office, working at home computer or in internet cafe.
   • It even comes with you when you are offline in subway or on a plane.
   
   If you happen to work on several machines during a day, it's like a blessing. Give it a try... You might also take a look at some cool firefox addons for Google Reader.
   
   
   
   
   
3. Thunderbird News & Blogs
   If you have just found yourself reading 100+ feeds every day, you need a heavy-duty tool! Here comes the RSS functionality of well known "birdy". Just click file - new account and pick RSS News & Blogs.
   Suprisingly it has no more functionality than Google Reader above, but is a little faster and smoother when dealing with a large amount of articles and feeds. And it also does a pretty good job as an email client;)
   
   
   
   
   
   
   
4. Feedblitz
   If you're too lazy or busy to keep configuring, monitoring and playing with the above toys, here's the easiest way to stay up-to-date.
   You give them your e-mail address, they send you the news every few hours. It's free, it's convenient and is working like a charm (unless you don't know what email is ;)

What are your favorite RSS toys ?

Why would I need RSS ?

It's incredible to find out, how many Internet professionals and engineers don't use RSS readers. Many of them don't use RSS at all. While it's not a crime, it is extremely inefficient at a long term.
For the past 10 years news sites like "Anandtech", "Slashdot" and others have been primary source for news and buzz in the industry. With a group of professional editors, and "human-Internet-spiders" (E-Spiderman ?), these people crawled the net searching for information, stories and press releases.
What happened? I believe that most important changes are:

• SPEED of information.
  For an IT guy like me, it was about ~20 news / day on my favorite sites, all taking about 30 minutes to read... But I have to admit, that the next say 3 hours were spent on visiting a sequence of speciality websites, manufacturers, developers and corporate sites. Slashdot does a great job, so does Gadgeteer and others, but it was simply not enough. I remember those geeky news readers, which given a list of urls, grabbed HTML and ran it through regexp parser (if you don't know what I mean, that's good :) I even had this cute little tools that read my Netscape bookmarks, checked page modification times and coloured them, so I knew which pages have been updated.
  Here comes syndication - they take from us, those take from them and these here keep leeching from them and everyone has it :)
  
  There is also a natural trend of changing publishing scheme from pull to push - it is developer or manufacturer's job to update major news sites and syndication points. Publisher are having a good time now.
  
  44 new feeds have appeared in my Google Reader as I write this paragraph...
  
  
  
  
• AMOUNT of information
  Hey - I remember 1996 - I remember the "far end of Internet".... I remember the listing of websites in Poland fit on one page! I remember that time spend on one website was under a minute, not because I was bored, but because there was little more to see. Number of RSS feeds has passed 5 million (try this and this). Editors publishing for my favorite sites have to pick only the best ones out of the crowd, but it is well known, that the pieces create the whole. There is number of blogs and tech resources you wouldn't ever know about, but these can sometimes give you a unique perspective.