UK Ruby on Rails, Exalead, AWS, Consultancy - Video thumbnails with FFmpeg and Paperclip Comments

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2011-05-19T09:17:42Z

It might be worth putting a breakpoint in the rescue block of the make method and inspecting the values of the original exception message and output attributes (before the exception is re-raised) to be sure that it is the warning that is causing the error and not something else.

I've had FFmpeg warnings before that haven't stopped Paperclip from doing its thing - so I'd be interested to know why it's breaking your code.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by erwin

2011-05-19T07:34:48Z

thanks for all these info .. it's running so well... However I got a, issue trying to generate thumbnail for an .m4v video file the ffmpeg command line on console works well .. ( it generates the thumbnail) but display a warning ...

ffmpeg -ss 2 -i clipout12-iPhone.m4v -y -vcodec mjpeg -f image2 -vframes 1 -s 100x74 clipout12-iPhone.jpg .. Seems stream 1 codec frame rate differs from container frame rate: 2000.00 (2000/1) -> 1000.00 (2000/2)

running the command line within Paperclip processor raises a PaperclipCommandLineError and the thumbnail is not generated.. is there anyway to avoid this 'frame rate' checking or should I bypass the error in the rescue ? ...

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Stijn Mathysen

2011-02-23T09:46:06Z

Seems like the newer versions of Paperclip changed the way interpolation works. Here is what I did to make it compatible again;

Paperclip.interpolates :extension do |attachment, style_name|
  case
    when ((style = attachment.styles[style_name]) && !style[:format].blank?) then style[:format]
    when attachment.instance.respond_to?(:video?) && attachment.instance.video? && style_name.to_s == 'transcoded' then 'flv'
    when attachment.instance.respond_to?(:video?) && attachment.instance.video? && style_name.to_s != 'original' then 'jpg'
  else
    File.extname(attachment.original_filename).gsub(/^\.+/, "")
  end
end

Comment on 'Video thumbnails with FFmpeg and Paperclip' by EH

2010-10-20T22:43:26Z

If you didn't already know, this article is being reprinted above you in Google results: http://www.railsfire.com/article/video-thumbnails-ffmpeg-and-paperclip

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Bert Goethals

2010-08-16T09:46:30Z

Recently i posted a git about how we handle videos with (and without) paperclip. The original solution was based on this post, so i thought it would be nice to share it here.

It has some cool things like

Thumbnails, but only once from video
Geometry for video (2px rule included)
conversion to flash

Hope it helps: http://gist.github.com/507804

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2010-08-16T08:55:51Z

We've not had a problem, although the site is over a year old and is running on Paperclip 2.2.4 (patched to allow Procs for processors) so maybe the problems you're seeing have been added more recently?

When I get chance I'll take a look at how it works with the current Paperclip version and see if there are any patches I can contribute.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2010-08-15T16:32:49Z

Hi,

Rob thanks for your message. Does the lambda which defines dynamic processors work for you? Because it does not for styles. Moreover if one defines styles using a hash, eg: :small => {:geometry => "64x64", :format => "png"}, then the geometry gets deleted by paperclip and results in a failure which is being discussed in the mailing-list.

Paperclip has a few bugs that make it unsuitable for production unless people double-check everything as prod/dev environments result in different behaviors.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2010-08-03T11:17:16Z

I'd agree Paperclip is not best suited to this kind of thing, it's just not flexible enough which is why the video thumbnailer and transcoder processors I've written aren't particularly nice pieces of code. I haven't tried anything too advanced yet, but I'm hoping CarrierWave will be better at this kind of thing.

My video transcoding is done in a Paperclip processor that looks almost exactly like the thumbnailer, the main difference being the ffmpeg command line which looks like this:

cmd = %Q[-i "#{File.expand_path(file.path)}" -y -f flv -vcodec flv -b 200000 -r 30 -acodec libfaac -ac 2 -ab 32000 -ar 22050 ]

My Paperclip definition defines a special 'transcoded' style which I handle in the content_type_extension interpolation with this extra condition:

when attachment.instance.video? && style_name.to_s == 'transcoded' then 'flv'

Our transcoding is dealt with offline using a queue, as it could be lengthy process, part of which involves forcing the file for the transcoded style to be set to the processed video:

original_file = asset.to_file
asset.queued_for_write[:transcoded] = Paperclip.processor(:video_transcode).make(original_file, { :whiny => true, :geometry => '480x360' }, asset)
asset.save

This probably doesn't make too much sense, but that only highlights it's messiness! It's the main reason I've not been able to do a proper post on it yet as it's not very easy to explain. Hopefully I've given you some ideas to get you going though :)
Rob

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas

2010-07-31T08:42:02Z

Although paperclip looked promising at first sight, once we start digging further it quickly reaches its design limits:

it's not well suited for "polymorphic" attachments like attaching a video, audio or video.
background re-encoding is not possible within processors

I'm still keeping it, but I have to bypass the processors and handle the transcoding myself within the model.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2010-07-29T19:59:23Z

Hi Rob it's me again :),

In your comment #14 you talk about an additional processor that converts the video to flv. How would you add that?

Because the way paperclip is made, is that if you upload let's say an mp4 file it's tricky to get .jpg and .flv files.

I didn't manage to tie the processor chooser to a specific style using the lambda. At runtime paperclip wants to inject the processors definition but the lambda doesn't get execute and it fails.

Thanks for your insight.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2010-07-29T17:16:48Z

Ok I got it, it's the contenttypeextension interpolation that does the magic. Their documentation on processors is awful!

PS again: what's that stupid comment filtering system you have? I couldn't post a second comment after my previous one. Take it off, it's pain.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2010-07-29T16:35:05Z

I am getting closer. So ffmpeg gets fired, the thumbnail is certainly generated, but Paperclip doesn't seem to get it and won't save it along with the original video.

Any reason for that?

PS: By the way I had issues running: Paperclip.run('ffmpeg', cmd), for some reason paperclip surrounds cmd with additional quotes which makes ffmpeg fail.

If I do: cmd = "ffmpeg -i input.avi ..." and Paperclip.run(cmd) ffmpeg can now do its job (but still paperclip won't sve the thumbnail).

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2010-06-21T21:16:23Z

Hi Thomas,
When you attach a new file the thumbnails should be regenerated. You can access a thumbnail file using the to_file method, for example:

attachment.to_file(:thumbnail)

And you could then assign this to a different attachment if you wanted to preserve the old thumbnail for some reason.

Rob

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2010-06-20T20:15:57Z

Hi Rob,

So I'm back in the processor business.

When the processor is done, is there a way to programmatically access the generated file? Like assign it to has_attached_file :thumbnail?

My concern is what happens when I upload and attach a new and different file to the object? Does the old thumbnail get destroyed? I'm not sure about that.

Cheers,

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Teresa

2010-06-15T16:28:31Z

I cannot thank you enough for posting this article: I wanted to do precisely this (use the same model for both video and other graphic attachments), but thought I might need to make a separate model to distinguish between content types. After a 5 minute implementation (copy and paste), your code worked flawlessly.

Really, I couldn't be more appreciative. Thanks so much.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2010-05-27T23:10:30Z

Just tried the same code and it worked without error. It looks like you need to make sure the user that you're running your Rails server as has write permissions to the temp directory where upload files are being stored prior to processing.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Chris

2010-05-26T20:14:43Z

ok... gotta correct my last post. Sorry for causing confusion. I forgot that i fixed it by setting the callback to :aftersave. Here is my method called on :beforesave, where 'audio' is my model attribute

def set_mp3_tags
    Mp3Info.open(audio.to_file.path) do |mp3|
      mp3.tag.title = 'foo'
    end if audio?
  rescue Mp3InfoError => e
    errors.add(:audio, "unable to process file (#{e.message})")
  end

Causes: Permission denied - C:/[..]/Temp/stream.1440.0

Greetings, Chris

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Chris

2010-05-26T18:16:19Z

The error must have been on my side, as a server restart fixed it...

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2010-05-26T10:55:30Z

Hi Chris, what does your code look like at the moment?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Chris

2010-05-26T06:28:29Z

Hi,

I tried to modify mp3-tags with mp3info in a before_save callback. Which gives me a 'access denied' exception. Any ideas how to handle this?

Greetings, Chris

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-11-27T10:38:18Z

Hi Glenn,
I've not posted it yet but it is my plan to (although it's been my plan to for about 6 months now). I'll see if I can put it together for next week as I'm long overdue a new blog post (as certain other WebFellas keep reminding me) :)

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Glenn

2009-11-21T16:09:50Z

Rob, very useful tutorial. You mention in the comments that you have another processor to handle the FLV transcoding. I've been been piecing my own such process together from various other blogs, but I was wondering if you have already posted, or will in the near future post, another tutorial detailing your transcoder processing.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-11-19T10:28:54Z

In one of our projects I added a VideoGeometry class (as a subclass of Paperclip's own Geometry class). To get the dimensions of the video it uses ffmpeg with just the -i input file argument and captures the output via stderr.

It then uses a simple regular expression to look for the video stream information from the captured output and matches the wxh dimensions it contains.

You could also look at the RVideo gem, it has an Inspector class that does a similar thing: I didn't use it as my needs were simple, but if you want to get even more information about your videos it's probably the way to go.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Bert Goethals

2009-11-18T15:13:26Z

Great plugin, however this solution will always create squared thumbnails.

This is not always desired. FFMPEG only supports wxh format, including a >, < or # modifier will break the command.

Ideally the Recalculation of the Geometry should know about the original dimensions, but how should those be obtained? I know of the -vstats argument, but that writes to files, and that just doesn't seems the way to go?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Leandro Pedroni

2009-10-01T10:03:57Z

Cool! I saw a bit of the Building a Video Portal in Rails - or How the Teenage Mutant Ninja Turtles Started Streaming presentation and they suggested flixcloud. It could be very interesting to do a writeup for a processor that uses the flixcloud API (there's a ruby gem) which by the way plays well with S3.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Marc Lainez

2009-09-25T12:21:05Z

Hello guys,

Great article, it helped me a lot. I didn't really understand what was the custom interpolation :contenttypeextension for. If you just want the generated files extensions to be handled has jpg, adding a :jpg next to my styles formats did the trick...

:styles => { :big => ['540x406>', :jpg], :thumb => ['270x203>', :jpg]}

That way the original file has its original extension, and the two generated files have a jpg extension.

But maybe I'm missing the point behind this :contenttypeextension thing...

Cheers

Comment on 'Video thumbnails with FFmpeg and Paperclip' by kadoudal

2009-07-21T10:55:23Z

thanks very helpful.. note : I am not sure that the itoffset parameters works fine ...

I tried 2 commands with ffmpeg (last version may, 2009)

ffmpeg -i #{vpath} -itsoffset -3 -s 150x100 -vframes 1 -f image2 -an #{tpath} and ffmpeg -i #{vpath} -ss 00:00:03 -s 150x100 -vframes 1 -f image2 -an #{tpath}

only the 2nd one gives me a shot at 3sec after initial frame ...

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-05-12T07:19:19Z

Yes it runs the processor in-process, so it'll block until it is done. We've found thumbnailing video with ffmpeg to be as quick as thumbnailing images with ImageMagick. We also have a Paperclip processor that uses FFmpeg to transcode the video to FLV format: this is a much longer process and for that we use a queue to handle it out-of-process.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2009-05-11T20:30:43Z

Using Paperclip.run, does it block the process will converting the video? Shouldn't we use backgroundrb for that purpose?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2009-04-22T11:04:41Z

Hi Rob,

Mplayer tries to play the video instead of extract a frame from it. From what I found, mplayer thinks that the file is corrupted and tries to play it and skip bad frames but it makes things even worse and kills the cpu. That behavior only happened with a particular file when the passed file came from Paperclip, so I preferred to simply drop the processor and use a callback that works on the real file and not from Tempfile just to be safer in the future.

For the processor, it's strange, because if I return the thumbnail, then Paperclip would save the original video as a 0Kb file on disk. If I return the video file then it gets saved properly.

Thanks for your reply.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-04-21T22:06:14Z

Hi Thomas,
Yes the processor is in use on the site that it was originally written for, happily thumbnailing videos 24/7 :)

As you can see from the Paperclip source the make method of a processor should “return an instance of File (Tempfile is acceptable) which contains the results of the processing”. This is also what Paperclip’s own thumbnail processor does and it was the basis for this video thumbnailer.

The final piece of the puzzle is the post_process_styles method in the Attachment class: it iterates over each processor and injects the returned (temp)file returned by the processor in the queued_for_write array which is then flushed to disk (or written to S3 depending on the storage method being used) when the attachment is saved.

Hope this helps clear things up. As for your mplayer problem, what exactly does “goes wack” mean?!
Rob

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2009-04-21T21:56:22Z

Wow! I run into a pretty nasty bug. I am using mplayer to extract the frames, and on one video which mplayer goes wack maybe because of the TempFile! Rewriting the processor into a model callback that works on the real file on disk works without any problem.

Did anyone run into such problem?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Thomas Glasgow

2009-04-21T20:13:18Z

Hi Rob,

Are you actually using this processor? Because from what I see, your make method returns a Tempfile instance that points to the generated thumbnail, but that's not what Paperclip expects.

In the case of a thumbnail generator, Paperclip is expecting the original video to be returned, otherwise it wouldn't save it. Don't you have your video saved as a 0Kb file?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by nap

2009-02-28T02:30:03Z

Great tutorial guys. A very useful processor.

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-02-26T17:31:14Z

You should be able to get the original file name and content type from the attachment instance that's passed to the processor: in my code above I'd do this in the initialize method of the processor using attachment.original_filename and attachment.instance_read(:content_type). The downside of this approach is that if multiple processors are chained together then the content_type of the file passed to the processor might not be the same as that of the original file.

Another option would be to see if @file.respond_to?(:content_type) and if so use @file.content_type to read it, but again this might not work if multiple processors are used.

A final option would be to use something like the RVideo Inspector class on the file which uses ffmpeg to analyse the content of the file and tell you what it contains: this would work even with chained processors but does of course add a bit of processing overhead.

Hope that helps,
Rob

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Moodang

2009-02-26T15:33:30Z

Let me try again. My @file looks like this:

File:/var/folders/4q/4qf80RBVH40ijDl9aQomck+++TI/-Tmp-/CGI2146-9

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Moodang

2009-02-26T15:31:54Z

I really like your example. I am in fact extending it to transcode videos into various formats. However, I run into some odd problems.

I am trying to do this

@current_format = File.extname(@file.path)

in my processor code, but it's not working.

Currently my @file has the form:

<file: />

Which essentially has no extensions.

The original file names are not accessible from within the processor.

Is there a way I can properly access @current_format from within the processor?

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Jon Yurek

2009-02-23T15:07:59Z

Nice work, Rob! I love seeing new and interesting Processors getting made!

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Rob Anderton

2009-02-23T08:44:41Z

Hey Mike,

If you're running on the newest Paperclip then your processor gets passed the attachment as the third parameter to the constructor like this: def initialize(file, options = {}, attachment = nil), so in the constructor code you could replace @instance = options[:instance] with @instance = attachment.instance and wouldn't need to patch.

I like the idea of the liquidize and compress processors, I hadn't even thought of doing something clever like that! I guess a virus checker would be another idea along the same lines.

And thanks for the links, glad you like the blog :D
Rob

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Michael Deering

2009-02-23T01:53:07Z

I shared my processors over here in a blog post to better put my comment into context.

Paperclip Processors: Doing so much more with your attachments

Comment on 'Video thumbnails with FFmpeg and Paperclip' by Michael Deering

2009-02-23T00:27:27Z

Paperclip rocks! I was not going to go cargo-cult to it away from attachment_fu until I came across your original paperclip post over here.

I was also creating processors this week. One wall I hit was getting access to the ActiveRecord instance inside of the processors.

I patched paperclip crudely like this.

# config/initilizers/paperclip_patch.rb
module Paperclip
  class Attachment

    # I need access to the AR instance in order to scope thing by the site the attachment belongs to.
    def post_process_styles
      log("Post-processing #{name}")
      @styles.each do |name, args|
        begin
          raise RuntimeError.new("Style #{name} has no processors defined.") if args[:processors].blank?
          @queued_for_write[name] = args[:processors].inject(@queued_for_write[:original]) do |file, processor|
            log("Processing #{name} #{file} in the #{processor} processor.")
            Paperclip.processor(processor).make(file, args.merge(:instance => @instance))
          end
        rescue PaperclipError => e
          log("An error was received while processing: #{e.inspect}")
          (@errors[:processing] ||= []) << e.message if @whiny
        end
      end
    end    

  end
end

This gave me access to the ActiveRecord instance through options[:instance] inside of the processor.

You had to do this yet? Have a better recommendation on the approach to access the AR instances? I have been in the docs and code, did I miss the obvious?

Keep up the quality posts.

Cheers, Mike D.