We build Web & Mobile Applications.
Way back last November when I first blogged about Paperclip I included a brief mention of hiding files behind a controller rather than simply putting them in the public directory for all to see. Since then I’ve noticed that the question of how to actually do this has come up regularly over on Rails Forum and a couple of weeks ago I had to figure out how to update some of our code to protect assets that we had migrated from local file system to Amazon S3 storage. So I figured it’s probably a worthwhile technique to share.
I’ll use a slightly updated version of the Track model from the Getting clever with validations section of my original Paperclip blog. Here’s the code:
class Track < ActiveRecord::Base
has_attached_file :mp3,
:url => '/:class/:id/:style.:extension',
:path => ':rails_root/assets/:class/:id_partition/:style.:extension'
validates_attachment_presence :mp3
validates_attachment_content_type :mp3, :content_type => ['application/mp3', 'application/x-mp3', 'audio/mpeg', 'audio/mp3']
validates_attachment_size :mp3, :less_than => 20.megabytes
def downloadable?(user)
user != :guest
end
end
This does the following:
Defines a Track model that has a Paperclip attachment called mp3.
Configures the URL used to access the mp3 asset, for example /track/1/original.mp3. Leaving the style attribute in the URL makes it possible for future versions of the code to do things like generating 10 second previews of the track (using a custom Paperclip processor) that could then be accessed using a URL like /track/1/preview.mp3.
Configures the path where Paperclip will store uploaded files (for example RAILS_ROOT/assets/tracks/000/000/001/original.mp3 where RAILS_ROOT is the path to the root directory of the Rails app) – the important thing here is that the files are stored outside of the /public directory.
Defines validations to ensure an mp3 is uploaded, it has a valid content type and it isn’t too big.
Defines a downloadable? method that can be used to implement user access rights to the track. For simplicity it just allows all logged in users to access the track, however you can replace this with whatever logic your app requires.
Now if you try to provide a link to download the mp3 in your view with something like:
link_to('Listen', track.mp3.url)
You’ll get a routing error like this when you click it:
Routing Error
No route matches "/tracks/1/original.mp3" with {:method=>:get}
I typically map the Paperclip URL to a download action in a controller using map.connect in my routes.rb file like this:
ActionController::Routing::Routes.draw do |map|
map.connect 'tracks/:id/:style.:format', :controller => 'tracks', :action => 'download', :conditions => { :method => :get }
end
There’s no need to use a named route as it’s unlikely you’ll need to reference the route by name, and by mapping to a custom download action you can also provide the standard resourceful CRUD actions in the controller if needed. Puritanical REST fans might insist that downloads are mapped to a separate resource and that you create a new download resource using a POST request: this is probably worth considering if you’re intending doing more than simply streaming a file to the client (like logging statistics, updating billing information) otherwise why complicate things?
The TracksController then needs a download action:
class TracksController < ApplicationController
SEND_FILE_METHOD = :default
def download
head(:not_found) and return if (track = Track.find_by_id(params[:id])).nil?
head(:forbidden) and return unless track.downloadable?(current_user)
path = track.mp3.path(params[:style])
head(:bad_request) and return unless File.exist?(path) && params[:format].to_s == File.extname(path).gsub(/^\.+/, '')
send_file_options = { :type => File.mime_type?(path) }
case SEND_FILE_METHOD
when :apache then send_file_options[:x_sendfile] = true
when :nginx then head(:x_accel_redirect => path.gsub(Rails.root, ''), :content_type => send_file_options[:type]) and return
end
send_file(path, send_file_options)
end
end
There’s quite a lot going on in the controller, here’s a breakdown of the download action:
An attempt is made to find the Track model using the :id parameter, returning a 404 Not Found response if the track doesn’t exist.
The current user is passed to the downloadable? method to determine if the user is allowed to download the track.
Assume here that current_user is a method provided by the application’s authentication code and returns either a User object or the :guest symbol if no user is logged in. The actual code you’ll use here and in the downloadable? method will probably need to be adapted to suit your own authentication code.
The controller returns a 403 Forbidden response if the user is not allowed to download the track.
The path to the mp3 file on the server is then generated for the passed :style parameter.
The controller then ensures that the file exists (it wouldn’t if, for example, an invalid :style parameter was used) and that the requested file extension matches the actual extension of the file. It returns a 400 Bad Request if necessary.
A hash of options to be used with the send_file method is initialised with the MIME type of the mp3 file.
I’m using the mimetype-fu plugin here (installed using ruby script/plugin install git://github.com/mattetti/mimetype-fu.git) rather than the content_type attribute of the mp3 attachment to allow for future functionality where different styles may use different file types.
The controller supports standard streaming, using the Lighttpd/Apache X-Sendfile or the Nginx X-Accel-Redirect methods for downloading the file.
For simplicity the SEND_FILE_METHOD constant is used here to configure the method to use, but in a real application it should be stored in a configuration setting of some sort (I recommend Luke Redpath’s SimpleConfig gemplugin for this kind of thing).
Depending on the configured streaming method the controller then either uses send_file (for standard and X-Sendfile streaming) or responds with a header (for X-Accel-Redirect streaming).
We usually use Nginx for our Rails apps so we use the X-Accel-Redirect method for streaming files with something like this in our Nginx configuration:
location /assets/ {
root /path/to/rails_root;
internal;
}
where /path/to/rails_root contains the full path to our Rails application root.
With the Track model providing access control logic via the downloadable? method and a download action in the TracksController handling the streaming of the mp3 file, it is now possible to provide download links in views using the link_to helper with the url method provided by Paperclip. For example, an index view could look like this:
<ul>
<% @tracks.each do |track| %>
<li><%= link_to('Listen', track.mp3.url) %></li>
<% end %>
</ul>
UPDATE! Later versions of Paperclip include expiring URL support.
The code above works well if your mp3s are being stored on the local file system, but if your site starts to grow and you need to scale both in terms of storage space and download capacity you’ll probably want to move to S3.
Paperclip provides an S3 storage module that until recently made use of the RightAWS gem however version 2.3.1 and newer use AWS::S3 instead. Be warned that this change does cause some problems if you want to use S3 storage buckets located in Europe, so you might want to stick with 2.3.0 if this will be a problem for you. I’ll cover both versions in the code below.
The first thing to change is the has_attached_file definition in the Track model:
has_attached_file :mp3,
:url => ':s3_domain_url',
:path => 'assets/:class/:id/:style.:extension',
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_permissions => 'authenticated-read',
:s3_protocol => 'http'
The most obvious changes here are the new :storage, :s3_credentials, :s3_permissions and :s3_protocol options which specify the storage module, credentials file location, permissions for uploaded files and communications protocol respectively.
The credentials YAML file is used to specify the access key, secret access key and bucket name for your S3 accounts and should look something like this:
common: &common
access_key_id: your_access_key_id_goes_here
secret_access_key: your_secret_access_key_goes_here
development:
<<: *common
bucket: app-name-development
test:
<<: *common
bucket: app-name-test
production:
<<: *common
bucket: app-name-production
This configures separate S3 buckets for each environment which is a good idea to prevent accidently mixing up your test files with your production files!
The :s3_permissions option allows you to specify one of four canned access policies, in this case authenticated-read is the one to use so that read access is only available to the object owner or an authenticated user.
Slightly less obvious are the changes to the :url and :path options: :url is set to ':s3_domain_url' a Paperclip interpolation that uses virtual hosted style bucket access (especially important if using European buckets as this is the preferred method of access); and the :path is used by Paperclip to generate a key name for the object being stored on S3.
With these changes in place new files uploaded to the Track model will be stored in the appropriate S3 bucket using a key generated from the :path option, for example http://app-name-development.s3.amazonaws.com/assets/tracks/1/original.mp3.
One of the good things about moving to S3 is that your server no longer has to concern itself with the streaming of data to the clients (using techniques like X-SendFile and X-Accel-Redirect help unburden your Rails processes but the server still has to do all the work). Instead of sending file data to the client the download action in the controller now needs to handle permissions checking but then redirect the client to the file on S3 for download.
This does however present a problem as the uploaded files are using the authenticated-read canned access policy which prevents public access to them. Fortunately S3 provides a way to generate an authenticated URL for private content that only works for a specified period of time: our controller can then redirect the client to this temporary URL to initiate the download but if a someone was to obtain the details of the URL and try to access the file at a later date then they’d be out of luck as the URL would have expired.
The updated code looks like this:
def download
head(:not_found) and return if (track = Track.find_by_id(params[:id])).nil?
head(:forbidden) and return unless track.downloadable?(current_user)
path = track.mp3.path(params[:style])
head(:bad_request) and return unless params[:format].to_s == File.extname(path).gsub(/^\.+/, '')
redirect_to(AWS::S3::S3Object.url_for(path, track.mp3.bucket_name, :expires_in => 10.seconds))
end
There are two main differences in the controller:
No check is made to ensure that the object exists as this would add the extra overhead of an additional request to S3 – let S3 worry about returning an appropriate response for non-existent objects.
Instead of streaming the file it redirects to a temporary URL generated using the url_for method. The temporary URL is set to expire after 10 seconds which should be long enough for the redirect to initiate the download: it doesn’t matter if the download takes longer than 10 seconds as long as it has started it will continue until it’s done.
If you’re using Paperclip 2.3.0 or older with RightAWS the controller redirect should look like this:
redirect_to(track.mp3.s3.interface.get_link(track.mp3.s3_bucket.to_s, path, 10.seconds))
When using local file system storage the url method of the Paperclip attachment could be used to link to the download action of the TracksController but now that the files are hosted on S3 the url method now returns the URL of the S3 bucket, which is not what we want.
There are a couple of choices here, the route could be changed to a named route and then the view could use something like this:
link_to('Listen', download_track_path(track.id, 'original', 'mp3')
I think a nicer approach is to add a new method to the Track model that uses Paperclip interpolations to generate the download URL. And while I’m at it, I can also move the authenticated S3 URL generation out of the controller and into the model too. Here’s the complete Track model:
class Track < ActiveRecord::Base
has_attached_file :mp3,
:url => ':s3_domain_url',
:path => 'assets/:class/:id/:style.:extension',
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_permissions => 'authenticated-read',
:s3_protocol => 'http'
validates_attachment_presence :mp3
validates_attachment_content_type :mp3, :content_type => ['application/mp3', 'application/x-mp3', 'audio/mpeg', 'audio/mp3']
validates_attachment_size :mp3, :less_than => 20.megabytes
def downloadable?(user)
user != :guest
end
def download_url(style = nil, include_updated_timestamp = true)
url = Paperclip::Interpolations.interpolate('/:class/:id/:style.:extension', mp3, style || mp3.default_style)
include_updated_timestamp && mp3.updated_at ? [url, mp3.updated_at].compact.join(url.include?("?") ? "&" : "?") : url
end
def authenticated_url(style = nil, expires_in = 10.seconds)
AWS::S3::S3Object.url_for(mp3.path(style || mp3.default_style), mp3.bucket_name, :expires_in => expires_in, :use_ssl => mp3.s3_protocol == 'https')
end
end
Again for RightAWS (Paperclip <= 2.3.0) the authenticated URL generation is slightly different:
def authenticated_url(style = nil, expires_in = 10.seconds)
mp3.s3.interface.get_link(mp3.s3_bucket.to_s, mp3.path(style || mp3.default_style), expires_in)
end
This then tidies up the download action, the redirect changes to:
redirect_to(track.authenticated_url(params[:style]))
And the view can now link to downloads:
link_to('Listen', track.download_url)
Hopefully I’ve given you enough here for you to start implementing protected downloads in your own Rails apps. Don’t forget that if you’re using European S3 buckets you’ll need to do some patching otherwise you’ll get strange errors like:
AWS::S3::PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
when you try and upload – fixing that properly is a job for another day!
How can I force the file to be downloaded instead of have it display in the browser?
If you’re using the file system storage module then you shouldn’t need to do anything as Rails sets the Content-Disposition header to attachment when using the send_file method. Internet Explorer may still insist on showing the file inline though, so you could also use a bit of brute force and set the Content-Type header to application/octet-stream in the controller:
send_file_options = { :type => 'application/octet-stream' }
If you were feeling clever you could of course use a bit of simple browser sniffing to return the correct Content-Type for intelligent browsers and the application/octet-stream header for IE.
If you’re using the S3 storage module then you need to add the s3_headers option to your has_attached_file definition in your models:
has_attached_file :mp3,
:url => ':s3_domain_url',
:path => 'assets/:class/:id/:style.:extension',
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_permissions => 'authenticated-read',
:s3_protocol => 'http',
:s3_headers => { :content_disposition => 'attachment' }
When the controller redirects to the S3 URL, Amazon will send the header you specify here, forcing the download. As with file system storage you can also force the Content-Type header using this method, although you won’t be able to use any kind of browser sniffing to select content type based on the user’s browser:
has_attached_file :mp3,
:url => ':s3_domain_url',
:path => 'assets/:class/:id/:style.:extension',
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_permissions => 'authenticated-read',
:s3_protocol => 'http',
:s3_headers => { :content_type => 'application/octet-stream', :content_disposition => 'attachment' }