Storing and Encoding Videos With Ruby on Rails, Lambda, and S3 | by Paulo Carvalho | Apr, 2022

A easy however scalable strategy to including video add and processing performance to your new app

As person choice shifts from picture to video primarily based content material, it has grow to be extra widespread for shopper apps to should add and reproduce video.

The necessity to take care of video over unreliable web connections requires that video be processed upon add and served (downloaded by others) in a top quality commensurate with out there bandwidth.

A number of options have been developed to realize these targets together with SaaS choices reminiscent of Mux and AWS Elemental.

Right here, we are going to go over a easy strategy that may work for each small and medium purposes built-in to a customized backend however utilizing AWS infrastructure for processing and serving of movies.

The strategy we are going to develop will make use of three key elements:

  1. AWS S3: We’ll use S3 (with transfer acceleration) to offer a scalable and quick location to add and retailer recordsdata to.
  2. AWS Lambda: Shall be used to re-encode the video recordsdata into any variety of desired codecs and high quality.
  3. Customized Backend: A Ruby on Rails (any language ought to work) backend to retailer metadata on the movies and serve to purchasers.

Uploads will observe a step related of the one offered within the article under the place the shopper requests a signed URL from the Rails backend after which makes use of the supplied URL to instantly add the video file to S3.

As soon as uploaded, we are going to make use of an S3 Notification Event to set off a Lambda to course of the video. The ensuing metadata (reminiscent of S3 key) will likely be saved to the customized backend. The ensuing movement is depicted under.

Information movement in add of recent video file

Half 1: Video Document (Object) in Backend

With a purpose to hold monitor of the uploaded movies we are going to create a mannequin in our backend with just a few vital properties:

  • A key string to retailer the S3 key of the unique uploaded file and the same attribute for every desired video spinoff.
  • A low_res_key to retailer the S3 key to a low decision spinoff of the unique file processed by the Lambda.

In Ruby on Rails, the required migration is proven under:

The corresponding video mannequin ought to embrace 2 strategies:

  1. A way to assign a singular key to a brand new video (you might use the id of the video object).
  2. A way to retrieve a cryptographically signed add URL to an AWS S3 bucket.

The ensuing mannequin is proven under:

Half 2: Create and Retrieve Video Document from Backend

We want a option to create the video information and for the shopper to retrieve the required info to add the video within the first place.

To allow this, we would require just a few controller strategies:

  1. A POST methodology to create a brand new video document. Be aware that upon creation the video object can have an auto-assigned key however this key won’t instantly level to an current S3 file for the reason that precise video file could not have been uploaded by the shopper but.
  2. A GET methodology to retrieve the video document.
  3. A PUT methodology that the lambda that may create the video derivatives will use to report again to the server with the keys for the generated spinoff recordsdata.

Half 3: Course of Uploaded Video

We’ll use S3’s notification event to robotically set off the execution of a Lambda to course of a video file that has simply been uploaded to S3.

We’ll use AWS SAM to declare and deploy our Lambda and S3 in a lot the identical means because the article under. Nonetheless, you should utilize the supplied code within the remaining of the article and manually create your Lambda and S3 from AWS console.

Create a folder and file construction such because the one under:

video-processor/
├── cmd/
│ └── deploy.sh
├── src/
│ ├── s3-util.js
│ ├── child-process-promise.js
│ └── index.js
├── .gitignore
└── template.yaml

The child-process-promise.js defines a helper perform that spins a brand new course of inside a promise. We will likely be utilizing it to name FFMPEG inside our Lambda’s predominant code.

s3-util.js defines a helper methodology for downloading video recordsdata from S3.

The total code for the Lambda is proven under in index.js. It may be damaged down into 5parts:

  1. Obtain video asset from S3 into the working listing of the Lambda. Information are uploaded to /uploads by the shopper utility.
  2. Course of the downloaded video asset utilizing ffmpeg.
  3. Add the newly processed video file to S3. Lambda uploads spinoff recordsdata to /processed.
  4. Inform server the brand new file is prepared.
  5. Take away file from Lambda. That is required for the reason that cupboard space of the Lambdas could also be shared between executions and if recordsdata are usually not deleted they might accumulate and replenish the out there house.

The AWS Serverless Application Model (SAM) permits defining the Lambda and its related assets in a YAML file and robotically deploy and replace it with AWS supplied command line instruments.

The template.yaml file declares the assets we will likely be utilizing and the set off on the creation of a file throughout the /uploads folder (aka prefix).

We have now gone over create the infrastructure to assist the add and processing of video recordsdata for a cell (or related) utility. The system ought to scale properly because it makes use of AWS’ infrastructure for heavier duties reminiscent of processing video recordsdata.

Some factors value mentioning:

  1. The server-side processing of video recordsdata doesn’t preclude the need to course of the recordsdata on the shopper for the reason that unique recordsdata could also be too massive to add over an affordable web connection.
  2. Further video derivatives will be added by altering the Lambda code to incorporate the creation of extra video recordsdata primarily based on the required specs. As an illustration, chances are you’ll wish to have a time-limited model of the uploaded video recordsdata to make use of in previews.

More Posts