User Tools

Site Tools


user:smeas:portfolio:hpc2project1

Project: Downloading YouTube Videos

YouTube is a wonderful resource. But, what if you want to keep those videos saved somewhere? Sure, there are websites that will do it for you… but isn't it more fun to make your own tool to do it?

Objectives

The objective of this project is to write a script which will call the youtube-dl tool to download YouTube videos. The script will be dynamic based on the type of video, and will allow the user to easily choose quality options for their downloads.

Background

YouTube is great and all, but, for those less blessed in the bandwidth department, such as myself, it can be a difficult thing sometimes. Since YouTube switched over to DASH (Dynamic Adaptive Streaming over HTTP) to stream it's videos, you can't just sit there and let an entire video buffer as I used to. Since I enjoy watching most longer videos in 720p, I came up with this way to download the videos.

Scope

This project will walk through the basic steps of writing a script to use the youtube-dl tool more easily.

Procedure

Step 1

Get the required tools

The only tool required for this project (other than a Linux machine running bash) is the youtube-dl program. The program can be obtained from the Debian/Ubuntu repositories by running the following command:

sudo apt-get install youtube-dl

Or the program can also be obtained from the GitHub repository, https://github.com/rg3/youtube-dl (which also has some wonderful documentation).

Step 2

Creating and linking the script

The next step is to create our script. I'm creating my script in the ~/bin directory and naming it youtube, giving it the path /home/shawn/bin/youtube. Be sure to change the permissions of the script to make it executable.

chmod +x youtube

Since we want to be able to call our script from anywhere, we want to make sure our ~/bin directory is accessible from anywhere. We do this by modifying the $PATH system variable. First, check to see if your ~/bin directory is listed in your $PATH. Do this by typing the following:

echo $PATH

Which will return something like this:

/home/shawn/bin:/usr/local/sbin:/usr/local/bin:/usr.sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

If you don't see “/home/username/bin” listed, we need to add it to our $PATH. This can be done by editing the ~/.profile file, and adding the following lines to the bottom.

if [ -d "$HOME/bin" ] ; then
    PATH="$HOME/bin:$PATH"
fi

This will check the home directory on login, and if it finds a ~/bin folder, it will add it to the path. After adding this line, save the ~/.profile and restart your shell. After this, we should be able to call the script from anywhere. So, let's test it. Edit your script file so it says the following:

#!/bin/bash

echo "It works!"

Now, change your directory, and type the name of your script, and hit enter. If you did everything correctly, it should echo “It works!” to the terminal.

Step 3

Writing the script

So, now we have all of our initial setup done, and we can get into writing the script. The first step in our script will be checking the number of arguments given by the user, and continuing on based upon the number of arguments. The program can be called in any of the following ways:

youtube
youtube https://www.youtube.com/watch?v=hNPrA2fghis
youtube 4 2 https://www.youtube.com/watch?v=hNPrA2fghis

The first option will display the syntax for the program, the second will download the supplied video with the default settings of the script, and the third will allow the user to choose which qualities to use for both video and audio.

First, we'll write the code for the syntax option (be sure to remove the echo line).

if [ "$#" -lt 1 ]; then
	echo "Usage: youtube [video quality] [audio quality] url"
	echo -e
	echo "Video Quality | Audio Quality"
	echo "1: 240p       | 1: 48kbps"
	echo "2: 360p       | 2: 128kbps"
	echo "3: 480p       | 3: 256kbps"
	echo "4: 720p       |"
	echo "5: 1080p      |"
	echo -e
	echo "Default settings: Video - 720p, Audio - 128kbps"

What we do here is pretty simple. We check to see if the program was run with no additional arguments with the line

if [ "$#" -lt 1 ]; then

If so, we simple echo out the proper usage of the program. Nothing too crazy going on here! Next, we'll apply the same logic to the second action, and download based on the default settings.

elif [ "$#" -eq 1 ]; then
	if [[ $1 = *playlist?list* ]]; then
		youtube-dl -ci -f 140 -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $1
		youtube-dl -ci -f 136 -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $1
	else
		youtube-dl -ci -f 140 -o "%(title)s.%(ext)s" $1
		youtube-dl -ci -f 136 -o "%(title)s.%(ext)s" $1
	fi

Again, we first check the number of arguments, and if it equals one, we run this. The next thing the loop does is parses the argument and checks to see if it contains a link to a playlist, or a video. In the case it is a playlist, I like my titling to be a little different. Also, notice that we call the youtube-dl program twice. This is because with DASH, there are separate audio and video streams.

So, now what is actually going on when youtube-dl is called? I'll break it down piece by piece.

  • youtube-dl: Pretty basic, the name of the program we're running.
  • -ci: The -c is a very vital option. This allows partial video downloading, and continuing of previously downloaded videos. The -i is used to ignore errors, so if a playlist video isn't found, it will continue on in the playlist.
  • -f: -f is the format specifier option. I'll touch on these a little more later, but for now, we're using 140 as the specifier for 128kbps audio, and 136 for 720p video.
  • -o: -o provides the output format for the file. As I said before, I like to have different formatting depending on whether I download a playlist or a single video. These settings are just personal taste. More info on output formatting can be found here: https://github.com/rg3/youtube-dl#output-template

Pretty simple, right? We check to see if it's a playlist or video, and change the output format based on that. Then, we download the audio stream, and the video stream.

Next, we move on to the user specified quality settings.

elif [ "$#" -eq 3 ]; then
	video=$1
	audio=$2

	case "$video" in
		"1")
			video=133
			;;
		"2")
			video=134
			;;
		"3")
			video=135
			;;
		"4")
			video=136
			;;
		"5")
			video=137
			;;
		*)
			;;
	esac

	case "$audio" in
		"1")
			audio=139
			;;
		"2")
			audio=140
			;;
		"3")
			audio=141
			;;
		*)
			;;
	esac
	
	if [[ $3 = *playlist?list* ]]; then
		youtube-dl -ci -f $audio -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $3
		youtube-dl -ci -f $video -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $3
	else
		youtube-dl -ci -f $audio -o "%(title)s.%(ext)s" $3
		youtube-dl -ci -f $video -o "%(title)s.%(ext)s" $3
	fi

The first thing done here is setting the variables of video and audio equal to whichever number the user entered (1-5 for video, 1-3 for audio, with higher numbers being higher quality, as stated in the usage syntax). After this, we run a simple case statement, to change this number into a format specifier recognized by youtube-dl (133-137 for video, 139-141 for audio). Then, we run a playlist or video check much like the default download settings, except instead of passing in a constant for the -f value, we pass in our two variables. Again, pretty simple, right?

Then, the final step is just a little bit of error checking.

<case> elif [ “$#” -eq 2 ] || [ “$#” -gt 3 ]; then

echo "Invalid number of arguments"

fi </case>

Pretty self explanatory. If the number of arguments equals 2, or is greater than 3, it tells the user the program was run incorrectly!

Now, let's test it. First, we'll try with the default settings.

youtube https://www.youtube.com/watch?v=hNPrA2fghis

This should download both the 128kbps audio stream, and the 720p video stream. Next, we'll try the user defined.

youtube 3 2 https://www.youtube.com/watch?v=hNPrA2fghis

Again, this will download the 128 kbps audio stream, but it will download the video stream in 480p. And we're all done!

Issues

The biggest issue with this script is that because of the way DASH works, two separate files are downloaded for each stream. In order to play back the video, the files need to be muxed together. This can be done through using ffmpeg, although at the moment, the version of ffmpeg in Debian/Ubuntu appears to have some issues. As such, I haven't included the ffmpeg command in the script itself. The command to mux audio and video is quite simple however.

ffmpeg -i video.mp4 -i audio.m4a -vcodec copy -acodec copy output.mp4

Where video.mp4 is the downloaded video file, audio.m4a is the downloaded audio, and output.mp4 is your output filename. Try this at your own leisure, to see if it works on your distro.

An alternative is that some video players are smart enough to mux audio and video files if they have the same name. For me personally, I use Media Player Classic on my Windows machine. With MPC, running the video file will mux in the audio file, and it'll play perfectly fine.

Additional Reading

Full Source

#!/bin/bash


if [ "$#" -lt 1 ]; then
	echo "Usage: youtube [video quality] [audio quality] url"
	echo -e
	echo "Video Quality | Audio Quality"
	echo "1: 240p       | 1: 48kbps"
	echo "2: 360p       | 2: 128kbps"
	echo "3: 480p       | 3: 256kbps"
	echo "4: 720p       |"
	echo "5: 1080p      |"
	echo -e
	echo "Default settings: Video - 720p, Audio - 128kbps"

elif [ "$#" -eq 1 ]; then
	if [[ $1 = *playlist?list* ]]; then
		youtube-dl -ci -f 140 -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $1
		youtube-dl -ci -f 136 -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $1
	else
		youtube-dl -ci -f 140 -o "%(title)s.%(ext)s" $1
		youtube-dl -ci -f 136 -o "%(title)s.%(ext)s" $1
	fi

elif [ "$#" -eq 3 ]; then
	video=$1
	audio=$2

	case "$video" in
		"1")
			video=133
			;;
		"2")
			video=134
			;;
		"3")
			video=135
			;;
		"4")
			video=136
			;;
		"5")
			video=137
			;;
		*)
			;;
	esac

	case "$audio" in
		"1")
			audio=139
			;;
		"2")
			audio=140
			;;
		"3")
			audio=141
			;;
		*)
			;;
	esac
	
	if [[ $3 = *playlist?list* ]]; then
		youtube-dl -ci -f $audio -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $3
		youtube-dl -ci -f $video -o "%(playlist)s - %(playlist_index)s - %(title)s.%(ext)s" $3
	else
		youtube-dl -ci -f $audio -o "%(title)s.%(ext)s" $3
		youtube-dl -ci -f $video -o "%(title)s.%(ext)s" $3
	fi

elif [ "$#" -eq 2 ] || [ "$#" -gt 3 ]; then
	echo "Invalid number of arguments"
fi
user/smeas/portfolio/hpc2project1.txt · Last modified: 2014/05/05 20:44 by smeas