YT-DLP Scripting

About the weirdness that is YouTube naming conventions

When looking at a channel page:

https://www.youtube.com/@Tinker001

The bold purple part is the channel name

When watching a video:

https://www.youtube.com/watch?v=y8o7qkmiDso

The bold purple part is the video name

When watching a playlist:

https://www.youtube.com/watch?v=oizvS01ovHE&list=PLLG8IgBGD4eUG2n6p3GGfkQ-KPceDuuWj&pp=gAQB

The bold purple part is the playlist name

Bonus fun bit: The bold green part is actually the name of the video being watched inside that playlist.

Cookies!

For the potentially required cookie file, see Exporting YouTube cookies

One script with multiple personalities

Yup...

Looks even more complicated. In fact, it might even be considered kinda silly.

(assumes your base for downloading YouTube videos is at /mnt/Download_Space/YTDL/ and contains subfolders named _Individual_ and _Playlists_)

Save this script as YTall

Then make symbolic links to it.

• ln -s YTall YTc
• ln -s YTall YTv
• ln -s YTall YTl

Call it as YTc for doing whole channels, YTv for individual videos, & YTl for playlists.

#!/bin/bash

BASENAME=$(basename "$0")

echo -e "~~~~~~~ $BASENAME $@ ~~~~~~~ "\\n

case $BASENAME in
  "YTv")
    TYPE="VideoHash"
    DESC="$TYPE = The name YouTube has given to the individual video"
    DESC+="\n\\t\\t (This is gonna take some work to explain...)"
    DESC+="\n\\t\\t You can specify multiple videos"
  ;;
  "YTc")
    TYPE="ChannelName"
    DESC="$TYPE = The name of the channel \(as defined by YouTube\)"
    DESC+="\n\\t\\t You can specify multiple channels"
  ;;
  "YTl")
    TYPE="ListHash"
    DESC="$TYPE = The name YouTube has given to the playlist"
    DESC+="\n\\t\\t (This is gonna take some work to explain...)"
    DESC+="\n\\t\\t You can specify multiple lists"
  ;;
  *)
    TYPE="poop"
    DESC="$TYPE = Some crap"
  ;;
esac

if [ $# -eq 0 ] || [ $1 = "-u" ] || [ $1 = "-?" ]; then
  echo -e USAGE: $(basename "$0") \[...option\(s\)...\] $TYPE $TYPE ...
  echo
  echo -e OPTIONS:
  echo -e \\t -BOB = Grant Style Folder Structure
  echo -e \\t -a = Download audio only \(into MP3 files\)
  echo
  echo -e \\t -s = simulate this run
  echo -e \\t -c = use a cookie file exported from your browser to identify yourself.
  echo -e \\t -r RATE = Limit bandwidth
  echo -e \\t\\t Maximum download rate in bytes per second,
  echo -e \\t\\t\\t e.g. 50K or 4.2M
  echo -e \\t -p MIN MAX = Sleep interval \(pause\) between downloads \(in seconds\)
  echo -e \\t\\t \(will result in a random pause between MIN \& MAX seconds long\)
  echo -e \\t -e DATE = Earliest video upload date to grab.
  echo -e \\t\\t see https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#video-selection for format info
  echo -e \\t\\t \(Simplest is YYYYMMDD but fancier options exist\)
  echo -e \\t $DESC
  echo
  echo -e The following \"ERROR\"s will also appear in 00000000-ERRORS
  echo -e \(Unfortunately, without indications of which files caused them...\)
  echo
  echo -e \"ERROR: unable to download video data: HTTP Error 403: Forbidden\"
  echo -e \\t Means you may need to supply cookies.
  echo -e \\t Often caused by things like age restrictions.
  echo -e \\t May also be caused by YouTube noticing you\'ve been downloading a bunch...
case $BASENAME in
  "YTc")
    echo
    echo -e \"ERROR: \[youtube\] oizvS01ovH0: Video unavailable\"
    echo -e \\t Most likely caused by a typo in the VideoHash
  ;;
  "YTv")
    echo
    echo -e  \"ERROR: \[youtube:tab\] \@ChannelName: This channel does not have a videos tab\"
    echo -e  \"ERROR: \[youtube:tab\] \@ChannelName: This channel does not have a shorts tab\"
    echo -e  \"ERROR: \[youtube:tab\] \@ChannelName: This channel does not have a streams tab\"
    echo -e \\t Is simply indicating that the channel doesn\'t have that type of content.
    echo -e \\t \(All 3 means the channel doesn\'t actually have any content... :P \)
  ;;
  "YTl")
    echo
    echo -e \"mv: cannot stat \'*\': No such file or directory\"
    echo -e \\t Don\'t panic. This one\'s just a bit of residue from the playlist folder cleanup
  ;;
esac
echo

  exit
fi

  BASEdest=/mnt/Download_Space/YTDL/
  COOKIEfile=/mnt/Download_Space/YTDL/www.youtube.com_cookies.txt

  SIMULATE=""
  WITHcookies=""
  LIMITrate=""
  PAUSEbetween=""
  TIMEspan=""
  BOBstyle=""
  AUDIOonly=""
  FILETYPE="-t mp4"

while (( "$#" ));
do
  case $1 in
    "-c")
      echo Using Cookies: $COOKIEfile
      WITHcookies="--cookies "$COOKIEfile
      shift
    ;;

    "-s")
      echo Simulating
      SIMULATE="-s"
      shift
    ;;

    "-r")
      LIMITrate="-r $2"
      echo Rate Limiting $2
      shift 2
    ;;

    "-p")
      PAUSEbetween="--sleep-interval $2 --max-sleep-interval $3"
      echo Random pause between $2 \& $3 seconds
      shift 3
    ;;

    "-e")
      TIMEspan="--dateafter $2"
      echo Time Span $2 to current
      shift 2
    ;;

    "-BOB")
      BOBstyle="True"
      echo Using Grant Style Folder Structure
      shift
    ;;

    "-a")
      AUDIOonly="--extract-audio --audio-format mp3 --audio-quality 0"
      FILETYPE="-t mp3"
      echo Extracting audio only
      shift
    ;;

    *)
      case $BASENAME in
        "YTv") # Video
          echo Working on video: $1
          echo ========================================
          WeirdID=$1
          URL=https://www.youtube.com/watch\?v=$1
          TYPES=""
          DEST="$BASEdest""_Individual_/"
          FORMAT="%(uploader_id)s - %(upload_date)s - %(title)s.%(ext)s"
        ;;

        "YTc") # Channel
          echo Working on Channel: $1
          echo ========================================
          WeirdID=$1
          URL=https://www.youtube.com/\@$1
          TYPES="videos shorts streams"
#          TYPES="videos shorts streams playlists podcasts courses"
          DEST="$BASEdest""$1"
          FORMAT="%(upload_date)s - %(title)s.%(ext)s"
        ;;

        "YTl") # Playlist
          echo Working on playlist: $1
          echo ========================================
          WeirdID=$1
          URL=https://www.youtube.com/playlist\?list=$1
          TYPES=""
          DEST="$BASEdest""_Playlists_/"
          FORMAT="%(playlist_index)s_%(playlist_count)s %(uploader_id)s - %(title)s.%(ext)s"
        ;;
      esac

      ERRfile=$DEST/00000000-ERRORS
      ARCHfile=$DEST/11111111-ARCHIVE

      OPTIONS="$SIMULATE $LIMITrate $TIMEspan $WITHcookies $PAUSEbetween $AUDIOonly"
      OPTIONS+=" --no-overwrites --write-description $FILETYPE"

      echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
      echo $URL
      echo $WeirdID
      echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"


      if wget --spider --quiet $URL > /dev/null 2>&1; then
        # Weirdly, for individual videos, YouTube says the URL is valid even if it doesn't exist when asking about videos...
        # Then it gives an error while attempting the download.
        # Not fatal, just annoying...

        echo -en "(storing in $DEST"

        case $BASENAME in
          "YTv")
            echo ")"
            echo VID $WeirdID >> $ARCHfile
            cd $DEST
            pwd
            /usr/local/bin/yt-dlp $OPTIONS --download-archive $ARCHfile -o "$FORMAT" $URL 2> >(/usr/bin/tee -a $ERRfile)
            rm -f *.part
          ;;
          "YTl")
            echo "$WeirdID)"
            echo LIST $WeirdID >> $ARCHfile
            cd $DEST
            pwd
            if [ -d $DEST/$WeirdID ]; then
              echo $DEST exists
            else
              echo no $DEST$WeirdID... Building it...
              mkdir $DEST$WeirdID
            fi
            cd $WeirdID
            pwd
            /usr/local/bin/yt-dlp $OPTIONS --download-archive $ARCHfile -o "$FORMAT" $URL 2> >(/usr/bin/tee -a $ERRfile)
            rm -f *.part

            NAME=`find . -maxdepth 1 -regextype posix-extended -regex './0+_.*' -type f   `

            Listname=`echo ${NAME::-12} | cut -d'@' -f 2`

            if [ -d ../"$Listname" ]; then
              echo "$Listname" already exists
            else
              mkdir ../"$Listname"
            fi

            mv "$NAME" ../"$Listname/000 $Listname.description"
            mv * ../"$Listname"
            cd ..
            rmdir $WeirdID
          ;;
          "YTc")
            echo ")"
            if [ -d $DEST ]; then
              echo $DEST exists
            else
              echo no $DEST... Building it...
              mkdir $DEST
              if [ -z $BOBstyle ]; then
                for ACK in $TYPES
                  do
                    mkdir $DEST/$ACK
                  done
              fi
            fi

            cd $DEST
            pwd

            if [ -z $BOBstyle ]; then
              echo -e "(using subfolders...)"
            else
              echo -e "(using Grant format...)"
            fi
            echo

            for ACK in $TYPES
              do
                echo ">>>>>>>> " $URL/$ACK
                if [ -z $BOBstyle ]; then
                  cd $ACK
                  echo CHAN $WeirdID $ACK >> $ARCHfile
                fi

                /usr/local/bin/yt-dlp $OPTIONS --download-archive $ARCHfile -o "$FORMAT" $URL/$ACK 2> >(/usr/bin/tee -a $ERRfile)

                # Clear out .part files
                rm -f *.part

                if [ -z $BOBstyle ]; then
                  cd ..
                fi
              done
          ;;
        esac
      else
        echo YouTube says this channel does not exist.
      fi   
      shift
    ;;

  esac
done

Even more options?

WIP

Errors noted, but not researched

In console output:

• [download] 57.7% of 15.24MiB at 65.83KiB/s ETA 01:40[download] Got error: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')). Retrying (1/10)...

In 00000000-ERRORS:

• WARNING: [youtube] ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')). Retrying (1/3)...