Skip to content

Instantly share code, notes, and snippets.

@Seneral
Last active August 7, 2022 16:57
Show Gist options
  • Save Seneral/bd64281ae30a6010e97d956ecd63238e to your computer and use it in GitHub Desktop.
Save Seneral/bd64281ae30a6010e97d956ecd63238e to your computer and use it in GitHub Desktop.
DOESN'T WORK ANYMORE - for workaround see https://gist.github.com/p3g4asus/597050997e01f8fd1fcf473fe6545a4fModified VLC YouTube playlist parsing script to support playlists with more than 200 videos. Verified it works with 1000+ videos, no duplicates and all included (minus deleted videos). Official version claims to support 100+, but actually o…
--[[
Youtube playlist importer for VLC media player 1.1 and 2.0
Copyright 2012 Guillaume Le Maout
Authors: Guillaume Le Maout
Contact: http://addons.videolan.org/messages/?action=newmessage&username=exebetche
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
--]]
--[[
MODified by Kai Gillmann, 19.01.2013, kaigillmann@googlemail.com:
VLC HAS already a youtube importer, but not for playlists. IMO this mentioned one is
better than this one, because it opens the video in the best possible video resolution.
So i decided to remove all parts of the code which is not responsible for list handling.
Now this lua script parses the list, as wanted, but for each video opened, the vlc default
Youtube script is used, so the videos will be displayed properly.
--]]
--[[
Patched by Aaron Hill (https://github.com/seraku24), 2018-05-16:
The original script was failing in VLC 3.x due to an overzealous probe function.
This patch makes the probe function more restrictive to avoid false positives.
--]]
-- Helper function to get a parameter's value in a URL
function get_url_param( url, name )
local _, _, res = string.find( url, "[&?]"..name.."=([^&]*)" )
return res
end
-- Probe function.
function probe()
if vlc.access ~= "http" and vlc.access ~= "https" then
return false
end
return string.match(vlc.path:match("([^/]+)"),"%w+.youtube.com") and (
not string.match(vlc.path, "list_ajax") and string.match(vlc.path, "[?&]list="))
end
-- Parse function.
function parse()
if string.match( vlc.path, "list=" ) then
local playlist_parsed, playlistData, line, s, item
local p = {}
local id_ref = {}
local index = 0
local playlistID = get_url_param( vlc.path, "list" )
local videoID = get_url_param( vlc.path, "v" )
local playlistURL = "https://www.youtube.com/list_ajax?action_get_list=1&style=xml&list="..playlistID
local prevLoaded = 0
while true do
playlistData = ""
line = ""
s = nil
s = vlc.stream(playlistURL.."&index="..index)
while line do
playlistData = playlistData..line
line = s:readline()
end
playlist_parsed = nil
playlist_parsed = parse_xml(playlistData).root.video
if playlist_parsed == nil then
playlist_parsed = {}
end
for i, video in ipairs(playlist_parsed) do
if not id_ref[video.encrypted_id.CDATA] then
vlc.msg.dbg(i.." "..video.encrypted_id.CDATA)
id_ref[video.encrypted_id.CDATA] = true
item = nil
item = {}
if video.encrypted_id
and video.encrypted_id.CDATA then
item.path = "http://www.youtube.com/watch?v="..video.encrypted_id.CDATA
end
if video.title
and video.title.CDATA then
item.title = video.title.CDATA
end
if video.artist
and video.artist.CDATA then
item.artist = video.artist.CDATA
end
if video.thumbnail
and video.thumbnail.CDATA then
item.arturl = video.thumbnail.CDATA
end
if video.description
and video.description.CDATA then
item.description = video.description.CDATA
end
--~ item.rating = video.rating
table.insert (p, item)
end
end
if #p > prevLoaded or index == 100 then
vlc.msg.dbg("Playlist-Youtube: Loaded " ..#p.. " videos...")
index = index + 100
prevLoaded = #p
else
vlc.msg.dbg("Playlist-Youtube: Finished loading " ..#p.. " videos!")
return p
end
end
end
end
function parse_xml(data)
local tree = {}
local stack = {}
local tmp = {}
local tmpTag = ""
local level = 0
table.insert(stack, tree)
for op, tag, attr, empty, val in string.gmatch(
data,
"<(%p?)([^%s>/]+)([^>]-)(%/?)>[%s\r\n\t]*([^<]*)[%s\r\n\t]*") do
if op=="?" then
--~ DOCTYPE
elseif op=="/" then
if level>0 then
level = level - 1
table.remove(stack)
end
else
level = level + 1
if op=="!" then
stack[level]['CDATA'] = vlc.strings.resolve_xml_special_chars(
string.gsub(tag..attr, "%[CDATA%[(.+)%]%]", "%1"))
attr = ""
level = level - 1
elseif type(stack[level][tag]) == "nil" then
stack[level][tag] = {}
table.insert(stack, stack[level][tag])
else
if type(stack[level][tag][1]) == "nil" then
tmp = nil
tmp = stack[level][tag]
stack[level][tag] = nil
stack[level][tag] = {}
table.insert(stack[level][tag], tmp)
end
tmp = nil
tmp = {}
table.insert(stack[level][tag], tmp)
table.insert(stack, tmp)
end
if val~="" then
stack[level][tag]['CDATA'] = {}
stack[level][tag]['CDATA'] = vlc.strings.resolve_xml_special_chars(val)
end
if attr ~= "" then
stack[level][tag]['ATTR'] = {}
string.gsub(attr,
"(%w+)=([\"'])(.-)%2",
function (name, _, value)
stack[level][tag]['ATTR'][name] = value
end)
end
if empty ~= "" then
level = level - 1
table.remove(stack)
end
end
end
return tree
end
@Seneral
Copy link
Author

Seneral commented Jul 21, 2020

Ok after a quick look, the list_ajax simply ignores the index parameter now, for some user lists only. I do not know if there is a different parameter now, but I really hope there is. Alternative is the browse_ajax way, which the default playlist site uses. I have a version of that implemented for FlagPlayer, I could adapt it, but considering this only affects a few user playlists I probably won't be putting in too much effort into that, at least not for an old lua script I've only used once haha
Basically you'd have to load the playlist page, get the initial data out of the html file, parse the video data in there, and check if there's a "nextContinuationData" with "continuation" and "clickTrackingParams" (aka ctoken and itct). If there is, you need to parse some additional stuff from the page, set some cookies, call browse_ajax with these parameters and some special content (along with the cookies), and rinse and repeat. Again, I have this implemented for browse content, however I have no motivation to convert this javascript code to lua.
If you need want, it's implemented here:
Initial load, most importantly yt_browse.
And continuing the load, yt_loadChannelPageUploads.
Again, it's complex, so I wouldn't count on it.

Edit: to see how it works, and see the network requests, you can load the following page and see that while the playlist only loads 100 videos, if you scroll down it does actually load all videos:
https://flagplayer.seneral.dev/?list=UU-lHJZR3Gqxm24_Vd_AJ5Yw&u=PewDiePie

@inunah
Copy link

inunah commented Jul 21, 2020

Unfortunately it's not only a few playlists, it's every "uploads by [user]" playlist with over around 104-111 videos (can't find an exact number due to being unable to find users who have uploaded between 105 and 111 videos, all I know is that 104 works and 112 does not).

Even the "Uploads by PewDiePie" one you stated worked does not load more than 100 videos.

@Seneral
Copy link
Author

Seneral commented Jul 21, 2020

Yh true, as I said list_ajax method doesn't seem to work anymore, maybe there is a fix for that, would have to do some more research for that though.
Also described another, MUCH more complicated solution, using browse_ajax, which I've got working in FlagPlayer as seen above. Thing is, right now I've got exams and one major project of mine to worry about (MakersVR), and I don't even program in Lua, so the chances of me basically rewriting this script to use a much more complicated method, even though I'm not even using it anymore, are near null. Sorry:/

@inunah
Copy link

inunah commented Jul 21, 2020

It's okay. There are always alternate playlists I can grab that have the same videos, even if they aren't as reliably updated.

@p3g4asus
Copy link

Today links like http://www.youtube.com/list_ajax?action_get_list=1&style=xml&index=1&list=<playlist_id> give all 404 error. Will it be temporary or permanent?

@Seneral
Copy link
Author

Seneral commented Feb 12, 2021

Thanks for the heads up, you're right, that's annoying. They have been phasing out this ajax interface before, starting with search I believe, so it is most likely permanent.
However I'll definitely find away around this for FlagPlayer. Most likely we'll have to use the browse interface which is a lot more complicated (but using it for some stuff already).
But most likely I'll not update this script, so somebody would have to port it.

@Seneral
Copy link
Author

Seneral commented Feb 12, 2021

Theres references to embeds_wexit_list_ajax_migration_killswitch in the code of YouTube so I'll likely find some clues there

@p3g4asus
Copy link

FlagPlayer is a very interesting project. Congratulations and thank you

@p3g4asus
Copy link

Theres references to embeds_wexit_list_ajax_migration_killswitch in the code of YouTube so I'll likely find some clues there

Do you mean that you found that string inside a js file used in the youtube page?

@p3g4asus
Copy link

Thanks for the heads up, you're right, that's annoying. They have been phasing out this ajax interface before, starting with search I believe, so it is most likely permanent.
However I'll definitely find away around this for FlagPlayer. Most likely we'll have to use the browse interface which is a lot more complicated (but using it for some stuff already).
But most likely I'll not update this script, so somebody would have to port it.

I think I can port it. Is there a way I can learn more about browse interface? Any open source code that is using it or any sort of documentation?

@Seneral
Copy link
Author

Seneral commented Feb 12, 2021

Theres references to embeds_wexit_list_ajax_migration_killswitch in the code of YouTube so I'll likely find some clues there

Do you mean that you found that string inside a js file used in the youtube page?

Exactly, so it's likely planned, but nothing to fix that ofc.

Thanks for the heads up, you're right, that's annoying. They have been phasing out this ajax interface before, starting with search I believe, so it is most likely permanent.
However I'll definitely find away around this for FlagPlayer. Most likely we'll have to use the browse interface which is a lot more complicated (but using it for some stuff already).
But most likely I'll not update this script, so somebody would have to port it.

I think I can port it. Is there a way I can learn more about browse interface? Any open source code that is using it or any sort of documentation?

Not a whole lot, will have to look into it anew, but seems like we'll have to load their page completely and parse that. Now I'll likely do this today. In short, they seem to be using a new very simple API like this:
https://www.youtube.com/youtubei/v1/browse?key=******
with a single key, and this is new to me. Also, it's POST, and with cookies, else it won't work. But, I have worked with similar stuff before for FlagPlayer so maybe my existing code works already. Still a lot of new stuff to sift through. Also worth a look might be youtubedl, they'll probably be having to figure this out as well (or maybe long have)

@Seneral
Copy link
Author

Seneral commented Feb 12, 2021

Ok apparently youtube-dl had this long figured out, so I'll take a look at their stuff

@mothership1101
Copy link

Did the youtube playlist parser stop working for anyone else? It stopped working as of 2/11/21

@Seneral
Copy link
Author

Seneral commented Feb 12, 2021

Yeah working on it as you can see. Unfortunately they cut out both list_ajax and browse_ajax at the same time (after they previously ditched search_ajax) so I'll have to completely rewrite my code (for another project). Then somebody with more knowledge than me in Lua can port it. I personally won't, in the first place I only did a slight edit to the original script for a one time use case.
youtube_dl is still working though, so tthey apparently prepared for this.

@Seneral
Copy link
Author

Seneral commented Feb 15, 2021

I think I can port it. Is there a way I can learn more about browse interface? Any open source code that is using it or any sort of documentation?

So I've finished updating FlagPlayer, it now uses the browse method for all loaded content (search, lists, related videos, etc) except for comments, where the ajax API is still used.
So if you want, take a look at yt_loadPlaylistData if you want to port it.

Essentially, yt_browse loads the webpage just like a browser does - important parts is to extract the innertube API key and some other 'secret' data and the cookies it sends, as well as the initialData JSON object stored in the webpage which contains not only the first batch of videos in the list but also data on the continuation.

From then on, yt_generateContinuationLoader generates a function (just an abstraction since it's used a lot, don't mind it) that loads the continuation and updates the state with new contination data. That usually consists of a conToken and itctToken. The request ist done by sending a POST request to a fixed URL with the innertube API key in the URL. The headers contain some other secret-not-so-secret data from the page and cookies that have been sent by youtube previously. The body contains the actual continuation info, including itctToken, conToken, and some fake webbrowser identification.
The response should be a JSON object formatted similarily to initialData, details in the code, with an updated conToken and itctToken.

Also note that in contrast to the ajax API, this method keeps deleted/private videos in, so you'll have to filter them out.

Some implementation-specific info for FlagPlayer: It supports mobile as well as desktop, so at some places when parsing, you'll find it takes multiple options - don't mind that too much. Also, since it's a website and youtube is cross-origin, it can't actually access the cookies itself, instead a custom CORS server does the cookie handling, that's why the headers x-cookies/x-set-cookies are used. I expect with the lua API you'll have direct access to the cookies, so don't mind that either.

Hope that helps you get on the right path.

@p3g4asus
Copy link

p3g4asus commented Feb 15, 2021

I think I can port it. Is there a way I can learn more about browse interface? Any open source code that is using it or any sort of documentation?

So I've finished updating FlagPlayer, it now uses the browse method for all loaded content (search, lists, related videos, etc) except for comments, where the ajax API is still used.
So if you want, take a look at yt_loadPlaylistData if you want to port it.

Essentially, yt_browse loads the webpage just like a browser does - important parts is to extract the innertube API key and some other 'secret' data and the cookies it sends, as well as the initialData JSON object stored in the webpage which contains not only the first batch of videos in the list but also data on the continuation.

From then on, yt_generateContinuationLoader generates a function (just an abstraction since it's used a lot, don't mind it) that loads the continuation and updates the state with new contination data. That usually consists of a conToken and itctToken. The request ist done by sending a POST request to a fixed URL with the innertube API key in the URL. The headers contain some other secret-not-so-secret data from the page and cookies that have been sent by youtube previously. The body contains the actual continuation info, including itctToken, conToken, and some fake webbrowser identification.
The response should be a JSON object formatted similarily to initialData, details in the code, with an updated conToken and itctToken.

Also note that in contrast to the ajax API, this method keeps deleted/private videos in, so you'll have to filter them out.

Some implementation-specific info for FlagPlayer: It supports mobile as well as desktop, so at some places when parsing, you'll find it takes multiple options - don't mind that too much. Also, since it's a website and youtube is cross-origin, it can't actually access the cookies itself, instead a custom CORS server does the cookie handling, that's why the headers x-cookies/x-set-cookies are used. I expect with the lua API you'll have direct access to the cookies, so don't mind that either.

Hope that helps you get on the right path.

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

@mothership1101
Copy link

@Seneral thanks for sharing. I've noticed the OG parser extracted the Google Video link (https://xxx.googlevideo.com/videoplayback) instead of the YouTube url (https://www.youtube.com/watch?v=xxxxxxxxxxx). I'm curious if there a way to only play the YouTube url? The Google Video links expire after about 6 hours of starting a playlist.

@Seneral
Copy link
Author

Seneral commented Feb 15, 2021

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

Huh, I checked again and it's not necessary. I just kept sending them because previous APIs needed them (not sure which exactly - EDIT: comment aajx API needed them).
So yeah, should work without! Sorry for the confusion.

@Seneral
Copy link
Author

Seneral commented Feb 15, 2021

@Seneral thanks for sharing. I've noticed the OG parser extracted the Google Video link (https://xxx.googlevideo.com/videoplayback) instead of the YouTube url (https://www.youtube.com/watch?v=xxxxxxxxxxx). I'm curious if there a way to only play the YouTube url? The Google Video links expire after about 6 hours of starting a playlist.

This one also only extracts the youtube link, not the raw file. AFAIK VLC can already parse these and extract the video on demand, doesn't it?

@p3g4asus
Copy link

@Seneral thanks for sharing. I've noticed the OG parser extracted the Google Video link (https://xxx.googlevideo.com/videoplayback) instead of the YouTube url (https://www.youtube.com/watch?v=xxxxxxxxxxx). I'm curious if there a way to only play the YouTube url? The Google Video links expire after about 6 hours of starting a playlist.

This one also only extracts the youtube link, not the raw file. AFAIK VLC can already parse these and extract the video on demand, doesn't it?

Yes. I can confirm it already does.

@p3g4asus
Copy link

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

Huh, I checked again and it's not necessary. I just kept sending them because previous APIs needed them (not sure which exactly - EDIT: comment aajx API needed them).
So yeah, should work without! Sorry for the confusion.

Thank you for testing. Another problem is the POST request. VLC lua api does not support it. Would GET work anyway? If I click show more link in the youtube playlist page for mobile, a GET is sent (for example: https://m.youtube.com/playlist?ctoken=4qmFsgI8EiRWTFBMOEJ3aXJhTVFMVktlZk02WVl3WHd1YTVnV2tvbVhIcS0aFENBRjZCbEJVT2tOQ1VRJTNEJTNE&pbj=1) so I assume GET should work anyway. Do you think it can work?

@Seneral
Copy link
Author

Seneral commented Feb 15, 2021

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

Huh, I checked again and it's not necessary. I just kept sending them because previous APIs needed them (not sure which exactly - EDIT: comment aajx API needed them).
So yeah, should work without! Sorry for the confusion.

Thank you for testing. Another problem is the POST request. VLC lua api does not support it. Would GET work anyway? If I click show more link in the youtube playlist page for mobile, a GET is sent (for example: https://m.youtube.com/playlist?ctoken=4qmFsgI8EiRWTFBMOEJ3aXJhTVFMVktlZk02WVl3WHd1YTVnV2tvbVhIcS0aFENBRjZCbEJVT2tOQ1VRJTNEJTNE&pbj=1) so I assume GET should work anyway. Do you think it can work?

Oh nice find. This API is apparently still open for both the desktop and mobile domain. Quick tests show it'll work just fine, even without cookies. You do need custom headers though with stuff extractred from the page (take a look at yt_getRequestHeadersYoutube without the cookie parameter).
Then, in javascript terms, the following returns the correct result you can parse:

fetch("https://youtube.com/playlist?ctoken=" + conToken + "&pbj=1", {
	method: "GET",
	headers: yt_getRequestHeadersYoutube("application/x-www-form-urlencoded", false)
});

Pretty easy. Although I wouldn't trust on it working forever, since they seem to be moving towards a different interface now (same but with POST, with a generic key in the query and the conToken in the body).

@mothership1101
Copy link

Thanks, keep us posted if someone updates the .lua .. I still can't get it to work in VLC

@p3g4asus
Copy link

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

Huh, I checked again and it's not necessary. I just kept sending them because previous APIs needed them (not sure which exactly - EDIT: comment aajx API needed them).
So yeah, should work without! Sorry for the confusion.

Thank you for testing. Another problem is the POST request. VLC lua api does not support it. Would GET work anyway? If I click show more link in the youtube playlist page for mobile, a GET is sent (for example: https://m.youtube.com/playlist?ctoken=4qmFsgI8EiRWTFBMOEJ3aXJhTVFMVktlZk02WVl3WHd1YTVnV2tvbVhIcS0aFENBRjZCbEJVT2tOQ1VRJTNEJTNE&pbj=1) so I assume GET should work anyway. Do you think it can work?

Thank you for your instructions. Is re-sending Youtube-received cookies mandatory? That would be a very nasty problem to solve in vlc lua interface. Vlc lua api does not support specifying cookies and vlc lua plugin is compiled with LUA_DL_DLL NOT defined and so it does not allow using lua standard socket.http dll lib. Honestly I don't know how to solve this issue without re-compiling vlc.

Huh, I checked again and it's not necessary. I just kept sending them because previous APIs needed them (not sure which exactly - EDIT: comment aajx API needed them).
So yeah, should work without! Sorry for the confusion.

Thank you for testing. Another problem is the POST request. VLC lua api does not support it. Would GET work anyway? If I click show more link in the youtube playlist page for mobile, a GET is sent (for example: https://m.youtube.com/playlist?ctoken=4qmFsgI8EiRWTFBMOEJ3aXJhTVFMVktlZk02WVl3WHd1YTVnV2tvbVhIcS0aFENBRjZCbEJVT2tOQ1VRJTNEJTNE&pbj=1) so I assume GET should work anyway. Do you think it can work?

Oh nice find. This API is apparently still open for both the desktop and mobile domain. Quick tests show it'll work just fine, even without cookies. You do need custom headers though with stuff extractred from the page (take a look at yt_getRequestHeadersYoutube without the cookie parameter).
Then, in javascript terms, the following returns the correct result you can parse:

fetch("https://youtube.com/playlist?ctoken=" + conToken + "&pbj=1", {
	method: "GET",
	headers: yt_getRequestHeadersYoutube("application/x-www-form-urlencoded", false)
});

Pretty easy. Although I wouldn't trust on it working forever, since they seem to be moving towards a different interface now (same but with POST, with a generic key in the query and the conToken in the body).

Unfortunately even this type of GET is impossible to reproduce with vlc lua API. I just realized that it supports only customizing referrer and user agent in http GET requests. And unfortunately additional x-youtube headers are mandatory. GET requests without them do not work. Would it be possible to make a simple GET request to FlagPlayer web interface to obtain youtube playlist details in json format?
Thanks in advance

@Seneral
Copy link
Author

Seneral commented Feb 16, 2021

Oh nice find. This API is apparently still open for both the desktop and mobile domain. Quick tests show it'll work just fine, even without cookies. You do need custom headers though with stuff extractred from the page (take a look at yt_getRequestHeadersYoutube without the cookie parameter).
Then, in javascript terms, the following returns the correct result you can parse:

fetch("https://youtube.com/playlist?ctoken=" + conToken + "&pbj=1", {
	method: "GET",
	headers: yt_getRequestHeadersYoutube("application/x-www-form-urlencoded", false)
});

Pretty easy. Although I wouldn't trust on it working forever, since they seem to be moving towards a different interface now (same but with POST, with a generic key in the query and the conToken in the body).

Unfortunately even this type of GET is impossible to reproduce with vlc lua API. I just realized that it supports only customizing referrer and user agent in http GET requests. And unfortunately additional x-youtube headers are mandatory. GET requests without them do not work. Would it be possible to make a simple GET request to FlagPlayer web interface to obtain youtube playlist details in json format?
Thanks in advance

That is quite worrying. I'd recommend creating an issue on VLC to try to fix this, because setting headers should really be in their radar.
Technically you could write a server that does these requests for you, but it'd be a lot of rewriting for the one I'm using and I'd have to open it up to other sources, which is generally a bad idea for puplic cors servers which so many options enabled as I have.
And finally, even if you did, YouTube has IP matching protection enabled on some copyrighted videos - VLC wouldnt be able to play back the URL you get, which means you'd have to stream the audio/video file over that server as well.
While there is really no sensible reason hlto have a separate server running for a host application like VLC. For a website, sure, they have unavoidable restrictions set by the browser, but in this case I'd push towards VLC improving their web request interface.

@p3g4asus
Copy link

Oh nice find. This API is apparently still open for both the desktop and mobile domain. Quick tests show it'll work just fine, even without cookies. You do need custom headers though with stuff extractred from the page (take a look at yt_getRequestHeadersYoutube without the cookie parameter).
Then, in javascript terms, the following returns the correct result you can parse:

fetch("https://youtube.com/playlist?ctoken=" + conToken + "&pbj=1", {
	method: "GET",
	headers: yt_getRequestHeadersYoutube("application/x-www-form-urlencoded", false)
});

Pretty easy. Although I wouldn't trust on it working forever, since they seem to be moving towards a different interface now (same but with POST, with a generic key in the query and the conToken in the body).

Unfortunately even this type of GET is impossible to reproduce with vlc lua API. I just realized that it supports only customizing referrer and user agent in http GET requests. And unfortunately additional x-youtube headers are mandatory. GET requests without them do not work. Would it be possible to make a simple GET request to FlagPlayer web interface to obtain youtube playlist details in json format?
Thanks in advance

That is quite worrying. I'd recommend creating an issue on VLC to try to fix this, because setting headers should really be in their radar.
Technically you could write a server that does these requests for you, but it'd be a lot of rewriting for the one I'm using and I'd have to open it up to other sources, which is generally a bad idea for puplic cors servers which so many options enabled as I have.
And finally, even if you did, YouTube has IP matching protection enabled on some copyrighted videos - VLC wouldnt be able to play back the URL you get, which means you'd have to stream the audio/video file over that server as well.
While there is really no sensible reason hlto have a separate server running for a host application like VLC. For a website, sure, they have unavoidable restrictions set by the browser, but in this case I'd push towards VLC improving their web request interface.

You are definitely right. I did not think about the implications my request would have. VLC http interface should definitely be extended to support at least setting headers. Before they add that support, playing youtube palylist will be nearly impossible.
The only way I can think of is using os.execute() api to execute:
youtube-dl -j --flat-playlist "https://<yourYoutubePlaylist>" > outfile.txt
and parsing outfile.txt
Obviously it will be ugly, slow and it will need youtube-dl.exe to be copied in the same dir as the lua playlist plugin file. But it should work. I will try this solution and I will post it when ready.

@p3g4asus
Copy link

p3g4asus commented Feb 17, 2021

Here you can find the updated version of this playlist script.
Disclaimer: this version works only under Windows. It can be easily ported but I have no way to test it on other OS at the moment.
Installation (under Windows): place this file in the lua playlist vlc folder together with JSON.lua and youtube-dl.exe

@Seneral
Copy link
Author

Seneral commented Feb 17, 2021

Looking good, will try it out later and somehow attempt to forward everyone there.
(Btw currently there seem to be two copies of the script in there)

@NeuroCPP
Copy link

@p3g4asus is it broken OR is it due to some youtube api updates recently?
doesn't seem to work for me though..
Stay Strong

@p3g4asus
Copy link

p3g4asus commented Jun 24, 2021

@p3g4asus is it broken OR is it due to some youtube api updates recently?
doesn't seem to work for me though..
Stay Strong

It still does work for most videos. I recommend you keep your youtube-dl executable file updated downloading it periodically from its official website

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment