Skip to content

Instantly share code, notes, and snippets.

@jamiephan
Last active December 20, 2024 07:04
Show Gist options
  • Save jamiephan/0c04986c7f2e62d5c87c4e8c8ce115fc to your computer and use it in GitHub Desktop.
Save jamiephan/0c04986c7f2e62d5c87c4e8c8ce115fc to your computer and use it in GitHub Desktop.
A script to automatically add ALL items to your account in quixel

Script to add all items from quixel

As quixel is being removed, all items are free to aquire. This script is to automate the process to add items to your account (As of writing, a total of 18874 items)

Note: This script only tested in the latest version of Chrome.

How to use

  1. Copy the script from below (run.js)
  2. Login into https://quixel.com
  3. Go to https://quixel.com/megascans/collections
  4. Open devtools (F12) -> Go to "Console" tab
  5. Paste in the script and press Enter.
  6. A dialog should popup confirming the execution, click "OK"
  7. Sit back and wait

Common issues

  • Getting "Forbidden" error. (Even after refresh, the whole page just shows "Forbidden")
    • There is a chance that the API adding too fast and you hit the rate limit of the API. (My testing is around after 10 pages, so ~10k items).
    • Wait after ~10-20 minutes and continue. See Common Fixes -> Restart script to continue the execution after you can load https://quixel.com.
  • The script seems to be paused/hang
    • It could be too much logging going it. Try monitor the script, if it says "END PAGE X", note the page number down (in case need restart) and clear the console by clicking the "🚫" icon in devtools.
    • See Common Fixes -> Restart script for fixing.
  • Getting the error **UNABLE TO ADD ITEM**
    • There should have the error message shown in ( ). If it is user already owns specified asset at a higher or equal resolution, then its already in your account.
  • Getting the error cannot find authentication token. Please login again
    • Clear browser cookies and re-login quixel again. Try just simply add 1 item manully. If it success, then see Common Fixes -> Restart script for continue the execution.

Common Fixes

Restart Script

  1. Note which page it was running
  2. Copy the run.js script
  3. Update the startPage = 0 on the first line to startPage = 10 (assuming page 10 was hanged)

Change Log

  1. Initial Script launch
  2. Update to clear logs to reduce chance of hanging
  3. [CURRENT] Skip adding items that already was acquired. Reduced logs. Added more info after script completion to show purchased item count. Due to now skipping purchased items, you technically don't need to specify the startPage anymore.
((async (startPage = 0, autoClearConsole = true) => {
const getCookie = (name) => {
const value = `; ${document.cookie}`;
const parts = value.split(`; ${name}=`);
if (parts.length === 2) return parts.pop().split(';').shift();
}
const callCacheApi = async (params = {}) => {
const defaultParams = {
page: 0,
maxValuesPerFacet: 1000,
hitsPerPage: 1000,
attributesToRetrieve: ["id", "name"].join(",")
}
const response = await fetch("https://proxy-algolia-prod.quixel.com/algolia/cache", {
"headers": {
"x-api-key": "2Zg8!d2WAHIUW?pCO28cVjfOt9seOWPx@2j"
},
"body": JSON.stringify({
url: "https://6UJ1I5A072-2.algolianet.com/1/indexes/assets/query?x-algolia-application-id=6UJ1I5A072&x-algolia-api-key=e93907f4f65fb1d9f813957bdc344892",
params: new URLSearchParams({ ...defaultParams, ...params }).toString()
}),
"method": "POST",
})
return await response.json()
}
const callAcl = async ({ id, name }) => {
const response = await fetch("https://quixel.com/v1/acl", {
"headers": {
"authorization": "Bearer " + authToken,
"content-type": "application/json;charset=UTF-8",
},
"body": JSON.stringify({ assetID: id }),
"method": "POST",
});
const json = await response.json()
if (json?.isError) {
console.error(` --> **UNABLE TO ADD ITEM** Item ${id} | ${name} (${json?.msg})`)
} else {
console.log(` --> ADDED ITEM Item ${id} | ${name}`)
}
}
const callAcquired = async () => {
const response = await fetch("https://quixel.com/v1/assets/acquired", {
"headers": {
"authorization": "Bearer " + authToken,
"content-type": "application/json;charset=UTF-8",
},
"method": "GET",
});
return await response.json()
}
// 1. Check token exist, quixel API needs it
console.log("-> Checking Auth API Token...")
let authToken = ""
try {
const authCookie = getCookie("auth") ?? "{}"
authToken = JSON.parse(decodeURIComponent(authCookie))?.token
if (!authToken) {
return console.error("-> Error: cannot find authentication token. Please login again.")
}
} catch (_) {
return console.error("-> Error: cannot find authentication token. Please login again.")
}
// 2. Get all currently acquired items
console.log("-> Get Acquired Items...")
const acquiredItems = (await callAcquired()).map(a => a.assetID)
// 3. Get total count of items
console.log("-> Getting Total Number of Pages....")
const { nbPages: totalPages, hitsPerPage: itemsPerPage, nbHits: totalItems } = await callCacheApi()
console.log("-> ==============================================")
console.log(`-> Total # of items: ${totalItems}`)
console.log(`-> ${totalPages} total pages with ${itemsPerPage} per page`)
console.log(`-> Total Items to add: ${(totalItems - acquiredItems.length)}.`)
console.log("-> ==============================================")
if (!confirm(`Click OK to start adding ${(totalItems - acquiredItems.length)} items in your account.`)) return
// Loop
for (let pageIdx = startPage || 0; pageIdx < totalPages; pageIdx++) {
console.log(`-> ======================= PAGE ${pageIdx + 1}/${totalPages} START =======================`)
console.log("-> Getting Items from page " + (pageIdx + 1) + " ...")
const { hits: items } = await callCacheApi({ page: pageIdx })
console.log("-> Adding non-acquired items...")
// Filter out owned items
const unownedItems = items.filter(i => !acquiredItems.includes(i.id))
const aclPromises = unownedItems.map(callAcl)
await Promise.all(aclPromises)
console.log(`-> ======================= PAGE ${pageIdx + 1}/${totalPages} COMPLETED =======================`)
if (autoClearConsole) console.clear() // Fix the issue that too much log hangs the console. Set autoClearConsole = false to keep the logs
}
console.log("-> Getting new acquired info...")
// Get acquired items again
const newItemsAcquired = (await callAcquired()).length
const newTotalCount = (await callCacheApi()).nbHits
console.log(`-> Completed. Your account now have a total of ${newItemsAcquired} out of ${newTotalCount} items.`)
alert(`-> Your account now have a total of ${newItemsAcquired} out of ${newTotalCount} items.\n\nIf you find some items missing, try refresh the page and run the script again.`)
})())
@JohnHeatz
Copy link

Worked like a charm in Opera GX, thank you!

@HaidarYusuf-001
Copy link

HaidarYusuf-001 commented Oct 9, 2024

image
I'm using Microsoft Edge and it works so freaking well in just 5 minutes (really, i'm not kidding, my internet speed is 10 Mbps) with no single error for the first run.

@limdingwen
Copy link

As arwa99-lab said it has me worried it is malicious even if inadvertently.

Plugged the code into chatgpt and got this Untitleda

is algolia trustworthy? i realise they supply the search api, but saw they have a public bug bounty for penetration testing so has me worried if our tokens are going through their network traffic.

Can jamiephan comment on this?

I read the code btw, I can affirm that most of the code looks fine, but the Algolia part is the only potentially weird part. I know of them as the search/cache(?) company though, so it should be fine?

Also, I got this from an email:

From Fab launch until the end of 2024, Megascans will be free for everyone under Fab's Standard License, for all engines and tools. We’re releasing a solution for you to easily claim Megascans that are available for free during this period - it’s coming soon, so stay tuned!

@rdbreak
Copy link

rdbreak commented Oct 9, 2024

This is so great! Thanks for the effort it took to put this together.

@sam-david
Copy link

sam-david commented Oct 9, 2024

So, I've used the script and now I have 18k+ purchased, but they're still on my collection and I haven't downloaded to my pc. Will they stay there after all the changes and I'll be able to download any other moment or should I do it now? Thank you in advance

I'm wondering about this as well. Based on the faq from quixel I see this answer that worries me

Where can I find my Megascans assets after Fab launches?

Everything you have acquired on Quixel.com or Bridge will continue to be available to you on Quixel.com and Bridge after Fab launches. Your Megascans may also appear on Fab, if certain conditions are met—read more on that below.

Please note that Quixel.com and Bridge will eventually be sunset. When that happens, if you wish to have continued access to those assets, we recommend that you download them onto a local disk so that you can continue to use them. We will alert you in advance of any change to the Quixel.com and Bridge services.

Furthermore it seems that we have until end of 2024 for free unreal users like me

If you acquired the assets for free via the Unreal Engine subscription plan or under a free trial of a Quixel plan, they will not automatically transfer over to Fab. From Fab’s launch until the end of 2024, you may download Megascans—both those you may have acquired in the past and those that may be new to you—for free from Fab. Any Megascans that you add to your library on Fab during this promotion period, you will be entitled to use under the terms of the Fab Standard License even after the promotion period ends. Please be aware that simply downloading a free Megascan on Fab in 2024 will not grant you this entitlement - you need to use the “Add to library” option.

I think it makes sense to download all of these assets locally. I'm going to work on my own script separately to do that. If I do I will post here for everyone's benefit.

Keeping in mind that storage space needs to be considered for this option

@nurb2kea
Copy link

"Forbidden" came up as said, but when you open the desktop app, log in and then go back to the website,
the "Forbidden" is gone.
The whole process took only 4 min. with a 5MB/s line.

So just start the desktop app after running the script several time in a row, and you won't see the "Forbidden" for 10-20 min.

Anyways, big thanks !

@JustinSchneider
Copy link

JustinSchneider commented Oct 11, 2024

Just wanted to say thanks for sharing this!

@collidernyc
Copy link

This worked great, thank you! I did it on Apple Safari, The Develop menu has to be active (I'm not sure if it's because I have a developer account, or if that menu is always there?). Show JavaScript Console (option + command + C). I made it to page 15 of 19 before I got errors, ran it a couple more times with 15 as start, then a last time from the beginning to clean up a couple hundred that were missed. A lot less painful than doing this by hand, thanks again!

@ParaLizzard
Copy link

Could someone make a script for Fab after it launches to add Megascans to library?

@limdingwen
Copy link

limdingwen commented Oct 11, 2024 via email

@orion200215
Copy link

So I think I screwed myself over. After getting everything downloaded right? I refreshed....and I get a 403 forbidden error with a translation option between danish and english....I refresh, same error. And i close the window and then open a new tab and open through quixel...same error....so....ouch
Screenshot (3)

@nurb2kea
Copy link

So I think I screwed myself over. After getting everything downloaded right? I refreshed....and I get a 403 forbidden error with a translation option between danish and english....I refresh, same error. And i close the window and then open a new tab and open through quixel...same error....so....ouch Screenshot (3)

just read my post above or the instructions that say that it will be gone in 10-20 min. and the 'forbidden' is gone!

@xtremertx
Copy link

They will remove access to Quixel and Bridge in the future, if you don't store assets locally on disk you WILL LOSE ACCESS!!!

They talk that WE can get those assets on FAB.COM before years 2024 ends, but I don't see anything like that there..

See:
https://support.fab.com/s/article/Fab-Transition-FAQs

@aldenparker
Copy link

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to
jamiephan btw this script helped me understand what I needed to do for mine.
https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

@FabriceBazzaro
Copy link

Thank you for the script, really nice work, I've added a little function at the beginning of the script to avoid the 403 error due to the number of calls too fast :

const delay = (milliseconds) => new Promise(resolve => {
        setTimeout(resolve, milliseconds);
  });

And I call this function at the end of the loop :

    if (autoClearConsole) console.clear() // Fix the issue that too much log hangs the console. Set autoClearConsole = false to keep the logs
	await delay(10000);
  }

I've purchased all the assets in two successive calls (I just had a 502 - bad gateway - on an asset).

@joe777777777777
Copy link

I think I know what the problem might be for some, You have to enter "allow paste" first and then paste the code, it worked for me.
Desktop Screenshot 2024 10 14 - 19 43 16 80

@Arwa99-lab
Copy link

I can't paste the code it keeps telling me this (Warning: Don’t paste code into the DevTools Console that you don’t understand or haven’t reviewed yourself. This could allow attackers to steal your identity or take control of your computer. Please type ‘allow pasting’ below and hit Enter to allow pasting.) even if I wrote 'allow pasting' like it said and pressed enter it still would not paste

did you check if you had the protection mode/safe browsing enabled? That could be doing it idk, not too familiar with chrome.

you literally saved me that really was my problem thank uuu

@IONADS
Copy link

IONADS commented Oct 15, 2024

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

That worked perfect for me. Thank you. I first tried that via PowerShell, but it has problems to handle the asset-id and amend it to the correct download-URL, python was my next thought but you've been faster :-)
Downloading a few hours now, ordered a new harddrive to save all the data (must be around 4,5TB; don't know exactly, because it is still running)

@sam-david
Copy link

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

Thank you @aldenparker for your work on this script. I made one similar in ruby that allowed a bit more flexibility to choose which resolution and components included. But also overly complex. I like yours though so i am running it separately from mine. I'm curious how long the download took you for all the assets? Or even you @IONADS?

@panzerjk
Copy link

good

@IONADS
Copy link

IONADS commented Oct 17, 2024

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

Thank you @aldenparker for your work on this script. I made one similar in ruby that allowed a bit more flexibility to choose which resolution and components included. But also overly complex. I like yours though so i am running it separately from mine. I'm curious how long the download took you for all the assets? Or even you @IONADS?

I had to restart it multiple times, because it gots issues from time to time (every 20 downloads ca.) telling the asset isn't available because the asset-id was not found or does not match.
I had a look into some of the assets, it looks like (for now!) it downloaded the textures and it doesn't import the assets as i expected, got some issues.
I will try to download a special one with the script AND this special one via quixel to recognize the difference and report it here when I know more.

@metaleap
Copy link

metaleap commented Oct 17, 2024

Downloading a few hours now, ordered a new harddrive to save all the data (must be around 4,5TB; don't know exactly, because it is still running)

@IONADS you estimate 4-5TB for the total based on your ongoing downloads? Or did you only download 4-5TB so far, and still ongoing?

If you downed ~1800 already we can all extrapolate to 18k from that size. So what's the latest figures at your end, so far: how many GB/TB for how many assets, right now? Thank ya, would rather buy a 5TB than a 20TB 😁

@martinka-h
Copy link

Thank you so much!

@dariopagliaricci
Copy link

dariopagliaricci commented Oct 20, 2024

I downloaded today 1200 assets in a 500 GB drive. Half of the assets didn’t download due to two specific issues (that I commented on gist of Alden Parker)
I’m going to keep it simple and say 1000 assets = 500 GB. 18000 (rounded) should be 9000 GB (9 TB).

@ZyronJustBlank
Copy link

I can't paste the code it keeps telling me this (Warning: Don’t paste code into the DevTools Console that you don’t understand or haven’t reviewed yourself. This could allow attackers to steal your identity or take control of your computer. Please type ‘allow pasting’ below and hit Enter to allow pasting.) even if I wrote 'allow pasting' like it said and pressed enter it still would not paste

did you check if you had the protection mode/safe browsing enabled? That could be doing it idk, not too familiar with chrome.

In console don't literally add colons just the words 'allow pasting' and you can be able to paste the script.

@DamienBlackwood
Copy link

DamienBlackwood commented Oct 21, 2024 via email

@pseudocolor
Copy link

pseudocolor commented Oct 22, 2024

I got the 400 bad request 'unable to add item' warning after click the OK button
Tried on firefox and opera
Is the script still working?

edited
turns out I have to click on blue 'subscribe' button first, until I got unreal engine access, and then run the script, and its works fine

@dariopagliaricci
Copy link

dariopagliaricci commented Oct 22, 2024

wait hold up how do you take it to a drive

I copy the script to an external drive.
I’m on a Mac. In terminal, I went to the directory of said external drive, using cd and the corresponding path.
Then I run the script like this python3 download_megascans.py
It will ask for Quixel auth token, then if you want to copy ALL, and then how many agents (4 by default).
you need to follow the instructions of the script in the gist.

@martyrot
Copy link

I Only get this type of error message, and no assets are downloading:
--> UNABLE TO ADD ITEM Item uhukeheew | Burnt Forest Floor (Asset acquire have been restricted. Go to FAB to download this asset)
Ive tried Chrome, Edge and FireFox, and get the same error.

@metaleap
Copy link

metaleap commented Oct 22, 2024

Go to FAB to download this asset

There's your answer @martyrot .. you can claim them all with 1 button click on fab.com — when their servers aren't down, which they seem to be right now. But I did so earlier today, and it was all listed then in fab's My Library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment