Skip to content

Instantly share code, notes, and snippets.

@jamiephan
Last active December 20, 2024 07:04
Show Gist options
  • Save jamiephan/0c04986c7f2e62d5c87c4e8c8ce115fc to your computer and use it in GitHub Desktop.
Save jamiephan/0c04986c7f2e62d5c87c4e8c8ce115fc to your computer and use it in GitHub Desktop.
A script to automatically add ALL items to your account in quixel

Script to add all items from quixel

As quixel is being removed, all items are free to aquire. This script is to automate the process to add items to your account (As of writing, a total of 18874 items)

Note: This script only tested in the latest version of Chrome.

How to use

  1. Copy the script from below (run.js)
  2. Login into https://quixel.com
  3. Go to https://quixel.com/megascans/collections
  4. Open devtools (F12) -> Go to "Console" tab
  5. Paste in the script and press Enter.
  6. A dialog should popup confirming the execution, click "OK"
  7. Sit back and wait

Common issues

  • Getting "Forbidden" error. (Even after refresh, the whole page just shows "Forbidden")
    • There is a chance that the API adding too fast and you hit the rate limit of the API. (My testing is around after 10 pages, so ~10k items).
    • Wait after ~10-20 minutes and continue. See Common Fixes -> Restart script to continue the execution after you can load https://quixel.com.
  • The script seems to be paused/hang
    • It could be too much logging going it. Try monitor the script, if it says "END PAGE X", note the page number down (in case need restart) and clear the console by clicking the "🚫" icon in devtools.
    • See Common Fixes -> Restart script for fixing.
  • Getting the error **UNABLE TO ADD ITEM**
    • There should have the error message shown in ( ). If it is user already owns specified asset at a higher or equal resolution, then its already in your account.
  • Getting the error cannot find authentication token. Please login again
    • Clear browser cookies and re-login quixel again. Try just simply add 1 item manully. If it success, then see Common Fixes -> Restart script for continue the execution.

Common Fixes

Restart Script

  1. Note which page it was running
  2. Copy the run.js script
  3. Update the startPage = 0 on the first line to startPage = 10 (assuming page 10 was hanged)

Change Log

  1. Initial Script launch
  2. Update to clear logs to reduce chance of hanging
  3. [CURRENT] Skip adding items that already was acquired. Reduced logs. Added more info after script completion to show purchased item count. Due to now skipping purchased items, you technically don't need to specify the startPage anymore.
((async (startPage = 0, autoClearConsole = true) => {
const getCookie = (name) => {
const value = `; ${document.cookie}`;
const parts = value.split(`; ${name}=`);
if (parts.length === 2) return parts.pop().split(';').shift();
}
const callCacheApi = async (params = {}) => {
const defaultParams = {
page: 0,
maxValuesPerFacet: 1000,
hitsPerPage: 1000,
attributesToRetrieve: ["id", "name"].join(",")
}
const response = await fetch("https://proxy-algolia-prod.quixel.com/algolia/cache", {
"headers": {
"x-api-key": "2Zg8!d2WAHIUW?pCO28cVjfOt9seOWPx@2j"
},
"body": JSON.stringify({
url: "https://6UJ1I5A072-2.algolianet.com/1/indexes/assets/query?x-algolia-application-id=6UJ1I5A072&x-algolia-api-key=e93907f4f65fb1d9f813957bdc344892",
params: new URLSearchParams({ ...defaultParams, ...params }).toString()
}),
"method": "POST",
})
return await response.json()
}
const callAcl = async ({ id, name }) => {
const response = await fetch("https://quixel.com/v1/acl", {
"headers": {
"authorization": "Bearer " + authToken,
"content-type": "application/json;charset=UTF-8",
},
"body": JSON.stringify({ assetID: id }),
"method": "POST",
});
const json = await response.json()
if (json?.isError) {
console.error(` --> **UNABLE TO ADD ITEM** Item ${id} | ${name} (${json?.msg})`)
} else {
console.log(` --> ADDED ITEM Item ${id} | ${name}`)
}
}
const callAcquired = async () => {
const response = await fetch("https://quixel.com/v1/assets/acquired", {
"headers": {
"authorization": "Bearer " + authToken,
"content-type": "application/json;charset=UTF-8",
},
"method": "GET",
});
return await response.json()
}
// 1. Check token exist, quixel API needs it
console.log("-> Checking Auth API Token...")
let authToken = ""
try {
const authCookie = getCookie("auth") ?? "{}"
authToken = JSON.parse(decodeURIComponent(authCookie))?.token
if (!authToken) {
return console.error("-> Error: cannot find authentication token. Please login again.")
}
} catch (_) {
return console.error("-> Error: cannot find authentication token. Please login again.")
}
// 2. Get all currently acquired items
console.log("-> Get Acquired Items...")
const acquiredItems = (await callAcquired()).map(a => a.assetID)
// 3. Get total count of items
console.log("-> Getting Total Number of Pages....")
const { nbPages: totalPages, hitsPerPage: itemsPerPage, nbHits: totalItems } = await callCacheApi()
console.log("-> ==============================================")
console.log(`-> Total # of items: ${totalItems}`)
console.log(`-> ${totalPages} total pages with ${itemsPerPage} per page`)
console.log(`-> Total Items to add: ${(totalItems - acquiredItems.length)}.`)
console.log("-> ==============================================")
if (!confirm(`Click OK to start adding ${(totalItems - acquiredItems.length)} items in your account.`)) return
// Loop
for (let pageIdx = startPage || 0; pageIdx < totalPages; pageIdx++) {
console.log(`-> ======================= PAGE ${pageIdx + 1}/${totalPages} START =======================`)
console.log("-> Getting Items from page " + (pageIdx + 1) + " ...")
const { hits: items } = await callCacheApi({ page: pageIdx })
console.log("-> Adding non-acquired items...")
// Filter out owned items
const unownedItems = items.filter(i => !acquiredItems.includes(i.id))
const aclPromises = unownedItems.map(callAcl)
await Promise.all(aclPromises)
console.log(`-> ======================= PAGE ${pageIdx + 1}/${totalPages} COMPLETED =======================`)
if (autoClearConsole) console.clear() // Fix the issue that too much log hangs the console. Set autoClearConsole = false to keep the logs
}
console.log("-> Getting new acquired info...")
// Get acquired items again
const newItemsAcquired = (await callAcquired()).length
const newTotalCount = (await callCacheApi()).nbHits
console.log(`-> Completed. Your account now have a total of ${newItemsAcquired} out of ${newTotalCount} items.`)
alert(`-> Your account now have a total of ${newItemsAcquired} out of ${newTotalCount} items.\n\nIf you find some items missing, try refresh the page and run the script again.`)
})())
@ParaLizzard
Copy link

Could someone make a script for Fab after it launches to add Megascans to library?

@limdingwen
Copy link

limdingwen commented Oct 11, 2024 via email

@orion200215
Copy link

So I think I screwed myself over. After getting everything downloaded right? I refreshed....and I get a 403 forbidden error with a translation option between danish and english....I refresh, same error. And i close the window and then open a new tab and open through quixel...same error....so....ouch
Screenshot (3)

@nurb2kea
Copy link

So I think I screwed myself over. After getting everything downloaded right? I refreshed....and I get a 403 forbidden error with a translation option between danish and english....I refresh, same error. And i close the window and then open a new tab and open through quixel...same error....so....ouch Screenshot (3)

just read my post above or the instructions that say that it will be gone in 10-20 min. and the 'forbidden' is gone!

@xtremertx
Copy link

They will remove access to Quixel and Bridge in the future, if you don't store assets locally on disk you WILL LOSE ACCESS!!!

They talk that WE can get those assets on FAB.COM before years 2024 ends, but I don't see anything like that there..

See:
https://support.fab.com/s/article/Fab-Transition-FAQs

@aldenparker
Copy link

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to
jamiephan btw this script helped me understand what I needed to do for mine.
https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

@FabriceBazzaro
Copy link

Thank you for the script, really nice work, I've added a little function at the beginning of the script to avoid the 403 error due to the number of calls too fast :

const delay = (milliseconds) => new Promise(resolve => {
        setTimeout(resolve, milliseconds);
  });

And I call this function at the end of the loop :

    if (autoClearConsole) console.clear() // Fix the issue that too much log hangs the console. Set autoClearConsole = false to keep the logs
	await delay(10000);
  }

I've purchased all the assets in two successive calls (I just had a 502 - bad gateway - on an asset).

@joe777777777777
Copy link

I think I know what the problem might be for some, You have to enter "allow paste" first and then paste the code, it worked for me.
Desktop Screenshot 2024 10 14 - 19 43 16 80

@Arwa99-lab
Copy link

I can't paste the code it keeps telling me this (Warning: Don’t paste code into the DevTools Console that you don’t understand or haven’t reviewed yourself. This could allow attackers to steal your identity or take control of your computer. Please type ‘allow pasting’ below and hit Enter to allow pasting.) even if I wrote 'allow pasting' like it said and pressed enter it still would not paste

did you check if you had the protection mode/safe browsing enabled? That could be doing it idk, not too familiar with chrome.

you literally saved me that really was my problem thank uuu

@IONADS
Copy link

IONADS commented Oct 15, 2024

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

That worked perfect for me. Thank you. I first tried that via PowerShell, but it has problems to handle the asset-id and amend it to the correct download-URL, python was my next thought but you've been faster :-)
Downloading a few hours now, ordered a new harddrive to save all the data (must be around 4,5TB; don't know exactly, because it is still running)

@sam-david
Copy link

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

Thank you @aldenparker for your work on this script. I made one similar in ruby that allowed a bit more flexibility to choose which resolution and components included. But also overly complex. I like yours though so i am running it separately from mine. I'm curious how long the download took you for all the assets? Or even you @IONADS?

@panzerjk
Copy link

good

@IONADS
Copy link

IONADS commented Oct 17, 2024

As a follow up to this I made this script to help download everything. Just in case anyone is paranoid about them taking it away. Much love to jamiephan btw this script helped me understand what I needed to do for mine. https://gist.github.com/aldenparker/0d8fee85469d3561bc3a772a03d642cb

Thank you @aldenparker for your work on this script. I made one similar in ruby that allowed a bit more flexibility to choose which resolution and components included. But also overly complex. I like yours though so i am running it separately from mine. I'm curious how long the download took you for all the assets? Or even you @IONADS?

I had to restart it multiple times, because it gots issues from time to time (every 20 downloads ca.) telling the asset isn't available because the asset-id was not found or does not match.
I had a look into some of the assets, it looks like (for now!) it downloaded the textures and it doesn't import the assets as i expected, got some issues.
I will try to download a special one with the script AND this special one via quixel to recognize the difference and report it here when I know more.

@metaleap
Copy link

metaleap commented Oct 17, 2024

Downloading a few hours now, ordered a new harddrive to save all the data (must be around 4,5TB; don't know exactly, because it is still running)

@IONADS you estimate 4-5TB for the total based on your ongoing downloads? Or did you only download 4-5TB so far, and still ongoing?

If you downed ~1800 already we can all extrapolate to 18k from that size. So what's the latest figures at your end, so far: how many GB/TB for how many assets, right now? Thank ya, would rather buy a 5TB than a 20TB 😁

@martinka-h
Copy link

Thank you so much!

@dariopagliaricci
Copy link

dariopagliaricci commented Oct 20, 2024

I downloaded today 1200 assets in a 500 GB drive. Half of the assets didn’t download due to two specific issues (that I commented on gist of Alden Parker)
I’m going to keep it simple and say 1000 assets = 500 GB. 18000 (rounded) should be 9000 GB (9 TB).

@ZyronJustBlank
Copy link

I can't paste the code it keeps telling me this (Warning: Don’t paste code into the DevTools Console that you don’t understand or haven’t reviewed yourself. This could allow attackers to steal your identity or take control of your computer. Please type ‘allow pasting’ below and hit Enter to allow pasting.) even if I wrote 'allow pasting' like it said and pressed enter it still would not paste

did you check if you had the protection mode/safe browsing enabled? That could be doing it idk, not too familiar with chrome.

In console don't literally add colons just the words 'allow pasting' and you can be able to paste the script.

@DamienBlackwood
Copy link

DamienBlackwood commented Oct 21, 2024 via email

@pseudocolor
Copy link

pseudocolor commented Oct 22, 2024

I got the 400 bad request 'unable to add item' warning after click the OK button
Tried on firefox and opera
Is the script still working?

edited
turns out I have to click on blue 'subscribe' button first, until I got unreal engine access, and then run the script, and its works fine

@dariopagliaricci
Copy link

dariopagliaricci commented Oct 22, 2024

wait hold up how do you take it to a drive

I copy the script to an external drive.
I’m on a Mac. In terminal, I went to the directory of said external drive, using cd and the corresponding path.
Then I run the script like this python3 download_megascans.py
It will ask for Quixel auth token, then if you want to copy ALL, and then how many agents (4 by default).
you need to follow the instructions of the script in the gist.

@martyrot
Copy link

I Only get this type of error message, and no assets are downloading:
--> UNABLE TO ADD ITEM Item uhukeheew | Burnt Forest Floor (Asset acquire have been restricted. Go to FAB to download this asset)
Ive tried Chrome, Edge and FireFox, and get the same error.

@metaleap
Copy link

metaleap commented Oct 22, 2024

Go to FAB to download this asset

There's your answer @martyrot .. you can claim them all with 1 button click on fab.com — when their servers aren't down, which they seem to be right now. But I did so earlier today, and it was all listed then in fab's My Library.

@DisneyUA
Copy link

Screenshot_1
They did something so that all assets are loaded through the Fab, how to deal with it?

@Dusaral
Copy link

Dusaral commented Oct 22, 2024

Same here - it seems that's impossible to add those since Fab got online today ...

@polynook
Copy link

Epic allows to claim all assets with a single click of a button if you go to their site: https://www.fab.com/megascans-free

@dsoap96
Copy link

dsoap96 commented Oct 22, 2024

image
For all requests

@hyperion2022
Copy link

Could you please adapt this code to claim all free assets on Fab? (That being, Unity, Unreal and UEFN)

@hakodev
Copy link

hakodev commented Oct 22, 2024

Could you please adapt this code to claim all free assets on Fab? (That being, Unity, Unreal and UEFN)

https://www.fab.com/megascans-free
All it takes is one click.

@melnation-com
Copy link

Still thanks for all your hardwork, the Claim Megascans Free button was just an afterthought and wouldn't even be born if it weren't for the what I call leet code community and shown it's possible in due time. If it weren't for you, they'll probably chatGPT some PR excuse that all assets were 5-digit GB in size and have to ROI and make it paid if not make it available for free directly.

@melnation-com
Copy link

Downloading a few hours now, ordered a new harddrive to save all the data (must be around 4,5TB; don't know exactly, because it is still running)

@IONADS you estimate 4-5TB for the total based on your ongoing downloads? Or did you only download 4-5TB so far, and still ongoing?

If you downed ~1800 already we can all extrapolate to 18k from that size. So what's the latest figures at your end, so far: how many GB/TB for how many assets, right now? Thank ya, would rather buy a 5TB than a 20TB 😁

Still a better failsafe than ToS'es that they can retroactively change at will like Unity. For my hobby and this quality and this flexible assets, for world building and for peace of mind, it's a small price to pay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment