Skip to content

Instantly share code, notes, and snippets.

@wikiti
Last active December 25, 2024 19:40
Show Gist options
  • Save wikiti/f64aa14ee65fd194467d8c8944ed8372 to your computer and use it in GitHub Desktop.
Save wikiti/f64aa14ee65fd194467d8c8944ed8372 to your computer and use it in GitHub Desktop.
A script to automatically add all Quixel MegaScans items to your FAB account

Important

This script was built for educational purposes. Due to how FAB APIs are built to handle search pagination, all assets might not be claimable through this script. With the last script update, a total of ~600 scanned pages, with ~14k assets. Please note that, by clicking on the "Claim All" quixel button, you'll already have claimed all assets for future free usage, beyond 2024 (official response). This script simply makes sure that assets are added to a library.

Script to add all quixel megascan items to your FAB account

As quixel megascan items will stop being free after 2024. So, this script will go through all Quixel MegaScan items and add them to your account. Quixel would be working on a tool to automatically add all resources to your FAB account, but I couldn't help myself and I wrote a script to do it.

How to use

  1. Copy the contents of the script below (claim.js)
  2. Log in into your fab dashboard
  3. Make sure you have accepted the terms and conditions (e.g. try claiming any free asset)
  4. Go to the Quixel seller list at https://www.fab.com/sellers/Quixel
  5. Open devtools (F12) -> Go to "Console" tab
  6. Paste the script and press enter

Feel free to tweak the constants at the very top of the script to reduce wait times, or to prevent tiemouts or throttlings.

Known issues

The script stopped working after X claims

Try running the script again. Before doing so, take note on the cursor and filter index values, and update the script with those, so you can resume the process at the same point where the script stopped executing. For example, if the script printed this message before stopping execution:

Getting content from page cursor bz0yMDQw, with filter index 1...

Update the top variables to match those values:

async function claimQuixelAssets() {
  // Configurable constants to resume
  const INITIAL_PAGE_CURSOR = "bz0yMDQw";
  const INITIAL_FILTER_INDEX = 1;
  
  // ...

Some claims constantly fail with a 500 error

For some reason, some assets cannot be claimed, not even through the web UI. Add the item IDs to the SKIP_ITEM_IDS constant at the top of the script. The following items were throwing errors when running the script:

3e441085-0d1d-4476-93bc-e68d184026f2
f74671f7-d8a5-42e7-9dd4-e76639ca0ef0
d808f867-2560-4476-ad05-a1d999bd940b
29c89901-9659-4a33-909c-fc320ef9c4fd
d63d8359-6bd6-417d-aa91-6c7ce6cd4f84
e9858851-471b-4ab5-b9c9-b7183ec477eb
25702175-05c5-41d0-913c-551b36f987de
dd67528e-0673-42fd-847c-8be873e70b4c
5434aac2-0abb-4af6-bbd8-42db0bbe2137
9eea8387-a859-4df9-9298-65e0cfb14509
08102c5c-5a57-4873-bd2c-61d6b91f55a5

So, I would update the SKIP_ITEM_IDS to be:

const SKIP_ITEM_IDS = [
  '3e441085-0d1d-4476-93bc-e68d184026f2',
  'f74671f7-d8a5-42e7-9dd4-e76639ca0ef0',
  'd808f867-2560-4476-ad05-a1d999bd940b',
  '29c89901-9659-4a33-909c-fc320ef9c4fd',
  'd63d8359-6bd6-417d-aa91-6c7ce6cd4f84',
  'e9858851-471b-4ab5-b9c9-b7183ec477eb',
  '25702175-05c5-41d0-913c-551b36f987de',
  'dd67528e-0673-42fd-847c-8be873e70b4c',
  '5434aac2-0abb-4af6-bbd8-42db0bbe2137',
  '9eea8387-a859-4df9-9298-65e0cfb14509',
  '08102c5c-5a57-4873-bd2c-61d6b91f55a5',
];

Script is a bit slow between pages

The script sleeps by default for 5 seconds to prevent request throttling (not sure if there is any to be honest). If you don't want to wait that much, set the value of WAIT_TIME_BETWEEN_PAGES to 0.

Script stops after X pages

For some reason, the search query might stop depending on the sorting key that's used. Or, in some cases, repeat the same page over and over again. I'm currently trying to find a solution to this, but it totally depends on how the server responds to the search queries made by this script.

I'm getting HTTP 401 errors on all claim requests, even if I add them to SKIP_ITEMS_ID

You must accept the terms and conditions first from FAB. For example, try manually adding through the web interface one free item to your library. Then, try running the script.

Changelog

  • [V1 23/10/24] Initial version
  • [V2 24/10/24] Changed query parameters for retrieving item list
  • [V3 25/10/24] Add error detection for cursor loops (around ~210 pages)
  • [V4 25/10/24] Add multiple search filters for covering as many assets as possible
  • [V5 26/10/24] Corrected an issue where the last filter (3d models) was not executed
  • [V6 26/10/24] Add Set check for claimable assets, so they are claimed once, in case filters are repeated
  • [V7 16/11/24] Rename CSRF cookie to fab_csrftoken
  • [V8 19/12/24] Add extra failing assets to skip list (suggested in comments)

Credits

I wrote the script by myself, but it was heavily inspired by this script, coded by jamiephan. Thanks to all the folks in the comment section who pointed out bugs, suggested minor improvements, and fixed some edge cases!

async function claimQuixelAssets() {
// Configurable constants to resume
const INITIAL_PAGE_CURSOR = null;
const INITIAL_FILTER_INDEX = 0;
const WAIT_TIME_BETWEEN_PAGES = 5; // in seconds
const WAIT_TIME_BETWEEN_CLAIMS = 0; // in seconds
// Debugging constants
const CLEAR_CONSOLE = true;
const PARALLEL_CLAIM = true;
const CLAIM = true;
// Hacky constants to overcome API limitations
const SKIP_ITEM_IDS = [
'3e441085-0d1d-4476-93bc-e68d184026f2',
'f74671f7-d8a5-42e7-9dd4-e76639ca0ef0',
'd808f867-2560-4476-ad05-a1d999bd940b',
'29c89901-9659-4a33-909c-fc320ef9c4fd',
'd63d8359-6bd6-417d-aa91-6c7ce6cd4f84',
'e9858851-471b-4ab5-b9c9-b7183ec477eb',
'25702175-05c5-41d0-913c-551b36f987de',
'dd67528e-0673-42fd-847c-8be873e70b4c',
'5434aac2-0abb-4af6-bbd8-42db0bbe2137',
'9eea8387-a859-4df9-9298-65e0cfb14509',
'08102c5c-5a57-4873-bd2c-61d6b91f55a5',
]; // A list of strings, in case an item constantly fails to claim due to 500 errors
const FILTERS = [
// Environments
'listing_types=environment',
// Materials
'listing_types=material',
// 'categories=art-traditional&listing_types=material',
// 'categories=building-human-made&listing_types=material',
// 'categories=damage-grunge&listing_types=material',
// 'categories=fabric-clothing&listing_types=material',
// 'categories=nature-terrain&listing_types=material',
// 'categories=organic&listing_types=material',
// Decals
'listing_types=decal',
// 'categories=art-traditional&listing_types=decal',
// 'categories=building-human-made&listing_types=decal',
// 'categories=damage-grunge&listing_types=decal',
// 'categories=fabric-clothing&listing_types=decal',
// 'categories=nature-terrain&listing_types=decal',
// 'categories=organic&listing_types=decal',
// Brushes
'listing_types=brush',
// 'categories=art-traditional&listing_types=brush',
// 'categories=building-human-made&listing_types=brush',
// 'categories=damage-grunge&listing_types=brush',
// Atlas
'listing_types=atlas',
// 'categories=building-human-made&listing_types=atlas',
// 'categories=damage-grunge&listing_types=atlas',
// 'categories=nature-terrain&listing_types=atlas',
// 3D models
'listing_types=3d-model',
// 'categories=buildings-architecture&listing_types=3d-model',
// 'categories=electronics-technology&listing_types=3d-model',
// 'categories=food-drink&listing_types=3d-model',
// 'categories=furniture-fixtures&listing_types=3d-model',
// 'categories=nature-plants&listing_types=3d-model',
// 'categories=environments-scenes&listing_types=3d-model',
// 'categories=objects-decor&listing_types=3d-model',
// ...
]
// Counters
let totalPages = 0;
let claimedAssets = new Set();
// Functions
const clearConsole = () => {
if (CLEAR_CONSOLE) {
console.clear()
}
else {
console.log(`-------------------------------------------------------------------`);
}
}
const sleep = (seconds) => {
return new Promise(r => setTimeout(r, seconds * 1000));
}
const getCSRFToken = () => {
const cookieName = 'fab_csrftoken';
return document.cookie.match('(^|;)\\s*' + cookieName + '\\s*=\\s*([^;]+)')?.pop() || '';
}
const postResource = async (path, body) => {
const baseUrl = "https://www.fab.com/i/listings";
const formData = new FormData();
for (const name in body) {
formData.append(name, body[name]);
}
const response = await fetch(`${baseUrl}/${path}`, {
headers: {
"accept": "application/json, text/plain, */*",
"x-csrftoken": getCSRFToken(),
},
method: "POST",
body: formData,
});
if (!response.ok) {
throw new Error(`Failed request with ${response.status} status code`);
}
return await response.text();
}
const getResource = async (path) => {
const baseUrl = "https://www.fab.com/i/listings";
const response = await fetch(`${baseUrl}/${path}`, {
headers: {
"accept": "application/json, text/plain, */*",
"x-csrftoken": getCSRFToken(),
},
method: "GET",
});
if (!response.ok) {
throw new Error(`Failed request with ${response.status} status code`);
}
return await response.json();
}
const fetchItem = async (itemId) => {
try {
return await getResource(itemId);
}
catch (error) {
throw new Error(`Failed to fetch item ${itemId} due to: ${error}`)
}
}
const claimItem = async (item) => {
let offer_id = item.licenses.find((license) => license.slug == "professional").offerId;
// offer_id = item.licenses[0].offerId;
try {
await postResource(`${item.uid}/add-to-library`, { offer_id });
}
catch (error) {
throw new Error(`Failed to claim item ${item.title} (${item.uid}) due to: ${error}`)
}
}
const handleItem = async (listItem) => {
if (SKIP_ITEM_IDS.includes(listItem.uid)) {
console.log(`-> Skipping ${listItem.title} (id: ${listItem.uid})`);
return;
}
if (claimedAssets.has(listItem.uid)) {
console.log(`-> Already claimed ${listItem.title} (id: ${listItem.uid})`);
return;
}
console.log(`-> Claiming ${listItem.title} (id: ${listItem.uid})`);
if (CLAIM) {
const item = await fetchItem(listItem.uid);
await claimItem(item);
}
claimedAssets.add(listItem.uid);
await sleep(WAIT_TIME_BETWEEN_CLAIMS);
}
const fetchItems = async (filterIndex, pageCursor) => {
const filters = FILTERS[filterIndex];
try {
const cursorQuery = pageCursor ? `&cursor=${pageCursor}` : ''
return await getResource(`search?seller=Quixel&${filters}${cursorQuery}`);
}
catch (error) {
throw new Error(`Failed to get page ${pageCursor || 'null'} due to: ${error}`);
}
}
const handleListPage = async (filterIndex, pageCursor) => {
console.log(`-> Getting content from page cursor ${pageCursor || '<not-set>'}, with filter index ${filterIndex}...`);
let page = await fetchItems(filterIndex, pageCursor);
console.log(`-> Found ${page.results.length} results on this page`);
if (PARALLEL_CLAIM) {
const promises = page.results.map((item) => handleItem(item));
await Promise.all(promises)
}
else {
for (const item of page.results) {
await handleItem(item);
}
}
await sleep(WAIT_TIME_BETWEEN_PAGES);
clearConsole();
totalPages += 1;
if (pageCursor != null && pageCursor == page.cursors.next) {
throw new Error(`Cursor loop found on page ${pageCursor}; this is a problem on FAB's API, not this script`);
}
return page.cursors.next
}
// Main loop function
const main = async () => {
clearConsole();
try {
for (let filterIndex = INITIAL_FILTER_INDEX; filterIndex < FILTERS.length; filterIndex += 1) {
let pageCursor = INITIAL_PAGE_CURSOR;
do {
console.log(`-> Pages processed so far: ${totalPages} pages`);
console.log(`-> Assets claimed so far: ${claimedAssets.size} assets`);
pageCursor = await handleListPage(filterIndex, pageCursor)
} while (pageCursor != null);
}
console.log('-> Script execution completed successfully!')
console.log(`-> Total pages processed: ${totalPages} pages`);
console.log(`-> Total assets claimed: ${claimedAssets.size} assets`);
}
catch (error) {
console.error(`-> ${error.message}`);
console.error("-> Try waiting for a couple of minutes, then re-running the script by changing INITIAL_FILTER_INDEX and INITIAL_PAGE_CURSOR");
console.error(error);
}
}
return await main();
}
claimQuixelAssets();

Copyright 2024 Daniel herzog

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

@kaefjke
Copy link

kaefjke commented Nov 7, 2024

BRO! Thank you A LOT!!! You're doing the GREAT job!!!

@Shtremann
Copy link

Shtremann commented Nov 8, 2024

Hello!
I managed to get 8335 items
In the posts above the guys managed to get more items, about 14000.
Maybe I'm doing something wrong, please tell me.

Ha7g5bFR7Y

@wocmultimedia
Copy link

wocmultimedia commented Nov 8, 2024

@Shtremann Did you claimed the items before from the quixel.com site? I had previously claimed the items in the quixel.com website then I used this script to get them from Fab site. Maybe it's this that I got almost all the elements on my library. Try to re run the script.

@Fabio1981OK
Copy link

Fabio1981OK commented Nov 8, 2024

but they are not present in the bridge, right ?
dsdsds

@Shtremann
Copy link

Shtremann commented Nov 8, 2024

@Shtremann Did you claimed the items before from the quixel.com site? I had previously claimed the items in the quixel.com website then I used this script to get them from Fab site. Maybe it's this that I got almost all the elements on my library. Try to re run the script.

Thank you for your reply
I restarted the script after some time and I have about 13500 assets accumulated

Apparently you can't get all 19,000 of them)
Thanks to the author of the script

@wocmultimedia
Copy link

wocmultimedia commented Nov 8, 2024

@Shtremann
That's the same amount I got. No, you can't get the all 19,000 because neither all are free nor all items have been transferred to Fab site. If do you visit the quixel.com site you will find that not all items have been migrated.
But I think we can be satisfied the same

@Shtremann
Copy link

@Shtremann That's the same amount I got. No, you can't get the all 19,000 because neither all are free nor all items have been transferred to Fab site. If do you visit the quixel.com site you will find that not all items have been migrated. But I think we can be satisfied the same

That's for sure!
And on that, thanks a lot to the author for the script!

@Trucabulles
Copy link

Hi, Thanks for this script. It works very well for "owning" but are you aware of a way to add all items to the lilbray at once since it does not do so ?

@HyliGun174
Copy link

image

@Kaptila
Copy link

Kaptila commented Nov 16, 2024

Hello everyone! Who has encountered this error? How did you solve it? (403)
Screenshot_2

@Kaptila
Copy link

Kaptila commented Nov 16, 2024

@HyliGun174

image

were you able to solve it?

@HarkTu
Copy link

HarkTu commented Nov 16, 2024

I added manually assets which gave that error then run script again. got that error for 3-4 assets.

@HyliGun174
Copy link

@HyliGun174

изображение

удалось ли вам решить эту проблему?

I haven't realized it yet

@shlakshadegrok
Copy link

change const cookieName = 'sb_csrftoken'; to const cookieName = 'fab_csrftoken';
and it work again

@liza2499
Copy link

error

Unable to download, can somebody please help

@Ahtaler
Copy link

Ahtaler commented Nov 19, 2024

Can this script use to claim other sellers free items?

@franz529
Copy link

can't be use 401

@Octaviusbjr
Copy link

Octaviusbjr commented Nov 25, 2024

How to fix it:
Uncaught SyntaxError: Unexpected identifier 'assíncrona'

@franz529
Copy link

@Octaviusbjr can you show where is it?

@Joyhanka
Copy link

Joyhanka commented Dec 18, 2024

image

Please help

@skinself
Copy link

skinself commented Dec 19, 2024

image
every time stop on this moment...
somebody help?

@skinself
Copy link

skinself commented Dec 19, 2024

there, ID is given in the error msg, works for me

image

Please help

image
Fixed it. Just add ID of this Mossy Grass in SKIP_ITEM_IDS there already 3 wrong things.

@skinself
Copy link

image
const SKIP_ITEM_IDS = [
'3e441085-0d1d-4476-93bc-e68d184026f2',
'f74671f7-d8a5-42e7-9dd4-e76639ca0ef0',
'd808f867-2560-4476-ad05-a1d999bd940b',
'29c89901-9659-4a33-909c-fc320ef9c4fd',
'd63d8359-6bd6-417d-aa91-6c7ce6cd4f84',
'e9858851-471b-4ab5-b9c9-b7183ec477eb',
'25702175-05c5-41d0-913c-551b36f987de',
'dd67528e-0673-42fd-847c-8be873e70b4c',
'5434aac2-0abb-4af6-bbd8-42db0bbe2137',
'9eea8387-a859-4df9-9298-65e0cfb14509',
'08102c5c-5a57-4873-bd2c-61d6b91f55a5',

@NasaNhak
Copy link

change to this, script not fail:
const WAIT_TIME_BETWEEN_CLAIMS = 5; // in seconds

-> Script execution completed successfully!
VM146:240 -> Total pages processed: 622 pages
VM146:241 -> Total assets claimed: 14869 assets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment