Skip to content

Instantly share code, notes, and snippets.

@katowulf
Last active September 21, 2023 20:28
Show Gist options
  • Save katowulf/6383103 to your computer and use it in GitHub Desktop.
Save katowulf/6383103 to your computer and use it in GitHub Desktop.
Get only new items from Firebase
// assumes you add a timestamp field to each record (see Firebase.ServerValue.TIMESTAMP)
// pros: fast and done server-side (less bandwidth, faster response), simple
// cons: a few bytes on each record for the timestamp
var ref = new Firebase(...);
ref.orderByChild('timestamp').startAt(Date.now()).on('child_added', function(snapshot) {
console.log('new record', snap.key());
});
// pros: works?
// cons: fetches entire record set, requires discarding the first record
// credits: http://stackoverflow.com/questions/12850789/is-there-a-way-to-know-in-what-context-child-added-is-called-particularly-page/12851236#12851236
var ref = new Firebase(...);
ref.once("value", function(snap) {
//TODO: display initial state...
// Object.keys not supported in IE 8, but has a polyfill: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys
var keys = Object.keys(snap.val()||{});
var lastIdInSnapshot = keys[keys.length-1]
ref.orderByKey().startAt(lastIdInSnapshot).on("child_added", function(newMessSnapshot) {
if( snap.key() === lastIdInSnapshot ) { return; }
console.log('new record', snap.key());
}
}
// pros: does not fetch entire record set
// cons: triggers child_removed events when last item changes, grabs the last record which needs to be ignored
var first = true;
var ref = new Firebase(...);
ref.limitToLast(1).on("child_added", function(snap) {
if( first ) {
first = false;
}
else {
console.log('new record', snap.key());
}
});
@ActiveHawk
Copy link

This doesn't work if you add children in this script directly after you put on the listener. What happens is the cb fires for the new items first before ever sending the exiting one. This means assuming that the first call to the callback will be the one that should get tossed is a bad assumption. I am looking for a solution but haven't found an elegant one.

@ActiveHawk
Copy link

I'm referring to the endAt().limit(1) method

@meetwudi
Copy link

I am confused why Firebase do not support this natively. This is somehow a hacky way, though.

@meetwudi
Copy link

Also, in the second method, you are still going to pull all previous data from server, which is not efficient.

@katowulf
Copy link
Author

Updated for 2.x queries.

@lhazell
Copy link

lhazell commented Aug 31, 2015

How would I use orderByChild('timestamp') with AngularFire's $watch?

@misterdev
Copy link

Very useful! Thanks

@gylippus
Copy link

@katowulf, any thoughts on why 3_limit_endat.js doesn't work well on larger datasets? It triggers as expected on a node with 10 records, but does not trigger on a node with 400,000 records.

@ngerritsen
Copy link

ngerritsen commented Aug 13, 2016

I think I like the 1_query_timestamp.js method best. But is there a way to not use the client side date but the server side date at startAt?

@pomber
Copy link

pomber commented Sep 11, 2016

@ngerritsen firebase.database.ServerValue.TIMESTAMP

Edit: this does not work on startAt
http://stackoverflow.com/q/39437845/1325646

@dougamos
Copy link

The problem with 1_query_timestamp.js is that client time may be out of sync with server time. As mentioned firebase.database.ServerValue.TIMESTAMP cannot be used with startAt.

There is an approach to get the servertime, see: http://stackoverflow.com/questions/23128027/retrieve-firebase-server-time-without-setting-it-first.

@kofifus
Copy link

kofifus commented Feb 17, 2017

Kato, on 3 you remark 'triggers child_removed events when last item changes'

can you explain that ? what does child_removed has to do with it ?

also isn't 3 missing orderByKey() ?

@Venryx
Copy link

Venryx commented Mar 22, 2017

@kofifus I think he means that when you first run the code, it returns the last already-existing item in the database. Whereas, you actually only want to know about new items that will be added in the future. Hence, it requires that if statement in the handler to ignore the first entry.

@Sarasranglt
Copy link

Thank you ! That worked for me.

@carlosvillu
Copy link

It works! Awesome! Respect 🙇‍♂️

@AhmadSalmanKhan
Copy link

how can i use timestamp with python? I am going to save image from python to database and want to retrieve the latest image uploaded from database

@PhanSon95
Copy link

Thanks for your solution ! It's work for me
🥇

@ctf0
Copy link

ctf0 commented Jul 26, 2019

dont forget to add an index for the timestamp to make the filtration quicker https://www.firebase.com/docs/security/guide/indexing-data.html

@Pushpavel
Copy link

Pushpavel commented Aug 24, 2020

This is also one solution,
ref.startAt(null,ref.push().key).on('child-added,...'
works only when keys are generated by push()

@nwparker
Copy link

This is also one solution,
ref.startAt(null,ref.push().key).on('child-added,...'
works only when keys are generated by push()

This is interesting, do you know why this works? Are keys generated in order by RTDB?

@SrBrahma
Copy link

SrBrahma commented Oct 9, 2020

On 2), ref.orderByKey().startAt(lastIdInSnapshot).on("child_added"..., I found out that instead of throwing away the first value, you can add a '-' to the lastIdSnapshot. It's just an character with a low ASCII value, so any push'ed item will go after the previous one + '-'.

@Pushpavel
Copy link

Pushpavel commented Oct 10, 2020

This is interesting, do you know why this works?

if we were to use ref.push().key to put a new entry on ref, it would be stored as the last child.So, using second parameter of startAt function, we retrive childs with key greater than or equal to ref.push().key
https://firebase.google.com/docs/reference/js/firebase.database.Query#startat

Are keys generated in order by RTDB?

yes, https://firebase.google.com/docs/database/admin/save-data#push-vs-transaction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment