Skip to content

Instantly share code, notes, and snippets.

@dj-nitehawk
Created September 7, 2023 03:27
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dj-nitehawk/02420788fb0a72c4be4752be8bd4c40b to your computer and use it in GitHub Desktop.
Save dj-nitehawk/02420788fb0a72c4be4752be8bd4c40b to your computer and use it in GitHub Desktop.
Storing `IJobStorageRecord` and `IEventStorageRecord` via EntityFramework Core
using FastEndpoints;
using MessagePack;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
public class JobRecord : IJobStorageRecord
{
public Guid Id { get; set; }
public string QueueID { get; set; }
public object Command { get; set; }
public DateTime ExecuteAfter { get; set; }
public DateTime ExpireOn { get; set; }
public bool IsComplete { get; set; }
}
public class MyDB : DbContext
{
public DbSet<JobRecord> Jobs { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder o)
=> o.UseSqlite("Data Source=JobDatabase");
protected override void OnModelCreating(ModelBuilder b)
{
MessagePackSerializer.DefaultOptions
= MessagePack.Resolvers.ContractlessStandardResolver.Options;
var converter = new ValueConverter<object, byte[]>(
v => Serialize(v),
v => Deserialize(v));
b.Entity<JobRecord>()
.Property(e => e.Command)
.HasConversion(converter)
.HasColumnType("BLOB"); //VARBINARY for sql server
}
static byte[] Serialize(object cmd) => MessagePackSerializer.Serialize(cmd);
static object Deserialize(byte[] cmdBytes) => MessagePackSerializer.Deserialize<object>(cmdBytes);
}
@dj-nitehawk
Copy link
Author

here's an alternative method without using a ValueConverter or involving the DbContext at all.

public class JobRecord : IJobStorageRecord
{
    public Guid Id { get; set; }
    public string QueueID { get; set; }
    public DateTime ExecuteAfter { get; set; }
    public DateTime ExpireOn { get; set; }
    public bool IsComplete { get; set; }
    
    //add this
    public string CommandJson { get; set; } 

    //do this
    TCommand IJobStorageRecord.GetCommand<TCommand>()
         => JsonSerializer.Deserialize<TCommand>(CommandJson)!;

    //do this
    void IJobStorageRecord.SetCommand<TCommand>(TCommand command)
         => CommandJson = JsonSerializer.Serialize(command);

    [NotMapped] //add this
    public object Command { get; set; }
}

the sample demonstrates serializing to/from JSON but for best performance consider using MessagePack and store the data as a byte array.

@JakeCroz1
Copy link

Is there anyway to get the status of the job that is in queue? Most job queues show the user the status of the job they placed in queue such as uploading media. Ideally, the client-side could fetch the status every 5 or 10 seconds since we are not using web sockets.

@dj-nitehawk
Copy link
Author

@JakeCroz1
there's no built-in mechanism yet to get a progress update.
what i typically do is, i'll have a "progress status" entity in the db which stores any progress/status values for each job/task.
the job handler itself sets these values while the job is being executed.
then anything (most likely a progress reporting endpoint) can query that "progress status" entity periodically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment