| Field Type | Personal Info |
|---|---|
| Student Name | Fares Atef |
| Github Profile | FaresAtef1 |
| LinkedIn Profile | Fares Atef |
| Organization | ENIGMA |
| Project Title | Better (de)serialization support for objects |
| Mentors | Josh Ventura, Greg Williamson |
| Primary Repository/Branch | enigma-dev |
This is the final documentation of the work done in Better (de)serialization support for objects project through GSoC'23. The ENIGMA project employs a hierarchical structure of objects to represent entities. These objects undergo serialization and deserialization processes facilitated by specific routines. However, the initial implementation had certain drawbacks, as it involved checking the type of each argument through a series of if-else statements. This approach led to code that was hard to comprehend and expand upon. In my contribution to the project, I enhanced the serialization framework, making it more user-friendly and extensible. This was achieved by redefining the serialization functions to enable automatic dispatch to supported types, eliminating the need for lengthy if-else constructs.
Additionally, I incorporated support for new data types, including generic routines. These enhancements facilitate the seamless serialization of ENIGMA's internal state, allowing for easy transfer of state either over a network or into a file.
I also introduced support for the JSON format within the serialization system, offering it as a new option alongside the existing bytes format. This addition enhances the readability and debugging capability of the serialized data.
Balloon_Pop (It is the same game in the ENIGMA site but with modified scripts).
PR #1
- This pull request is about "Changing the way the serialization functions work".
- The main serialization functions that ENIGMA has are:
enigma_internal_sizeof,serialize_into,serialize,deserialize,resize_buffer_forandenigma_internal_deserialize. - Updated the 6 serialization functions to work in a new way.
- Moved the functions from a structure similar to this:
template <typename T>
inline void internal_serialize_into(std::byte *iter, T &&value) {
if constexpr (std::is_pointer_v<std::decay_t<T>>) {
serialize_into<std::size_t>(iter, 0);
} else if constexpr (std::is_same_v<std::string, std::decay_t<T>>) {
serialize_into<std::size_t>(iter, value.size());
std::transform(value.begin(), value.end(), iter + sizeof(std::size_t), [](char c) { return static_cast<std::byte>(c); });
} else if constexpr (std::is_same_v<bool, std::decay_t<T>>) {
*iter = static_cast<std::byte>(value);
} else if constexpr (std::is_same_v<var, std::decay_t<T>>) {
serialize_var_into(iter, value);
} else if constexpr (std::is_base_of_v<variant, std::decay_t<T>>) {
serialize_variant_into(iter, value);
} else if constexpr (std::is_integral_v<std::decay_t<T>> || std::is_floating_point_v<std::decay_t<T>>){
serialize_numeric_into(iter, value);
} else {
static_assert(always_false<T>,
"'serialize_into' takes 'variant', 'var', 'std::string', bool, integral or floating types");
}
}To a structure like this:
template <typename T>
inline void internal_serialize_into(std::byte *iter, T &&value) {
if constexpr (_bytes_serialization_has_internal_serialize_into_fn_<std::decay_t<T>>) {
enigma::bytes_serialization::internal_serialize_into_fn(iter, value);
} else {
static_assert(always_false<T>,
"'internal_serialize_into' takes 'variant', 'var', 'std::string', bool, integral, floating types, "
"std::vector, std::map, std::complex, std::set, std::tuple, std::queue, std::stack and std::pair");
}
}- Implemented a mechanism to identify the presence of a free function based on its signature using type traits and SFINAE techniques.
- Introduced new naming conventions for all serialization functions and established a new directory for housing all serialization files.
- Implemented comprehensive suite of SOG tests for all the new functions.
PR #2
- This pull request is about "Implementing (de)serialization routines for generic data types".
- Implemented support for
std::vector,std::map,std::complex,std::set,std::tuple,std::queue,std::stackandstd::pair. - Created 20 type traits.
- Implemented comprehensive suite of unit tests for all the new types using GoogleTest.
PR #3
- This pull request is about "Adding support for JSON format in the serialization system".
- Added JSON support as another option with the existing bytes format.
- Implemented serialization and deserialization functions for all the types ENIGMA has.
- Implemented comprehensive suite of unit tests for all the new functions using GoogleTest.
- Implemented comprehensive suite of SOG/unit tests for the new format.
- The serialized data now looks like:
Map object
{"false":76920.59869,"true":98610.58768}Vector object
[19700.598698,10097.59867,1220.598644]complete game state
{
"instance_list": [
{
"obj": {
"event_parent": {
"object_type": "object_collisions",
"parent": {
"object_type": "object_transform",
"parent": {
"object_type": "object_graphics",
"parent": {
"object_type": "object_timelines",
"parent": {
"object_type": "object_planar",
"parent": {
"object_type": "object_basic",
"id": 100001,
"object_index": 0
},
"x": 0,
"y": 0,
"xprevious": 0,
"yprevious": 0,
"xstart": 0,
"ystart": 0,
"persistent": "false",
"direction": {
"type": "real",
"value": 0
},
"speed": {
"type": "real",
"value": 0
},
"hspeed": {
"type": "real",
"value": 0
},
"vspeed": {
"type": "real",
"value": 0
},
"gravity": 0,
"gravity_direction": 270,
"friction": 0
},
"timeline_moments_maps": [],
"timeline_index": -1,
"timeline_running": "false",
"timeline_speed": 1,
"timeline_position": 0,
"timeline_loop": "false"
},
"sprite_index": -1,
"image_index": 0,
"image_speed": 1,
"image_single": {
"type": "real",
"value": -1
},
"depth": {
"type": "real",
"value": 0
},
"visible": "true",
"image_xscale": 1,
"image_yscale": 1,
"image_angle": 0
},
"image_alpha": 1,
"image_blend": 16777215
},
"mask_index": -1,
"solid": "false",
"polygon_index": -1,
"polygon_xscale": 1,
"polygon_yscale": 1,
"polygon_angle": 0
},
"vmap": {}
},
"locals": [
{
"name": "mydouble",
"data": {
"variant": {
"type": "real",
"value": 1.2345
},
"array1d": [],
"array2d": []
}
},
{
"name": "myuint8_t",
"data": {
"variant": {
"type": "real",
"value": 255
},
"array1d": [],
"array2d": []
}
}
]
}
],
"instance_deactivated_list": [],
"backgrounds": [],
"room_index": 0
}or:
Map object
0 0 0 0 0 0 0 2 0 40 f2 c7 89 94 3b f7 27 1 40 f8 13 29 67 23 24 c8
Vector object
0 0 0 0 0 0 0 3 40 d3 3d 26 51 11 6a 8c 40 c3 b8 cc a1 37 f3 8c 40 93 12 65 2 ee c7 c9
complete game state
00 00 00 00 00 00 00 01 aa 00 01 86 a1 00 00 00 00 ab 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 70 e0 00 00 00 00 00 00 00 00 00 00 00 00 00 ac
00 00 00 00 00 00 00 00 ff ff ff ff 00 3f 80 00 00 00 00 00 00 00 ad ff ff ff ff 00 00 00 00 3f
80 00 00 00 bf f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 3f 80 00 00 3f 80 00 00 00 00
00 00 ae 3f f0 00 00 00 00 00 00 00 ff ff ff af ff ff ff ff 00 ff ff ff ff 3f 80 00 00 3f 80 00
00 00 00 00 00 00 00 00 00 00 00 00 00 bb 00 00 00 00 00 00 00 09 00 00 00 00 00 00 00 09 6d 79
75 69 6e 74 38 5f 74 00 00 00 00 00 00 01 c8 00 00 00 00 00 00 00 0a 6d 79 75 69 6e 74 36 34 5f
74 00 00 00 00 00 00 01 8f 00 00 00 00 00 00 00 0a 6d 79 75 69 6e 74 33 32 5f 74 00 00 00 00 00
00 01 56 00 00 00 00 00 00 00 0a 6d 79 75 69 6e 74 31 36 5f 74 00 00 00 00 00 00 01 1d 00 00 00
00 00 00 00 08 6d 79 69 6e 74 38 5f 74 00 00 00 00 00 00 00 e4 00 00 00 00 00 00 00 09 6d 79 69
6e 74 33 32 5f 74 00 00 00 00 00 00 00 72 00 00 00 00 00 00 00 09 6d 79 69 6e 74 31 36 5f 74 00
00 00 00 00 00 00 39 00 00 00 00 00 00 00 09 6d 79 69 6e 74 36 34 5f 74 00 00 00 00 00 00 00 ab
00 00 00 00 00 00 00 08 6d 79 64 6f 75 62 6c 65 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 01
00 3f f3 c0 ca 42 8c 59 fb 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 df ff c0 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 df ff ff ff c0 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 41 df ff ff ff c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 40 5f c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 ef
ff e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 ef ff ff ff e0 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 ef ff ff ff e0 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 40 6f e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 ee 00 00 00 00 00 00 00 00 ef 00 00 00 00
I expanded my project by adding many things more than the required for this project like:
- Implemented support for serializing
std::set,std::queueandstd::stack. - Implemented support for serializing
std::tupleandstd::pair. - Added support for JSON format in ENIGMA serialization system.
- Gained experience in TemplateMetaprogramming(TMP) in C++, using type traits and SFINAE(Substitution Failure Is Not An Error) techniques.
- Worked with JSON format.
- Worked with
write_object_data.cppfile that is responsible for code generation during compile time. - Developed expertise in handling macros.
- Conducted unit testing using GoogleTest and the unique SOG testing technique for ENIGMA.
- Worked with complex data structres like
var. - Worked with Makefiles.
- Gained an understanding of the serialization and deserialization processes.
- Deepened my understanding of game engine architecture.
- Adding new serialization formats like XML, YAML, or any suggested format from the mentors.
- Adding support for new types.
I am grateful to Dhruv Chawla for his assistance and guidance. Dhruv was a GSoC'22 contributor to The Enigma Team, and he initiated the serialization system last year. This year, I am completing the serialization system under his guidance. As well, I am grateful to my mentors (Robert Colton, Josh Ventura and Greg Williamson) for guiding me and trusting me to complete this project.