Last time I wrote a little bit about my current GUI research progress. There are some things that were misunderstood so I want to clarify some fundamental design decisions and finally write up some current problems and their solutions.
First up I want to clarify a component is not another term for what is usually refered to as widget.
Instead wigets in this implementation are made out of n >= 1 components. Components themself are just
rectangles with attributes left, right, top, bottom, center_x, center_y, width and height some behavior flags
and an optional surface to draw into. For example a scroll regions is made up out of at least three
components: Content, Scrollbar and Cursor (optional five or more components with button up and down).
However components are not only part of a widget they are also what is refered to containers in
other retain mode GUI frameworks or other rectangle instances.
Definitions are compile time data counterparts to components and just define initial flags, size and
callbacks for drawing, layouting and input handling for their counterpart. The mapping between component
and definition are one to one just like with nodes. Nodes are tree nodes making up a component hierarchy
(notice component NOT widget hierarchy). Each node has a parent node except the root node which
has itself as parent and k >= 0 child nodes. So nodes and to an extend components can be both widget
as well as component container as well as widget container.
Components are placed on screen by constraints. Each constraint is a combinations of a simple condition to check if it even needs to be executed and a simple linear functions for changing one attribute of a component. This particular implementation has three different linear functions. The default constraint is:
destination.attribute = multiplier * source.attribute + offset
With both destination and source being components while attribute is one of each components attributes (left,right,top,bottom,center_x,center_y,width and height). The other two variables, multiplier and offset, are both constants values.
The default constraint can be extended. For example the = could be changed to support
some additional assignment operators like += or -= or like the two other constraints in this implementation
with min and max.
Important is as soon as a constraint was executed the rectangle in the destination component is unbalanced and needs to be put back into correct state since we have both relative as well as absolute attributes. A single axis looks like:
min center max
|============|==============|
<----------- len ----------->
If one of these attributes changes then all the other attributes in that axis need to be recalculated. However
each attributes can be calculated in multiple ways. So each recalculation requires an anchor and the attribute
that changed to rebalance (solve in my implementation).
Finally to address a special component flag COMPONENT_LAYER. Layers are special components used with scroll
components. All components are directly blitted to screen under normal circumstances. However the only exceptions
are components with flag COMPONENT_LAYER. The child hierarchy of these components are directly drawn into these
specific components surface/canvas and then finally blitted to screen. This allows blitting without clipping rectangles
and even more interesting it is possible to have immediate mode UI inside of COMPONENT_LAYER or in other words
scrollable components.
This is basically already everything needed to understand the core library. The rest are functions to call callbacks, handle interactions, hash table insert/lookup and fold + commit which I already described in the last post. So inherently there is not much to it but this design does quite a lot with very little.
One example of the power of this design are popups. Popups can be implemented without additional code inside the core library. Instead you just have to introduce some additional components/nodes. Currently there is only one fixed component which is the root component at the top of the component hierarchy. However you could add two additional high level components spanning the whole screen for popup control.
The main high-level popup component contains all popup component hierarchies which are all inactive by default and therefore are hidden and not interactive. The second high-level popup component is always in the background of the first high-level popup component and is used for blocking and non-blocking popup behavior. By default this component is interactive and allows underlying actual UI components to get input. However if a blocking popup was activated it will be set to non-interactive and blocks input from there on. For non-blocking components it will check if it was clicked and if so will close all non-blocking components in its parent component. Finally the underlying actual UI is a child component of the second high-level popup component.
Layers:
______________
|Root(overlay)|
--------------
|
_____v______
| Popup |
------------
|
____v_______
| Block |
------------
|
____v_______
| UI |
------------
I talked a little bit about imgui last time but not very much. The reason so far was that I was not happy with just saying "add pure immediate mode library here". It requires both quite a lot of work and code to write these pure imgui versions doing something that should already be possible with already existing code in this implementation.
The reason I did not talk about it before in more depth were resource concerns. The fold model
I proposed last time can already be abused as an immediate mode UI by recommiting each frame and keeping
and working on the last and current layout versions (you could of course even work with more layouts but
a double buffer is required at minimum). While this could make sense for very graphical heavy applications that only
use this implementation for debugging purposes on machines with good hardware, it is way to resource consuming under normal
circumstances. As a small rough estimate 10,000 components (paint.net has around 500) would cost about 5-10 MB of memory
that would be needed to be copied and filled each frame (you could parallelize some operations like copy to achieve
better speed). Granted many GUIs probably won't have these high number of components but we are talking about the
general case. So going full non-pure immediate mode is possible but cannot be used as an easy answer for everything.
However I thought about it and noticed that it is possible to nest layouts. So you could create multiple layouts
with each being either non-changing/declarative UI or full on immediate mode UI without having to write a pure imgui
library ontop of this library. So you would have one code base for editor generated compile time tables,
declarative UI and immediate mode UI. All this can be done with same core library.
Downside however is that widgets themself become more complex since they need to be able to handle all these different
use cases.
Sidenote: A layout is an instance of a tree with components, defintions and nodes. Of course if I
describe it this way it seems to be obvious that you can outsource parts of the tree as a sub-tree
but there were some uses cases like popups which I was not sure would work out.
Furthmore I just want to make clear there are differences between "pure" imgui libraries like nuklear and dear imgui and "normal" imgui libraries like this implementation. While pure imgui UI libraries pretend to be stateless on the API side they still keep track of some UI state internally but only selectively. On the other side this implementation keeps track of all UI state over time (theoretically indefinitely) and can go back to and lookup previous state. A concept which simplifies and solves a lot of nasty problems I encountered in nuklear.
However reusing layouts for immediate mode requires some small changes. The main change is that there is no free hash
table (since using an index as key only works for static layouts) anymore. However since we still know the final
number of components accumulate by the reducer we can just allocate a big int array to map between ids and indicies.
Interestingly on the API side there are no differences between immediate mode and retain mode. Static UI can use
a growing ID for identifications (generated by enum for example). Immediate mode however needs more extensive
identification since widgets need to be identifiable between changes. Still the passed ID is a unique int identifier
on the API side of things just like in retain mode but it is easier for the library user to generate a hash value
from string. So in this implementation I added a compile time
string hash macro.
Once again the biggest downside is that each widget has to support all possible ways of using it. For immediate mode
for example it has to lookup its past state or for declarations it has to construct itself initially from components.
Still Interestingly these core changes resulted in less/same code size since I could remove some parts meant for pure immediate mode. So the core implementation is still ~800 LOC composed out of ~280 lines of header and ~500 LOC implementation. So at this point the UI core already supports UI creation by:
- compile time tables generated by external editor or directly through code
- declarations run at initialization
- Immediate mode UI declared and run each frame or on each update
- Pure immediate mode UI after processing in the event handling stage (see last post)
Of course you could also combine these and use each depending on your use cases. In general the APi still look like previously stated:
QK_DECLARATION_BEGIN(app)
QK_PANEL(APP_PANEL),
QK_SCROLL_REGION(APP_PANEL_SCROLL),
QK_BUTTON(APP_OK)
QK_BUTTON(APP_CANCEL)
QK_DECLARATION_END
int main (...)
{
while (1) {
/* I.) Declarations stage */
/* retain mode */
qk_panel_begin(APP_PANEL,...);
{
qk_scroll_region_begin(APP_PANEL_SCROLL, ...);
qk_button(APP_OK...);
qk_button(APP_CANCEL...);
qk_scroll_region_end(...);
}
qk_panel_end(...);
/* immediate mode */
qk_panel_begin(qk_id("Demo"),...);
{
qk_scroll_region_begin(qk_id("Demo Scroll"), ...);
qk_button(qk_id("BUTTON_OK"),...);
qk_button(qk_id("BUTTON_CANCLE"),...);
qk_scroll_region_end(...);
}
qk_panel_end(...);
/* II.) Processing stage */
qk_blueprint(...);
qk_layouting(...);
update(...); /* update ui by input events */
qk_layouting(...);
qk_paint(...);
/* III.) Handling + pure imgui */
/* handle click of previously declared buttons */
if (button(APP_OK,...)) {
/* event handling */
}
if (button(qk_id("BUTTON_OK"),...)) {
/* event handling */
}
/* pure immediate mode list */
struct list_view view;
if (list_begin(LIST_ID, &view, ...)) {
for (int i = view.from, i < view.to; ++i)
list_push(...);
list_end(&view, ...)
}
}
}Finally I want to quickly show how widgets are constructed. All widgets are composed out of modules which each are themself
made out of component, definition, node and constraints. So widgets are not constructed by code but instead through data
tables that will be pushed into a qk_reducer buffer. Furthermore it is also possible to further push optional data as
extensions.
qk_intern void
qk_scroll_region_construct(struct qk_reducer *buf, unsigned parent, unsigned id)
{
qk_storage const int sz = 8;
qk_storage const int pad = 5;
const struct qk_module template[] = {
/* vertical scrollbar */
{.def = {QK_TBL(0, scroll_paint, 0), .flags = QK_INTERACTIVE|QK_PAINTABLE},
.id = id + 1, .parent = parent, .cnt = 4, .con = {
{{.eq = QK_CONSTRAINT_SET, .dst = {id+1,T}, .src = {parent,T}, .cons = {1.00f, +pad}}, .anch = 0},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+1,B}, .src = {parent,B}, .cons = {1.00f, -pad}}, .anch = T},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+1,W}, .src = {parent,W}, .cons = {0.00f, +sz}}, .anch = 0},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+1,R}, .src = {parent,R}, .cons = {1.00f, -pad}}, .anch = W},
}}, /* vertical scrollbar cursor */
{.def = {QK_TBL(0, scroll_cursor_paint, 0),
.flags = QK_INTERACTIVE|QK_PAINTABLE|QK_MOVABLE_Y},
.id = id + 2, .parent = id + 1, .cnt = 4, .con = {
{{.eq = QK_CONSTRAINT_CPY, .dst = {id+2,L}, .src = {id+1,L}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_CPY, .dst = {id+2,R}, .src = {id+1,R}, .cons = {1.00f, 0}}, .anch = L},
{{.eq = QK_CONSTRAINT_MAX, .dst = {id+2,T}, .src = {id+1,T}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_MIN, .dst = {id+2,B}, .src = {id+1,B}, .cons = {1.00f, 0}}, .anch = T},
}}, /* horizontal scrollbar */
{.def = {QK_TBL(0, scroll_paint, 0), .flags = QK_INTERACTIVE|QK_PAINTABLE},
.id = id + 3, .parent = parent, .cnt = 4, .con = {
{{.eq = QK_CONSTRAINT_SET, .dst = {id+3,H}, .src = {parent,H}, .cons = {0.00f, +sz}}, .anch = 0},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+3,B}, .src = {parent,B}, .cons = {1.00f, +pad}}, .anch = H},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+3,L}, .src = {parent,L}, .cons = {1.00f, +pad}}, .anch = 0},
{{.eq = QK_CONSTRAINT_SET, .dst = {id+3,R}, .src = {parent,R}, .cons = {1.00f, -pad}}, .anch = L},
}}, /* horizontal scrollbar cursor */
{.def = {QK_TBL(0,scroll_cursor_paint,0),
.flags = QK_INTERACTIVE|QK_PAINTABLE|QK_MOVABLE_X},
.id = id + 4, .parent = id + 3, .cnt = 4, .con = {
{{.eq = QK_CONSTRAINT_CPY, .dst = {id+4,T}, .src = {id+3,L}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_CPY, .dst = {id+4,B}, .src = {id+3,R}, .cons = {1.00f, 0}}, .anch = T},
{{.eq = QK_CONSTRAINT_MAX, .dst = {id+4,L}, .src = {id+3,T}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_MIN, .dst = {id+4,R}, .src = {id+3,B}, .cons = {1.00f, 0}}, .anch = L},
}}, /* content region */
{.def = {QK_TBL(qk_scroll_region_dispatch, qk_scroll_panel_paint, 0),
.flags = QK_INTERACTIVE|QK_LAYER|QK_PAINTABLE},
.id = id, .parent = parent, .cnt = 6, .con = {
{{.eq = QK_CONSTRAINT_CPY, .dst = {id,T}, .src = {id+1,T}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_CPY, .dst = {id,L}, .src = {id+3,L}, .cons = {1.00f, 0}}, .anch = 0},
{{.eq = QK_CONSTRAINT_SET, .dst = {id,B}, .src = {id+3,T}, .cons = {1.00f, -pad}}, .anch = T,
.cond = {.eq = QK_CONDITION_NEQ, .dst = {id+3,ACT}, .cons = {0,0}}},
{{.eq = QK_CONSTRAINT_SET, .dst = {id,B}, .src = {parent,B}, .cons = {1.00f, -pad}}, .anch = T,
.cond = {.eq = QK_CONDITION_EQ, .dst = {id+3,ACT}, .cons = {0,0}}},
{{.eq = QK_CONSTRAINT_SET, .dst = {id,R}, .src = {id+1,L}, .cons = {1.00f, -pad}}, .anch = L,
.cond = {.eq = QK_CONDITION_NEQ, .dst = {id+1,ACT}, .cons = {0,0}}},
{{.eq = QK_CONSTRAINT_SET, .dst = {id,R}, .src = {parent,R}, .cons = {1.00f, -pad}}, .anch = L,
.cond = {.eq = QK_CONDITION_EQ, .dst = {id+1,ACT}, .cons = {0,0}}},
}}
}; /* custom scroll region extension */
qk_storage const int rsz = qk_szof(struct qk_scroll_region);
qk_storage const int ralign = qk_alignof(struct qk_scroll_region);
const struct qk_scroll_region custom = {
.comp = id, .cnt = 2, .flags = QK_SCROLL_XY,
.s = {{id+1, id+2, QK_SCROLL_Y}, {id+3, id+4, QK_SCROLL_X}}};
qk_reducer_push(buf, template, qk_cntof(template));
qk_reducer_add(buf, QK_SCROLL_IDENTIFIER, &custom, rsz, ralign);
}
On the subject of having to copy all state every frame, would it not be possible to run some kind of diff algorithm and then only store the changes between the current and the previous state? Persistent data structures such as this are what I'm talking about. I believe this would be fairly similar to what a library like React does in combination with the browser's own DOM implementation.
Computing the diff could become costly but perhaps this could be optimized by detecting which of the (sub-)layouts was affected.