Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@dsyer
Created November 2, 2017 10:25
Show Gist options
  • Star 44 You must be signed in to star a gist
  • Fork 8 You must be signed in to fork a gist
  • Save dsyer/a43fe5f74427b371519af68c5c4904c7 to your computer and use it in GitHub Desktop.
Save dsyer/a43fe5f74427b371519af68c5c4904c7 to your computer and use it in GitHub Desktop.
Notes on Dynamic Configuration Properties

Dynamic Configuration Properties in Spring Boot and Spring Cloud

TL;DR

  • Use @ConfigurationProperties and always get state from the bean.
  • The Environment can change at runtime and Spring Cloud does this for you using RefreshEvent.
  • Changes are propagated to beans in Spring Cloud in 2 ways (@ConfigurationProperties and @RefreshScope).
  • If you care about the state of @ConfigurationProperties being consistent on concurrent access, put it or the consumer @Bean in @RefreshScope.

Typical Scenarios

  • Feature flags and canaries
  • Timeouts
  • Pool sizes
  • Polling or export frequency
  • Temporary log level changes

Features

Spring Environment

The Spring Environment is the canonical source of property values for configuration of Spring Boot applications. Spring Boot also goes the extra mile and includes properties files included by @PropertySource declarations. When we talk iof the Environment below, mostly we mean "the combined property sources managed by Spring Boot". But there is a distinction in practice and implementations sometimes need to be ware of them. The @PropertySources are not dynamic property sources (they come only from the classpath), so for the purposes of discussing dynamic configuration, we are talking about the PropertySources instance inside the Environment. It is always a ConfigurableEnvironment in a Spring Boot application (unless the user explicitly changes it), and it is always mutable, but adding and removing a PropertySource is atomic. Changes are available at runtime through the Environment itself, and also through the various public APIs provided by Spring Boot and Spring Cloud. Spring Boot in particular puts the PropertySources in a clever wrapper that detects changes and audits the access to the various property sources, so that it can report on binding errors with precise details.

ConfigurationPropertiesBinder

Spring Boot 2.0 exposes ConfigurationPropertiesBinder as the API underneath @ConfigurationProperties binding. It uses it on start up (or more precisely whenever a bean is initialized) but makes it available to Spring Cloud to use if the Environment changes.

The /refresh Endpoint

Spring Cloud has an Actuator endpoint which computes changes in the Environment and sends an EnvironmentChangeEvent with the keys that changed. It does not detect deleted keys, or changes in array lengths. It actually delegates the to a ContextRefresher, which is a public API that can be used by other interested components. There is another ApplicationEvent (RefreshEvent) that can be used to trigger a context refresh in exactly the same way (i.e. a RefreshEvent can trigger an EnvironmentChangeEvent).

ConfigurationPropertiesRebinder

Spring Cloud responds to EnvironmentChangeEvent by locating and rebinding the @ConfigurationProperties. It doesn't currently optimize this in any way (like only refreshing the beans whose keys it kows has changed) - they just all get rebound using the public API provided by Spring Boot. The rebinding actually happens just by applying the bean lifecycle callbacks using AutowireCapableBeanFactory.initializeBean(), so users can put validation and initialization logic in a @PostConstruct for example, and rely on BeanFactoryPostProcessors all being applied to the new state.

There is one exception. Any @ConfigurationProperties bean that is also in @RefreshScope is not rebound when the event is consumed. They could be rebound, but in the light of what happens in @RefreshScope, it would be redundant. Instead, they follow the usual path of @RefreshScope beans.

RefreshScope

A bean that is declared in @RefreshScope is created as a proxy. The actual target bean is also created on startup and stored in a cache with a key equal to its bean name. When a method call arrives at the proxy, it is passed down to the target. When the EnvironmentChangeEvent is consumed the cache is cleared and the BeanFactory callbacks on bean disposal are called by Spring, so the next method call on the proxy results in the target being re-created (the full Spring lifecycle, just as with any Scope).

Other Scopes

Refresh Scope is a little bit special, because it has some public APIs and events associated with it (and some other stuff). But a bean in any scope other than singleton can also result in the bean factory initializing the bean during the lifetime of the application context. For example, with @Scope("prototype") the bean is initialized every time there is a call to BeanFactory.getBean() (for that bean definition). In @RequestScope the bean is initialized on first usage per HTTP request. These beans will also pick up changes to the Environment (this has been a source of quite a lot of confusion and a few bugs in Spring Boot and Spring Cloud 2.0).

EnvironmentManager and /env Endpoint

Spring Cloud has a POSTable /env endpoint backed by an EnvironmentManager. It creates and updates a high priority PropertySource in the Environment. Changes persist in memory only, so it is useful for temporary changes, and experiments, but not for permanent

Encryption and Decryption

Spring Cloud supports decryption of Environment properties through an ApplicationContextInitializer (EnvironmentDecryptApplicationInitializer). It adds a high priority PropertySource to the Environment with decrypted values. It uses SystemEnvironmentPropertySource even if the decrypted properties do not come from the System environment variables (as a convenience so that foo.bar and FOO_BAR can both be used to bind to foo.bar, the logic for which is contained in the PropertySource in Spring Framework).

Log Levels

Since 1.5.x Spring Boot supports changing log levels dynamically via a special Actuator endpoint. Spring Cloud continues to support log level changes via the Environment.

Spring Cloud Bus

Users can fire a RefreshEvent by sending a message on the Spring Cloud Bus. It can be a "broadcast" (targetting all applications and instances) or it can target individual apps or instances via pattern matching.

Third Party Tools

There are a couple of libraries in the ecosystem that deal with Feature Flags, notably Tooglz and FF4J. Neither was designed for, nor is in use by, Spring users, but both are friendly to Spring Boot, and happily accept contributions. Most of the code is both libraries is about providing back ends for storing configuration properties, and front ends for managing the flags at runtime. Neither of these features is very interesting for Spring Boot: for configuration properties we prefer the Environment as an abstraction, and for runtime management of that there are plenty of options. The GUIs provided by the libraries are nice for demos, but probably not practical in a production system with multiple scaled up applications.

In summary: both Tooglz and FF4J can be used idiomatically by Spring Boot users as long as they stick to the Environment for configuration, and the state is updated at runtime if there is a RefreshEvent.

Concurrency

Because the Environment can be updated at runtime, there are clearly implications for components that operate concurrently. For a Spring Cloud app the trigger is nearly always the RefreshEvent. If a bean is @ConfigurationProperties it gets rebound, or if it is in @RefreshScope it gets destroyed, all in the same thread. For a vanilla Spring Boot app, only scoped proxies (e.g. @RequestScope) receive additional callbacks at runtime, by virtue of the normal Spring lifecycle. Other components can access the Environment directly, if they choose, but it isn't idiomatic and isn't really encouraged by the Spring Boot programming model.

Users of those components (other components usually) might need to be aware of the changes, or their implications at least. At a minimum they should access configuration properties that they expect to change via an injected @ConfigurationProperties bean. But callers can not rely on the state being the same from one moment to another. This means, for example, that state changes might occur in between method executions, or even during a single method execution. If things do change while a component is in use, callers might be surprised, but there is nothing we can do at present to help them generically at the framework level.

Environment

The Environment implementations that Spring uses under the hood are all backed by a MutablePropertySource that has a CopyOnWriteArrayList of PropertySource. The copy-on-write features mean that the a new PropertySource can usually be safely appended or removed at runtime. This happens, for example, when consuming a RefreshEvent, where the PropertySources are updated by replacing them with new ones that are built from the same sources. Other changes can come from mutating the PropertySource instances themselves - most of them have Map or Properties as source data (less common, but happens when the user POSTs to /env for example).

@ConfigurationProperties

Most @ConfigurationProperties beans are pure data holders and do not care or need to care about their internal state and its consistency or lack thereof, but it may be important to be aware of the implications of the state changing at runtime.

No special treatment is given to accommodate concurrent access by the ConfigurationPropertiesRebinder. On application startup all the property binding happens in a single thread which is part of the ApplicationContext lifecycle, and callers can rely on the internal state being fixed as soon as they have a reference to the bean instance. After startup things are different. Without @RefreshScope, any @ConfigurationProperties bean with a @PostConstruct that modifies its state can rely on the callback being made, but not on when it happens, so the internal state may be inconsistent when a property is accessed. Callers may need to be aware of that, or else the author of the @ConfigurationProperties could protect access to critical parts of the state of the bean using locks and/or synchronized blocks behind its public API.

@RefreshScope

Remember a bean in @RefreshScope is a proxy, wrapping a target instance of the desired type. Two consecutive method executions on the same bean in the same thread may be applied to different targets, if things get really busy. There's nothing that the scope can do itself to protect against that.

That's the bad news. The good news is that each method execution is applied to a target that is fully initialized, and therefore has consistent state (to the extent that this is required by the target). If your method execution is the one that triggers the initialization, then it all even happens in the same thread. There's not much that can go wrong with a single method execution, but the framework does have to do some work to protect callers from state changes.

The bean initialization and removal (on refresh) inside @RefreshScope is protected by a synchronized block. In addition, a disposable bean (one with a destruction callback) is detected by RefreshScope and the proxy is created in a special way, so that the call to the destruction callback takes place within a WriteLock, whereas all other method access is protected with a ReadLock from the same ReadWriteLock. There is a single instance of the lock per bean (per proxy in other words). The implication is that callers of a method in a @RefreshScope bean can be sure that they have a single, stable instance of the target bean for the duration of the method execution, and it will not be destroyed, and its state probably changed, by the BeanFactory until after the method has finished execution.

NOTE: The interceptor that applies the lock has to be inserted into the proxy before the interceptor that handles the target method calls, so that there is no window where the scope has started the destruction but the lock has not yet been acquired (there is a bug in older versions of Spring Cloud that has that flaw, but actually no-one noticed).

@ConfigurationProperties beans in @RefreshScope get special treatment by Spring Cloud (as mentioned already), so they behave just as any other @RefreshScope bean, including the callback to bind to the Environment, which comes from the Spring Boot ConfigurationPropertiesBindingPostProcessor not from the ConfigurationPropertiesRebinder.

@KirillKurdyukov
Copy link

Thanks!

@nikita2206
Copy link

@dsyer thanks a lot for this thorough and concise explainer here!

I’m looking into options for implementing feature flags for our applications in the company where I work, and I see you mentioned ff4j and Togglz but those seem to be quite stale now. If you know of any more actively supported alternatives, I would love to try them out.

More on the topic, quite a few years have passed since you wrote this, we are currently on Spring Boot 3, did anything fundamental changed here? Is there maybe now a way to ensure that all invocations in the scope of a request X will see consistent property values?

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment