json-ux-study.md

System.Text.Json Usability Study

We've performed a usability study on the new System.Text.Json APIs which will replace JSON.NET as the default JSON experience in .NET Core 3.0:

• We focused on the primary "getting started" scenarios to validate that we picked the right type- and method names.
• We simulated what we believe to be common error cases to validate that our exception messages are understandable and actionable.
• Since we were short on time, we couldn't use the recruiting provided by the UX lab but instead brought in our own developers, which we recruited from the .NET engineering team

This has worked out well for us as we were able to get short term feedback which resulted in significant improvements to the overall API shape and usability. In the future, we should do more API focused UX studies 😊

Tasks

#	Description	Code
1	Serialize and pretty print	Link
2	Deserialize with trailing comma	Link
3	Deserialize with custom property names	Link
4	Read with DOM	Link
5	Pretty print an existing JSON file	Link

Hypotheses

We expect developers to...

#	Description	Result
1	...assume the APIs are similar/identical to JSON.NET	Validated
2	...complete tasks 1-4 successfully	Validated
3	...struggle/unable to complete the task 5, because it requires working with two different components which is very different from JSON.NET	Invalidated
4	...not look at documentation for most of their coding as IntelliSense is good enough	Invalidated
5	...be happy with the experience of using the API	Validated
6	...plan on using System.Text.Json for their next JSON task	Partially Validated

Key Observations

• Everyone completed scenarios #1-#3, 2 didn't complete #4, 6 didn't complete #5
  • The reason was always "out of time". Every person who started a scenario successfully finished it.
  • We feel this validates that we picked the right scope for a 60 minute study of people from various backgrounds
• Overall people liked the new API
  • Key value prop for most people was the fact that "it's now built in"
  • Very few mentioned perf and the ones that did thought that it probably wouldn't make a difference in their scenarios.
  • Most people said they would prefer System.Text.Json for new APIs, but would stick to JSON.NET in code that already uses it.
  • [Action] None, this is in line with what we expected.
• Virtually everyone tried to google for sample code in the documentation
  • Everyone found our blog post and online documentation
  • People new to .NET were confused that JSON.NET and System.Text.Json weren't the same thing
  • Everyone complained that the current docs are lacking, especially sample code.
  • [Action] Docs are work in progress. We plan on shipping with a fully documented API surface and sample code.
• Virtually everyone found the key type relatively quickly
  • Some people were confused when types were static, rather than something that one can new up
  • [Action] None. This is by-design as we try to reduce the number of allocations.
• Virtually everyone had trouble selecting the correct method on JsonSerializer
  • We chose a very complicated naming convention
  • [Action] We simplified it by using names that people in the study told us were looking for.
• Most of our error messages were well received because they told people what the problem is & how they can fix them.
  • One error message was unclear because we reported an unexpected character when we could have reported that we didn't expect a comment.
  • [Action] We've reworded the error message.
  • We had one error message that was generated by a lower layer and provided an action item that doesn't make sense when people receive it when using the higher layer component.
  • [Action] We've reworded the error message to be more generic so it makes sense for both kind of consumers.
• Virtually everyone used the new Roslyn completion feature which will also show types from namespaces not imported yet.
  • However, virtually nobody realized that they were doing it.
  • This caused confusion when people found other APIs that happen to include Json in them, specifically the old data contract serialization.
  • [Action] Talk to the Roslyn IDE team to see if this something more people will run into. This could prove to be especially problematic for technologies that share type names (e.g. UserControl in WinForms and WPF) and both are referenced by the project.
• Optional parameters and async/await are usability cliffs
  • Many were confused by optional parameters and tried to pass them in, even though they didn't have to.
  • [Action] Talk to the Roslyn IDE team to see if we can visualize them better in VS.
  • [Action] Try to avoid heavy use of optional parameters for mainline APIs.
  • Async is a very steep cliff. People have no idea what to do in their code to make async work. While IntelliSense shows the usage by prefixing it with await, people didn't notice that.
  • [Action] Unclear; this is a broader issue.
• Virtually nobody used F1 to get help for APIs
  • Surprise. I expected more people to use F1.
• Virtually everyone used F12 to explore the API ("metadata view")
  • Surprise. I expected most people to explore the type via the IntelliSense completion
• Nobody used Object Browser to explorer the APIs
  • Surprise. I expected at least some people to.

I really want see the source code in Visual Studio. Can F12 show full source code?

I can confirm I never use F1 nor Object Browser. Google All The Time.

Wait, what F1 ? I never knew about F1, I usually use F12 and then Google.
F12 is much better than object browser.

after serialization can we encrpt the file