learning-django.md

Where people struggle learning Django

Over the last 3 years or so I've helped a bunch of companies, small and large, switch to Django. As part of that, I've done a lot of teaching Django (and Python) to people new to the platform (and language). I'd estimate I've trained something around 200-250 people so far. These aren't people new to programming — indeed, almost all of them are were currently employed as software developers — but they were new to Python, or to Django, or to web development, or all three.

In doing so, I've observed some patterns about what works and what doesn't. Many (most) of the failings have been my own pedagogical failings, but as I've honed my coursework and my skill I'm seeing, time and again, certain ways that Django makes itself difficult to certain groups of users.

This document is my attempt at organizing some notes around what ways different groups struggle. It's not particularly actionable — I'm not making any arguments about what Django should or shouldn't do (at least not yet) — it's simply observational, an attempt to get down in writing these themes and trends that I've noticed. The organization here is by background: how do people with different backgrounds struggle when exposed to Django?

This is a work in progress. I hope to update it as I notice new trends, and I hope to document solutions should I find them.

Windows users

That is, people whose primary computing platform to date has been Windows. This includes people who develop for the Windows platform (ASP, .NET, etc.) as well as people who develop for other platforms (Java, mostly) but do so on Windows. Some have at least a small amount of Unix exposure, and most are comfortable at the Windows command shell.

These people struggle with:

• Basic shell operations. Despite being somewhat familiar with shells in general, most have trouble with Django's reliance on using the shell, and with manage.py commands in particular. They don't very much understand what something like python manage.py startapp <app> means; terms like "arguments" and "flags" in a shell context draw blank looks. They struggle with understanding how paths fit in with manage.py, often trying to run python manage.py ... from the wrong directory and getting confused with the errors about not finding manage.py
• Environment variables. I completely avoid mentioning DJANGO_SETTINGS_MODULE because I've been unable to explain adequately what the "environ" is and how it interacts with programs they run.
• Paths and general path name concepts, especially the distinction between absolute and relative paths. Something like setting DATABASES[NAME] correctly can take many attempts and trips to the shell to run pwd to get correct.
• The Python shell versus the Unix shell. Many try to type python manage.py runserver from the Python shell, or from django import ... from the Unix shell.

Java and .NET programmers

People whose primary programming experience has been in Java or .NET often struggle with:

• Missing an IDE. Many Java/.NET developers struggle with the lack of things like file trees, a "run" button, etc. It's possible this is just a failure of my class (I teach Django without the use of an IDE, just a simple text editor and the shell); perhaps I should better meet these students where they are and help them use PyDev, PyCharm, or one of the other IDEs.
• Configuration in Python. Using Python to configure databases, logging, routing, etc. is a real shock to developers used to config files (most of them with matching IDE-provided config GUIs).
• Front-end vs. backend form handling: e.g. wanting to know if Django provides a data grid or client-side validation, then not quite understanding the idea of plugging something like jQuery into a Django site. I think especially .NET users are used to "heavier" widgets and things like auto-generated JS validation.

An interesting sidebar is that .NET programmers who've been exposed to ASP.NET MVC don't have any of these problems, and in fact are usually among the fastest learners. They often report that they find Django very similar to ASP.NET MVC, and have no trouble grasping the concepts.

"Document-based" programmers (HTML, CGI, PHP, Classic ASP, etc.)

That is, people used to do doing web development in static HTML, old-school CGI, PHP (without a framework) etc. These people struggle with:

• Routing in general. The idea that a URL doesn't correspond to a file is hard to explain; many try to load URLs like /appname/views.py or /appname/name_of_a_view_function or something similar.
• The template engine, especially template inheritance.

People new to web development

Again, not people new to programming, but people newly making the leap into web development. They have trouble with:

• HTML! Simple things like making links between views are difficult if you've never seen an <a> tag. They often say things like "it's not showing up" when they've missed a closing tag, put the entire markup into their <head>, etc.
• HTTP, in particular the distinction between GET data, POST data, headers, and metadata. Further, some developers have never been exposed to the concept of the request/response cycle, and find it difficult to understand what pieces are happening in the browser and what's happening on the server.
• The breakdown between Django, HTML, CSS, and JavaScript. I'll often get questions like "How do I make a popup in Django?".

DBAs

A sizable minority of the people I've trained are transitioning from a strict DBA role to a more middle-tier app-and-database developer. Many of them are new to web development (see above) but on top of that they struggle with:

• Django's "model in Python" philosophy. These people have spent their careers nitpicking SQL, and they find the idea of having a framework generate it for them repugnant. They want to know how to map to an existing schema, and are especially put out by Django's lack of support for composite primary keys.
• ORMs in general (and Django's in particular). They want to know about performance, how to execute raw SQL, how to use views/stored procedures/triggers etc. They tend to be intensely suspicious of Django's query generation, and particularly the "automatic joining" which they see as inevitably fail-some.

People who don't know regexes

They struggle with regexes :)

Most people

Sadly, there's also a bunch of stuff that the majority of my students continue to struggle with, despite my best efforts:

• "Too many files" — by the time we've started our second app, there are over a dozen files created between the project, models, views, and templates. Students struggle with finding where to put code, how to import stuff from other files, and how to find work they've already done.
• Too many files with the same name — questions like "which models.py does that go in?" are very common, as is a frustration about not being able to tell which of a couple same-named files are which in an editor window.
• Too many settings — tasks like adding something to INSTALLED_APPS can take minutes while people sift through the file. (I suspect the new-on-truck simplified project layout should help with this.)
• The admin versus views we write. Many try loading URLs like /admin/<someroute> instead of just /<someroute>/. Perhaps this is a confusion about routing, but I think there's something specifically about the admin here because when I don't cover the admin most students don't have trouble with routing to their views.
• Views versus templates, and particularly how a view is "tied" to a template, and how the template "knows" what data to render (e.g. they miss the linkage between get_template(), the context, and the template itself).
• The overall "big picture" of how a request is actually handled and how everything fits together. They do OK with the URL -> view part, but fail after that. Most struggle to understand how model data gets into templates (for example, some think that the list of Product instances appear in the product_list.html template because of the coincidental naming similarity). Most struggle to understand how templates link to views (see above). Most fail to distinguish between the things that the framework requires (URLs route to views, views return responses) and the things that are triggered by code in the views themselves (running queries, loading and rendering templates).

Very interesting stuff, thanks for compiling. I haven't taught nearly as much as you have, but I taught a one-semester web-dev university course using Python/Django, and a lot of these points had me nodding in recognition, particularly the very first section (shell and paths) and the "document-based" section (routing and template inheritance); most of my students had done some basic web work in PHP.

I notice (from the reference to get_template) that you teach people to write their own views using the CBV framework. Do you start right off with that, or do you start with simple function views first? Since you've been doing this 4-5 years, I wonder if you've noticed any difference in how quickly people "get" the request/response stuff and the view/context/template relationship since you've made that switch?

I taught my course before the CBV framework existed, and I did spend an entire 3-hour class period on the request/response cycle in detail, but I found that students very quickly grasped the basic concept of a function view "request comes in, response goes out." The first views we did manually constructed an HttpResponse using a string, and only after that introduced template-rendering as an optional implementation detail of how a view might choose to construct the response content, and I didn't have any trouble with students not understanding that relationship. It seems like the CBV framework could really obfuscate all of that for newcomers; have you seen any correlation there?

Ah, sorry, that wasn't clear - I'm referring to template.loader.get_template (and by extension django.shortcuts.render). I don't teach CBVs yet, haven't really figured out whether I should. So it's the translation from e.g. render(request, "foo.html", ...) and /home/me/mysite/templates/foo.html that some people get thrown by.

Great points! I remember that I had some of the same thoughts when I started a few years ago.

@carljm: contributing the plural anecdote, I've noticed similar trends just in the discussion at work or in our local user group. The CBV docs were not up to the challenge, which has made me wonder whether the API could be changed - even things like the Foo.as_view() seem to radiate an aura of black magic to someone starting from scratch. TemplateResponse + wrappers/decorators solves some of the same use-cases and seems to be a much easier concept to grasp (“Oh, I get the response and change it slightly before returning it?”).

Ah, my bad assumption. In fact now that I check, there isn't even a get_template() method in the CBVs; shows how often I use them :-) Interesting though, I don't recall that being a point of confusion in my class; I think I just had them set TEMPLATE_DIRS and use render (well, back then it was render_to_response, which meant I had to explain RequestContext) and glossed over everything else (loaders etc). It definitely gets more confusing if you get into the details of template loading.

@acdha I agree that TemplateResponse + decorators is an easier pattern to understand and work with than CBVs for factoring out custom view code. My question was an honest one (if based on a bad assumption), but I do have my own opinion on it :-)

People who don't know regexes
They struggle with regexes :)

I totally agree with you.

I guess it's just hard for a programmer to grasp a framework (any) without writing a significant amount of code without one before. Why? Because when you tackle a larger project, you reinvent (poorly) all the patterns found in frameworks. You would expect a programmer to immediately recognize and understand how the various parts of the framework fit together after that, because they know the pain points (e.g., routing let me decouple code organization from URLs, templates let me decouple data model from presentation, etc.).

Maybe it would help to present how each component works in isolation by working with the API and why it was created (motivation behind it, which problems does it solve), and only later give a high-level vision of how everything ties together by writing a complete app.

A lot of these things (Too many files with the same name, too many settings, etc...) are an artifact of Django giving you full visibility of the settings by default. A lot of web frameworks that focus on minimalism have a default behavior which then is 'configured' by adding code. The 'code by difference' metaphor is really good for 80% of the problems they have covered but fail to teach people the tools they need to solve the remaining ones.

The 'clutter' is overwhelming for students but the alternative of hiding it does no favors. I know a big part of why I chose Django over other frameworks when I started learning it was because I liked the visibility of the 'request comes in, response goes out' views (and I was already an experienced web-developer). Django chooses to place large chunks of it's code and configuration directly into the user's project. I really like this more than an 'overridable defaults' kind of framework.

The places where students don't understand the linkage are places where you have to understand a hidden convention/behavior of the framework:

• I put a function in the url routes and when a request comes in then the framework calls it
• I return a response from a view and then the framework calls the template with the context
• I write a validation, and then the framework calls it

Nobody is confused by how the views and the forms are tied together because they are responsible for instantiating them. They can see it right there.

By this measure, the CBV's are a big step backwards (although very powerful and admittedly well-loved by many people). People focus on the power they give you by all of the 'code you don't have to write.' An empty class with 3 or 4 mixins will give you all the behavior you need but you have to fully understand how Django's generic class based views work to understand that code. Adding a method with a certain name, means the framework will call it. Your only visibility into it is reading the django source, or the docs (which I think is why there was a lot of complaining about the docs being insufficient when they launched).

I think it gives the appearance of simplicity (by there not being much code) but actually it's hiding the inner workings from you. Maybe a good metaphor would be the BMW idrive system[1] which seemingly hides the complexity from you, but really it's all still there and now harder to find.

[1] [http://en.wikipedia.org/wiki/IDrive]

@acdha - For the 1.5 release, te documentation on CBVs has been dramatically updated. Can you take a look at them and see if they address the difficulties encountered during your workshop?

I think when you are teaching stuff like using many different files - urls.py, models.py, views.py, templates - it's important to always start with 'why'. Obviously, you have to start with a simple example, and the natural question is, why can't I specify the url pattern on the model itself and add the view code as a method, and the template as an attribute with a few lines of text? It's good to acknowledge that for a simple example, it would make a lot more sense than splitting things up. It's best to say upfront that in a typical app you may have a couple dozen url patterns, models, views, templates and an average template may be 100-400 lines. Suddenly it makes tons of sense to split things up. And when something makes sense, it's 10x easier to remember!

Really important stuff. Django sometimes feels awesome - clean and sane like Python - sometimes au contraire.
I'm quite new to Django and programming in general. What holds me back the most is how to really kickstart your own project (real project not a toy project). Most Django tutorials stop when the really important part should kick in. After a long search I've found Two Scoops Of Django really helpful, thanks @pydanny.
In a real project you are generally encouraged to reuse existing apps and try not to reinvent the wheel, right? But, it is not easy for me to get my head around integrating my code with apps written by somone else (not only in terms of docs shortcomings but also best practices - how hould I go about it). If you are knew to Django it almost feels easier to write everything on your own. I think new users would really benefit from some guidelines for typical use cases, bast practices and working examples. It would also help with the following issue.
The number of files you have to manage is also overwhelming, I think it's because due to a lack of understanding of inner workings of the framework for a novice it's just it - files. How they are interconnected and what they 'do' needs more thorough explanation from the very get go, imho.

I'm also giving Django tutorials/courses for quite a while now. There are two different kinds of courses I do. First there are free tutorials I give at conferences to introduce people to Django. Usually I have three or four hours to do this. So the goal is to build a simple app and give a basic understanding of Django. Many things can't be explained in depth because of the small amount of time. I usually start with my django-introduction slides. I think the architecture slide helps people at lot to understand what is going on behind the scenes.

I also do longer courses (usally six days) where people have to pay for. The material I use for this courses goes much deeper than the first one. Usually it takes the whole first day to setup the environment, explain the Django basics and build a simple app. I try to go as deep as possible into the Django stack to give the people an idea what is happening when they later use render().

But in general I had similar experiences like @jacobian during my courses.

A frequent problem I've had teaching "people new to web development" is that the distinction between "Django form class" and "HTML forms" is often unclear. I find using the term "forms" is often confusing for new users when they're learning about two separate but related concepts at once that are both referred to as "forms".

Since Regex is tough, why not have DSL or library like werkzeug.route which is easier ?