• watch their intro video

And they’re celebrating by also announcing a $10 million developer challenge, two be handed out in two $5 million dollar rounds with at least 50 individual recipients each round. Pretty exciting, eh?

So does anyone have any suggestions for “i wish my phone could do this”? I’d love to hear your thoughts…

We now no longer require a patch to Django. One import statement in settings.py allows our program to fake it via the very crafty Python language.

The new website is here: http://code.google.com/p/deseb/
The discussion list us here: http://groups.google.com/group/deseb-discuss

Plus there is an introductory video available here: http://kered.org/deseb_demo.mpeg

$ cd /<path_to_python_dir>/site-packages/django/
$ patch -p1 < ~/<download_dir>/django_schema_evolution-v096patch.txt

It should output the following:

patching file core/management.py
patching file db/backends/mysql/base.py
patching file db/backends/mysql/introspection.py
patching file db/backends/postgresql/base.py
patching file db/backends/postgresql/introspection.py
patching file db/backends/sqlite3/base.py
patching file db/backends/sqlite3/introspection.py
patching file db/models/fields/__init__.py
patching file db/models/options.py

To use it:

$ cd /<path_to_project_dir>/
$ ./manage.py sqlevolve <app_name>

It should output something like this:

BEGIN;
ALTER TABLE `main_query` CHANGE COLUMN `accuracy` `accuracynew` numeric(10, 6) NULL;
ALTER TABLE `main_query` ADD COLUMN `price` varchar(256) NULL;
COMMIT;

Assuming you have a model such as this:

class Query(models.Model):
    query = models.CharField(maxlength=256, blank=False)
    accuracynew = models.FloatField(max_digits=10, decimal_places=6, null=True, blank=True, aka='accuracy')
    price = models.CharField(maxlength=256, null=True, blank=True) # new column

Note the aka field where I changed the name of “accuracy” to “accuracynew”.

Source code:

• django_schema_evolution-v096patch.txt

Documentation:

• http://code.djangoproject.com/wiki/SchemaEvolutionDocumentation

Let me know if you find any bugs.

UPDATE: This project is now on Google Code: http://code.google.com/p/deseb/

The skinny: I will be implementing schema evolution for the Django project. Watch this category for further updates.

The meat of my proposal:

I would like to implement the introspection and migration schema evolution suggestions detailed here: [http://code.djangoproject.com/wiki/SchemaEvolution]

I am in a unique position for this proposal because I have implemented much of this functionality before. I am the PM/tech lead for a large (~400k loc) java based web application. We have a 200+ table schema, and last December I wrote a schema manager that implements much of what is described in your schema evolution proposal. (this is no coincidence – Django’s need was my catalyst/inspiration) Obviously I would not be able to use any of the code (as it was written for an Air Force contract), but the techniques would be transferable.

The schema manager I wrote (in Java, manipulating an Oracle DB, and likely on the outer difficulty edge of any similar implementation) was developed independently from your wiki’s proposal. (it was inspired by the one-line ‘this would be hard but nice to have’ comment in the 0.9.0 documentation, IIRC) However it implements everything listed in your ‘Automatically applied migration code’ suggestion. Additionally, it supports the following:

• automatic downgrades are supported (in addition to upgrades)
• the application can automatically identify a schema without a ’schema version’ table
• the application can verify the accuracy of a schema
• multiple schema identification algorithms are simultaneously supported (for instance: primary keys, foreign keys and constraints can be named in Oracle – should a constraint name change be considered a schema version change? you decide)
• multiple different schemas with ‘equivalent’ functionality can be mapped back to a single logical schema version (‘equivalency’ obviously being determined by the developer)

I would like to note that this would not be simply a porting project. For obvious legal reasons I would not have access to my previous code base. It would be in a different language. (I am familiar with Python, but not yet competent) It would be designed against a different database. And it would be for an object framework very different from my previous application experience. Nonetheless, many of the ‘hard’ problems associated with this type of functionality I have already solved, proven and deployed.

As far as the introspection functionality, this was prototyped, but never deployed for the following reasons:

• my application does not have a central, unified object definition mechanism from which to base such a function
• converting our application to such would have been prohibitive
• other development requirements took priority

However my exploratory code towards this was promising. (you all would be proud of my tuple-based model implementation in Java – made quite the use of the new 1.5 features and had full compile-time checking of all parameters)

BTW, I briefly considered a full Java port of Django last winter. (I’ve owned django4j.com for a while now) I still think it would be a good long-term idea, but a bit much for a summer project.

As far as any legal concerns you might have, I’ve contributed to open source projects before with both my company’s and the government’s knowledge and approval. (including my own project, buglist) I know the landscape well. Plus my contract will be ending soon. (on 6/30)

Anyway, I look forward to working with you all.
– Derek Anderson

P.S. Yes, getting *everyone* to save their migration scripts in a VCS should be an absolute, unwavering requirement. Good god it scares me how few people do this.